Multi-source method and system for coded media - Patents.com

Abstract

Embodiments are disclosed relating to multi-source methods and systems for coded media. In some embodiments, the method at a first device includes: receiving media data representing a media asset; obtaining a first plurality of data elements including at least one of bitstream identification data, content specific encoding data, and media segment data; encoding at least a portion of the media data according to a first coding process to obtain coded data corresponding to the media asset; generating a second plurality of data elements different from the first plurality of data elements based on information related to the first coding process; combining the first plurality of data elements and the second plurality of data elements into one or more coded bitstreams representing the media asset; and transmitting the one or more coded bitstreams to one or more second devices over one or more network paths.

Description

[Technical field]

[0001]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 332,210, filed April 18, 2022, U.S. Provisional Patent Application No. 63 / 332,212, filed April 18, 2022, and U.S. Provisional Patent Application No. 63 / 443,933, filed February 7, 2023, each of which is incorporated by reference in its entirety.

[0002]

[0002] TECHNICAL FIELD TECHNICAL FIELD This disclosure relates generally to encoding and decoding media. [Background technology]

[0003]

[0003] Application layer coding is an approach that can be used to mitigate the effects of lower layer protocols, networks and channels on the overall quality of data transmission from one or more sources to one or more receivers. The rate at which data is transferred over a network is highly dependent on the network characteristics (e.g., congestion, packet loss, delay, etc.). Application layer coding attempts to optimize data transfer by selecting an appropriate coding type given knowledge of the network characteristics (e.g., network capacity, delay, etc.), the specific use case (e.g., online streaming) and the overall objective (e.g., minimizing latency). Summary of the Invention

[0004] SUMMARY OF THE DISCLOSURE Embodiments of a multi-source method and system for coded media are disclosed.

[0005]

[0005] In some embodiments, a method includes, at a first device, the steps of: receiving media data representing a media asset; obtaining a first plurality of data elements including at least one of bitstream identification data, content specific encoding data, and media segment data; encoding at least a portion of the media data according to a first coding process to obtain coded data corresponding to the media asset; generating a second plurality of data elements different from the first plurality of data elements based on information related to the first coding process; combining the first plurality of data elements and the second plurality of data elements into one or more coded bitstreams representing the media asset; and transmitting the one or more coded bitstreams to one or more second devices using one or more network paths.

[0006] In some embodiments, the first coding process is a linear coding process to obtain coded data corresponding to the media asset.

[0007] In some embodiments, the first coding process is a non-linear coding process. A linear coding process that converts media data into coded data that is compatible with another linear coding process.

[0007]

[0008] In some embodiments, the data element is a data structure.

[0008]

[0009] In some embodiments, the first plurality of data elements includes a synchronization data element, and each of the one or more coded bitstreams begins with the synchronization data element.

[0009]

[0010] In some embodiments, the synchronization data element is: identification data identifying each of the one or more coded bitstreams as having been generated from a common source media asset; synchronization data for synchronizing each of the one or more coded bitstreams with other coded bitstreams generated from a common source media asset; and lineage data for each of the one or more coded bitstreams; Includes at least one of the following:

[0010]

[0011] In some embodiments, the genealogy data comprises: data identifying the original or coded symbols used to generate the coded symbols included in each coded bitstream; data identifying how the original or coded symbols were combined to generate the coded symbols included in each coded bitstream; and Each coded bitstream includes data identifying the coding coefficients used to generate the coded symbols included therein.

[0011]

[0012] In some embodiments, the method further includes interleaving synchronization data elements throughout the one or more coded bitstreams, the synchronization data elements enabling a second electronic device receiving one of the one or more coded bitstreams to begin decoding from multiple points within one of the one or more coded bitstreams.

[0012]

[0013] In some embodiments, the first plurality of data elements or the second plurality of data elements are generated based at least in part on system data associated with the received media data.

[0013]

[0014] In some embodiments, the method includes: obtaining the first plurality of data elements includes generating the first plurality of data elements from the received media data or system data related to the received media data.

[0014]

[0015] In some embodiments, encoding the media data according to a first coding process includes splitting the media data into a plurality of original symbols and generating a plurality of coded symbols based on the plurality of original symbols.

[0015]

[0016] In some embodiments, the coded symbols are generated by linearly combining a subset of the original symbols according to a network coding technique.

[0016]

[0017] In some embodiments, the media data comprises a linearly coded bitstream.

[0017]

[0018] In some embodiments, obtaining the first plurality of data elements comprises: extracting metadata from the media data; and computing a first plurality of data elements from the retrieved metadata;

[0019] In some embodiments, the metadata is extracted from the media data without performing linear decoding.

[0018]

[0020] In some embodiments, encoding the media data according to the first coding process includes re-encoding linearly coded symbols in the coded bitstream into new linearly coded symbols representing the media asset.

[0019]

[0021] In some embodiments, the method further comprises, prior to the combining step, splitting one or more data elements having a first size into data elements having a second size smaller than the first size.

[0020]

[0022] In some embodiments, the method further includes adding hash data to one or more data elements of the second plurality of data elements, thereby enabling validation, authentication, or integrity by a device receiving the one or more bitstreams. authentication, or integrity).

[0021]

[0023] In some embodiments, the method further includes adding hash data to one or more data elements of the second plurality of data elements that are associated with one or more of a file corresponding to the media asset, a block of the media asset, a segment of the media asset, and a packet of the media asset.

[0022]

[0024] In some embodiments, the combining step includes inserting the directory data element into one or more of the coded bitstreams.

[0023]

[0025] In some embodiments, the directory data element is used to assist receiving devices in identifying subatoms within each bitstream.

[0024]

[0026] In some embodiments, the one or more coded bitstreams include a first coded bitstream that includes data elements of a first type and a second coded bitstream that does not include data elements of the first type.

[0025]

[0027] In some embodiments, the data elements or subatoms used to initialize the decoder at the client are not latency sensitive data.

[0028] In some embodiments, the bitstream may include a bitstream_header(), block_header(), encoder_content_info(), or media_segment_info(). At least one of the subatoms and sync() Includes subatom.

[0026]

[0029] In some embodiments, the data elements or subatoms carry encoded data or delay sensitive data.

[0027]

[0030] In some embodiments, the bitstream contains packet() subatoms and sync() subatoms, but does not include any of the following: bitstream_header(), block_header(), encoder_content_info(), or media_segment_info() Does not contain at least one of the subatoms.

[0028]

[0031] In some embodiments, the transmitting step comprises: transmitting the first coded bitstream using a first type of communication channel; and The method includes transmitting the second coded bitstream using a communication channel of a second type different from the first type.

[0029]

[0032] In some embodiments, the second type of communication channel is a high latency channel, a low bandwidth channel, a reliable channel, or a channel using a second type of transmission protocol that is different from the first type of transmission protocol.

[0030]

[0033] In some embodiments, the method comprises, in the first device: receiving one or more coded bitstreams associated with a media asset; storing the one or more coded bitstreams in a buffer; processing one or more coded bitstreams stored in the buffer, the processing comprising: extracting a first plurality of data elements from the one or more coded bitstreams, the data elements including at least one of bitstream identification data, content-specific encode data, and media segment data; extracting a second plurality of data elements from the one or more coded bitstreams, the data elements including information related to respective coding processes used to generate the one or more coded bitstreams; and The method includes the step of performing one of: generating decoded data representing the media asset based on the second plurality of data elements; and generating newly coded data representing the media asset based on the second plurality of data elements.

[0031]

[0034] In some embodiments, generating the decoded data includes decoding the coded data using a respective decoder of the plurality of decoders in accordance with a determination of an encoding type used to generate the coded data.

[0032]

[0035] In some embodiments, the encoding type is determined from the retrieved first plurality of data elements.

[0033]

[0036] In some embodiments, the encoding type is determined from the retrieved second plurality of data elements.

[0034]

[0037] In some embodiments, the encoding type is determined from the first plurality of data elements without using the retrieved second plurality of data elements.

[0035]

[0038] In some embodiments, the encoding type is determined from the second plurality of data elements without using the retrieved first plurality of data elements.

[0036]

[0039] In some embodiments, the method further comprises deriving consumable data corresponding to the media asset from the decoded data and the first plurality of data elements. The method further includes generating a set of data.

[0037]

[0040] In some embodiments, the consumable data is compatible with an application executing on the first electronic device or a media file type supported by the application.

[0038]

[0041] In some embodiments, generating the decrypted data includes: processing the first type of coded data into a second type of coded data without performing linear coding; and Decoding the second type of coded data using a decoder capable of decoding the second type of coded data.

[0039]

[0042] In some embodiments, processing the coded data is performed by a joint coding device that aligns coefficients or projects the coded data into a common finite field.

[0040]

[0043] In some embodiments, generating the newly coded data includes processing the coded data included in the second plurality of data elements using a respective re-encoder of the plurality of re-encoders in accordance with a determination of the encoding type used to generate the coded data.

[0041]

[0044] In some embodiments, the encoding type is determined from the retrieved first plurality of data elements.

[0042]

[0045] In some embodiments, the encoding type is determined from the retrieved second plurality of data elements.

[0043]

[0046] In some embodiments, the encoding type is determined from the first plurality of data elements without using the retrieved second plurality of data elements.

[0044]

[0047] In some embodiments, the encoding type is determined from the second plurality of data elements without using the retrieved first plurality of data elements.

[0045]

[0048] In some embodiments, generating the new coded data comprises: processing the coded data of the first type into coded data of a second type without performing linear decoding; and Re-encoding the second type of coded data with a re-encoder capable of encoding the second type of coded data.

[0046]

[0049] In some embodiments, processing the coded data is performed by a joint coding device that aligns coefficients or projects the coded data into a common finite field.

[0047]

[0050] In some embodiments, the method further includes storing the newly coded data or generating one or more bitstreams from the newly coded data.

[0048]

[0051] In some embodiments, the method further includes, prior to processing the one or more encoded bitstreams stored in the buffer, combining a first quantity of data elements of the first plurality of data elements or the second plurality of data elements, each having a first size, into a smaller quantity of data elements having a second size larger than the first size.

[0049]

[0052] In some embodiments, the combining step is based on data within the CHUNKED_SUBATOM subatom.

[0050]

[0053] In some embodiments, the method comprises: retrieving integrity data from the second plurality of data elements prior to decoding or re-encoding; performing at least one of decoding or re-encoding in response to each data element determining successful verification based on the respective integrity data; and The method further includes refraining from performing at least one of decoding or re-encoding in response to each data element determining unsuccessful verification based on the respective integrity data.

[0051]

[0054] In some embodiments, the integrity data includes hash data, and verification of the hash data is based on the key information.

[0052] In some embodiments, the key information includes at least one of a public key or a private key.

[0053]

[0055] In some embodiments, the key information is received from an authentication server.

[0054]

[0056] In some embodiments, the method further includes, prior to decrypting or re-encoding, decrypting each encrypted data using a decryption key in accordance with a determination that the second plurality of data elements comprises each encrypted data.

[0055]

[0057] In some embodiments, the key information or hash data is associated with a bitstream_encryption_key_id.

[0056]

[0058] In some embodiments, encryption is applied by the media asset owner or creator to one or more received coded bitstreams associated with a media asset to prevent unauthorized access / consumption / decoding / re-encoding.

[0057]

[0059] In some embodiments, the method comprises: transmitting identification information associated with the first electronic device to a second electronic device different from the first electronic device; and The method further includes receiving a decryption key from the second electronic device that enables accessing, consuming, decoding, or re-encoding one or more coded bitstreams associated with the media asset.

[0058]

[0060] In some embodiments, a method includes, in an electronic device: obtaining, using at least one processor, a first coded bitstream of a first type; generating, using at least one processor, a second coded bitstream of a second type different from the first type; and Transmitting, using at least one processor, the second coded bitstream to one or more second electronic devices over one or more network paths.

[0059]

[0061] In some embodiments, the first bitstream of a first type is generated according to a first type of linear coding protocol or standard.

[0060]

[0062] In some embodiments, the first or second bitstreams may include systematic packets associated with the data segments and non-systematic packets associated with the data segments. packets).

[0061]

[0063] In some embodiments, the first or second bitstream includes systematic packets associated with the data segment and does not include non-systematic packets associated with the data segment.

[0062]

[0064] In some embodiments, the first or second bitstream includes non-systematic packets associated with the data segment and does not include systematic packets associated with the data segment.

[0063]

[0065] In some embodiments, the second bitstream of a second type is generated according to a linear coding protocol or standard of a second type.

[0064]

[0066] In some embodiments, the one or more network paths include more than one network, more than one network type, more than one network transmission medium, or communication channels having different performance characteristics.

[0065]

[0067] In some embodiments, the method comprises, in the first device: receiving a first bitstream associated with a media asset encoded using a first type of coding; generating a second bitstream of a second type of linear coding from the first bitstream in accordance with a determination that the first bitstream should be decoded with a decoder associated with the second type of coding, the steps including: removing data from a set of fields of the first bitstream and mapping the data to a second corresponding set of fields of the second bitstream; extracting metadata associated with the media asset from the first bitstream; and decoding the second bitstream with a decoder associated with the second type of coding into an uncoded representation of the media asset; Processing the first bitstream in accordance with a determination that the first bitstream should not be decoded with a decoder associated with the second type of coding, the step including: decoding the first bitstream into an uncoded representation of the media asset using metadata associated with the media asset and a decoder associated with the first type of coding; and distributing the uncoded representation of the media asset together with the retrieved metadata associated with the media asset to an application.

[0066]

[0068] In some embodiments, a non-transitory computer readable storage medium includes instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0067]

[0069] In some embodiments, a computing device includes at least one processor and memory storing instructions that, when executed by the at least one processor, cause the computing device to perform any of the methods described above.

[0068]

[0070] Other embodiments disclosed herein are directed to systems, devices, and computer-readable media. Details of the disclosed embodiments are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

[0069]

[0071] Certain embodiments disclosed herein provide one or more of the following advantages: Combining data elements related to a coding process (e.g., a linear coding process) with data elements including at least one of bitstream identification data, content specific encoding data, and media segment data prior to transmission allows for more efficient distribution of media data to / from devices with different processing capabilities (e.g., encoding / decoding capabilities) over networks with different performance characteristics. The combined data provides more efficient decoding and reduces the need to generate and transmit multiple redundant types of bitstreams to ensure compatibility, thereby conserving network and processing resources.

[0070]

[0072] The disclosed multisource methods and systems support application layer coding using a unique bitstream container format (hereinafter also referred to as the "disclosed bitstream") that supports different applications with a common representation and interface. The disclosed bitstream enables a scalable and interoperable method that supports the use of one or more linear codes, improving the network side efficiency and reliability of cloud and network connected processing, storage, distribution, delivery and playback of multimedia. The disclosed bitstream is also designed to carry metadata and other information (e.g., coded symbols, block count, number of symbols per block, information about coding matrices, field sizes of coding matrix coefficients) in a compatible format to comply with standardized encoder and decoder implementations of different coding types.

[0071]

[0073] The disclosed methods and systems also support interchange bitstream formats, which enable novel uses of a number of code types and use cases including single-source and multi-source media distribution over one or more networks and / or access paths.

[0072]

[0074] The disclosed methods and systems also support embodiments in which a single processing entity is capable of seamlessly processing and / or decoding multiple signaled code types on-the-fly. [Brief description of the drawings]

[0073] [Figure 1A]

[0075] 1A and 1B show the concepts of linear encoding and decoding, respectively. [Figure 1B]

[0075] Figures 1A and 1B show the concepts of linear encoding and decoding, respectively. [Figure 2A]

[0076] 2A and 2B are block diagrams of bitstream encoding and decoding, respectively, according to some embodiments. [Figure 2B]

[0076] Figures 2A and 2B are block diagrams of bitstream encoding and decoding, respectively, according to some embodiments. [Figure 3A]

[0077] FIG. 3A illustrates a hierarchical organization of data elements in the disclosed bitstream according to some embodiments. [Figure 3B]

[0078] FIG. 3B illustrates an example of data elements in a sub-atom of the disclosed bitstream according to some embodiments. [Figure 4]

[0079] FIG. 4 is a block diagram of a media processing and distribution system according to some embodiments. [Diagram 5]

[0080] FIG. 5 is a block diagram of the bitstream generator shown in FIG. 4 according to some embodiments. [Figure 6]

[0081] FIG. 6 is a block diagram of the bitstream decoder shown in FIG. 4 according to some embodiments. [Figure 7]

[0082] FIG. 7 is a block diagram of a bitstream re-encoder according to some embodiments. [Figure 8A]

[0083] FIG. 8A is a block diagram of a unified decoder according to some embodiments. [Figure 8B]

[0084] FIG. 8B is a block diagram of an integrated decoder with a decoder switch according to some embodiments. [Figure 9A]

[0085] FIG. 9A is a block diagram of a unified bitstream re-encoder according to some embodiments. [Figure 9B]

[0086] FIG. 9B is a block diagram of a unified bitstream re-encoder with a re-encoder switch in accordance with some embodiments. [Figure 10]

[0087] FIG. 10 illustrates bitstream mapping for compliance with other coding bitstream standards according to some embodiments. [Figure 11]

[0088] FIG. 11 is an example bitstream for multi-source distribution according to some embodiments. [Figure 12]

[0089] FIG. 12 illustrates a multi-source, multi-path distribution system in accordance with some embodiments. [Figure 13]

[0090] FIG. 13 illustrates a low latency unicast bitstream delivery system according to some embodiments. [Figure 14A]

[0091] FIG. 14A is a block diagram of a system for low latency unicast encoding and decoding in accordance with some embodiments. [Figure 14B]

[0092] FIG. 14B illustrates off-band processing of variable data sizes according to some embodiments. [Figure 15A]

[0093] FIG. 15A illustrates low latency unicast encoding with separate linear encoder-decoder pairs according to some embodiments. [Figure 15B]

[0094] FIG. 15B illustrates low-latency unicast encoding with an inter-generational decoder according to some embodiments. [Figure 16]

[0095] FIG. 16 illustrates an example of the use of the disclosed bitstreams for digital rights management (DRM) according to some embodiments. [Figure 17]

[0096] FIG. 17 is a block diagram of a distributed media distribution system with data authentication enabled by the disclosed bitstream in accordance with some embodiments. [Figure 18]

[0097] FIG. 18 is a flow diagram of a process for generating a bitstream from non-linear or linear coded data according to some embodiments. [Figure 19]

[0098] FIG. 19 is a flow chart of a decoding / re-encoding process according to some embodiments. [Figure 20]

[0099] FIG. 20 is a flow chart of a process for converting new code types for downstream compatibility according to some embodiments. [Figure 21]

[0100] FIG. 21 is a flow chart of another process for converting new code types for downstream compatibility according to some embodiments. [Figure 22]

[0101] FIG. 22 is a block diagram of an exemplary hardware architecture suitable for implementing the systems and methods described with reference to FIGS. 1-21.

[0102] In the drawings, a specific arrangement or order of schematic elements, such as those representing devices, units, instruction blocks, and data elements, is shown for ease of explanation. However, it should be understood by those skilled in the art that the specific arrangement or order of schematic elements in the drawings is not intended to imply that a particular order or sequence of processing, or separation of multiple processes, is required. Furthermore, the inclusion of schematic elements in the drawings is not intended to imply that such elements are required in all embodiments, or that features represented by such elements may not be included or combined with other elements in some implementations.

[0074]

[0103] Furthermore, in the drawings, when connecting elements such as solid or dashed lines or arrows are used to indicate a connection, relationship or association between or among two or more other schematic elements, the absence of any such connecting elements is not intended to imply that no connection, relationship or association may exist. In other words, some connections, relationships or associations between elements are not shown in the drawings so as not to obscure the present disclosure. Furthermore, for ease of explanation, a single connecting element is used to represent multiple connections, relationships or associations between elements. For example, when a connecting element represents communication of signals, data, or instructions, it should be understood by those skilled in the art that such element represents one or more signal paths affecting the communication, as appropriate.

[0075]

[0104] The same reference symbols used in the various drawings indicate similar elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0076]

[0105] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the various embodiments described. It will be apparent to one of ordinary skill in the art that the various embodiments described may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. Some features described below can be used independently of each other or in any combination with other features.

[0077]

[0106] term As used herein, the term "including" and variations thereof should be read as open-ended terms meaning "including, but not limited to." The term "or" should be read as "and / or" unless the context clearly indicates otherwise. The term "based on" should be read as "based at least in part on." The terms "one implementation" and "embodiment" should be read as "at least one implementation." The term "another implementation" should be read as "at least one other implementation." The terms "determined," "determine," or "determine" should be read as obtain, receive, compute, calculate, estimate, predict, or derive. Moreover, in the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0078]

[0107] In addition to the above definitions, the following additional definitions shall apply to the following description:

[0079]

[0108] "Encoding" refers to the encoding of source-coded / non-linearly-coded media data input, or the re-coding of linearly-coded media input using a linear coding scheme.

[0080]

[0109] A "finite field" is a field that has finite degree, and the numbers in the field are arithmetically closed and complete.

[0081]

[0110] A "homomorphic hash" is a family of cryptographic hashing algorithms in which a hash of a compound data block (e.g., by linearly combining or concatenating the individual data blocks) can be derived using the hashes of the individual data blocks.

[0082]

[0111] "Linear codes" are a family of redundancy codes in which a linear combination of data segments is created as redundant information. Examples of linear codes include, but are not limited to, Reed-Solomon codes, RaptorQ codes, random linear (network) codes, Hamming codes, convolutional codes, low-density parity check (LDPC) codes, and Bose-Chaudhuri-Hocquenghem (BCH) codes.

[0083]

[0112] "Linear encoding" is the operation of generating linear redundancy by linearly combining data segments.

[0084]

[0113] "Linear decoding" is the operation of solving linear equations to recover the original data.

[0085]

[0114] "Linear re-encoding" is the operation of generating new linear redundancy, either by decoding and encoding, or by further linearly combining linear combinations.

[0086]

[0115] "Metadata" is high-level descriptive data that helps a receiver process the media data, such as media type, frame rate, bit rate, etc.

[0087]

[0116] "Post-processing" is the process of converting the distributed data into consumable media, for example by applying unpackaging and / or decompression.

[0088]

[0117] "Preprocessing" refers to processing the raw media data into a suitable format, such as applying source coding techniques (e.g., compression) to the raw media data and / or packaging the media data using a specific representation such as HTTP-like streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH).

[0118] Application Layer Coding Overview To assist the reader in better understanding the disclosed embodiments, the following provides an overview of application layer coding, and in particular linear coding.

[0089]

[0119] The majority of application layer codes are linear codes, and encoding data typically follows three basic steps: 1) Divide the data into a number of blocks. The number of blocks can be small (e.g. 1) or large, depending on the size of the data; 2) Each block is divided into a number of source symbols X i where i is the index of the source symbol within the block, each source symbol is of a specified size (e.g., in bytes), the final source symbol is padded with zero-valued bytes if necessary, and the number of source symbols is typically selected based on the application and coding type used; and 3) Encode the source symbols to produce (depending on the coding type) an equal or greater number of coded symbols Y i Create a.

[0090]

[0120] The mathematical representation of the linear code encoding and linear code decoding steps are shown in Figures 1A and 1B, respectively. In the illustrated example, the original data file or stream is divided into multiple blocks. Although this example focuses on the linear encoding of one block, the linear code encoding / decoding steps are repeated for each block in the data file or stream.

[0091]

[0121] Referring to FIG. 1A, given a general linear code, the steps for encoding data are as follows:

[0092] Step 1: Split each block of data into a number of source symbols. In this example, four source symbols are created from the input block of data.

[0093] Step 2: Formulate a vector of source symbols (X ^ ).

[0094] Step 3: Generate a coding matrix (G). Each element of the matrix is ​​selected from a Galois field GF{q}, where q = 2m. In this example, the coding matrix is ​​6x4.

[0095] Step 4: The coding matrix (G) and the input source symbols (X ^ ) to compute the coded symbol (Y). The Galois field of GF{2} is used, and y1 ^ The coefficient vector of [a 1,1 ,a 1,2 ,a 1,3 ,a 1,4 ]=[1 0 0 0], then the first coded symbol y1 is equivalent to the first source symbol x1, because the coefficient vector has only one nonzero coefficient. In this case, y1 can be referred to as a systematic symbol.

[0096]

[0122] After encoding, the coded symbols and the coding coefficients of the coding matrix are transmitted to a decoder. In another embodiment, the coded symbols are cached in a network, CDN, etc., and delivered upon request to one or more client devices that contain a suitable decoder. The coding coefficients of the coding matrix are: (a) transmitted to a decoder; (b) the means for generating the coding coefficients is parameterized and transmitted to the decoder; or (c) It is possible that the coefficients are known and assumed in both the encoder and the decoder.

[0097]

[0123] With knowledge of the coding coefficients at the decoder, it is possible to disentangle the original symbols from the coded symbols, for example using approaches such as Gaussian elimination. This process is simpler in the presence of systematic symbols, since they are guaranteed to be linearly independent. At least as many coded symbols as there are source symbols should be received by the decoder for decoding. In some cases, more coded symbols are needed if the coding coefficients generated are not linearly independent of each other.

[0098]

[0124] Referring to FIG. 1B, the steps for decoding the above exemplary linear code are as follows.

[0099] Step 1: Determine the coefficient vector associated with each coded symbol to generate a received coding coefficient matrix A.

[0100] Step 2: Inverse of A, A -1 Calculate.

[0101] Step 3: The inverse of the received coding coefficient matrix (A -1 ) and the vector of received coded symbols (Y ^ ) to obtain the original vector of symbols (X ^ ) to restore the

[0102] Step 4:X ^ The original block data is reconstructed by concatenating each symbol in the block.

[0125] Exposed bitstream as a container 2A and 2B are block diagrams of bitstream encoding and decoding, respectively, according to some embodiments. The disclosed bitstream (see FIGS. 3A and 3B) can be thought of as a container that supports different application layer coding types. A particular coding type may be selected for a particular use case. Many coding types have their own formats that produce different kinds of bitstreams, but the goal of the disclosed bitstreams is to support different applications using a common representation and interface.

[0103]

[0126] If application layer coding is used, information is transmitted to the receiver which recovers the original data. For example, in addition to the coded symbols, the block count, the number of symbols per block, information about the coding matrix, and the field size of the coding matrix coefficients are transmitted to the receiver. The disclosed bitstream shown in Figures 3A and 3B is designed to carry the metadata and this information in a compatible format, following standardized encoder and decoder implementations for different coding types. A bitstream mapper is used to convert the information generated by the application layer encoder into data element types supported by the disclosed bitstream.

[0104]

[0127] 2A, system 200a includes media file 201, application layer encoders 202a...202n, bitstream mapper 203, and bitstream encoder 204. Bitstream mapper 203 converts information generated by bitstream encoders 202a...202n into data element types supported by a disclosed bitstream generated by bitstream encoder 204. Bitstream encoder 204 generates a disclosed bitstream including supported data element types and content metadata 205 that is output by bitstream mapper 203.

[0105]

[0128] Referring to FIG. 2B, system 200b includes a bitstream decoder / parser 206, a demapper 207, and application layer encoders 208a...208n. Bitstream decoder / parser 206 uses content metadata 205 to decode the received disclosed bitstream and parse data elements in the disclosed bitstream. Bitstream demapper 207 converts data element types into types required by standardized application layer decoders 208a...208n. The output of application layer decoders 208...208n, along with content metadata 205, are combined into a media file 209.

[0129] Example of disclosed bitstream structure FIG. 3A illustrates a hierarchical organizational structure 300 of the disclosed bitstream 301 according to some embodiments. The disclosed bitstream 301 provides an extensible container that enables several new, more efficient and reliable storage, caching, distribution, and delivery paradigms for multimedia applications than currently provided by existing methods. These new multimedia applications are not within reach utilizing today's source coding and highly redundant storage methods for delivering media over single and / or multiple communication channels / networks. More specifically, the disclosed bitstream 301 enables a scalable and interoperable method that supports the use of one or more linear codes to increase the efficiency and reliability of the network side of cloud and networked multimedia processing, storage, distribution, delivery, and playback.

[0106]

[0130] In an exemplary embodiment, a source coded multimedia asset is mapped by one or more linear coding processes into one or more functionally equivalent binary representations or "bitstreams" to generate multiple functionally equivalent (FE) bitstreams according to the format of the disclosed bitstream 301 shown in FIG. 3A. In some embodiments, each of the FE bitstreams generated by the linear coding processes begins with a SYNC subatom data structure 302. The information defined in the SYNC data structure 302 is used for subsequent processes to synchronize with, identify, and interpret the disclosed bitstream 301. In some embodiments, the SYNC data structure 302 and the information defined in the SYNC data structure 302 may be included within a subatom (bitstream data structure) and injected at any point within the disclosed bitstream 301 for applications requiring resynchronization. The information contained within the SYNC data structure 302 provides a universal entry point into bitstreams generated for applications described in the disclosed embodiments, including, but not limited to, forward compatibility with future applications having future linear code types and subatoms.

[0107]

[0131] Following the SYNC data structure 302, each disclosed bitstream 301 includes a number of subatom data elements 303a...302n (hereinafter referred to as "subatoms") that contain information necessary to enable one or more applications. Each subatom 303a...303n includes a subatom_id field (not shown) that enables the disclosed bitstream 301 to describe and carry one or more unique subatoms and the information necessary to enable one or more applications. As discussed above, the use of the disclosed subatom_id bitfield enables the definition and use of several unique subatom data structures for the disclosed bitstream 301 format.

[0108]

[0132] As shown in Figure 3A, each sub-atom 303a, 303b, 303c...303n is a generic container in which different types of data can be placed. Since a sub-atom can carry only one type of data, the type of data carried within the sub-atom determines the sub-atom type and associated data structure. For example, the BITSTREAM_HEADER sub-atom 305 is an instance of the sub-atom 303a syntactical structure that carries the bitstream header syntonical structure (BITSTREAM_HEADER).

[0109]

[0133] Hereinafter, the sub-atoms and their respective bit fields may be referred to more generally as "data elements." Additionally, a sub-atom type may be referred to by its name (e.g., bitstream sub-atom) or by its associated syntactic structure (BITSTREAM_HEADER()). In some embodiments, each sub-atom 303a, 303b, 303c...303n has the following syntactic structure: SYNC 304, BITSTREAM_HEADER 305, MEDIA_SEGMENT INFO 306, ENCODER_CONTENT_INFO 307, BLOCK_HEADER 308, PACKET 309, BLOCK_DIRECTORY 310, CHUNKED_SUBATOM 311 and PACKET_HEADER_ONLY 312 A valid disclosed bitstream 301 may contain a single subatom, but a decoder may receive multiple subatoms, including BITSTREAM_HEADER, BLOCK_HEADER(S), and PACKET(S), to successfully decode the disclosed bitstream 301. The syntax allows additional subatom data types to be defined in the future for new applications.

[0110]

[0134] 3B further illustrates the data elements in each sub-atom of the disclosed bitstream 301 container format according to some embodiments. The data carried by a unique instance of a sub-atom, described below, is either independent or dependent on other sub-atoms. Correct parsing / interpretation and / or decoding of a sub-atom may require that data from previous sub-atom instances have been correctly parsed and / or processed. The data elements of a sub-atom are described below.

[0111]

[0135] The SYNC sub-atom 304 is used to identify and synchronize two bitstreams that are generated from the same source media asset. Some examples of the meaning / usage of the data elements (e.g., bit fields) of the SYNC sub-atom 304 are described below.

[0112] syncword: A unique byte sequence that is used to identify that the disclosed bitstream conforms to the format of the disclosed bitstream 301.

[0113] version: the specific version of the bitstream syntax that follows. This bitfield allows the bitstream to be extended in a backward and forward compatible manner.

[0114] ● content_encode_uuid: A unique identifier that, in one example, allows a multi-source media system (e.g., including encoders, decoders, re-encoders, parsers, processors, etc.) to implicitly signal a specific 'encode' of a media asset. In one example, this information is useful for downstream processes to efficiently (and independently) determine the lineage of coded bitstreams produced by one or more encoders or re-encoders. Bitstreams coded with this level of implicit signaling of lineage are advantageous in that they allow a multi-source media system to operate most efficiently by ensuring that all coded media sources (distributed and available across multiple nodes, including storage / caches in a network) contain unique and functionally equivalent coded media representations.

[0115]

[0136] The BITSTREAM_HEADER subatom 305 contains high level generic file information about the linearly encoded data such as the linear encoding method used, bitstream profile information, and an indication of whether any part of the bitstream is encrypted. Some examples of the meaning / usage of the bitstream_header bit fields are as follows:

[0116] - content_source_size: indicates the size of the linearly encoded media in bytes.

[0117] ● content_source_type: indicates more information about the subsequent linearly encoded media source, such as whether the encoded stream is sourced from the original content, an encoded variant, or a live stream.

[0118] - linear_code_type: Indicates what type of linear code is being used to encode the content.

[0119] profile_information: Processing of bitstreams conforming to a particular profile useful in applications allowing efficient determination of whether a particular bitstream is suitable for a particular decoder / parser with restricted or limited capabilities.

[0120] ● encrypted_coefficients: enables implicit signaling that encryption of coding coefficients is active.

[0121]

[0137] Encryption of the coding coefficients can be considered a form of digital rights management, and therefore can reduce or eliminate the need to fully encrypt the linearly coded media asset itself.

[0122]

[0138] The MEDIA_SEGMENT_INFO sub-atom 306 is used for media streaming applications, where content is divided into segments (e.g., audio and video segments) in network and over-the-top (OTT) media streaming applications / services. Hereafter, MEDIA_SEGMENT_INFO is also more generally referred to as "media segment data". The structure of the MEDIA_SEGMENET_INFO sub-atom 306 provides information about the media in the segment, such as audio / video properties (e.g., frame rate, bit rate, codec type, etc.). This information would typically be included in a manifest file (e.g., a DASH manifest or an HLS manifest). This information is included in this structure to avoid the need to decode the bitstream to inspect the linearly encoded data. Example fields of the MEDIA_SEGMENT_INFO sub-atom 306 are as follows: ● segment_index: Identifies the media index of the segment represented by the current subatom.

[0123] content_type: indicates the type of multimedia contained in this media segment, such as audio, video, text, or application data.

[0124] aspect_ratio: The ratio between the width and height of the video frame_rate: number of video frames per second audio_channel_config: Audio configuration (e.g. channel, location, object, virtualized, etc.).

[0125]

[0139] The ENCODER_CONTENT_INFO sub-atom 307 identifies the particular linear encoder used to create the bitstream, along with a unique identifier representing the underlying media content. The ENCODER_CONTENT_INFO sub-atom 307 also carries information that allows verification of the integrity (trustworthiness) of the underlying stream / file / block information, and information about the duration of the underlying stream / file / block information when it is decoded. Some examples of the meaning / usage of the ENCODER_CONTENT_INFO sub-atom 307 are as follows:

[0126] ● encoder_uuid: A unique identifier that, in one example, allows a multi-source media system (e.g., one that includes encoders, decoders, recorders, parsers, processors, etc.) to intrinsically signal the lineage of the coded bitstreams generated by one or more encoders or recorders. Coded bitstreams with intrinsic signaling of lineage are a key feature of the disclosed embodiments, allowing a multi-source media system to operate most efficiently by ensuring that all coded media sources (distributed and available across multiple nodes, including storage / caches in a network) consist of unique and functionally equivalent coded representations of the media.

[0127] ● content_id_type: indicates the type of identification format used to identify the content, such as EIDR (Entertainment Identification Registry), Ad-iD (Advertising Digital Identifier), or a user-defined value.

[0128] content_id: Holds the ID value according to the content_id bitfield.

[0129]

[0140] In some embodiments, the above data elements are distributed across a distributed system (including a content-centric network). It allows processing entities in the network to efficiently identify information about the underlying coded media without the need to decode the media, communicate with a centralized server, or look for stream / segment manifests (if applicable). This feature is one of several key elements enabling future media delivery paradigms that will not utilize HTTP delivery.file_integrity: a unique data structure that provides a mechanism to verify that the received and decoded stream / block matches that of the original source. The delivery.file_integrity structure is a mapping to the fb_integrity() bitfield, which specifies further information including, for example, the hash function used to hash the file / stream / block, the hash size, and the hash value. In one example, the fb_integrity() bitfield allows any processing node in the network to independently verify the authenticity of the coded multi-source bitstream. This feature can be leveraged to store multi-source coded media using traditionally untrusted media sources such as mobile devices, personal computers (PCs), set-top boxes, digital media adapters, and home gateways.

[0130]

[0141] The BLOCK_HEADER 308 contains information about the linear coding parameters that are applied to the data in the block of data, such as block size, symbol size, number of symbols, finite field size, etc. Examples of the meaning / usage of the BLOCK_HEADER data elements are as follows:

[0131] block_index: A multi-source coded media segment may be divided into blocks, each of which is coded separately. This field indicates the index of the block.

[0132] ● symbol_size: The size in bytes of the systematic and linearly coded symbols.

[0133] number_of_symbols: signals the number of coded symbols into which the block is divided for linear coding.

[0134] ● field_size: The size of the underlying finite field on which the linear coding operates.

[0135] ●Encryption parameters: These fields are used to decrypt the encrypted coding coefficients.

[0136] prng_seed: Linear codes can use a pseudo-random number generator to produce randomized coding decisions for the current block. Prng_seed is the seed used to initialize the pseudo-random number generator in the decoder or recorder.

[0137] ● block_integrity: A structure that provides a mechanism to verify that a received and decrypted block matches the block from the original source. The structure may include information such as the hash function used to hash the block, the hash size, and the actual hash. In one example, an implementation may hash all systematic symbols in the current block using SHA1 or a homomorphic hash, and then use a hash aggregator such as a Merkle tree to aggregate the symbol hashes to obtain the block hash.

[0138]

[0142] The PACKET subatom 309 contains the basic part of the linearly coded data, i.e. the coded symbol. It also contains a packet header data structure that provides information about this coded symbol, such as whether it is a systematic symbol, the window size, and the coding coefficients. Examples of data elements in the PACKET subatom 309 are: packet_header: A set of bit fields that aid in decoding the packet, such as: 1) block_index: A segment may be divided into blocks, and each block is coded separately. This field indicates the index of the block to which this packet belongs.

[0139] 2) b_systematic: indicates whether this packet carries systematic symbols or linearly coded symbols.

[0140] 3) poly_index: This is a generic field used to index packets. It can be used to systematically identify the original symbol index of a packet. It can also be used to synchronize packet subatoms with packet header-only subatoms.

[0141] 4) coeff_vector: the sequence of finite field scalars used to linearly combine the systematic packets to produce this symbol.

[0142] 5) window_start_index: A packet may be generated using a contiguous subset of systematic symbols (i.e., a window). This field indicates the index of the first systematic symbol of this window.

[0143] 6) window_start_index: A packet may be generated using a contiguous subset of systematic symbols (i.e., a window). This field indicates the index of the last systematic symbol in this window.

[0144] 7) prng_seed: The seed for the pseudorandom number generation k used to generate the coefficients for this symbol.

[0145] 8) packet_integrity: A structure that provides a mechanism to verify that a received packet is correctly generated before using this packet for decoding or re-encoding. The structure may contain information such as the hash function used to hash the packet, the hash size, and the actual hash.

[0146]

[0143] In some embodiments, a minimal packet_header may be included in the PACKET subatom 309, but the complete packet_header may be stored and distributed separately / independently (e.g., out of band, in a separate network session, etc.). With only the minimal packet_header, the payload may not be available to decrypt the data. This approach allows for an additional and simpler method of digital rights management, for example, by suppressing access to the complete packet_header.

[0147]

[0144] The BLOCK_DIRECTORY sub-atom 310 includes a field that indicates the start of a block header sub-atom instance within the disclosed bitstream 301. In some embodiments, the block directory sub-atom 310 includes one or more of the following fields: block_header_subatom_offsets(), which describes the offset in bytes from the end of the BLOCK_DIRECTORY sub-atom 310 to the beginning of the BLOCK_HEADER sub-atom 308 within the same disclosed bitstream 301.

[0148]

[0145] CHUNKED_SUBATOM 311 provides a mechanism to split large subatom data into smaller chunks. For example, if the coded symbol size in PACKET subatom 309 is too large for the transmission protocol used (e.g., a UDP packet has a payload of about 1400 bytes), the PACKET subatom 309 can be split into smaller chunks that fit the transmission requirements. Each chunk is then carried by one CHUNKED_SUBATOM 311. Some exemplary bit fields are as follows:

[0149] ● chunked_segment_id: Identifies a series of chunked_subatom instances as containing different portions of the same subatom data. When the subatom data is chunked, all CHUNKED_SUBATOM 311 instances created shall have the same chunk_segment_id field.

[0150] chunked_segment_index: identifies the index of the chunked segment of the subatom data contained within the chunked_subatom instance. The parser can use this information to reassemble the chunked subatom data into its original order.

[0151] num_chunked_segments: Describes the number of chunk segments into which the subatom data (associated with a given chunk_segment_id) is divided.

[0152] ●original_subatom_id: Specifies the subatom_id of the segment data contained within this chunked_subatom instance.

[0153] ● chunked_data: The chunked segment of data carried by this chunked_subatom instance.

[0154]

[0146] The PACKET_HEADER_ONLY subatom 312 is identical to the PACKET subatom 309, except that it does not contain any actual coded symbols.

[0147] Example Media Processing and Distribution System 4 is a block diagram of a media processing & distribution system 400 according to some embodiments. The system 400 includes pre-processing blocks 401a, 401b...401n, bitstream generators 402a, 402b...402n, a communication network 403, a bitstream decoder 404, and a post-processing block 405. The network / cloud 403 includes re-processing 406 and a bitstream regenerator 407. In some embodiments, the system 400 is connected to a network, e.g., a content delivery network (CDN), The RESTful service may be partially or fully implemented by one or more servers, such as a Content Delivery Network (CDN) server, a Point-of-Presence (PoP) server, an application server (e.g., a server hosting an electronic news gathering application, a content broadcasting application, a unicast application, etc.), a content cache, a peer device, a local caching device, an edge device (e.g., a 5G / 6G edge application server), or a client device (e.g., a media consumption device, computer, mobile device, gaming system, streaming device, audio / video equipment, etc.).

[0155] In this exemplary embodiment, media data is provided to one or more servers. The media data is obtained from a media asset and includes, but is not limited to, images, audio, video, game state, game commands, etc. In some embodiments, the media data is source coded data, data packaged according to a streaming protocol (e.g., HLS, DASH, etc.). In some embodiments, the media data is coded bitstreams and linear coded data associated with a media asset (e.g., coded bitstreams including coded content and associated metadata). In some embodiments, media data representing a media asset is received from multiple network paths.

[0156]

[0149] The pre-processing blocks 401a...401n in the server pre-process the raw media data. Examples of pre-processing include, but are not limited to, applying source compression techniques and / or packaging the raw media data, for example as defined by the HLS and DASH standards. The server also includes respective bitstream generators 402a...402n to generate bitstreams. Different bitstream generators 402a...402n can generate different bitstreams, for example with different formats or different payloads. The bitstreams are then transmitted using a communication network 403 that can cache, route, and / or forward the bitstreams. The communication network 403 can also include entities that can further modify the bitstreams before retransmitting them, such as by re-processing 406 and / or regenerating 407 the bitstreams.

[0157]

[0150] A receiver of the bitstreams (e.g., a consumer device) downloads one or more bitstreams and recovers the processed media data using a bitstream decoder 404. The processed media data is then post-processed by a post processor 405 and made ready for use, e.g., read for consumption by an end user. Optionally, the bitstream decoder 404 sends feedback 408 to the bitstream generators 402a...402n to help upstream processes in the system 400 adapt to dynamic conditions in the communication network 403. Example of bitstream generation Figure 5 is a block diagram of a bitstream generator 500 (e.g., one of the bitstream generators 402a...402n of Figure 4) according to some embodiments. Pre-processed media data and optionally system information that may aid in decision making are provided to the bitstream generator 500. The bitstream generator 500 generates the disclosed bitstream 301, which includes a number of sub-atoms a as described with reference to Figures 3A and 3B.

[0158] In this exemplary embodiment, the metadata generator 501 analyzes the pre-processed media data and generates the SYNC sub-atom 304 and the MEDIA_SEGMENT_INFO sub-atom 306 .

[0159] The linear encoder 502 performs linear encoding on the media data by splitting the media data into original (source) symbols and then generating linear combinations of the original symbols as linearly coded symbols, e.g., as described with reference to Figures 1A and 1B. The linear encoding information, together with the coded data, is carried by the data elements BITSTREAM_HEADER, ENCODER_CONTENT_INFO, BLOCK_HEADER, and PACKET subatoms described with reference to Figures 3A and 3B, where each packet subatom corresponds to one linearly coded symbol.

[0160]

[0152] In some embodiments, the security sealer 503 digitally signs the linearly coded media data by adding hashed information about the files, data blocks, and packets to the corresponding ENCODER_CONTENT_INFO, BLOCK_HEADER, and PACKET subatoms. This step allows for intrinsic signaling of information required for subsequent processing throughout the media system for independent verification / authentication of the coded bitstream / files generated in accordance with the disclosed invention.

[0161]

[0153] In another exemplary embodiment, the bitstream generator 500 can also separate packet header information from the packet subatoms and use a dedicated PACKET_HEADER_ONLY subatom 312 to carry the packet header information and control access to the corresponding PACKET subatom 309. The coding coefficients in either the PACKET subatom 309 or the PACKET_HEADER_ONLY subatom 312 are encrypted to add a layer of protection to the media content. In some embodiments, each subatom can be further truncated by a subatom truncator 504 and carried by multiple CHUNKED_SUBATOM subatoms 311 to reduce the size of the subatom. This embodiment is particularly useful when the transport protocol has limitations on the data unit size.

[0162]

[0154] The sub-atoms are then multiplexed by a multiplexer 505 ("Muxer") to form one or more bitstreams 301. Different bitstreams 301 may carry different sub-atoms and may be processed differently (e.g., cached, distributed). To help the receiver navigate the sub-atoms within each bitstream, the multiplexer 505 may generate and insert a BLOCK_DIRECTORY sub-atom 310 into each bitstream on the fly.

[0155] Disclosed bitstream decoding examples FIG. 6 is a block diagram of a bitstream decoder 600 (e.g., bitstream decoder 404 shown in FIG. 4) according to some embodiments. Bitstream decoder 600 receives data from one or more bitstreams having a disclosed bitstream format, where each bitstream includes one or more subatoms. Bitstream download is performed by a downloader 601 that manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing connections, listening to connections, and terminating connections, and controlling the rate and amount of data downloaded from each media source. In some embodiments, a subatom parser 602 parses the bitstream to extract and temporarily cache received CHUNKED_SUBATOM subatoms 311, and a chunked subatom merger 603 merges the CHUNKED_SUBATOM subatoms 311. The merged CHUNKED_SUBATOM subatom 311 is then verified by the security checker 604 if security / authentication / integrity data is present in the media asset.

[0163]

[0156] The verified sub-atoms are separated by a demultiplexer 605, the media metadata is extracted and fed to a post-processor 607 (e.g. an audio / video decoder) and the linearly coded data is fed to a linear decoder 606 to recover the original media data. The decoded media data together with the extracted media metadata are fed to the post-processor 607 for post-processing so that consumable media data is available at the output of the post-processor 607.

[0157] Examples of re-encoding of the disclosed bitstreams 7 is a block diagram of a bitstream re-encoder 700 according to some embodiments. The re-encoder 700 downloads data from one or more disclosed bitstreams 701, each bitstream 701 including one or more sub-atoms 304-309, 312. The download is performed by a downloader 702 that manages the download (e.g., using a buffer), including but not limited to discovering and selecting media content sources, establishing connections, listening to connections, and terminating connections, and controlling the rate and amount of data downloaded from each media source. A sub-atom parser 703 parses the bitstream 701 to extract and temporarily cache received CHUNKED_SUBATOM sub-atoms 311, and a chunked sub-atom merger 704 merges the CHUNKED_SUBATOM sub-atoms 311. The merged CHUNKED_SUBATOM subatom 311 is then verified by the security checker 705 if security / authentication / integrity data is present in the media asset.

[0164]

[0158] The verified sub-atoms are then separated by a demultiplexer 706 and the media metadata is used to recreate the SYNC sub-atom 304 and the MEDIA_SEGMENT_INFO sub-atom 306, while the linearly coded data is supplied to a linear re-encoder 707 to create new linearly coded data.

[0165]

[0159] The creation of the new linearly coded data can be performed in various ways. For example, the re-encoder 700 can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as the new linearly coded symbols. For another example, the re-encoder 700 can directly linearly combine the linearly coded symbols to create the new linearly coded symbols.

[0166]

[0160] In some embodiments, the linearly encoded information is carried along with the linearly encoded data in the BITSTERAM_HEADER, ENCODER_CONTENT_INFO, BLOCK_HEADER, PACKET, and PACKET_HEADER_ONLY subatoms, with each PACKET subatom corresponding to one new linearly coded symbol. The security sealer 708 digitally signs the media data by adding hashed information about the file, data block, and packet to the corresponding ENCODER_CONTENT_INFO, BLOCK_HEADER, and PACKET subatoms.

[0167]

[0161] In some embodiments, each subatom can be further truncated by a subatom truncator 709 and carried by multiple CHUNKED_SUBATOM subatoms to reduce the size of the subatom. This is particularly useful when the transport protocol has limitations on the data unit size. The subatoms are then multiplexed by a multiplexer 710 to form and output one or more bitstreams 711. Different bitstreams can carry different subatoms and can be processed differently (e.g., cached, distributed). To help the receiver navigate the subatoms within each bitstream, the multiplexer 710 generates and inserts a BLOCK_DIRECTORY subatom into each bitstream.

[0162] Example of an integrated decoder 8A is a block diagram of a unified decoder 800a capable of recovering media data from multiple bitstreams implementing different linear coding methods according to some embodiments. The different bitstreams 801 can use different linear codes, which are signaled per the code_type bit field in the BITSTREAM_HEADER subatom 305 of the bitstream 801. The download is performed by a downloader 802 that manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing connections, listening to connections, and terminating connections, as well as controlling the rate and amount of data downloaded from each media source. A subatom parser 803 parses the bitstream to extract and temporarily cache received CHUNKED_SUBATOM subatoms 311, which are merged by a chunked subatom merger 804. The merged CHUNKED_SUBATOM subatom 311 is then verified by a security checker 805 if security sealing has been enabled for this media asset.

[0168]

[0163] The verified sub-atoms are separated by a demultiplexer 806, the media metadata is extracted and fed to a post-processor 809 (e.g. an audio / video decoder) and the linearly coded data from the different linear codecs is fed to a coding merging unit 807. The coding merging unit 807 merges the linearly coded data from the different linear codecs, for example by matching the coefficients and / or by projecting the coded data into the same finite field. The merged linearly coded data is then fed to a linear decoder 808 to recover the original media data. The decoded media data is then fed to the post-processor 809 for post-processing using the extracted media metadata, so that consumable media data can be obtained at the output of the post-processor 809.

[0169]

[0164] Figure 8A is a block diagram of a unified decoder 800b using a decoder switch 810 according to some embodiments. The unified decoder 800b includes a linear decoder module 808 that encapsulates multiple different decoder types. The module 808 allows access to a specific type of linear codec via the decoder switch 810. More specifically, the decoder 800b receives data from one or more bitstreams 801, each bitstream 801 being composed of one or more subatoms 303 (see Figures 3A and 3B). Different bitstreams 801 use different linear codecs. The download is performed by a downloader 802 that manages the download (e.g., using a buffer), including but not limited to discovering and selecting media sources, establishing connections, listening to connections, and terminating connections, as well as controlling the rate and amount of data downloaded from each media source.

[0170]

[0165] In some embodiments, the sub-atom parser 803 parses the bitstream 801 to extract and temporarily cache the received CHUNKED_SUBATOM sub-atoms 311, which are merged by the chunked sub-atom merger 804. The merged CHUNKED_SUBATOM sub-atom 311 is then verified by the security checker 805 if security sealing is enabled for this media asset. And, if security sealing is enabled for the media asset, the complete sub-atom is verified by the security checker 805.

[0171]

[0166] The verified sub-atoms are then separated by a demultiplexer 806 and the media metadata is extracted and fed to a post processor 809 (e.g. an audio / video decoder) and fed to an integrated linear decoder module 808 using a specific linear codec. The integrated linear decoder module 808 identifies the type of linear codec used, signaled per the code_type bit field in the bitstream_header() of the disclosed bitstream, and dynamically activates and switches the decoder switch 801 to the appropriate linear decoder type to decode the original media data. The decoded media data is then fed to the post processor 809 for post processing with the help of the extracted media metadata, so that consumable media data can be obtained from the output of the post processor 809.

[0167] Example of a Unified Bitstream Re-Encoder FIG. 9A is a block diagram of a unified bitstream re-encoder 900a according to some embodiments. The re-encoder 900a is capable of creating a new (including linearly independent) bitstream from one or more bitstreams using different linear codecs. More specifically, the re-encoder 900a downloads data from one or more bitstreams 901, each bitstream 901 containing one or more sub-atoms 303 (see FIG. 3A and FIG. 3B). The download is performed by a downloader 902 that manages the download (e.g., using a buffer), including but not limited to discovering and selecting sources, establishing connections, listening to connections, and terminating connections, as well as controlling the rate and amount of data downloaded from each media source. A sub-atom parser 903 parses the bitstream 901 to extract and temporarily cache CHUNKED_SUBATOM sub-atoms 311, which are merged by a chunked sub-atom merger 904. The CHUNKED_SUBATOM subatom 311 being merged is then verified by the security checker 905 if security sealing is enabled for the media asset.

[0172]

[0168] The verified sub-atoms are then separated by a demultiplexer 906, the media metadata is used to recreate the SYNC sub-atom 304 and the MEDIA_SEGMENT_INFO sub-atom 306, and the linearly coded data using the different linear coders is provided to a coding merging unit 907. The coding merging unit 907 merges the linearly coded data using the different linear codecs, for example by matching coefficients and / or by projecting the coded data into the same finite field. The merged linearly coded data is provided to a linear re-encoder 808 to create new linearly coded data.

[0173]

[0169] The generation of the new linearly coded data can be performed in various ways. For example, the re-encoder 900a can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as the new linearly coded symbols. As another example, the re-encoder 900a can directly linearly combine the linearly coded symbols to create the new linearly coded symbols.

[0174]

[0170] In some embodiments, the linearly encoded information is transmitted along with the linearly encoded data in the BITSTERAM_HEADER subatom 306, the ENCODER_CONTENT_INFO subatom 307, the BLOCK_HEADER subatom 308, the PACKET subatom 309, and the PACKET_HEADER_ONLY subatom 310, with each PACKET subatom 309 corresponding to one new linearly coded symbol. A security sealer 909 seals (e.g., digitally signs) the media data by adding hashed information about the file, data block, and packet to the corresponding ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, and PACKET subatom 309.

[0175]

[0171] In some embodiments, each subatom can be further truncated by a subatom truncator 910 and carried by multiple CHUNKED_SUBATOM subatoms 311 to reduce the size of the subatom. This is particularly useful when the transport protocol has limitations on the data unit size. The subatoms are then multiplexed by a multiplexer 911 to form one or more output bitstreams 912. Different bitstreams 912 can carry different subatoms and can be processed differently (e.g., cached, distributed). To help the receiver navigate the subatoms within each bitstream 901, the multiplexer 911 can generate and insert a BLOCK_DIRECTORY subatom 310 into each bitstream 901.

[0176]

[0172] Figure 9B is a block diagram of a unified bitstream re-encoder 900b with a re-encoder switch 914 according to some embodiments. The unified bitstream re-encoder 900b illustrates another implementation of the disclosed unified bitstream re-encoder 900a shown in Figure 9A, which implements re-encoders for multiple types of linear codecs and encapsulates them within a linear re-encoder module 913. The linear re-encoder module 913 allows access to a particular type of linear codec via a switch 914.

[0177]

[0173] More specifically, the re-encoder 900b downloads data from one or more bitstreams 901, each bitstream 901 containing one or more subatoms 303 (see Figures 3A and 3B). The download is performed by a downloader 902, which manages the download (e.g., using a buffer), including but not limited to discovering and selecting sources, establishing connections, listening to connections, and terminating connections, as well as controlling the rate and amount of data downloaded from each media source.

[0178]

[0174] In some embodiments, a subatom parser 903 parses the bitstream 901 to extract and temporarily cache CHUNKED_SUBATOM subatoms 311, which are merged by a chunked subatom merger 904. The merged CHUNKED_SUBATOM subatoms 311 are then verified by a security checker 905 if security sealing is enabled for the media asset.

[0179]

[0175] The verified sub-atoms are then separated by a demultiplexer 906, the media metadata is used to recreate the SYNC sub-atom 304 and the MEDIA_SEGMENT_INFO sub-atom 306, and the linearly coded data using different linear coders is fed to a linear re-encoder module 913. The linear re-encoder module 913 activates a re-encoder switch 914 to select a particular re-encoder type, for example based on a switch signal in the bitstream metadata.

[0180]

[0176] The generation of the new linearly coded data can be performed in various ways. For example, the re-encoder 900b can first apply linear decoding to decode the original symbols, and then create a new linear combination of the original symbols as the new linearly coded symbols. As another example, the re-encoder 900b can directly linearly combine the linearly coded symbols to create the new linearly coded symbols.

[0181]

[0177] In some embodiments, the linearly encoded information is transmitted along with the linearly encoded data in the BITSTERAM_HEADER subatom 306, the ENCODER_CONTENT_INFO subatom 307, the BLOCK_HEADER subatom 308, the PACKET subatom 309, and the PACKET_HEADER_ONLY subatom 312, with each PACKET subatom 309 corresponding to one linearly coded symbol. A security sealer 909 then security seals (e.g., digitally signs) the media data by adding hashed information about the file, data block, and packet to the corresponding ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, and PACKET subatom 309.

[0182]

[0178] In some embodiments, each subatom can be further truncated by a subatom truncator 910 and carried by multiple CHUNKED_SUBATOM subatoms to reduce the size of the subatom. This is particularly useful when the transport protocol has limitations on data unit size. The subatoms are then multiplexed by a multiplexer 911 to form one or more output bitstreams 912. Different bitstreams 912 can carry different subatoms and can be processed differently (e.g., cached, distributed). To help the receiver navigate the subatoms within each bitstream 901, the multiplexer 911 can generate and insert a BLOCK_DIRECTORY subatom into each bitstream 901.

[0179] Example of Bitstream Mapping for Standard Compliance FIG. 10 illustrates a bitstream mapping process 1000 for conformance with other coding bitstream standards according to some embodiments. The disclosed bitstream 301 and its implementations can also function as a container format for carrying linearly encoded data generated using various linear coding techniques, even if they have been previously defined or standardized. The process 1000 supports existing linear codes by using a bitstream mapper 1003 and a bitstream demapper 1007. The disclosed mapping process 1000 allows the disclosed bitstream to remain compliant with existing linear coding methods that have been standardized and implemented in existing media products.

[0183]

[0180] Referring to Fig. 10, the media data to be protected is first encoded using an existing standardized linear encoder 1002 of the considered linear coding technique. The output is the standardized bitstream of this technique / method. A bitstream mapper 1003 then extracts fields of this bitstream and maps them to the disclosed bitstream 301 having the format described with reference to Figs. 3A, 3B. The bitstream mapper 1003 also extracts rich media metadata from the application layer 1001 to fill metadata related fields in the disclosed bitstream 301. The bitstream is then ready for distribution over the communication network 1004.

[0184]

[0181] At the receiver side, there are two options depending on whether an existing standardized decoder is used or not. First, if an existing standardized decoder is used 1005, the bitstream demapper 1007 maps the coding related data to an existing standard bitstream that is input to the standardized linear decoder 1008, which outputs the media data to the application layer 1009. Meanwhile, the bitstream demapper 1007 also extracts the rich media metadata from the disclosed bitstream 301 and passes it to the receiver's application layer 1009. If an existing standard decoder does not need to be used 1005 or if there is no existing standard decoder, the receiver uses the bitstream decoder 1006 to decode the media data and metadata for the receiver's application layer 1009.

[0185] 11 illustrates an example of two FE coded copies of a media file generated using the disclosed bitstream and bit generator according to some embodiments, each of which includes instances of the disclosed subatoms, including all or part of the SYNC subatom 304, BITSTREAM_HEADER subatom 305, MEDIA_SEGMENT_INFO subatom 306, ENCODER_CONTENT_INFO subatom 307, BLOCK_HEADER subatom 308, PACKET subatom 309, BLOCK_DIRECTORY subatom 310, and PACKET_HEADER_ONLY 312 subatoms.

[0186]

[0183] In particular, the BLOCK_HEADER subatom 310 and the PACKET subatom 309 in the different copies are generated in a manner that provides maximum linear independence between the PACKET subatoms 309 across all media sources used. This is achieved by configuring the disclosed bitstream generators uniquely for the different media sources. Examples of configurations that can result in linear independence include applying different random seeds or using different coding coefficient vectors in the different bitstream generators. In this way, any coded media PACKET subatom 309 from any copy generated will be useful for recovery with a high probability and thus will be a FE copy (hereinafter also referred to as "FE copy(s)"). If a FE copy carries enough linearly independent PACKET subatoms 309 to allow media stream / file recovery, this copy is called a full FE coded copy. Otherwise it is called a partial FE coded copy.

[0187]

[0184] Figure 12 illustrates a distributed multi-source, multi-path distribution system 1200 enabled by the disclosed bitstream format according to some embodiments. The system 1200: 1) The original media file that was source coded; 2) several clusters 1202, 1203, 1204 of caches, each cluster including connected caches capable of storing, processing (e.g., re-encoding and decoding), and serving data files (some examples of clusters include, but are not limited to: a CDN, a set of edge compute nodes, an ISP node with caching capabilities, a set of peer users, a hybrid of different types of caches); and 3) end users 1205 who desire and consume the original media files;

[0188]

[0185] In the existing configuration of the system 1200, the origin server 1201 distributes an original media file for replication to all of the connected caches 1202, 1203, 1204. In contrast, in the disclosed system using the disclosed bitstream 301, the origin server 1201 uses the disclosed bitstream generator (see FIG. 4) to create different FE coded copies of the original media file and then distributes each to the different connected caches 1202, 1203, 1204.

[0189]

[0186] In the existing configuration of the system 1200, a cache that is not connected to the original server 1201 connects to its neighbors to create a copy of the original media file. In contrast, in the disclosed system using the disclosed bitstream 301, such a cache connects to multiple neighbors and efficiently utilizes the FE copies of the neighbors to create its own FE coded copy using the disclosed bitstream re-encoder described with reference to FIG. 9. Also, the newly created FE copy can be a full or partial copy. The caches 1202, 1203, 1204 can even be end users 1205 that want to contribute as peers.

[0190]

[0187] The system 1200 is a distributed media FE distribution system, where all caches store different but still the same FE coded copies of the original media file. This allows efficient multi-source multi-pass download: an end user 1205 can download in parallel from multiple caches 1202, 1203, 1204 over different communication links, and as long as the total amount of downloaded data is slightly more than the original media file (due to header overhead etc.), the end user 1205 can finish all the downloads and recover the original media file using the disclosed bitstream decoder 404 shown in Figure 4. In this way, the end user 1205 can enjoy the aggregated bandwidth and stability of all communication links without increasing the download cost.

[0191]

[0188] The above-described download process is superior to existing single-source, single-path download processes, in which an end user 1205 only connects to one of the caches 1202, 1203, 1204 to download the original media file. The performance of the existing download processes is limited by the single link between the cache and the end user 1205, and is vulnerable to cache or link failure. It is noted that the disclosed download process is superior to existing replication-based multi-source, multi-path download processes, at least because the existing replication-based multi-source, multi-path download processes require time-consuming and error-prone scheduling between downloads to avoid downloading duplicate data from the media sources.

[0192]

[0189] Figure 13 illustrates a low latency unicast bitstream delivery system 1300 according to some embodiments. The system 1300 illustrates an example of the use of the disclosed bitstream 301 in a low latency unicast application, where a receiver requires low latency and reliable delivery of a data stream over a lossy communication channel. In this use case, a media stream is pre-processed by a pre-processor unit 1301 into small chunks that can be sent to the receiver in a short time, such as video frames. A bitstream generator 1302 is applied to the pre-processed bitstream to generate the disclosed bitstream 301. Subatoms 1303 that carry metadata and do not change frequently, such as SYNC subatom 304, BITSTREAM_HEADER subatom 305, and MEDIA_SEGMENT_INFO subatom 306, are then delivered to the receiver over a reliable control channel that may have higher latency and lower bandwidth than other channels.

[0193] Meanwhile, the linearly coded associated subatoms 1304, such as the BLOCK_HEADER subatom 308 and the CHUNKED_PACKET_SUBATOM subatom 311, are transmitted to the receiver via a low-latency lossy data channel to enable low-latency decoding of the data chunks for post-processing. Optionally, the bitstream decoder 1305 observes the channel conditions and sends feedback to the bitstream generator 1302 via a feedback channel to help the bitstream generator 1302 make informed coding decisions.

[0194]

[0191] Figure 14A is a block diagram of a system 1400 for low-delay unicast encoding and decoding focusing on PACKET subatoms 309 according to some embodiments. A pre-processed media stream 1401 is provided to a bitstream generator 1402 in time ticks, each tick containing one or more equal-length data symbols to be linearly coded. In the illustrated example, ticks 1, 2 and 4 each contain one data symbol, while tick 3 contains two data symbols (3.2, 3.1). The bitstream generator 1402 creates one PACKET subatom 309 for each data symbol as a systematic PACKET subatom 309, and creates linearly coded packets by applying linear coding to these data symbols. The receiver-side bitstream decoder 1403 uses the received systematic PACKET subatom 309 and the coded packet, along with other supporting subatoms, to decode the tick data and pass the data to the receiver application layer.

[0195]

[0192] Figure 14B illustrates off-band processing of variable data sizes via size markers 1404 according to some embodiments. Pre-processor 1401 generates data on a tick-by-tick basis, and each tick's data may have a different size. In some embodiments, size markers 1404 measures the size of each tick's data and adds size information to the tick data before sending it to bitstream generator 1405. This size information is included in the data from bitstream generator 1405 and linearly encoded. In the illustrated example, there are three ticks of data: P1, P2, and P3. Their sizes are added by size markers 1404, which become part of the data.

[0196]

[0193] Inside the bitstream generator 1405, a zero padding unit 1406 pads the data with zeros to make their lengths equal for linear encoding. In some embodiments, the length is the largest tick data among the ticks being encoded together. In some embodiments, a global maximum length is applied. In the illustrated example, three ticks are being linearly encoded together, and P1 is the largest, so the symbol length is set to the length of P1 prepended with its size. Thus, P2 and P3 are padded with zeros before linear encoding.

[0197]

[0194] When transmitting the tick data as a systematic PACKET subatom 309, padding zeros do not need to be transmitted because when the bitstream decoder 1408 receives a systematic PACKET subatom 309, it can pass the symbols in the PACKET subatom 309 directly to a padding peeler 1409, described below. The padding peeler 1409 parses the size and ensures that there are no zeros to peel. The padding peeler 1409 removes the size prefix and passes the tick data to the application.

[0198]

[0195] In the illustrated example, assuming that the transmission of P3 as a systematic PACKET subatom 309 is successful, the bitstream decoder 1408 receives P3 with size information added without zero padding and passes it to the padding peeler 1407. Setting the b_systematic flag to True in the PACKET subatom 309 is used to make the decision. When the bitstream decoder 1408 receives the coded PACKET subatom 309, it knows through the size of the coded symbols how many zeros should be padded into the received systematic symbols to enable decoding.

[0199]

[0196] When the systematic symbols are obtained by decoding, the bitstream decoder 1408 does not know how many zeros were padded. In this case, the bitstream decoder 1408 passes the zero-padded symbols to the padding peeler 1409, which analyzes the size information and performs the peeling process accordingly. In the illustrated example, it is assumed that the transmission of the systematic PACKET subatom 309 of P2 has failed, and P2 is extracted by decoding. The decoded P2 has padded zeros, which are removed by the padding peeler 1409 after analyzing the size information.

[0200]

[0197] Figure 15A illustrates a low latency unicast individual linear encoder-decoder pair according to some embodiments. In particular, Figure 15A illustrates how the disclosed bitstream implementation can be used to protect multiple ticks of data 1501, focusing on linear encoding and decoding. In this way, during each tick, one linear encoder 1502 is generated. This linear encoder 1502 treats the data from this tick and several older ticks as a block and applies linear coding to the block. At the receiving side, each linear encoder 1502 is paired with a linear decoder 1503 that decodes the corresponding block of data, and passes the decoded tick data directly to the application or to a buffer 1504 that reorders the tick data and removes duplicates.

[0201]

[0198] The above method can be implemented in an adaptive manner. For example, the number of ticks that are encoded together can be dynamic based on factors such as network quality, system latency budget, and other suitable factors. As another example, the ticks at which the encoders should be generated can be dynamically determined based on factors such as network quality and system latency budget instead of one encoder per tick.

[0202]

[0199] Figure 15B illustrates low latency unicast using a cross generation decoder according to some embodiments. In particular, Figure 15B illustrates how the disclosed bitstream 301 can be used to protect multiple ticks of data 1501, focusing on linear encoding and decoding. In this method, during each tick, one linear encoder 1502 is generated. This linear encoder treats data from this tick and several older ticks as a block and applies linear coding to this block. At the receiving side, a cross block linear decoder 1505 is implemented, which collects PACKET subatoms 309 from one or more blocks and decodes them together, for example, by solving an expanded set of linear equations. The cross block decoder 1505 also reorders the tick data, removes duplicates, and then passes the data to the receiving application layer.

[0203]

[0200] In some embodiments, the above method may be implemented in an adaptive manner. For example, the number of ticks that are coded together may be dynamic, based on factors such as network quality, system latency budget, etc. As another example, the number of blocks that the cross block decoder 1505 decodes together may be dynamically determined, based on factors such as network quality, system latency budget, etc.

[0204]

[0201] Figure 16 illustrates a system 1600 using the disclosed bitstream 301 for digital rights management (DRM) according to some embodiments. In particular, Figure 16 illustrates how the proposed bitstream enables simple digital rights protection. To decode or re-encode linearly encoded media data, the receiver 1602 obtains linear coding information such as coding coefficients, generation size, packet size, random seed, etc. Otherwise, the receiver 1602 cannot recover the media data. Thus, by encrypting such information in the corresponding subatoms of the proposed bitstream, the content owner 1601 can prevent unauthorized access to the media data. This enables a simple and straightforward DRM system using the proposed bitstream.

[0205]

[0202] In some embodiments, when preparing the content, the content owner 1601 encrypts the linear coding information using a specific cryptographic encryption scheme (e.g., AES). The encrypted bitstream is then distributed to the receiver 1602 via a communication network. The content owner 1601 then stores a decryption key in the authentication server 1603. The authentication server 1603 authenticates the receiver 1602 based on its ID and sends the decryption key to the receiver 1602 if the authentication is passed. The receiver 1602 then uses the decryption key to decrypt the linear coding information and uses the information to decrypt or re-encode the media data.

[0206]

[0203] Figure 17 is a block diagram of a distributed media distribution system 1700 with data authentication according to some embodiments. In particular, Figure 17 illustrates how the disclosed bitstream enables data authentication in a distributed media distribution system where there may be untrusted caching nodes such as peers.

[0207] In the illustrated exemplary embodiment, a content producer (or content creator) 1701 encodes media data into multiple bitstreams using the disclosed bitstreams with security sealing enabled. For example, file integrity is included in the ENCODER_CONTENT_INFO subatom 307, block integrity is included in the BLOCK_HEADER subatom 308, and packet integrity is included in the PACKET subatom 309. The content producer 1701 then distributes the bitstreams to a network 1703 of caching nodes 1704-1706. Although three caching nodes are shown in this example, multiple caching nodes can be included in the network 1703.

[0208]

[0205] The caching nodes 1704-1706 can either store the bitstream unchanged or create new bitstreams via the re-encoding functionality enabled by the disclosed bitstream format, where untrusted caching nodes 1705, 1706 may be hostile and capable of generating corrupted bitstreams, in particular PACKET subatoms 309 that are not correctly linearly coded.

[0209]

[0206] A receiver node 1707 (which may be an end user or a new caching node) can authenticate a PACKET subatom 309 that it downloads from an untrusted caching node 1705, 1706 by first downloading the correct ENCODER_CONTENT_INFO subatom 307 and BLOCK_HEADER subatom 308 to obtain file and block integrity information. Such information, if properly generated (e.g., using homomorphic hashing), can be used to verify the integrity of a PACKET subatom 309 generated by an untrusted node.

[0210]

[0207] To download the correct ENCODER_CONTENT_INFO sub-atom 307 and BLOCK_HEADER sub-atom 308, the receiver node 1707 can use a trusted caching node 1704, such as one handled by an authorized party. Alternatively, since the ENCODER_CONTENT_INFO sub-atom 307 and the BLOCK_HEADER sub-atom 308 are static for each media asset, the content creator 1701 can cryptographically sign these two sub-atoms, so that the receiver 1707 can download these two sub-atoms from anywhere and verify them by the content creator's cryptographic signature.

[0211]

[0208] Furthermore, if the receiver node 1707 finds that the PACKET subatom 309 received from the node is corrupted, it may perform, for example, forensic fraud proof. The user may report this violation to the regulation manager 1702 of the system 1700 by submitting a fraud proof. The regulation manager 1702 may then penalize the cheating node by such means as fines, cancellation of membership, blacklisting the node from the network 1703, etc.

[0209] Process Example 18 is a flow diagram of a process 1800 for generating a bitstream from non-linearly or linearly coded data according to some embodiments. The process 1800 can be implemented, for example, using the electronic device architecture disclosed in connection with FIG.

[0212]

[0210] The process 1800 includes, at a first device, receiving media data representing a media asset (1801); obtaining a first plurality of data elements including at least one of bitstream identification data, content specific encoding data, and media segment data (1802); encoding at least a portion of the media data into coded data corresponding to the media asset according to a first linear coding process (1803); generating a second plurality of data elements different from the first plurality of data elements based on information related to the first linear coding process (1804); combining the first plurality of data elements and the second plurality of data elements into one or more coded bitstreams representing the media asset (1805); and transmitting the one or more coded bitstreams to one or more second devices using one or more network paths (1806).

[0213]

[0211] In some embodiments, process 1800 is performed by one or more devices, including, but not limited to, one or more servers: for example, a content delivery network (CDN) server, a point-of-presence (PoP) server, one or more application servers (e.g., servers hosting electronic news gathering applications, content broadcasting applications, unicast applications, etc.), content caches, peer devices, local caching devices, edge devices (e.g., 5G / 6G edge application servers), or client devices (e.g., media consumption devices, computers, mobile devices, gaming systems, streaming devices, audio / video equipment, etc.).

[0214]

[0212] In some embodiments, media assets include, but are not limited to, images, videos, game state, game command data, and the like.

[0215]

[0213] In some embodiments, the media data may be source coded data, data packaged according to a streaming protocol (e.g., HLS (HTTP-like streaming), DASH (dynamic Streaming)). In some embodiments, the media data may be received from multiple network paths, including, but not limited to, linear coded data or a coded bitstream associated with a media asset (e.g., a coded bitstream including coded content and associated metadata). In some embodiments, the media data representing a media asset is received from multiple network paths.

[0216]

[0214] In some embodiments, the media segment data identifies characteristics of the media asset in a respective segment or block (e.g., MEDIA_SEGMENT_INFO). In some embodiments, the first plurality of data elements includes metadata associated with the media asset (e.g., derived from, corresponding to, received together with) (SYNC and MEDIA_SEGMENT_INFO subatoms). In some embodiments, the first plurality of data elements does not include linearly coded payload data.

[0217]

[0215] In some embodiments, encoding with a first linear coding process (e.g., a linear coding process corresponding to a supported linear code type) includes applying a linear coding technique to a packet including at least a first portion of the media data (e.g., generating a coded packet that is a combination of a plurality of packets corresponding to at least a portion of the media data).

[0218] In some embodiments, the encoding with the first linear coding process is performed without applying a linear coding technique to at least a portion of the media data (e.g., the process does not include creating an encoded packet by combining multiple packets corresponding to at least a portion of the media data). For example, the encoding process may include applying a mapping function (e.g., one or more mapping rules or tables) associated with the first linear coding process to convert at least a portion of the media data into coded data without requiring application of the first linear coding process.

[0219]

[0217] In some embodiments, the second plurality of data elements includes the coded payload data and metadata associated with the media asset (e.g., one or more of the BITSTREAM_HEADER sub-atom 305, the ENCODER_CONTENT_INFO sub-atom 307, the BLOCK_HEADER sub-atom 308, the PACKET sub-atom 309, or the PACKET_HEADER_ONLY sub-atom 312 and their associated bit fields).

[0220]

[0218] In some embodiments, combining includes interleaving, multiplexing, or otherwise converting into a sequential signal or representation for transmission or storage.

[0221]

[0219] In some embodiments, one or more network paths include more than one network, more than one network type (e.g., LAN, VLAN, WAN, SAN, WLAN, VPN, etc.), more than one network standard (e.g., 1G / 2G / 3G / 4G / 5G / 6G Mobile, Bluetooth, 802.11 based WIFI, etc.), more than one network transmission medium (e.g., wireless, wired, fiber, etc.), communication channels having different performance characteristics (e.g., channels utilizing non-lossless protocols, reliable protocols, low latency).

[0222]

[0220] Figure 19 is a flow diagram of a decoding / re-encoding process 1900 according to some embodiments. The process 1900 can be implemented using, for example, the electronic device architecture disclosed in connection with Figure 22.

[0223]

[0221] The process 1900 includes, in a first device, a step of: receiving one or more coded bitstreams associated with a media asset (1901); a step of storing the one or more coded bitstreams in a buffer (1902); a step of processing the one or more coded bitstreams stored in the buffer including: extracting a first plurality of data elements from the one or more coded bitstreams including at least one of bitstream identification data, content specific encoding data, and media segment data (1903); extracting a second plurality of data elements including information from the one or more coded bitstreams associated with respective coding processes used to generate the one or more coded bitstreams (1904); and a step of performing one of generating decoded data representing the media asset based on the second plurality of data elements and generating new coded data representing the media asset based on the second plurality of data elements (1905).

[0224]

[0222] In some embodiments, the first device may be a server (e.g., a content delivery network (CDN) server, a point-of-presence (PoP) server, an application server (e.g., a server hosting an electronic news gathering application, a content broadcasting application, a unicast application, etc.), a content cache, a peer device, a local caching device, an edge device (e.g., a 5G / 6G edge application server) or a client device (e.g., a media consumption device, a computer, a mobile device, a gaming system, a streaming device, audio / video equipment, etc.).

[0225]

[0223] In some embodiments, the one or more coded bitstreams include bitstreams coded according to one of at least two different linear coding protocols / standards.

[0226] In some embodiments, the processing includes demultiplexing and / or parsing.

[0225] In some embodiments, media segment data is used to identify characteristics of the media asset in each segment or block (eg, MEDIA_SEGMENT_INFO subatom 306).

[0227] In some embodiments, the first plurality of data elements includes metadata associated with (e.g., derived from, corresponding to, received together with) the media asset (e.g., the SYNC sub-atom 304 and the MEDIA_SEGMENT_INFO sub-atom 306).

[0227] In some embodiments, the first plurality of data elements does not include linearly coded payload data.

[0228]

[0228] In some embodiments, the second plurality of data elements includes coded payload data and metadata associated with the media asset (e.g., one or more of the BITSTREAM_HEADER sub-atom 305, the ENCODER_CONTENT_INFO sub-atom 307, the BLOCK_HEADER sub-atom 308, the PACKET sub-atom 309, or the PACKET_HEADER_ONLY sub-atom 312).

[0229]

[0229] In some embodiments, the coded data corresponds to different of at least two different linear coding protocols / standards.

[0230]

[0230] Figure 20 is a flow diagram of a process 2000 for converting new code types for downstream compatibility according to some embodiments. The process 2000 can be implemented, for example, using the electronic device architecture disclosed in connection with Figure 22.

[0231]

[0231] The process 2000 includes, in a first electronic device: obtaining a first coded bitstream of a first type (2001); generating a second coded bitstream of a second type different from the first type (2002); and transmitting the second coded bitstream to one or more second electronic devices via one or more network paths (2002).

[0232] In some embodiments, the first bitstream of the first type is a first bitstream of a first type linear coding protocol or standard (e.g., RaptorQ TM ) is generated according to

[0233]

[0233] In some embodiments, the first or second bitstream includes systematic packets associated with the data segment and non-systematic packets associated with the data segment.

[0234]

[0234] In some embodiments, the first or second bitstream includes systematic packets associated with the data segment and does not include non-systematic packets associated with the data segment.

[0235]

[0235] In some embodiments, the first or second bitstream includes non-systematic packets associated with the data segment and does not include systematic packets associated with the data segment.

[0236]

[0236] In some embodiments, the second bitstream of the second type is generated according to a linear coding protocol or standard of a second type (eg, a Reed-Solomon code).

[0237]

[0237] In some embodiments, one or more network paths include more than one network, more than one network type (e.g., LAN, VLAN, WAN, SAN, WLAN, VPN, etc.), more than one network standard (e.g., 1G / 2G / 3G / 4G / 5G / 6G Mobile, Bluetooth, 802.11 based WIFI, etc.), more than one network transmission medium (e.g., wireless, wired, fiber, etc.), communication channels having different performance characteristics (e.g., channels utilizing non-lossless protocols, reliable protocols, low latency protocols), etc.

[0238]

[0238] Figure 21 is a flow diagram of another process 2100 for converting new code types for downstream compatibility according to some embodiments. The process 2100 can be implemented, for example, using the electronic device architecture disclosed in connection with Figure 22.

[0239]

[0239] Process 2100 includes, at a first electronic device, steps of: receiving a first bitstream associated with a media asset encoded using a first type of linear coding (2101); according to a determination that the first bitstream should be decoded with a decoder associated with a second type of linear coding, processing the first bitstream into a second bitstream of a second type of linear coding, comprising: removing data from a set of fields of the first bitstream and mapping the data to a second corresponding set of fields of the second bitstream (2102); extracting metadata associated with the media asset from the first bitstream (2103); and encoding the second bitstream into a second bitstream of a second type of linear coding according to a determination that the first bitstream should be decoded with a decoder associated with a second type of linear coding, comprising: removing data from a set of fields of the first bitstream and mapping the data to a second corresponding set of fields of the second bitstream (2104); extracting metadata associated with the media asset from the first bitstream (2105); , decoding the first bitstream using a decoder associated with the second type of coding into an uncoded representation of the media asset (2104); processing the first bitstream in accordance with a determination that the first bitstream should not be decoded with a decoder associated with the second type of coding, the processing including: decoding the first bitstream using metadata associated with the media asset and a decoder associated with the first type of coding into an uncoded representation of the media asset (2105); and distributing the uncoded representation of the media asset together with the retrieved metadata related to the media asset to an application (2106).

[0240]

[0240] The processes 1800, 1900, 2000 and 2100 disclosed above each enable combining data elements associated with a linear coding process with data elements including at least one of bitstream identification data, content specific encoding data, and media segment data prior to transmission, allowing for more efficient distribution of media data to / from devices having various processing capabilities (e.g., encoding / decoding capabilities) over networks with different performance characteristics. The combined data results in more efficient decoding and reduces the need to generate and transmit multiple redundant types of bitstreams to ensure compatibility, thereby saving network and processing resources.

[0241] System Architecture Example 22 illustrates a block diagram of an example electronic device architecture 2200 suitable for implementing example embodiments of the present disclosure. Architecture 2200 includes, but is not limited to, server and client devices as described above with reference to FIGS. 1-21.

[0241]

[0242] As shown, the architecture 2200 includes a central processing unit (CPU) 2201 capable of executing various processes according to a program stored in, for example, a read-only memory (ROM) 2202 or loaded into, for example, a random access memory (RAM) 2203 from a storage unit 2208. The RAM 2203 also stores data required by the CPU 2201 to execute various processes, as necessary. The CPU 2201, the ROM 2202, and the RAM 2203 are connected to each other via a bus 2204. An input / output (I / O) interface 2205 is also connected to the bus 2204.

[0242]

[0243] The following components are connected to the input / output interface 2205: an input unit 2206, which may include a keyboard, mouse, etc.; an output unit 2207, which may include a display, such as a liquid crystal display (LCD), and one or more speakers; a storage unit 2208, which may include a hard disk or other suitable storage device; and a communication unit 2209, which may include a network interface card, such as a network card (e.g., wired or wireless).

[0243]

[0244] In some embodiments, the input unit 2206 includes one or more microphones in different positions (depending on the host device) that enable capture of audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

[0244]

[0245] In some embodiments, the output unit 2207 includes a system having a variable number of speakers. The output unit 2207 can render audio signals in a variety of formats (e.g., mono, stereo, immersive, binaural, and other suitable formats) (depending on the capabilities of the host device).

[0245]

[0246] In some embodiments, the communication unit 2209 is configured to communicate with other devices (e.g., via a network). The drive 2210 is also connected to the I / O interface 2205 as needed. A removable medium 2211, such as a magnetic disk, optical disk, magneto-optical disk, flash drive, or other suitable removable medium, is attached to the drive 2210 so that computer programs read therefrom are installed in the storage unit 2208 as needed. Although the system 2200 is described as including the above-mentioned components, those skilled in the art will understand that in practical applications, some of these components can be added, removed, and / or replaced, and all such variations or modifications are within the scope of the present disclosure.

[0246]

[0247] According to an exemplary embodiment of the present disclosure, the above-described process may be implemented as a computer software program or in a computer readable storage medium. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied in a machine readable medium, the computer program including program code for performing the method. In such an embodiment, the computer program may be downloaded and loaded from a network via a communication unit 2209 and / or installed from a removable medium 2211, as shown in FIG. 22.

[0247]

[0248] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or special purpose circuits (e.g., control circuits), software, logic, or any combination thereof. For example, the above-mentioned units may be executed by a control circuit (e.g., CPU 2201 in combination with other components of FIG. 22), which may then perform the operations described in the present disclosure. Some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device (e.g., control circuit). Although various aspects of the exemplary embodiments of the present disclosure have been illustrated and described using block diagrams, flowcharts, or some other schematic representations, it will be understood that the blocks, apparatus, systems, techniques, or methods described herein may be implemented in, by way of non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers or other computing devices, or any combination thereof.

[0248]

[0249] Additionally, the various blocks illustrated in the flowcharts may be viewed as method steps and / or as operations resulting from the operations of the computer program code and / or as multiple coupled logic circuit elements configured to perform the associated functions. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied in a machine-readable medium, the computer program including program code configured to perform the methods described above.

[0249]

[0250] In the context of this disclosure, a machine-readable medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may be non-transitory and may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of these.

[0250]

[0251] The computer program codes for carrying out the methods of the present disclosure may be written in any combination of one or more programming languages. These computer program codes may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus having control circuitry, such that the program code, when executed by the processor of the computer or other programmable data processing apparatus, performs the functions / operations specified in the flowcharts and / or block diagrams. The program code may run entirely on the computer, partly on the computer, as a stand-alone software package, partly on the computer and partly on a remote computer, entirely on a remote computer or server, or distributed over one or more remote computers and / or servers.

[0251]

[0252] Other embodiments In some embodiments, encoding according to a first linear coding process (e.g., a linear coding process corresponding to a supported linear code type) includes applying a linear coding technique to a packet including at least a portion of the media data (e.g., generating a coded packet that is a combination of a plurality of packets corresponding to at least a portion of the media data).

[0252]

[0253] In some embodiments, the encoding with the first linear coding process is performed without applying a linear coding technique to at least a portion of the media data (e.g., the process does not include creating an encoded packet by combining multiple packets corresponding to at least a portion of the media data). For example, the encoding process may include applying a mapping function (e.g., one or more mapping rules or tables) associated with the first linear coding process to convert at least a portion of the media data into coded data without requiring application of the first linear coding process.

[0253]

[0254] Advantages of the Disclosed Embodiments The disclosed bitstream provides the following advantages: 1. The disclosed embodiments support the use of multiple linear coding techniques within the same bitstream, including any linear code, including but not limited to block codes.

[0254] 2. The disclosed embodiments include metadata that enables media assets to be encoded, distributed, re-encoded, cached, and decoded at distribution points via a multi-source, multi-path media data distribution system.

[0255] 3. The disclosed bitstream 301 (subatoms and metadata) enables multi-source, multi-path distribution without the need for centralized control, without the need for coordination between sources, and allows for minimal redundancy at the distribution points.

[0256] 4. The disclosed embodiments allow the sub-atom system to transmit different types of data using different methods, which allows for flexible heterogeneous distribution of different sub-atoms. For example, some sub-atoms, such as metadata, can be sent over trusted channels and other sub-atoms, such as media essence, can be sent over untrusted channels. Some sub-atoms can be encrypted and some sub-atoms can be unencrypted.

[0257] 5. The disclosed embodiments allow flexible bitstream subatom sizes to match communication protocol data unit sizes through the use of CHUNKED_SUBATOM subatoms.

[0258] 6. The disclosed embodiments allow for chunking a media asset into smaller blocks and then applying linear codes separately to different blocks by indicating different blocks using the block_index field of the BLOCK_HEADER subatom.

[0259] 7. The disclosed embodiments allow for linear re-encoding of encoded data within a particular subatom without changing the subatom structure (e.g., the output is the same subatom structure and can be consumed by a decoder without modification).

[0260] 8. The disclosed embodiments allow for the creation of functionally equivalent coded copies of various sizes to meet the storage constraints of different devices without modifying the subatom structure.

[0261] 9. The disclosed embodiments support the ability to encode a media asset using multiple different linear codes and distribute it via multiple sources (e.g., CDNs), such that when the multiple bitstreams are available at the decoding location, they can be decoded using the same bitstream decoder.

[0262] 10. The disclosed embodiments support the ability to linearly re-encode a media asset that is encoded using multiple different linear codes and distributed via multiple sources (e.g., CDNs), where the multiple bitstreams are re-encoded using the same bitstream codec if they are available at the re-encoding location.

[0263] 11. The disclosed embodiments allow for the conversion of a media asset that has been encoded into a bitstream using an existing standardized linear code format (e.g., RaptorQ, RS), and then this standardized bitstream can be mapped into a proposed bitstream by a mapping function. The demapping function is applied at a decoder of the proposed bitstream to restore the original standardized bitstream to the standardized format.

[0264] 12. The disclosed embodiments support the selection and use of linear codes that are sliding window or block codes. This format allows the decoder to decode using either block or sliding window if a sliding window code is used in the encoder.

[0265] 13. The disclosed embodiments support the delivery of low-latency, non-lossless media data unicast systems through the selection of appropriate linear codes (e.g., supporting sliding window codes) and sub-atoms that allow media assets to be transmitted over reliable channels while also being transmitted over unreliable channels.

[0266] 14. The disclosed embodiments enable encryption of selected sub-atoms and fields within the bitstream to prevent unauthorized decoders from recovering the media asset (eg, DRM).

[0267] 15. The disclosed embodiments support different levels of data integrity checks (e.g., file-level, block-level, and packet-level checks) using fields carrying authentication information. In particular, it supports the use of special hashing algorithms, such as homomorphic hashing, to enable integrity checking of linearly coded packets without decryption.

[0268]

[0255] Furthermore, the disclosed bitstream generator provides the following advantages:

[0269] 1. The disclosed bitstream generator can chunk subatoms into smaller pieces (CHUNKED_SUBATOM subatoms 311) to fit the communication protocol data unit size.

[0270] 2. The disclosed bitstream generator may chunk the media asset into smaller blocks, apply the linear code separately to the various blocks, and specify the various blocks using the block_index field of the BLOCK_HEADER subatom.

[0271] 3. The disclosed bitstream generator is capable of generating multiple different bitstreams using a sub-atom multiplexer, and the different bitstreams can be distributed to different network entities.

[0272] 4. The disclosed bitstream generator can receive channel information and use it to adapt the encoding scheme.

[0273] 5. The disclosed bitstream generator selects a linear encoding technique from among multiple linear encoding techniques for encoding media into a bitstream by signaling this in the subatom BITSTREAM_HEADER using the field linear_code_type.

[0274] 6. The disclosed bitstream generator can generate multiple linear coded variants of the same media asset by placing the same media information (e.g., the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO, MEDIA_SEG_INFO subatoms) in the different variants to uniquely identify this media asset, but different linear code information (e.g., a different prng_seed in the BLOCK_HEADER subatom, a different poly_index in the PACKET subatom, and different coefficients) to distinguish the variants. In this way, the variants can be distributed over a multi-source data distribution system and all can contribute to the decoding of the media asset.

[0275] 7. The disclosed bitstream generator can signal whether a block code or a sliding window code is used via various fields, such as by setting the linear_code_type field of the BITSTREAM_HEADER subatom to various values, setting the content_source_size field of the BITSTREAM_HEADER subatom to 0 for sliding window codes, and / or using dynamic window_start_index and window_end_index of the PACKET subatom for sliding window codes.

[0276] 8. The disclosed bitstream generator can dynamically change linear coding schemes and parameters based on receiver feedback.

[0277] 9. The disclosed bitstream generator can encrypt coding information (such as the coefficient_vector field of the PACKET subatom) and transmit the coding information separately (e.g., using the PACKET_HEADER_ONLY subatom) to prevent unauthorized decryption of the media asset.

[0278] 10. The disclosed bitstream generator can hash media assets in various ways (hash files, hash blocks, hash original symbols, etc.) using various hashing techniques (SHA2, SHA256, homomorphic hashing, etc.) to enable data integrity checking. In particular, specialized hashing algorithms such as homomorphic hashing can be used to enable integrity checking of linearly coded packets without decryption.

[0279]

[0256] Furthermore, embodiments of the disclosed bitstream re-encoder provide the following advantages:

[0280] 1. The disclosed bitstream re-encoder can be implemented in different network entities to generate functionally equivalent full and partial coded copies without implementing the bitstream re-encoders and coordinating with each other.

[0281] 2. The disclosed bitstream re-encoder can identify and re-encode using different linearly coded variants of the same media asset through the same media information (e.g., the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO, and MEDIA_SEG_INFO sub-atoms) and different linear code information (e.g., different prng_seed in the BLOCK_HEADER sub-atom, different poly_index and different coefficients in the PACKET sub-atom) from the different variants.

[0282] 3. The disclosed bitstream re-encoder is capable of combining and re-encoding different bitstreams of the same media asset that were generated using different linear codes.

[0283] 4. The disclosed bitstream encoder can decode (for re-encoding) and re-encode coding information (e.g., the coefficient_vector field of the PACKET subatom) and transmit the coding information separately (e.g., using the PACKET_HEADER_ONLY subatom) to prevent unauthorized decoding of the media asset.

[0284] 5. The disclosed bitstream re-encoder can verify the integrity of a media asset in various ways (file verification, block verification, packet verification, etc.) according to the hashing technique (SHA2, SHA256, homomorphic hashing, etc.) and the data integrity field specified in the subatom. In particular, if a special hashing technique such as homomorphic hashing is used, it can verify linearly coded packets without decryption.

[0285]

[0257] Furthermore, embodiments of the disclosed bitstream decoder provide the following features / advantages:

[0286] 1. The disclosed bitstream decoder is capable of identifying and downloading multiple distinct bitstreams of the same media asset and using them to decode the media asset.

[0287] 2. The disclosed bitstream decoder can use the block_index field of the BLOCK_HEADER subatom as an indicator to separately decode different blocks of the same media asset and concatenate them in the correct order to recover the original media asset.

[0288] 3. The disclosed bitstream decoder can recover a complete subatom using multiple CHUNKED_SUBATOM subatoms by using the relevant fields in the CHUNKED_SUBATOM subatom (e.g., original_subatom_id, chunk_size, chunked_segment_index, num_chunked_segments) as connectors, by using a buffer to temporarily store CHUNKED_SUBATOM subatoms that cannot yet be recovered, and by using a merger to attempt recovery.

[0289] 4. The disclosed bitstream decoder is capable of synthesizing and decoding media assets using bitstreams generated using different linear codes.

[0290] 5. The disclosed bitstream decoder supports various types of decoding, such as block decoding and sliding window decoding, based on the coding scheme and latency requirements.

[0291] 6. The disclosed bitstream decoder can identify whether a block code or a sliding window code is used via various fields such as the linear_code_type field of the BITSTREAM_HEADER subatom, the content_source_size field of the BITSTREAM_HEADER subatom, and / or the window_start_index and window_end_index fields of the PACKET subatom.

[0292] 7. The disclosed bitstream decoder can identify the linear code type from among multiple linear coding techniques to properly decode the bitstream by reading the BITSTREAM_HEADER subatom and field linear_code_type.

[0293] 8. The disclosed bitstream decoder can distinguish and decode different linearly coded variants of the same media asset using the same media information (e.g., the same SYNC, BITSTREAM_HEADER, ENCODER_CONTENT_INFO, and MEDIA_SEG_INFO subatoms) and different linear code information (e.g., a different prng_seed in the BLOCK_HEADER subatom, a different poly_index in the PACKET subatom, and different coefficients) between the different variants.

[0294] 9. The disclosed bitstream decoder uses a combination of window size and generation size to identify whether a linear code is a sliding window or a block code.

[0295] 10. The disclosed bitstream decoder can analyze channel information based on received subatoms and report the information to the generator for making informed coding decisions.

[0296] 11. The disclosed bitstream decoder can decode the coding information (e.g., the coefficient_vector field of a PACKET subatom) to enable decoding of the media asset.

[0297] 12. The disclosed bitstream decoder can verify the integrity of a media asset in various ways (file verification, block verification, packet verification, etc.) according to the data integrity fields specified in the subatoms and the hashing technique (SHA2, SHA256, homomorphic hashing, etc.), particularly when special hashing techniques such as homomorphic hashing are used, linearly coded packets can be verified without decryption.

[0298]

[0258] Furthermore, the disclosed embodiments provide: 1. A mechanism for handling variable-sized data packets, using preprocessing of the data before linear encoding, so that the bitstream does not know that the media is in variable-sized packets.

[0299] 2. A system capable of enabling low-latency delivery of a media stream by sending certain subatoms that are not time-sensitive but important for decoding over a reliable but delay-insensitive connection (TCP), and sending the encoded media stream over an unreliable (and low-latency) connection.

[0300] 3. Multi-path multi-source media delivery systems, in which an origin server generates different functionally equivalent linearly-encoded copies of the same media asset to different caches, new caches re-encode their existing copies to obtain new functionally equivalent linearly-encoded copies, and recipients download data from multiple sources / caches to recover the original media asset.

[0301] 4. A DRM system in which the content creator encrypts the linear coding information and only provides the decryption key to authorized parties, so that only authorized parties can re-encode and decrypt the content.

[0302] 5. A decentralized media distribution system enabling data integrity authentication, in which network entities can use the data integrity field of the disclosed bitstream to verify the integrity of the data they receive and submit fraud proofs against hostile nodes to a regulatory manager for penalties.

[0303] 6. Enabling coded multi-source distribution of media and / or data in both wireless uplink and downlink channels. An exemplary embodiment of the uplink channel includes electronic news gathering applications.

[0304] 7. Enables an end-to-end solution supporting coded multi-source media generated and formulated in accordance with the disclosed invention for efficient / scalable processing in 5G / 6G edge application servers, while enabling the same 5G / 6G edge application servers to support real-time conversion to coded unicast or broadcast traffic for delay-sensitive applications.

[0305]

[0259] Although the present specification includes many specific implementation details, these should not be construed as limitations on the scope of what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, even if features are described above as operating in a particular combination and initially claimed as such, one or more features in the claimed combination may, in some cases, be removed from the combination, and the claimed combination may relate to a subcombination or a variation of the subcombination. The logic flows depicted in the figures do not require the particular order or sequential order depicted to achieve the desired results. Furthermore, other steps may be provided or steps may be removed from the described flow, and other components may be added to or removed from the described system. Accordingly, other implementations are within the scope of the following claims.

Claims

1. A method in a first device, which: A step of receiving media data representing media assets using at least one processor; A step of using the at least one processor to obtain a first plurality of data elements relating to the media asset, the first plurality of data elements including at least one of bitstream identification data, content-specific encoded data, and media segment data; A step of using at least one processor to encode at least a portion of the media data according to a first coding process to obtain coded data corresponding to the media asset; A step of generating a second plurality of data elements different from the first plurality of data elements using at least one of the processors, wherein the second plurality of data elements carry encoded information related to the first coding process and the coded data; The steps of using at least one processor to combine the first plurality of data elements and the second plurality of data elements into one or more coded bitstreams representing the media asset; and A step of using at least one processor to transmit the one or more coded bitstreams to one or more second devices via one or more network paths; A method that includes this.

2. The method according to claim 1, wherein the first coding process is a linear coding process for obtaining coded data corresponding to the media asset.

3. The method according to claim 1, wherein the first coding process is a nonlinear coding process that converts the media data into coded data that conforms to another linear coding process.

4. The method according to claim 1, wherein the first plurality of data elements include at least one of content-specific encoded data or media segment data and bitstream identification data.

5. A method according to claim 1, wherein the data element is a data structure.

6. The method according to claim 1, wherein the first plurality of data elements include a synchronization data element, and each of the one or more coded bitstreams begins with a synchronization data element.

7. In the method according to claim 1, the synchronization data element is: Identification data that identifies each of the one or more coded bitstreams as having been generated from a common source media asset; Synchronization data for synchronizing each of the one or more coded bitstreams with other coded bitstreams generated from the common source media asset; and Genealogy data for each of the one or more coded bitstreams; A method that includes at least one of the following.

8. In the method of claim 7, the genealogical data is: Data identifying the original or coded symbol used to generate the coded symbol contained in each of the coded bitstreams; Data identifying how the original or coded symbols were combined in order to generate the coded symbols contained in each of the coded bitstreams; and Data identifying the coding coefficients used to generate the coded symbols contained in each of the coded bitstreams; Methods that include...

9. A method according to claim 1, further comprising the step of interleaving a synchronization data element across the entirety of the one or more coded bitstreams, wherein the synchronization data element enables a second electronic device receiving one of the one or more coded bitstreams to initiate decoding from a plurality of points within the one or more coded bitstreams.

10. A method according to claim 1, wherein the first plurality of data elements or the second plurality of data elements are generated at least in part based on system data relating to received media data.

11. A method according to claim 1, wherein the step of obtaining the first plurality of data elements includes the step of generating the first plurality of data elements from received media data or system data related to received media data.

12. A method according to claim 1, wherein encoding the media data according to a first coding process includes dividing the media data into a plurality of original symbols and generating a plurality of coded symbols based on the plurality of original symbols.

13. A method according to claim 12, wherein coded symbols are generated by linearly combining subsets of the plurality of original symbols according to a network coding technique.

14. A method according to claim 1, wherein the media data includes a linearly coded bitstream.

15. In the method according to claim 1, the step of obtaining a first plurality of data elements is: Steps to extract metadata from the media data; and A step of performing calculations on the first set of data elements from the extracted metadata; Methods that include...

16. A method according to claim 1, wherein metadata is extracted from media data without performing linear decoding.

17. A method according to claim 1, wherein encoding the media data according to a first coding process includes re-encoding linearly coded symbols in the coded bitstream into new linearly coded symbols representing the media asset.

18. A method according to claim 1, further comprising, before the joining step, dividing one or more data elements having a first size into data elements having a second size smaller than the first size.

19. A method according to claim 1, further comprising the step of adding hash data to one or more data elements of the second plurality of data elements, thereby enabling at least one of verification, authentication, or integrity by a device receiving the one or more bitstreams.

20. A method according to claim 1, further comprising the step of adding hash data to one or more data elements of the second plurality of data elements that are associated with one or more files, blocks, segments, and packets of the media asset that correspond to the media asset.

21. A method according to claim 1, wherein the joining step includes inserting directory data elements into the one or more coded bitstreams.

22. A method according to claim 21, wherein the directory data elements are used to assist a receiving device in identifying subatoms within each bitstream.

23. The method according to claim 1, wherein the one or more coded bitstreams include a first coded bitstream containing a first type of data element and a second coded bitstream not containing the first type of data element.

24. The method according to claim 1, wherein the data element or sub-atom used to initialize the decoder in the client is not delay-sensitive data.

25. The method according to claim 1, wherein the bitstream includes at least one of the bitstream_header(), block_header(), encoder_content_info() or media_segment_info() sub-atoms and a sync() sub-atom.

26. A method according to claim 1, wherein the data element or sub-atom carries encoded data or delay-sensitive data.

27. The method according to claim 1, wherein the bitstream includes a packet() sub-atom and a sync() sub-atom, but does not include at least one of the bitstream_header(), block_header(), encoder_content_info(), or media_segment_info() sub-atoms.

28. In the method according to claim 1, the steps of transmission are: The steps of: transmitting the first coded bitstream using a first type of communication channel; and A step of transmitting a second coded bitstream using a second type of communication channel different from the first type; Methods that include...

29. The method according to claim 28, wherein the second type of communication channel is a high-latency channel, a low-bandwidth channel, a reliable channel, or a channel using a second type of transmission protocol different from the first type of transmission protocol.

30. A non-temporary, computer-readable storage medium for storing instructions, wherein, when executed by an arithmetic unit, the instructions cause the arithmetic unit to perform the method according to any one of claims 1 to 29.

31. At least one processor; and Memory for storing instructions; An arithmetic unit comprising a processor, wherein, when the instruction is executed by the at least one processor, it causes the arithmetic unit to perform the method according to any one of claims 1 to 29.

Images