Techniques for delivering encoded multi-source media to legacy devices via edge agents

Abstract

A method for delivering multi-source media content to a legacy client device via an edge proxy is disclosed. The method includes determining one or more criteria that meet a multi-source relay mode based on a determination that the client device requesting the media content does not support decoding of the multi-source media data. In response thereto, the method may include instantiating, at a first server in communication with the client device, a multi-source media decoder associated with the client device. The method may also include (i) concurrently receiving, at the first server, multi-source media data corresponding to the media content from the plurality of multi-source media sources; (ii) decoding the multi-source media data into unencoded media content data corresponding to the media content using a multi-source media decoder; and (iii) communicating at least a portion of the unencoded media content data from the first server to the client device.

Description

Cross-references to related applications

[0001] This application claims priority to European application No. 23306438.5, filed on 30 August 2023, and European application No. 24151187.2, filed on 10 January 2024, each of which is incorporated herein by reference in its entirety. Technical Field

[0002] This application generally relates to distributing media content (e.g., video, audio, images, applications, firmware, application data, and / or any other content that is expected to be delivered in an improved manner) to one or more client devices via a distribution network. Summary of the Invention

[0003] Various aspects of this disclosure relate to devices, systems, and methods for distributing media content over a distribution network to one or more client devices using an edge agent to provide multi-source media content to conventional client devices that are not configured to request or receive multi-source media content.

[0004] In one aspect of this disclosure, a method is provided that may include determining whether one or more criteria for a multi-source relay mode are met. The method may further include determining the satisfaction of one or more criteria for a multi-source relay mode based on determining that a client device requesting media content does not support decoding multi-source media data. The method may further include instantiating a multi-source media decoder associated with the client device requesting the media content at a first server communicating with the client device. The method may further include concurrently receiving multi-source media data corresponding to the media content from multiple multi-source media sources at the first server. The method may further include decoding the multi-source media data corresponding to the media content into unencoded media content data corresponding to the media content using the multi-source media decoder. The method may further include transmitting at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device.

[0005] In addition to any combination of the features mentioned above, one or more criteria for satisfying the multi-source relay mode may include determining that the number of client devices within the communication range of the first server that do not support decoding of multi-source media data exceeds a threshold number.

[0006] In addition to any combination of the features described above, one or more criteria for satisfying the multi-source relay mode may include determining that one or more performance parameters associated with delivering data corresponding to the media content to the client device requesting the media content exceed one or more performance criteria.

[0007] In addition to any combination of the features described above, the method may include a second client device that, based on the determination of requested media content, supports decoding of multi-source media data and transmits multi-source media data corresponding to at least a portion of the media content from the first server to the second client device without using a multi-source media decoder.

[0008] In addition to any combination of the features mentioned above, determining that the client device requesting the media content does not support decoding multi-source media data can be based on data included in the request for the media content.

[0009] In addition to any combination of the features mentioned above, multi-source media data corresponding to media content corresponds to DASH or HLS segments.

[0010] In addition to any combination of the features described above, the plurality of multi-source media sources may include a cache populated via an on-the-fly encoding process associated with the client device requesting the media content.

[0011] In addition to any combination of the features described above, multiple multi-source media sources include caches populated via encoding processes associated with client devices other than the client device requesting the media content.

[0012] In addition to any combination of the features described above, the plurality of multi-source media sources include peer devices, local caches, or both peer devices and local caches.

[0013] In addition to any combination of the features described above, the method may include instantiating different multi-source media decoders associated with the client device requesting the media content at a second server. The method may also include concurrently receiving multi-source media data corresponding to the media content from a second plurality of multi-source media sources at the second server. The method may further include decoding the multi-source media data corresponding to the media content into unencoded media content data corresponding to the media content using the different multi-source media decoders. The method may further include transmitting at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device.

[0014] In addition to any combination of the features described above, instantiating different multi-source media decoders associated with the client device requesting media content at the second server can be performed in response to motion data associated with the movement of the client device. This motion data could indicate that the client device is moving away from the first server and toward the second server.

[0015] In addition to any combination of the features described above, transmitting at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device and transmitting at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device can be performed concurrently. Alternatively, in addition to any combination of the features described above, the method may include stopping the transmission of at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device before transmitting at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device.

[0016] In another aspect of this disclosure, a computing device is provided, which may include at least one electronic processor. The computing device may also include a memory storing instructions that, when executed by the at least one electronic processor, cause the computing device to perform any combination of the methods and / or features described above.

[0017] In another aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores instructions which, when executed by a computing device, cause the computing device to perform any combination of the methods and / or features described above.

[0018] Other aspects of these embodiments will become clear by taking into account the specific implementation and the accompanying drawings. Attached Figure Description

[0019] Figure 1 An example distribution network for distributing media content is illustrated according to an embodiment described herein.

[0020] Figure 2 Included in the embodiments described herein Figure 1 Hardware block diagram of an example client device in a distribution network.

[0021] Figure 3 Included in the embodiments described herein Figure 1 A block diagram of an example edge server in the distribution network.

[0022] Figures 4A-4B The illustration shows an embodiment of the invention described herein for use in Figure 1 On edge computing resources available in the distribution network (e.g., in) Figure 3 A flowchart of an example method for instantiating a Multi-Source Media Format (MMF) compatible proxy on an edge server. Detailed Implementation

[0023] Media content (e.g., video and / or audio) can be delivered to one or more client devices via a distribution network for output by the client devices. Delivering media content via a distribution network can include a unicast model (a source provides media content to a requesting client device through a single communication path). Alternatively, delivering media content via a distribution network can include a single client device requesting media content (all or different portions of the media content) from a single source using multiple unicast models (e.g., different communication channels, paths, modalities, etc.). However, these different communication channels, paths, modalities, etc. (i.e., communication links) are typically asynchronous, making data received from one communication link unusable for decoding data from another communication link, and vice versa.

[0024] Therefore, resource management and scheduling of media content requests and delivery are required based on perceived communication link quality to orchestrate the delivery of complete media content requested by client devices. This resource management and scheduling consumes power and processing resources on both client devices and within the distribution network. Furthermore, this resource management and scheduling is based on perceived communication link quality, which is variable and may change during media content requests and / or delivery (e.g., when client devices move in or out of buildings, when client devices move between different coverage areas of different edge networks, when a type of communication link becomes heavily used by other nearby client devices, when a type of communication link is affected by some type of interference, etc.). Updating resource management and scheduling to reflect changes in perceived communication link quality further consumes power and processing resources on client devices and / or the distribution network. Additionally, media content delivery may be delayed because the original resource management and scheduling are based on communication link quality perceived before changes in the communication link's condition. Therefore, there are technical challenges in delivering media content via asynchronous communication links within the distribution network.

[0025] To address the aforementioned technical challenges, a Multi-Source Media Format (MMF) can be used to deliver media content to client devices. Unlike traditional or single-source media formats designed to carry a single encoded version of source media, a Multi-Source Media Format (MMF) is a format designed to carry multiple representations of source media. For example, media content delivered via an MMF includes one or more encoded representations and corresponding signaling data that enables downstream processing across various representations (e.g., joint decoding, re-encoding, etc., utilizing more than one representation). For instance, a Coded Multi-Source Media Format (CMMF) can be used, where rate-free codes (e.g., a code that allows the generation of any number of encoded packets from a set of raw unencoded packets) are encapsulated within an MMF container containing signaling data. By using MMF to deliver media content to client devices, each client device, when equipped with an MMF decoder, can synchronously decode MMF packets received from any source in the distribution network via any communication link. Therefore, resource management and scheduling for media content requests and delivery can be reduced or eliminated, as equally useful information from multiple communication links can be received and decoded by each client device. In other words, if an unexpected delay occurs on a communication link, the client device will continue to receive media content via other communication links / channels until enough MMF packets have been received overall (from any communication link) to fully decode the media segment, which is then output by the client device. Therefore, not only can the resource management and scheduling of media content requests and delivery be reduced or eliminated, but media content can also be delivered to the client device efficiently, thereby reducing or eliminating delays caused by variations in communication link quality.

[0026] As described above, in order to take advantage of the benefits of using MMF to deliver media content, client devices must typically be specifically configured to participate in MMF functionality (i.e., be MMF-compatible). For example, client devices must typically be specifically configured to include an MMF client component that acts as (i) an MMF request generator for providing MMF requests to upstream devices in the distribution network and (ii) an MMF decoder for decoding MMF packets received from upstream devices in the distribution network.

[0027] However, many client devices may be legacy client devices that are not compatible with MMF. Furthermore, in some instances, configuring and / or upgrading all legacy client devices to be MMF-compatible may be impractical, as each client device is configured to use many different operating systems, media players, etc. Therefore, there are technical issues that may prevent non-MMF-compatible devices (i.e., legacy devices) from requesting or receiving media content using MMF.

[0028] To address this technical challenge, an MMF-compatible proxy can be instantiated on edge computing resources available in the distribution network. For example, an MMF-compatible proxy (e.g., a Hypertext Transfer Protocol (HTTP) proxy) can be instantiated on an edge server and can include an MMF client component comprising (i) an MMF request generator for converting requests from non-MMF-compatible client devices into requests for MMF media content and (ii) an MMF decoder for decoding MMF packets received from upstream devices into unencoded media content to be delivered to the non-MMF-compatible client devices.

[0029] MMF-compatible proxies instantiated on edge computing resources address the aforementioned technical challenges by allowing traditional non-MMF-compatible client devices to receive MMF media content from multiple devices upstream of the edge server using MMF. This can (i) reduce or eliminate resource management and scheduling for media content requests and delivery, and (ii) allow media content to be delivered efficiently from multiple sources to non-MMF-compatible client devices, thereby reducing or eliminating latency caused by variations in communication link quality. Furthermore, implementing an MMF-compatible proxy instantiated on edge computing resources may be more practical than configuring or upgrading numerous client devices to MMF compatibility, as the MMF-compatible proxy can be instantiated temporarily, and multiple different instances of the MMF-compatible proxy can be instantiated for each of several different non-MMF-compatible client devices, as explained in more detail herein.

[0030] Furthermore, using an MMF-compatible proxy allows for the request of additional MMF media content and its caching within the distribution network 100, enabling other client devices (such as MMF-compatible client devices) to obtain improved Quality of Service / Experience (QoS / E), for example, by obtaining MMF media content cached further downstream and / or earlier than when the MMF-compatible proxy had not requested the MMF media content.

[0031] Figure 1An example distribution network 100 for distributing media content is illustrated. Network 100 includes a content server 105 configured to store and / or serve raw media content or source content (e.g., audio / video (A / V) encoded content, images, applications, firmware, application data, etc., prepared for distribution and playback using adaptive bitrate (ABR) (e.g., HTTP-based Dynamic Adaptive Streaming (DASH) or HTTP Real-Time Streaming (HLS)). Distribution network 100 also includes an MMF encoder 110. In some instances, the MMF encoder 110 is a processing server configured to (i) ingest media content from content server 105 in various representations and (ii) prepare network encoded blocks with different versions (e.g., MMF packets). The network encoded blocks can be arbitrarily recombined and decoded to output raw media content (e.g., received via an asynchronous communication link), as explained previously herein. The MMF encoder 110 can encode the raw media content into multiple MMF packets for distribution to one or more origin servers 115.

[0032] In some embodiments, the data corresponding to the media content is encoded using rate-free codes (e.g., rate-free erasure codes, digital fountain codes, fountain codes, etc.) encapsulated in an MMF container (e.g., CMMF). For example, the content is encoded using rate-free codes such as Dolby xCD-1 codes, digital fountain codes, LT codes, Raptor codes, RaptorQ codes, online codes, linear network codes, random linear network codes, etc., or other codes that allow the generation of any number of encoded packets from a set of original uncoded packets. In some embodiments, the rate-free code is a network code. For example, the content is encoded using simple network codes, linear network codes, random linear network codes (RLNC), or other forms of network codes that allow the generation of functionally equivalent encoded representations at intermediate nodes without decoding and re-encoding the encoded packets. Regardless of the code type, the encoded output (e.g., the encoded output of a linear network encoder) may include system packets in addition to non-system packets, and the amount of each system packet based on partition / tile can be adjusted according to various factors (e.g., tile priority, network conditions, cost, performance requirements, quality of experience (QoE), etc.). The techniques described in this article can be applied to any type of multi-source media (MMF) content / data, including but not limited to CMMF content / data.

[0033] In some instances, origin servers 115 are HTTP servers each corresponding to a specific Content Delivery Network (CDN) 120. Each origin server 115 stores network-encoded blocks (e.g., MMF packets) of encoded content. Depending on the configuration, all versions of the network-encoded blocks or only a subset of them (e.g., one version) are available in each origin server 115. MMF clients downstream of origin servers 115 can issue "GET" requests to these origin servers 115 to retrieve the network-encoded blocks. A subset of MMF packets associated with a given media segment can be referred to as an encoded version of the media segment. Thus, each origin server 115 can store different encoded versions of a given media segment; however, as explained earlier herein, because the encoded versions include MMF packets, any MMF packets from any of the origin servers 115 can be used by receiving devices to reconstruct the media segment by synchronizing with each other, assuming that the receiving devices include MMF decoders for decoding the received MMF packets (i.e., assuming the receiving devices are MMF-compatible). In some instances, CDN proxy caches are HTTP proxy caches, which are used to distribute and store media content obtained from the origin server 115 in the corresponding CDN 120.

[0034] In some instances, content providers launch a Just-In-Time (JIT) MMF service, configuring it to fetch packaged content from its existing origin server 115 and reconfiguring its CDN 120 to use JIT as its origin server. The JIT encoding proxy is a simple, containerized service capable of running in any cloud environment via a variety of orchestration tools. JIT is stateless, allowing for easy horizontal scaling. JIT determines whether to encode content based on arguments provided in the Uniform Resource Locator (URL) path. Upon receiving each request for media content from a downstream device, the JIT encoding proxy resolves the URL path to obtain information on how to encode the response. The original content is downloaded from the origin server and streamed to an MMF encoder (which acts as a reverse proxy) running in the cloud, which has access to the original source-encoded and packaged media. In other words, the JIT encoding proxy can encode unencoded media content data into MMF for downstream transmission to one or more components of the distribution network 100 requesting MMF data.

[0035] The edge network 125 of the distribution network 100 may include one or more edge servers 130. The one or more edge servers 130 may communicate bidirectionally with one or more of the CDN 120, for example, to request and / or receive media data from one or more origin servers 115, as explained in more detail herein. In some instances, the edge network 125 is the core network of an operator, which acts as a network operating on a network basis, providing services to one or more client devices 135 through various access types: 3G, 4G, 5G, 6G wireless access networks 140B; home broadband; Wi-Fi networks 140A (Wireless Local Area Network (WLAN), Wired Ethernet (LAN), etc.); satellite links 140C, etc. The edge network 125 grants one or more client devices 135 access to the Internet via a gateway and CDN 120. As explained in more detail below, edge server 130 can be configured to include an MMF proxy (i.e., a smart HTTP proxy caching server), which is instantiated on demand in the data center and has high-bandwidth and low-latency connections to edge network 125 and access network 140. As illustrated, Wi-Fi network 140A, 4G / 5G network 140B, and satellite network 140C are wireless access networks to which one or more client devices 135 are attached to access the operator's core network 125 (i.e., edge network 125).

[0036] In some instances, client device 135 (i.e., UE (User Equipment) 135) is a multi-access capable device that includes a media player. Client device 135 may be a playback device configured to output audio and / or video media content to a user. Example client devices 135 include, but are not limited to, televisions, smartphones, computers, tablet PCs, audio and / or video rendering devices, etc. Each client device 135 may or may not include an MMF decoder. In other words, each client device 135 may or may not be MMF-compatible. Therefore, as explained in more detail herein, edge server 130 may provide media data to each client device 135 in the form of unencoded media data or multi-source media data (e.g., MMF packets) depending on whether each client device 135 is an MMF-compatible client device or a non-MMF-compatible client device (i.e., a legacy device).

[0037] In some instances, client device 135 can be configured to communicate directly with one or more CDNs 120 without traversing the edge network 125. For example, an MMF-compatible client device 135 may be able to communicate directly with one or more CDNs 120 using some communication channels / modalities. However, in some instances, MMF-compatible client device 135 may be configured to communicate with CDNs 120 via the edge network 125, either additionally or alternatively, using other communication channels / modalities. In some instances, MMF-compatible client device 135 may communicate directly with CDNs 120 using a first communication channel / modal while simultaneously communicating indirectly with another CDN 120 via the edge network 125 using a second communication channel / modal.

[0038] Figure 1 The distribution network 100 shown is merely an example. In some instances, the distribution network 100 includes more or fewer components. For example, the distribution network 100 may include more or fewer content servers 105, MMF encoders 110, origin servers 115, CDNs 120, client devices 135, and / or access networks 140. Additionally, although... Figure 1 Only one edge network 125 and one edge server 130 are shown, but in some instances, distribution network 100 may include more edge networks 125 and edge servers 130. For example, different edge networks 125 may have different geographical coverage areas to provide services to client devices 135 within the communication range of edge network 125 when client devices 135 move geographically. Although edge network 125 is shown as including a single edge server 130, in some instances, multiple computing devices with multiple electronic processors may be collectively referred to as edge server 130, provided that all such computing devices and electronic processors are part of edge network 125 (i.e., the operator's core network). In some instances, the electronic device considered as edge server 130 is implemented as part of a distributed system comprising one or more components located in different locations. For example, in some instances, the electronic device includes local hardware components and one or more remote hardware components (e.g., cloud-based components hosted on public and / or private cloud infrastructure, SaaS, PaaS, etc.), provided that all such components are part of edge network 125. Similarly, in some instances, the origin server 115 of each CDN 120 may include multiple computing devices with multiple electronic processors (e.g., as part of a distributed system), provided that all such computing devices and electronic processors are part of the corresponding CDN 120.

[0039] in addition, Figure 1This does not necessarily illustrate all communication paths existing within distribution network 100. For example, as explained earlier in this document, MMF-compatible client device 135 can communicate directly with one or more CDNs 120. As another example, although Figure 1 Some arrows in the diagram are bidirectional arrows while others are unidirectional arrows, but in some instances, components between unidirectional arrows can communicate bidirectionally with each other.

[0040] Figure 2 This is a hardware block diagram of a client device 135 according to an example embodiment. As illustrated in the example, the client device 135 includes a first electronic processor 205 (e.g., a microprocessor or other electronic device). The first electronic processor 205 includes input and output interfaces (not shown) and is electrically coupled to a first memory 210, a first network interface 215, an optional microphone 220, a speaker 225, and a display 230. In some embodiments, the client device 135 interacts with... Figure 2 The different configurations illustrated in the figures include fewer or more components. For example, client device 135 may not include microphone 220. As another example, client device 135 may include one or more additional input devices, such as a computer mouse and / or keyboard that receive input from a user of client device 135. As yet another example, client device 135 may include environmental sensors (such as an ambient light sensor) or motion sensors (such as an accelerometer and / or a location tracking device (e.g., a Global Positioning System (GPS) receiver)). As yet another example, the first electronic processor 205 may include multiple electronic processors within client device 135 that work together to control various aspects of client device 135. Therefore, in the claims, if the apparatus or system is claimed to include electronic processors or other elements configured in a certain way, then, for example for multiple determinations, the claims or claim elements should be interpreted as referring to one or more electronic processors (or other elements), wherein any one or more electronic processors (or other elements) is configured, as claimed, to perform, for example, some or all of the multiple determinations. In some embodiments, client device 135 performs functions other than those described below.

[0041] The first memory 210 may include read-only memory (ROM), random access memory (RAM), other non-transitory computer-readable media, or combinations thereof. The first electronic processor 205 is configured to receive instructions and data from the first memory 210 and execute those instructions, etc. Specifically, the first electronic processor 205 executes instructions stored in the first memory 210 to implement the methods described herein.

[0042] The first network interface 215 sends and receives data to other devices within the distribution network 100 (e.g., edge network 125 and / or CDN 120, for example, via one or more of the access networks 140). In some embodiments, the first network interface 215 includes one or more transceivers for wireless communication with other devices within the distribution network 100. Alternatively or additionally, the first network interface 215 may include a connector or port for receiving a wired connection (e.g., an Ethernet cable) to one or more of the other devices in the distribution network. The first electronic processor 205 may receive media content (e.g., audio, video, etc.) from other devices (e.g., edge server 130) via the first network interface 215. The first electronic processor 205 may output the media content received from the edge server 130 via a speaker 225, a display 230, or a combination thereof. Additionally, the first electronic processor 205 may transmit data generated by the client device 135 (e.g., a request for media content) to upstream devices in the distribution network 100 (e.g., edge server 130, one or more CDNs 120, etc.).

[0043] In some instances, the first electronic processor 205 is configured to implement an MMF decoder to decode multi-source media data (e.g., MMF packets) received from other devices, such as edge server 130. In some instances, the first electronic processor 205 is also configured to issue an "acquisition" request for multi-source media data (e.g., MMF packets) to other devices, such as edge server 130. When the first electronic processor 205 of client device 135 is configured to request and decode multi-source media data, client device 135 may be referred to as MMF-compatible client device 135. On the other hand, when the first electronic processor 205 of client device 135 is not configured to request or decode multi-source media data, client device 135 may be referred to as non-MMF-compatible client device 135 (i.e., conventional client device 135). For example, client device 135 may be non-MMF-compatible when it does not support MMF decoding locally or via a media player application installed on an operator access network.

[0044] Display 230 is configured to display images, videos, text, and / or data to a user. Display 230 may be a liquid crystal display (LCD) screen or an organic light-emitting diode (OLED) display. In some embodiments, a touch-sensitive input interface may also be incorporated into display 230 to allow a user to interact with content provided on display 230. In some embodiments, display 230 includes a projector or a display technology developed in the future. In some embodiments, speaker 225 and display 230 are referred to as output devices for presenting media content and other information to a user of client device 135. In some embodiments, microphone 220, computer mouse, and / or keyboard or touch-sensitive display are referred to as input devices for receiving input from a user of client device 135.

[0045] Figure 3 This is a block diagram of an edge server 130 according to an example embodiment. In the illustrated example, the edge server 130 includes a second electronic processor 305 electrically connected to a second memory 310 and a second network interface 315. These components are similar to those described above. Figure 2 The components of the client device 135 are explained with similar names and function in a similar manner to those described above. For example, the second electronic processor 305 sends and receives data from one or more downstream client devices 135 and / or one or more upstream CDNs 120 via the second network interface 315. In some instances, the edge server 130 is connected to... Figure 3 The different configurations illustrated in the figures include fewer or more components. For example, edge server 130 may additionally include a display, such as a touchscreen, to allow backend users to reprogram the settings or rules of edge server 130. As another example, second electronic processor 305 may include multiple electronic processors that work together to control various aspects of client device 135. Therefore, in the claims, if the apparatus or system is claimed to include electronic processors or other elements configured in a certain way, then, for example for multiple determinations, the claims or claim elements should be interpreted as referring to one or more electronic processors (or other elements), wherein any one or more electronic processors (or other elements) is configured, as claimed, to perform, for example, some or all of the multiple determinations. In some instances, edge server 130 performs functions other than those described below.

[0046] As explained earlier in this document, client device 135 can be MMF-compatible or non-MMF-compatible. For MMF-compatible client device 135, the client components of the MMF system are integrated into an existing video streaming application serving on client device 135. The MMF client components include a networking component that handles issuing and decoding multi-source requests to upstream devices, such as edge server 130. This networking component is initialized once per application session. The MMF client components also include a metrics component that integrates with a media player (e.g., a video player) and collects QoE data. This metrics component can be reinitialized for each media file played.

[0047] During initialization, the networking component can download configuration information from a configuration service. This configuration information may include information about which information sources (e.g., CDN 120, etc.) to download media content from. The MMF client component is also configured to register a custom Uniform Resource Locator (URL) scheme with the operating system of client device 135. All media content requests using this scheme will be passed to the MMF client component for processing. For example, to play video via MMF, the service must provide a manifest via a URL with a custom Uniform Resource Identifier (URI) scheme and hostname.

[0048] Upon receiving a media content request, the MMF client component examines the URL and decides whether to issue a request for MMF-encoded multi-source content. In some instances, the MMF client component is configured to determine whether to issue a request for MMF-encoded multi-source content based on a predetermined set of rules regarding, for example, the size and / or content of the requested media content. For example, non-media files (e.g., manifests and captions) may be too small to see significant benefits from MMF encoding. Therefore, for such files, the MMF client component can request a full version of the original media content file from each of the available information sources, as doing so is unlikely to impact QoE (e.g., media transmission delays are unlikely to be significant enough to cause playback delays that affect QoE), and this approach also improves robustness by giving multiple sources the ability to provide a full version of the original media content file should one of the files be corrupted during transmission.

[0049] For media content requests concerning video and / or audio segments, the MMF client component can determine to issue a request for MMF-encoded multi-source content. To do this, the MMF client component adds arguments to the URL path to signal upstream devices (e.g., MMF encoding proxies) that the requested content should be MMF-encoded. The MMF client component uses hostnames from information sources (e.g., N CDNs) from configuration information to simultaneously (e.g., from multiple sources on multiple communication paths) issue N HTTP requests for the encoded content. Upon receiving the first response byte, which includes MMF-encoded multi-source media content, from any source, the MMF client component initializes the MMF decoder to decode the multi-source media content. In some instances, the multi-source media data / content (i.e., MMF packets) includes HTTP-based Dynamic Adaptive Streaming (DASH) or HLS segments.

[0050] The MMF decoder is configured using the MMF header information included in the received multi-source media content (i.e., MMF packets). Data from N sources is then transmitted to the MMF-configured decoder. When the MMF-compatible client device 135 receives a complete packet from any source, the MMF decoder is configured to add the complete packet to its internal matrix. In one type of deployment example and setup, system MMF packets can be received sequentially and streamed to the media player on client device 135. Once the decoder has enough data to decode the entire segment, the decoded data is streamed to the media player, and any incomplete HTTP requests are canceled.

[0051] Once the decoder has finished decoding the entire segment, to minimize the amount of data in flight, the download manager runs for each MMF source and issues byte-range requests for the multi-source media content. The purpose of this component is to minimize the amount of incomplete data requested from each source at any given time while maintaining full throughput.

[0052] The shift to a non-MMF-compatible client device 135 means that the client components of the MMF system are not integrated into existing video streaming applications serving on the client device 135. For example, the non-MMF-compatible client device 135 does not support MMF decoding, either locally or via a media player application installed on the carrier access network. The non-MMF-compatible client device 135 also does not support requesting multi-source media content (e.g., MMF packets). As explained earlier in this document, it would be useful for both the non-MMF-compatible client device 135 and other MMF-compatible client devices 135 if the non-MMF-compatible client device 135 could request MMF packets (i.e., multi-source media content) from at least some upstream devices without requiring the non-MMF-compatible client device 135 to be configured and / or upgraded to be MMF-compatible. (References: This document refers to...)Figures 4A-4B An example of how to implement this function is described.

[0053] Figures 4A-4B A flowchart illustrating a method 400 for instantiating an MMF-compatible agent on edge computing resources available in distribution network 100 (e.g., on edge server 130 of edge network 125) is shown. Method 400 is described as being primarily performed by edge server 130 of edge network 125. While in Figures 4A-4B The examples provided indicate a specific order of processing steps, message reception, and / or message transmission, but such timing and order of steps, reception, and transmission may be varied where appropriate without negating the purpose and advantages of the examples detailed throughout the remainder of this disclosure.

[0054] like Figures 4A-4B As shown and explained herein, in some instances, edge server 130 uses an MMF proxy to deliver MMF-optimized media content to a traditional non-MMF-compatible client device 135 by performing the following processes: (i) edge server 130 detects that client device 135 is MMF-incompatible and instantiates an MMF proxy on edge server 130; (ii) edge server 130 acts as a proxy (MMF proxy) for DASH / HLS requests; (iii) if the requested content is available, edge server 130 serves unencoded DASH / HLS content in its local cache; (iv) edge server 130 processes a content miss as an MMF request to origin server 115; and (v) edge server 130 receives encoded content from origin server 115, decodes it on edge server 130 via the MMF proxy, and delivers the decoded / unencoded media content to non-MMF-compatible client device 135.

[0055] At box 405, method 400 is initiated, i.e., a non-MMF-compatible client device 135 connects to a configuration server to obtain configuration information in order to request media content. In some instances, client device 135 connects to a service configuration endpoint when launching a media playback application. In some instances, edge server 130 acts as a configuration server or communicates with a configuration server to provide configuration information to client device 135.

[0056] At box 410, edge server 130 determines whether one or more criteria for instantiating a multi-source relay mode are met (i.e., whether one or more criteria for instantiating an MMF proxy for relaying MMF to non-MMF-compatible client device 135 are met). In some instances, one or more criteria include edge server 130 determining whether the client device 135 requesting configuration information and media content does not support decoding multi-source media data (i.e., whether client device 135 is a non-MMF-compatible client device 135). In some instances, one or more criteria include edge server 130 determining that the number of non-MMF-compatible client devices 135 within the communication range of edge server 130 exceeds a threshold number. In some instances, one or more criteria include edge server 130 determining that one or more performance parameters associated with delivering data corresponding to media content to client device 135 (or to other client devices 135) are outside one or more performance criteria.

[0057] When client device 135 is MMF-compatible, one or more criteria for instantiating a multi-source relay mode are not met because client device 135 already has its own MMF client component to utilize MMF communication. On the other hand, when client device 135 is not MMF-compatible, edge server 130 can determine that one or more criteria for instantiating a multi-source relay mode are met. In some instances, edge server 130 is configured to determine, based on data included in a request for media content received by edge server 130 from client device 135, that the client device 135 requesting the media content does not support decoding multi-source media data (i.e., the client device is a non-MMF-compatible client device 135). For example, data included in a request for media content from client device 135 may include a URL, a device ID associated with the client device, etc., indicating that client device 135 is non-MMF-compatible.

[0058] In some instances, edge server 130 may determine not to instantiate a multi-source relay mode until the number of non-MMF-compatible client devices 135 within the communication range of edge server 130 exceeds a threshold number and / or until one or more performance parameters associated with delivering data corresponding to media content to one or more client devices 135 are outside one or more performance criteria. For example, if conventional single-path media content distribution does not result in poor QoE for client device 135 and / or nearby client devices 135 (e.g., because the type or amount of the requested content is too small to cause poor QoE), edge server 130 may register non-MMF-compatible client devices 135 to conventional single-path distribution and participate in non-MMF-compatible media content delivery (at box 415). However, during conventional single-path media content distribution, edge server 130 may continue to evaluate one or more criteria to determine whether a multi-source relay mode should be instantiated, as explained below. In some instances, performance parameters for client device 135 include minimum requirements based on, for example, estimated latency, startup time, buffering behavior, and / or playback errors associated with the delivery of the requested media content. In some instances, edge server 130 can evaluate the performance parameters of non-MMF compatible client device 135 and / or MMF compatible client device, because the performance parameters of these two types of devices can be improved by instantiating an MMF agent for non-MMF compatible client device 135 as explained herein.

[0059] In addition to one or more of the criteria mentioned above, one or more criteria for determining whether to instantiate a multi-source relay mode at edge server 130 may include the availability of edge computing resources in the current local / regional area to provide an MMF proxy at edge server 130 to non-MMF-compatible client devices 135, thereby performing MMF-compatible tasks and relaying MMF information between non-MMF-compatible client devices 135 and upstream devices such as CDN 120. Edge server 130 may perform a service capability lookup to determine whether edge computing resources and capabilities are available for instantiating the MMF proxy. Instantiating the MMF proxy may require a certain amount of edge computing resources and capabilities, which may or may not be available at certain times in certain edge networks 125. While upstream resources can theoretically be used to instantiate the MMF proxy, doing so as downstream as possible (e.g., at edge network 125) provides the greatest benefit from an efficiency and QoE perspective, as MMF delivery can occur over more communication paths in distribution network 100.

[0060] When edge server 130 determines that one or more criteria for instantiating a multi-source relay mode for client device 135 at edge server 130 are not met, method 400 proceeds to block 415. At block 415, edge server 130 registers the non-MMF-compatible client device 135 to a traditional single-path distribution and participates in non-MMF-compatible media content delivery. For example, using a service configuration application programming interface (API), edge server 130 can register the non-MMF-compatible client device 135 to a service configuration server to participate in traditional single-path media content distribution. In this media content distribution, media content is requested and received on a single communication path. Unlike MMF media content, which utilizes multiple communication paths from multiple communication sources, single-path media content distribution does not utilize multiple communication paths and multiple media content sources. Therefore, client device 135 using single-path media content distribution may experience latency when receiving media content, which may result in lower QoE. However, in some cases (e.g., when client device 135 is not MMF-compatible and when edge server 130 determines that edge computing resources and / or capabilities are not available for instantiating the MMF agent), client device 135 and edge server 130 can participate in single-path media content distribution to receive media content from upstream devices such as CDN 120. As previously noted herein and as... Figure 4A As indicated, during conventional single-path media content distribution at box 415, edge server 130 may continue to evaluate one or more criteria to determine whether a multi-source relay mode should be instantiated (at box 410). For example, the characteristics of edge network 125 and / or (multiple) client devices 135 may be modified to later meet one or more criteria for instantiating the multi-source relay mode.

[0061] When edge server 130 determines that one or more criteria for instantiating a multi-source relay mode at edge server 130 are met (at box 410), method 400 proceeds to box 420. At box 420, edge server 130 instantiates a containerized instance of the MMF agent (i.e., the edge agent). In some instances, the MMF agent performs the same functions as the MMF client components on MMF-compatible client device 135, except that these functions are moved upstream to be performed by edge computing (i.e., by edge server 130) of edge network 125. In some instances, the MMF agent has two parts: (i) a networking component that handles issuing and decoding multi-source media content requests; and (ii) a metrics component that collects QoE data from client device 135. The networking component is initialized once per application session. In some instances, instantiation of the MMF proxy includes instantiating a multi-source media (e.g., MMF) decoder at the edge server 130 that is associated with a non-MMF-compatible client device 135 that requests configuration information (and later media content) from the edge server 130.

[0062] When the MMF proxy is instantiated on edge server 130, the networking component of the MMF proxy is configured to download configuration information from a configuration server. This configuration information includes information about which sources (e.g., CDNs, servers, etc.) to download media content from. Additionally, the networking component of the MMF proxy is configured to generate a unique URL (including (multiple) Domain Name System (DNS) records, if necessary) hostname for routing non-MMF-compatible clients to upstream MMF media content sources via the instantiated MMF proxy. In other words, the MMF proxy registers the non-MMF-compatible client device 135 (using its unique client ID) with the upstream device as a connection endpoint for receiving media content delivery. The MMF proxy generates a unique extension to the client ID of the non-MMF-compatible client device 135 and also registers this unique extension with the MMF proxy. Using this unique extension to the client ID uniquely associates the containerized MMF proxy with the specific non-MMF-compatible client device 135. Through this registration process of the MMF proxy, the MMF proxy inherits the security / access attributes of the non-MMF compatible client device 135, which enables the non-MMF compatible client device 135 to obtain the requested media content by transmitting MMF media content via the MMF proxy.

[0063] In one type of deployment example and setup, during the registration process, the non-MMF-compatible client device 135 registers a custom URL scheme with its operating system. All requests using this scheme are passed to the non-MMF-compatible client device 135, causing requests for MMF traffic to be routed to the MMF proxy based on a unique hostname generated during MMF proxy instantiation. In different types of deployment examples and setups (not involving changes to the client-side networking library), content bootstrapping mechanisms defined in MPEG-DASH (e.g., bootstrapping servers, Media Presentation Descriptions (MPDs) with a base URL pointing to CDN1:CDNn, bootstrapping servers, etc.) can be utilized to include extensions to or references to the MMF proxy's BaseURL in the service location element of the content bootstrapping manifest and in the bootstrapping server's response, which includes a path priority element. The path priority element can (to the client non-MMF-compatible client device 135) indicate which service location has the highest priority for requesting content. In a second type of deployment example and setup, the MMF / edge proxy is essentially added as another CDN. In some instances, the DASH MPD includes a content bootstrapping element that defines the URL of the bootstrapping server. In some instances, similar content bootstrapping mechanisms also exist for HLS (e.g., depending on the type of operating system of the non-MMF compatible client device 135).

[0064] As indicated by the above explanation of the registration process and MMF proxy instantiation, the MMF proxy instance is uniquely coupled to the non-MMF compatible client device 135, ensuring that no other non-MMF compatible device 135 can access the specific instance of the MMF proxy. In other words, any additional non-MMF compatible client device 135 will require its own MMF edge proxy, and the edge server 130 will execute method 400 separately to determine whether to instantiate an additional MMF proxy for the additional non-MMF compatible client device 135.

[0065] Upon receiving a media content request from a non-MMF-compatible client device 135 (at box 425), the MMF agent at the edge server 130 is configured to examine the URL and decide whether to issue a request for MMF multi-source media content (at box 430). For example, as explained earlier in this paper regarding the MMF client component on the MMF-compatible client device 135, non-media files (e.g., manifests and captions) may be too small to see significant benefits from MMF encoding. Therefore, for these files, the MMF agent can request a full version of the original media content file from each of the available information sources (at box 435), as doing so is unlikely to impact QoE (e.g., media transmission delays are unlikely to be significant enough to cause playback delays that would affect QoE), and this also improves robustness by giving multiple sources the ability to provide a full version of the original media content file should one of the files be corrupted during transmission. Once the MMF agent at edge server 130 has decided to process the media content request without using MMF encoding (at box 435), method 400 can return to box 425, where the MMF agent can wait for additional media content requests from non-MMF compatible client device 135.

[0066] For media content requests concerning video and / or audio segments, at box 430, the MMF proxy can determine to issue a request for MMF-encoded multi-source content. To this end, the MMF proxy performs functions similar to those of the MMF client component, as explained previously herein. For example, the MMF proxy adds arguments to the URL path to signal upstream devices (e.g., the MMF encoding proxy) that the requested content should be MMF-encoded. The MMF proxy uses the hostnames of the information sources (e.g., N CDNs) from the configuration information to simultaneously (e.g., from multiple sources on multiple communication paths) issue N HTTP requests for the encoded content (at box 440).

[0067] At box 445, edge server 130 concurrently receives multi-source media data (e.g., MMF packets) corresponding to media content requested by edge server 130 from multiple multi-source media sources (e.g., CDN 120 and / or other devices upstream of edge server 130). Upon receiving a first response byte including MMF-encoded multi-source media content from any source, the MMF agent initializes a multi-source media (e.g., MMF) decoder to decode the multi-source media content (e.g., MMF packets).

[0068] At box 450, the MMF agent on edge server 130 uses a multi-source media decoder to decode multi-source media data corresponding to the requested media content into unencoded media content data corresponding to the requested media content. In a manner similar to that explained earlier herein with respect to MMF-compatible client device 135, MMF header information included in the received multi-source media content (i.e., MMF packets) is used to configure the multi-source media (e.g., MMF) decoder. Data from N sources is then streamed to the MMF-configured decoder of the MMF agent. Upon receiving a complete packet from any source, the MMF decoder is configured to add the complete packet to its internal matrix. In one type of deployment example and setup, system MMF packets can be received sequentially and can be stored in a second memory 310 at edge server 130.

[0069] Once the decoder has enough data to decode the entire requested MMF segment, at box 455, the edge server 130 transmits at least a portion of the unencoded media content data corresponding to the requested media content to the non-MMF-compatible client device 135 that previously requested the media content. For example, transmitting at least a portion of the unencoded media content data corresponding to the media content from the edge server 130 to the non-MMF-compatible client device 135 is done using... Figure 1 This is performed by one or more access networks 140 as shown. In some instances, decoded / unencoded data is relayed or requested to the non-MMF-compatible client device 135 via HTTP and standard streaming protocols such as HLS or Moving Picture Experts Group-based Dynamic Adaptive Streaming (MPEG-DASH) using a previously generated unique URL hostname. In other words, although the unencoded media content data is unencoded (because it is not multi-source encoded content data), it can still be 'source' encoded and encapsulated by the edge server 130 using, for example, MPEG video encoding and MPEG-DASH or HLS encapsulation for transmission to the client device 135. Other example options for the non-MMF-compatible client device 135 to relay media content requests via an MMF edge proxy include using a Web Real-Time Communication (WebRTC) peer-to-peer connection. Additionally, once the decoder has enough data to decode the entire requested MMF segment, any incomplete HTTP requests from the MMF proxy are cancelled in a manner similar to that explained above regarding the client component of the MMF-compatible client device 135.

[0070] Once the entire MMF fragment of the request has been decoded and the corresponding unencoded data has been provided to the non-MMF compatible client device 135, method 400 can return to box 425, in which the MMF agent can wait for additional media content requests from the non-MMF compatible client device 135.

[0071] In some instances, after a predetermined period of time during which the MMF proxy has not been used to relay media content to the non-MMF-compatible client device 135, the edge server 130 is configured to stop the operation of the MMF proxy to free up edge computing resources. For example, the edge server 130 may stop the operation of the MMF proxy in response to not receiving a media content request from the non-MMF-compatible client device 135 within the predetermined period of time. In some instances, the edge server 130 may stop the operation of the MMF proxy in response to receiving an indication from the non-MMF-compatible client device 135 (e.g., an indication that the media player on client device 135 is no longer active).

[0072] As previously indicated herein, in some instances, edge server 130 is configured to instantiate multiple MMF proxies, each uniquely associated with a different non-MMF-compatible client device 135. Therefore, edge server 130 can concurrently relay media content (e.g., unencoded media content data) to multiple non-MMF-compatible client devices 135. Additionally, while delivering media content to multiple non-MMF-compatible client devices 135, edge server 130 can also deliver multi-source media content / data (e.g., MMF packets) to one or more MMF-compatible client devices 135. For example, if edge server 130 determines that a second client device 135 requesting media content includes capabilities to decode multi-source media data (i.e., the second client device is MMF-compatible), edge server 130 will deliver multi-source media data (e.g., MMF packets) corresponding to at least a portion of the media content to the second client device 135 without using an instantiated multi-source media (MMF) decoder, since the second client device 135 is capable of decoding the multi-source media data itself.

[0073] In some instances, during the execution of method 400, edge server 130 monitors whether non-MMF compatible client device 135 is moving / in transit (e.g., moving away from edge server 130 and moving out of the coverage area of ​​edge server 130). In some instances, edge server 130 monitors the movement of client device 135 by receiving movement data such as that recorded by motion sensors of client device 135 (e.g., accelerometers, location tracking devices, etc.).

[0074] In some instances, when a non-MMF-compatible client device 135 is in transit, the MMF proxy at a first edge server 130, uniquely paired with the non-MMF-compatible client device 135, dynamically migrates across available edge computing (e.g., migrates to different edge servers 130 and edge networks 125). For example, when the non-MMF-compatible client device 135 moves between coverage areas of adjacent edge networks 125, the first edge server 130 may communicate with a second edge server 130 toward which the client device 135 is moving. In response to determining that the movement data associated with the client device's movement indicates (i) that the client device 135 is approaching the outer limit of the communication range of the first edge server 130 and / or (ii) that the client device is moving away from the first edge server 130 and / or toward the second edge server 130, the first edge server 130 may instruct the second edge server 130 to instantiate a different multi-source media (e.g., MMF) proxy associated with the client device 135 requesting media content at the second edge server 130. In some instances, the different MMF agents on the second edge server 130 may be copies of the MMF agents previously generated by the first edge server 130.

[0075] The second edge server 130 may instantiate an MMF agent according to instructions and assuming the existence of edge computing resources and capabilities on the second edge server 130 as explained earlier herein, in accordance with the above regarding Figure 4A A similar approach, as illustrated in box 420, is uniquely associated with client device 135. For example, instantiation of an MMF proxy may include instantiating different multi-source media (e.g., MMF) decoders associated with client device 135. Second edge server 130 may concurrently receive multi-source media data corresponding to media content requested by client device 135 from first edge server 130 and / or second edge server 130 from a second plurality of multi-source media sources. Second edge server 130 may use different multi-source media decoders to decode the multi-source media data (e.g., MMF packets) corresponding to the media content into unencoded media content data corresponding to the media content requested by client device 135. Second edge server 130 may transmit at least a portion of the unencoded media content data corresponding to the media content from second edge server 130 to client device 135.

[0076] In some instances, the transfer of at least a portion of the unencoded media content data corresponding to the media content from a first edge server 130 to a client device 135 and the transfer of at least a portion of the unencoded media content data corresponding to the media content from a second edge server 130 to the client device 135 are performed concurrently. In other words, both the first edge server 130 and the second edge server 130 can transfer unencoded media content data to the client device 135 simultaneously (e.g., during a switchover / dynamic migration of MMF proxies at the two edge servers 130). For example, this parallel unencoded media content transfer can be performed unicast over a separate communication link. In some instances, the first edge server 130, whose coverage area is outside the client device 135, is configured to stop transferring at least a portion of the unencoded media content data corresponding to the media content to the client device 135 before the second edge server 130 transfers it. In other words, in some instances, only one edge server 130 can transfer unencoded media content to the client device 135 at a time. In some of these instances, the first edge server 130 can stop delivering unencoded media content to the client device 135 at the same time the second edge server 130 begins delivering unencoded media content to the client device 135. In some instances, the first and second edge servers 130 can communicate with each other during MMF proxy switching / dynamic migration to ensure a seamless transition of the client device 135's MMF proxy service. For example, the first edge server 130 can notify the second edge server 130 of the point in the unencoded media content where it sent its last packet, allowing the second edge server 130 to begin its delivery at the same point.

[0077] Performing the method 400 and related functions described herein yields numerous advantages. For example, media content previously only available to non-MMF-compatible client devices 135 via a single communication path through distribution network 100 can now be transmitted from more than one source to edge server 130 as concurrently transmitted multi-source media data corresponding to the requested media content via upstream portions of distribution network 100 (e.g., edge server 130 with an MMF proxy and devices upstream of edge server 130), and then the edge server ultimately provides the media content to client device 135. In some instances, more than one source may include a cache populated via a JIT encoding process associated with the client device 135 requesting the media content. For example, the cache may reside at one or more origin servers 115 of CDN 120. As another example, the cache may additionally or alternatively reside on edge server 130 itself. In the example, when the cache resides on edge server 130, edge server 130 is configured to update the permissions / permissions for cached media content such that the cached media content can be accessed by other client devices 135 even if it was requested by an MMF proxy uniquely paired with a single non-MMF-compatible client device 135. Therefore, (i) an MMF-compatible client device 135 or (ii) another non-MMF-compatible client device 135 for which an MMF proxy was created can utilize multi-source media content cached at edge server 130 or elsewhere in distribution network 100, content previously requested by other client devices 135 and populated via encoding processes associated with other client devices 135. In some instances, the cache may additionally or alternatively reside on a peer device (e.g., another client device 135) and / or a local cache (e.g., a device connected to the same local network or WLAN as the client device 135 requesting the media content).

[0078] Due to the general increase in caching of multi-source media content across the entire distribution network 100 by implementing an MMF proxy at edge server 130 for at least some non-MMF-compatible client devices 135, other client devices 135 (e.g., MMF-compatible client devices 135) benefit from the execution of method 400. For example, the additional caching of multi-source media content across the entire distribution network 100 means that all devices requesting such content can typically request it from a more downstream source than would otherwise be required. Therefore, passing such content from one or more sources to many client devices 135 (without necessarily causing downstream caches at the source to be filled with such content) can be achieved faster and more efficiently. In other words, the execution of method 400 and the relevant functionality herein involves filling the CDN cache with multi-source encoded media (e.g., CMMF packets) for all non-MMF-compatible and MMF-compatible client devices 135, thereby reducing or eliminating the need to cache both MMF media content and non-MMF media content. Reducing or eliminating the need to store non-MMF media content reduces the amount of storage space utilized by the origin server 115, such as in CDN 120, thereby increasing the amount of storage space available to store MMF media content for all client devices 135.

[0079] Furthermore, the techniques described herein enable multi-access and edge-optimized streaming of media content, wherein (i) an MMF-compliant client device 135 simultaneously streams via one or more access networks 140; (ii) an edge server 130 can be accessed only via a subset of the access networks (e.g., 4G / 5G only); (iii) the edge server 130 is instantiated on demand by the network operator (e.g., due to a large number of non-MMF devices, a large number of subscribers in the area, etc.); and (iv) the client device 135 probes various access networks 140 and selects one(s) to request. Benefits include improved QoS / E through multi-source and multi-path (statistical multiplexing) gains from MMF end-to-end without complex network orchestration and / or scheduling.

[0080] It should be understood that the embodiments are not intended to limit their application to the configuration details and component arrangements set forth herein or illustrated in the accompanying drawings. The embodiments can be practiced or implemented in various ways. Similarly, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “including,” “comprising,” or “having,” and variations thereof is intended to cover the items listed thereafter and their equivalents, as well as additional items. Unless otherwise specified or limited, the terms “installation,” “connection,” “support,” and “coupled,” and variations thereof, are widely used and cover direct and indirect installation, connection, support, and coupling.

[0081] Additionally, it should be understood that embodiments may include hardware, software, and electronic components or modules, which, for the purposes of discussion, may be illustrated and described as if most components were implemented solely in hardware. However, those skilled in the art, and based on a reading of this specific embodiment, will recognize that in at least one embodiment, the electronic aspects may be implemented in software (e.g., stored on a non-transitory computer-readable medium) executable by one or more electronic processors (such as microprocessors and / or application-specific integrated circuits (“ASICs”)). Thus, it should be noted that embodiments may be implemented using multiple hardware and software-based devices and multiple different structural components. For example, the “server” and “computing device” described in the specification may include one or more electronic processors, one or more computer-readable medium modules, one or more input / output interfaces, and various connectors (e.g., system buses) connecting the various components.

[0082] Various aspects of this disclosure can be understood from the following enumerated example embodiments (EEE): EEE 1. A method comprising: Determine whether one or more criteria for multi-source relay mode are met; and Based on determining that the multi-source relay mode is satisfied by one or more criteria, wherein satisfying the multi-source relay mode includes determining that the client device requesting the media content does not support decoding multi-source media data: A multi-source media decoder associated with the client device requesting the media content is instantiated at a first server communicating with the client device requesting the media content; The first server concurrently receives multi-source media data corresponding to the media content from multiple multi-source media sources; The multi-source media decoder is used to decode the multi-source media data corresponding to the media content into unencoded media content data corresponding to the media content; and At least a portion of the unencoded media content data corresponding to the media content is transmitted from the first server to the client device.

[0083] EEE 2. The method as described in EEE 1, wherein satisfying the one or more criteria of the multi-source relay mode includes determining that the number of client devices within the communication range of the first server and not supporting decoding of multi-source media data exceeds a threshold number.

[0084] EEE 3. The method of any one of EEE 1 to 2, wherein satisfying the one or more criteria of the multi-source relay mode includes determining that one or more performance parameters associated with delivering data corresponding to the media content to the client device requesting the media content exceed one or more performance criteria.

[0085] EEE 4. The method as described in any one of the preceding EEE methods further includes: The second client device that determines the requested media content includes support for decoding multi-source media data: Multi-source media data corresponding to at least a portion of the media content is transmitted from the first server to the second client device without using the multi-source media decoder.

[0086] EEE 5. The method as described in any of the preceding EEE methods, wherein determining that the client device requesting the media content does not support decoding multi-source media data is based on data included in the request for the media content.

[0087] EEE 6. The method as described in any of the preceding EEE methods, wherein the multi-source media data corresponding to the media content corresponds to a DASH or HLS segment.

[0088] EEE 7. The method as described in any of the preceding EEE methods, wherein the plurality of multi-source media sources include a cache populated via an on-the-fly encoding process associated with the client device requesting the media content.

[0089] EEE 8. The method as described in any of the preceding EEEs, wherein the plurality of multi-source media sources include a cache populated via an encoding process associated with a client device other than the client device requesting the media content.

[0090] EEE 9. The method as described in any of the preceding EEE methods, wherein the plurality of multi-source media sources include peer devices, local caches, or both the peer devices and the local caches.

[0091] EEE 10. The method as described in any one of the preceding EEE methods further comprises: Instantiate different multi-source media decoders associated with the client device requesting the media content at the second server; The second server concurrently receives the multi-source media data corresponding to the media content from a second plurality of multi-source media sources; The different multi-source media decoders are used to decode the multi-source media data corresponding to the media content into the unencoded media content data corresponding to the media content; and At least a portion of the unencoded media content data corresponding to the media content is transmitted from the second server to the client device.

[0092] EEE 11. The method as described in EEE 10, wherein instantiating the different multi-source media decoder associated with the client device requesting media content at the second server is performed in response to motion data associated with the movement of the client device, wherein the motion data indicates that the client device is moving away from the first server and toward the second server.

[0093] EEE 12. The method of any one of EEE 10 to 11, wherein transferring at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device and transferring at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device are performed concurrently.

[0094] EEE 13. The method as described in any one of EEE 10 to 11, further comprising: Before transmitting at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device, the transmission of at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device is stopped.

[0095] EE 14. A computing device comprising: At least one electronic processor; and A memory that stores instructions that, when executed by the at least one electronic processor, cause the computing device to perform a method as described in any one of EEE 1 to 13.

[0096] EEE 15. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform a method as described in any one of EEE 1 to 13.

[0097] Various features and advantages are set forth in the appended claims.

Claims

1. A method comprising: Receive requests for media content from the client device; Determine whether one or more criteria for configuring a multi-source relay mode are met at the first server, the multi-source relay mode enabling media data received from multiple multi-source media sources on multiple communication paths to be decoded and delivered to the client device; as well as Based on determining that the multi-source relay mode is satisfied by one or more criteria, wherein satisfying the multi-source relay mode includes determining that the client device requesting the media content does not support decoding multi-source media data: A multi-source media decoder associated with the client device requesting the media content is instantiated at the first server communicating with the client device requesting the media content; The first server concurrently receives multi-source media data corresponding to the media content from multiple multi-source media sources; The multi-source media decoder is used to decode the multi-source media data corresponding to the media content into unencoded media content data corresponding to the media content; and At least a portion of the unencoded media content data corresponding to the media content is transmitted from the first server to the client device.

2. The method as described in claim 1, wherein, The criteria for satisfying the multi-source relay mode include determining that the number of client devices within the communication range of the first server that do not support decoding multi-source media data exceeds a threshold number.

3. The method according to any one of claims 1 to 2, wherein, The criteria for satisfying the multi-source relay mode include determining that one or more performance parameters associated with delivering data corresponding to the media content to the client device requesting the media content exceed one or more performance criteria.

4. The method as described in any of the preceding claims, further comprising: The second client device that determines the requested media content includes support for decoding multi-source media data: Multi-source media data corresponding to at least a portion of the media content is transmitted from the first server to the second client device without using the multi-source media decoder.

5. The method as described in any one of the preceding claims, wherein, The determination that the client device requesting the media content does not support decoding multi-source media data is based on the data included in the request for the media content.

6. The method as described in any of the preceding claims, wherein, The multi-source media data corresponding to the media content corresponds to DASH or HLS segments.

7. The method as described in any of the preceding claims, wherein, The plurality of multi-source media sources include a cache populated via an on-the-fly encoding process associated with the client device that requests the media content.

8. The method as described in any of the preceding claims, wherein, The plurality of multi-source media sources include a cache populated via an encoding process associated with client devices other than the client device requesting the media content.

9. The method as described in any of the preceding claims, wherein, The plurality of multi-source media sources include peer devices, local caches, or both the peer devices and the local caches.

10. The method as described in any of the preceding claims, further comprising: Instantiate different multi-source media decoders associated with the client device requesting the media content at the second server; The second server concurrently receives the multi-source media data corresponding to the media content from a second plurality of multi-source media sources; The different multi-source media decoders are used to decode the multi-source media data corresponding to the media content into the unencoded media content data corresponding to the media content; as well as At least a portion of the unencoded media content data corresponding to the media content is transmitted from the second server to the client device.

11. The method of claim 10, wherein, Instantiating the different multi-source media decoders associated with the client device requesting media content at the second server is performed in response to motion data associated with the movement of the client device, wherein the motion data indicates that the client device is moving away from the first server and toward the second server.

12. The method according to any one of claims 10 to 11, wherein, The transfer of at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device and the transfer of at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device are performed concurrently.

13. The method of any one of claims 10 to 11, further comprising: Before transmitting at least a portion of the unencoded media content data corresponding to the media content from the second server to the client device, the transmission of at least a portion of the unencoded media content data corresponding to the media content from the first server to the client device is stopped.

14. A computing device, comprising: At least one electronic processor; as well as A memory that stores instructions that, when executed by the at least one electronic processor, cause the computing device to perform the method as described in any one of claims 1 to 13.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 13.

Images