System and Method for Lightweight Bitrate-Resolution Optimization for Live Streaming and Transcoding

Abstract

Systems and methods for transcoding at least a portion of a live media asset captured from a media content source are described. The systems and methods may be configured to determine, in real time, parameters of at least a portion of the live media asset after the step of capturing at least a portion of the live media asset. The systems and methods may further be configured to determine, in real time, a plurality of optimal bitrate-resolution pairs for at least a portion of the live media asset based on the parameters after the step of capturing at least a portion of the live media asset, and may be configured to cause at least a portion of the live media asset to be transcoded based on the plurality of optimal bitrate-resolution pairs in real time after the step of capturing at least a portion of the live media asset.

Description

Technical Field

[0001] The present disclosure is directed to systems and methods for transcoding media assets based on an optimal bitrate-resolution pair. In particular, such transcoding may be performed in real time after the step of capturing at least a portion of a live media asset.

Summary of the Invention

Means for Solving the Problems

[0002] Adaptive bitrate (ABR) streaming has been widely deployed to provide high-quality video and viewer experience. ABR streaming can be used in demanding scenarios, such as short-latency live streaming, in response to user and network events. Many service providers deploy HTTP Adaptive Streaming (HAS) through HTTP Dynamic Adaptive Streaming over HTTP (DASH) or HTTP Live Streaming (HLS).

[0003] In video streaming, there are many challenges, including network conditions, user requirements, and the heterogeneity of content compression performance. To help ensure a high quality of experience (QoE), videos are encoded at different resolutions and bitrates to produce a set of bitrate-resolution pairs for the video, which can be referred to as an ABR ladder, enabling adaptation to variable conditions. In one approach, a static ABR ladder is adopted for all content, a pre-defined bitrate point is used regardless of the content, which is called a general-purpose approach. In another approach, the defined bitrate points can be differentiated based on the genre of the content, i.e., a higher bitrate can be used for content with fast motion and rapid scene changes such as sports. However, such an approach does not consider the dependence of video compression performance on diverse content characteristics, resulting in significant coding artifacts and thus a decrease in QoE for certain content.

[0004] In another approach, content optimization solutions have been developed. In such multi-objective optimization, each video is split into short segments or chunks, and each chunk is encoded using optimized parameters such as resolution, quantization parameter, inter-frame distance, etc. The goal is to construct a Pareto frontier (PF) across all rate-distortion (RD) curves, analyze a set of target bitrates, and find the best encoded bitstream. Such an approach, which can be referred to as per-title and shot-optimized encoding, delivers higher-quality video in two ways. Under low-bandwidth conditions, this often delivers better video quality as more easily encodable content that is streamed at a higher resolution for the same bitrate. When the bandwidth is appropriate for high-bitrate encoding, this provides even better video quality for complex titles as it will be encoded at a higher maximum bitrate than in fixed-ladder production where it is not optimized.

[0005] Given the extensive parameter space in such optimizations and the need to repeat this process for each chunk, this approach requires a vast amount of computational resources. Thus, this technique is very expensive in terms of computational, monetary, and energy costs, assuming the need to provide content in different formats to different users in various locations using various types of devices with various connectivity and display capabilities. In addition to complexity and cost, the extensive iterative processing renders this approach infeasible for deployment in live ABR streaming applications that do not have any privileged access to gather post - knowledge for optimizing the ladder in live and low - latency use cases. In one approach for live ABR streaming, a fixed ladder where the bitrate is associated with a predefined resolution is used. However, there is a need to construct a low - complexity optimized ABR ladder that is responsive in transcoding and practical for live ABR streaming with reduced latency.

[0006] To overcome these problems, a computer - implemented system and method for capturing at least a portion of a live media asset from a media content source are provided herein. The system and method may perform, after the step of capturing the live media asset, in real - time, the step of determining at least some parameters of the captured live media asset, the step of determining a plurality of optimal bitrate - resolution pairs for at least a portion of the live media asset based on the parameters, and the step of transcoding at least a portion of the live media asset based on the plurality of optimal bitrate - resolution pairs.

[0007] Such aspects may enable low-complexity modules to estimate video complexity and generate a content-based ABR ladder in order to achieve real-time optimization of live content. In some embodiments, such portions of the media asset may already be encoded when the such portions of the media asset are captured, and the systems and methods provided herein may analyze and analyze bitstream-level statistics that are already available in connection with the captured media asset. Such analysis of bitstream-level encoding data can provide sufficient information while resulting in a very low complexity cost in processing. In some embodiments, the systems and methods provided may employ a prediction network to derive a best-estimated bitrate-resolution pair that is highly adaptive and content-dependent. In some embodiments, the content optimization ABR ladder may be included as metadata within the bitstream of the encoding data. Such metadata may be associated with a very small payload of optimized encoding parameters and further provides an effective means for improving transcoding, e.g., transcoding at an edge server or a central server. In some embodiments, the systems and methods provided may be employed in live ABR streaming and / or non-live ABR streaming where transcoding at the edge can often occur.

[0008] In some aspects of the present disclosure, the provided systems and methods may further be configured to generate a bitstream that includes metadata, the metadata including a plurality of optimal bitrate-resolution pairs. The provided systems and methods may transcode at least a portion of a live media asset by transmitting the bitstream from a central server to one or more edge servers. Such one or more edge servers may be configured to transcode at least a portion of the live media asset based on the plurality of optimal bitrate-resolution pairs indicated within the metadata.

[0009] In some embodiments, at least a portion of the live media asset is a segment of the live media asset, the live media asset includes a plurality of segments, and the transmitted bitstream includes a single indication of metadata for each individual segment of the plurality of segments.

[0010] In some aspects of the present disclosure, the central server performs an ingestion of at least a portion of a live media asset from a media content source. The provided systems and methods may cause the central server to perform transcoding of at least a portion of the live media asset based on a plurality of optimal bitrate-resolution pairs. Such transcoded at least a portion of the live media asset may be transmitted to one or more edge servers and / or client devices.

[0011] In some embodiments, the provided systems and methods may further be configured to train a machine learning model using training data that includes a plurality of parameters for at least individual portions of a plurality of media assets and corresponding bitrate-resolution pairs. The trained machine learning model may be configured to receive, as input, parameters for at least a portion of a captured live media asset and output a plurality of optimal bitrate-resolution pairs for the captured live media asset. In some aspects of the present disclosure, the parameters of the training data include an indication of a genre (e.g., action, comedy, sports, drama, documentary, or any other suitable categorization of content, or any combination thereof) for at least individual portions of the plurality of media assets of the training data.

[0012] In some aspects of the present disclosure, the step of determining parameters for at least a portion of a live media asset includes extracting scene and motion statistics from a bitstream corresponding to at least a portion of the captured live media asset.

[0013] In some embodiments, at least a portion of the live media asset is a segment of the live media asset, the live media asset includes a plurality of segments, and the step of determining parameters for at least a portion of the live media asset includes determining parameters for at least one segment of the plurality of segments. In some aspects of the present disclosure, the parameters include the genre of at least a portion of the live media asset or at least one segment thereof.

[0014] In some embodiments, the provided systems and methods may further be configured to transcode at least a portion of the live media asset based on a plurality of optimal bitrate-resolution pairs in response to receiving a request regarding at least a portion of the captured live media asset from a client device.

[0015] In some embodiments, at least a portion of the live media asset is encoded when it is captured.

[0016] In some embodiments, at least a portion of the live media asset is not encoded when it is captured, and the provided systems and methods may further be configured to encode at least a portion of the captured live media asset, and the parameters of the live media asset may be determined at least in part based on the step of performing the encoding.

Brief Description of the Drawings

[0017] The present disclosure will be described in detail with reference to the following figures in accordance with one or more various embodiments. The figures are provided for illustrative purposes only and depict only typical or exemplary embodiments. These figures are provided to facilitate understanding of the concepts disclosed herein and should not be construed as limiting the scope, range, or availability of these concepts. Note that these figures are not necessarily drawn to scale for clarity and ease of illustration.

[0018]

Figure 1

[0019]

Figure 2

[0020]

Figure 3

[0021]

Figure 4

[0022]

Figure 5

[0023]

Figure 6

[0024]

Figure 7

[0025]

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0026] Detailed Description FIG. 1 shows an illustrative block diagram 100 for transcoding at least a portion of a media asset 106 for an ABR streaming process. During a capture stage, at least a portion of the media asset 106 may be captured by one or more than one of a server 104, such as an origin server of a content delivery network (CDN), a central server, or an edge server, or by any other suitable computing device, or any combination thereof. In some embodiments, at least a portion of the media asset 106 may include a single segment or scene or chunk or other single portion of the media asset, the entire media asset, or any suitable duration of the media asset or some portions thereof. In some embodiments, at least a portion of the media asset 106 may be encoded in a particular format, such as in a pre-encoded media asset, when being captured. Alternatively, in some embodiments, at least a portion of the media asset 106 may not be encoded and / or compressed when being captured, and thus, encoding of at least a portion of the media asset 106 may be performed after the step of capturing at least a portion of the media asset 106. A single server 104 and a content source 102 are shown in FIG. 1, but any suitable number of servers and content servers (and / or edge servers or any other suitable computing device) may be utilized and computing tasks may be distributed across such individual groups of servers. In some embodiments, the server 104 may be referred to as and operate as a central server or central data center of a CDN.

[0027] At least a portion of media asset 106 may be received at 106 from any suitable content source 102 (e.g., via any suitable communication network 609 of FIG. 6 or other wireless or wired link). In some embodiments, such a content source or other external source may perform encoding on the uncompressed and / or raw version of the media asset in order to obtain at least a pre-encoded portion of the media asset 106. In some embodiments, the content source 102 may be a repository of the media asset and / or a portion thereof, or the location where a live media asset or a portion thereof is generated (e.g., a video feed from a sports game), or the location where such a live media asset or portion is otherwise received and further transmitted. The content sources may be collocated, or they may also reside at various locations. The content from each source may be in the same format, or the content from some or all of the sources may be in any other suitable format. In some embodiments, server 104 may be configured to perform at least a portion of the encoding, in addition to or as an alternative. In some embodiments, at least a portion of media asset 106 may be captured in an unencoded and / or uncompressed format. In some embodiments, at least a portion of media asset 106 may be in a compressed format when captured by server 104. In some embodiments, at least a portion of media asset 106 may include bitstream-level statistics (e.g., included in metadata) from which encoding parameters can be extracted, or may be transmitted separately with or associated with it, as will be described in more detail below.

[0028] In some embodiments, at least a portion of the captured media asset 106 may correspond to live content. Such a captured at least a portion of the media asset 106 corresponding to live content may be, for example, a high bitrate rendition of at least a portion of the media asset 106. In some embodiments, at least a portion of the media asset 106 may correspond to on-demand content. At least a portion of the media asset 106 may be received in any suitable format. Examples of formats of at least a portion of the media asset 106 may be a particular bitrate (such as those in kbps, etc.) at which at least a portion of the media asset 106 is to be streamed, a resolution (such as those in pixels or voxels, etc., such as 1,920×1,080 or any other suitable resolution, etc.), a frame rate, progressive or interlaced video, a video and / or audio codec, or any combination thereof, and / or any other suitable attribute of the media asset or a portion thereof may constitute a particular format.

[0029] As referred to herein, the term "media asset" is to be understood to refer to electronically consumable user assets such as live content, television programming, and paper view programming, on-demand programs (such as in a video on demand (VOD) system), Internet content (such as streaming content, downloadable content, webcasts, etc.), augmented reality content, virtual reality content, three-dimensional content, video clips, audio, playlists, websites, articles, e-books, blogs, social media, applications, games, and / or any other media or multimedia, and / or combinations of the foregoing.

[0030] As mentioned herein, at least partial compression and / or encoding of media asset 106 can be understood as the performance of any suitable combination of hardware and / or software of bit reduction techniques on at least a portion of the digital bits of the media asset to reduce the amount of storage space required to store at least a portion of the media asset. Such techniques can reduce the bandwidth or network resources required to transmit at least a portion of the media asset via a network or other suitable wireless or wired communication medium and / or enable bitrate savings related to downloading or uploading the media asset. Such techniques may encode at least a portion of the media asset such that the encoded media asset or its encoded portion is represented using fewer digital bits than the original representation while minimizing the impact of encoding or compression on the quality of at least a portion of the media asset. In some embodiments, the encoding of at least a portion of the media asset may employ a hybrid video coder such as, for example, the High Efficiency Video Coding (HEVC) H.265 standard, the Versatile Video Coding (VVC) H.266 standard, the H.264 standard, the H.263 standard, MPEG-4, MPEG-2, or any other suitable codec or standard, or any combination thereof.

[0031] In some embodiments, the ABR ladder generator system may be configured to perform the techniques described above and below, at least partially, on server 104 and / or on any other suitable computing device described herein (e.g., in FIGS. 1-5). In some embodiments, certain functionality provided by the ABR ladder generator system may be provided via an application programming interface (API) or software development kit (SDK). At 107, the ABR ladder generator system may be configured to perform transcoding operations on at least a portion of media asset 106. As used herein, "transcoding" refers to operating on at least a portion of the digitally compressed and encoded data of a media asset to convert such data from a first format (or specification) to a second format (or specification). For example, the first format may be a first encoding format and the second format may be a second encoding format. In some embodiments, at least a portion of the original, uncompressed media asset (rather than a digitally compressed version of at least a portion of the media asset) may be available to the transcoder (e.g., implemented by the ABR ladder generator system). In such cases, "transcoding" may additionally or alternatively refer to encoding at least a portion of the original, uncompressed format of the media asset into a new analog or digital format of at least a portion of the media asset. In some embodiments, transcoding may include the step of re-encoding the encoded media asset or a portion thereof into a different encoding format. In some embodiments, one or more than one of the transcoding or encoding processes may be lossless or lossy.

[0032] The ABR ladder generator system may perform transcoding for any suitable purpose. For example, the ABR ladder generator system may perform transcoding to generate an optimized bitrate-resolution pair or resolution-bitrate pair, at least in part, for a particular media asset or a portion thereof, for purposes such as facilitating ABR streaming to improve a user's QoE regarding consuming a media asset (or a portion thereof) via a network, even when network conditions are inconsistent. The ABR ladder generator system may employ any suitable ABR streaming technique. The ABR ladder generator system may perform transcoding to generate various formats of a media asset (or a portion thereof), at least in part, to adapt to the variable capabilities of different types of client devices and / or specific platforms or operating systems of client devices that may request access to the media asset (or a portion thereof).

[0033] The ABR ladder generator system may be configured to transcode at least a portion of media asset 106 into any suitable number of formats, such as an optimized bitrate-resolution pair, including, for example, a high bitrate format 108 for transmission to an edge server and / or a client device, a medium bitrate format 110, and a low bitrate format 112. The ABR ladder may include a set of segments of different qualities (e.g., based on the number of bits used to represent a media asset or a portion thereof and / or the rate of transmission of such bits) and a resolution available for streaming to a client within a CDN to enable dynamic adaptation to variable conditions and different types of segments or different types of media assets. In some embodiments, the ABR ladder is content-dependent and may vary, for example, based on attributes of at least a portion of the media asset. In some embodiments, the resolution may be predefined and an optimal bitrate may be identified for such resolution. In some embodiments, such at least a portion of the media asset transcoded based on a plurality of optimized bitrate-resolution pairs may be stored at server 104 and / or an edge server communicating with server 104. In some embodiments, at 114 (which may occur during the process of transcoding at 107 or as part of a different process), different renditions of the ABR stream may correspond to different periods within the media asset runtime and may be segmented, for example, into segments that are 2 to 10 seconds in length or any other suitable length. For example, the ABR ladder generator system may generate a segment 116 corresponding to rendition 108, a segment 118 corresponding to rendition 110, and a segment 120 corresponding to rendition 112. In some embodiments, the segments may be predefined, for example, by a content provider, prior to transcoding.

[0034] In some embodiments, one or more servers may initiate the step of distributing content via a network (e.g., network 609 of FIG. 6) in response to receiving a request from a client device. The format in which the content (or segments thereof) is distributed may be selected such that it is compatible with the client device's network and display capabilities. For example, such a selection process may include identifying whether the client device can play or view the format and / or whether there is sufficient bandwidth between the terminal and one or more servers for delivering that format. In some embodiments, the client device may receive, for example, a manifest file from server 104 and use the manifest file to request segments of the transcoded rendition that are currently optimal for its connectivity, display, and processing power. Such variables may change in the middle of a stream while the client device is playing at least a portion of the media asset, at which point the client device may detect the change and automatically request another step for a different rendition, e.g., for an ABR ladder generated by an ABR ladder generator application.

[0035] As referred to herein, the term "manifest" should be understood to refer to a file and / or data structure that contains information about sequential segments (including sequential frames) of a media asset that are available to a client device. Such information may include, for example, the number of segments in a playlist, the bitrate of each segment, the codec associated with each segment, the resolution of each segment, the timing of each segment, the location on the network where the segment can be read, the bandwidth of each segment, the video track of each segment, the audio track of each segment, the subtitle track of each segment, the caption of each segment, the language of each segment, other metadata associated with each segment, and / or any other suitable information. The manifest may be utilized in connection with various streaming protocols employed by an ABR ladder generator system, such as a media presentation description (MPD) file for HTTP-based dynamic adaptive streaming (MPEG-DASH), an m3u8 file for HTTP live streaming (HLS), an f4m file for HTTP dynamic streaming (HDS), an ingest file for CMAF (Common Media Application Format), and / or a manifest file for Microsoft Smooth Streaming (MSS), or any other suitable protocol, or any combination thereof. The manifest may be a standard manifest (e.g., an MPD file from MPEG-DASH), or a modified version of a standard manifest. A segment may include information about a particular interval of a media asset (e.g., encoded video, audio, subtitle information, error correction bits, error detection bits, etc.), and each segment may correspond to a file defined within the manifest that indicates an associated URL for reading the file.A segment may include a set or sequence of frames (e.g., still images that together constitute the moving pictures of a part of a media asset scene), and each segment may have a specific length (e.g., from 1 second to several seconds). In some embodiments, the manifest may be an XML file.

[0036] Figures 2A-2B show illustrative block diagrams 200, 201 for transcoding at least a portion of media asset 106 for an ABR streaming process, according to some embodiments of the present disclosure. In particular, FIGS. 2A-2B show two alternatives for video encoding by a central data center and edge servers in an extended scalable framework for ABR streaming, each configured to utilize cloud computing capabilities. As shown in FIG. 2A, at least a portion of the ingested media asset 106 may be received at server 104 (e.g., a central data center), which may perform transcoding of at least a portion of the ingested media asset 106 based on an optimized bitrate-resolution ladder for at least a portion of media asset 106. Such transcoded at least a portion of media asset 106 may be transmitted to any suitable number of edge servers, such as 202, 204, and then transmitted to any suitable number of client devices in response to requests according to a particular ABR streaming protocol. In some embodiments, multiple edge servers may be strategically located in various geographical locations to optimize content delivery. In some embodiments, one or more than one of the edge servers may be a mobile edge server configured to provide processing support for mobile devices in various geographical regions. Each edge server may be positioned at the edge of a CDN, cache certain content according to a certain caching policy, and facilitate quickly providing the content requested by client device 206. Client device 206 may correspond to any suitable device, such as a television, a mobile device (e.g., a smartphone, a tablet, a smartwatch, and / or any other suitable mobile device), an Internet of Things (IoT) device, a biometric device, a desktop computer, a laptop computer, a virtual reality (VR) device, an augmented reality (AR) device, and / or any other suitable device, or any combination thereof.

[0037] In some embodiments, the ABR ladder generator system may employ the arrangement shown in block diagram 201 of FIG. 2B, where at least a portion of the captured media asset 106 can be received at the server 104 (e.g., a central data center), and the server 104 can transmit a single high-bitrate rendition of at least a portion of the media asset 106 to the edge servers 202, 204. In some embodiments, the server 104 obtains such a high-bitrate rendition of at least a portion of the media asset 106 directly from the content source 102, or performs encoding and / or transcoding on the content received from the content source 102 to obtain such a high-bitrate rendition of at least a portion of the media asset 106. In the example of FIG. 2B, the edge servers 202 and / or 204 and / or any other suitable number of edge servers may be used to perform at least a portion of the transcoding of at least a portion of the captured media asset 106 based on an optimized bitrate-resolution ladder for at least a portion of the media asset 106. At least a portion of the transcoded media asset 106 may be transmitted to any suitable number of client devices 206 in response to requests according to a particular ABR streaming protocol. Compared to FIG. 2A, in the arrangement of FIG. 2B, the edge servers 202 and / or 204 may utilize more computing power and / or transcoding or encoding firmware to perform transcoding. On the other hand, in the arrangement of FIG. 2A, more bandwidth may be consumed to transmit multiple copies of at least a portion of the transcoded media asset 106 to the edge servers 202, 204. In some embodiments, the transcoding may be split between the server 104 and each of the edge servers 202 and 204, or the transcoding may be performed at the server 104 or at one of the edge servers 202 and 204.

[0038] Figures 3A-3B illustrate an exemplary block diagram for training a machine learning model 306, using the trained machine learning model 312, and generating a content optimization bitrate-resolution pair 314 for at least a portion of the media asset 106, according to some embodiments of the present disclosure. In some embodiments, the machine learning model 306 may be referred to as a bitrate-resolution prediction network that may be trained to obtain the trained machine learning model 312. The trained machine learning model 312 is used for predicting an optimal resolution per bitrate for each media asset (and / or individual portions thereof) associated with an input to the trained machine learning model 312 to generate an ABR ladder, and may facilitate real-time adaptive bitrate transcoding of such media assets, such as media asset 106 or a portion thereof. In some embodiments, the machine learning model 312 may be a neural network, or any other suitable machine learning model, or any combination thereof. Although the machine learning model is described in connection with FIGS. 3A-3B, any suitable computer implementation technique, such as heuristic-based analysis, may be used for predicting an optimal resolution per bitrate for each media asset (or portion thereof) to generate an ABR ladder. The machine learning model may be implemented by an ABR ladder generator system, for example, in server 104 and / or edge server 202 and / or edge server 204, and / or in content source 102, and / or in any other suitable computing device, or any combination thereof.

[0039] In some embodiments, training data from the encoded video dataset database 302 may be used to train the machine learning model 306. In some embodiments, the encoded video dataset database 302 may correspond to the media content source 102 and / or the server 104 and / or the database associated with the server 104. In some embodiments, the training data from the encoded video dataset database 302 may be associated with any suitable number of media assets (or individual portions thereof) of any suitable format and characteristics, some of which may be associated with various genres. In some embodiments, the encoded video dataset database 302 may include comprehensive combinations of various genres, bitrates, and resolutions of training content. In some embodiments, the encoded video dataset database 302 may include data associated with previously streamed live content and / or previously streamed non-live content.

[0040] The ABR ladder generator system may employ any suitable computer-implemented technique for assessing the complexity of a particular media asset (or one or more portions thereof). For example, the ABR ladder generator system may be configured to extract one or more parameters 304 regarding a particular media asset (or one or more portions thereof) that may be pre-encoded or otherwise encoded, such one or more parameters 304 being stored in an encoded video dataset database 302 in a state associated with the corresponding media asset (and / or a portion thereof). In some embodiments, the encoded video dataset database 302 may store the media asset (or portions thereof) themselves, data and parameters associated with the media asset, or any combination thereof. In some embodiments, non-invasive techniques may be employed by the ABR ladder generator system, e.g., an uncompressed rendition of at least a portion of the media asset may not be required to obtain one or more parameters 304. Instead, the ABR ladder generator system may extract, at low cost and optionally, bitstream level statistics associated with a pre-encoded media asset (or a portion thereof), such extracted bitstream level statistics corresponding to or otherwise being used to obtain one or more parameters 304 that may be used to estimate the complexity of the encoded video.Additionally, or alternatively, the ABR ladder generator system may be configured to identify or determine at least some parameters of media asset 106, such as by extracting parameters determined during an encoding process, for use in determining an optimal bitrate-resolution pair when at least a portion of media asset 106 is being captured in a format that is not encoded, or during or based on steps of performing encoding on at least a portion of media asset 106. In some embodiments, any suitable parametric model may be implemented by the ABR ladder generator system to perform such extraction and / or assessment of the complexity of a media asset (or a portion thereof) corresponding to one or more parameters 304. In some embodiments, the ABR ladder generator system may be configured to assess the complexity of the entire media asset and / or to assess the complexity of segments or portions of the media asset.

[0041] In some embodiments, the one or more extracted parameters 304 may be included in metadata that is associated with the media asset stored in database 302. In some embodiments, the one or more parameters 304 may be defined in a header that is associated with the bitstream of the encoding data of the media asset. The ABR ladder generator system may analyze and analyze the bitstream to optimize the bitrate-resolution pair for ABR streaming, for example, by using one or more parameters 304 extracted from the bitstream for training machine learning model 306.

[0042] In some embodiments, the machine learning model 306 may be trained using, for example, supervised learning to refine parameters such as weights and / or bias values and / or other internal model logic associated with the layers of the model 306, and to minimize a loss function, using labeled training examples, to help the model 306 converge within an acceptable error range. In some embodiments, each layer may include one or more nodes that may be associated with learned parameters (e.g., weights and / or biases), and / or the connections between nodes may represent learned parameters (e.g., weights and / or biases) during training (e.g., using backpropagation techniques and / or any other suitable techniques). In some embodiments, the nature of the connections may enable or prevent certain nodes of the network. In some embodiments, the ABR ladder generator system may be configured to receive a user specification (or an automatic selection thereof) of hyperparameters (e.g., the number of layers and / or nodes or neurons within each model) (e.g., prior to training). The ABR ladder generator system may automatically set or receive, for example, a manual selection of a learning rate that indicates the urgency with which parameters should be adjusted. In some embodiments, the machine learning model 306 may be trained using unsupervised learning to recognize and learn patterns based on, for example, unlabeled data.

[0043] In some embodiments, in the step of training the machine learning model 306 using supervised learning, the training data may be preferably formatted and / or labeled (e.g., labeled by a human annotator or editor, or otherwise via a computer-implemented process) to indicate that a particular bitrate-resolution ladder 308 corresponding to particular input training parameters 304 was previously determined to be optimal for such a media asset or a segment or portion thereof. As an example, such labels may be categorized metadata attributes stored in conjunction with or attached to the training parameters 304, and the model 306 may be trained using such a training data set for any suitable number of training cycles. In some embodiments, the input parameters 304 may include any suitable number and / or type of parameters, such as quantization parameters (QP), bits per pixel, the number of slices or tiles or other regions used when encoding at least a portion of the media asset, the number of reference frames used when encoding at least a portion of the media asset, motion vectors used when encoding at least a portion of the media asset, or any other suitable encoding parameters or other parameters, or any combination thereof. In some embodiments, the parameters may include an indication of the genre and / or any other suitable characteristics of the media asset or a portion thereof, or such genre or other characteristics may otherwise be input to the machine learning model 306 along with the parameters 304.

[0044] Any suitable network training patch size and batch size may be employed. Any suitable number of training runs may be used to train the machine learning model 306 and to adjust its internal parameters to improve its ability to output an optimal bitrate-resolution pair 308 for a given input parameter 304. The machine learning model 306 can be trained to learn a pattern indicative of the relationship between an input parameter and a bitrate-resolution pair for a certain type of content. For example, the machine learning model can learn over time to adjust a bitrate-resolution ladder differently for different types of content (e.g., an NFL game as opposed to a cooking show), and / or different types of devices or operating systems that may be associated with requirements for accessing media assets (or individual portions thereof). In some embodiments, the training data may at least in part correspond to historical instances of a particular bitrate-resolution ladder used for a particular type of content. In some embodiments, the parameter inputs to the machine learning model 306 for training and the inputs to the trained machine learning model 312 may be encoded as vectors and / or pre-processed (e.g., normalized) to facilitate the input to the machine learning model.

[0045] In some embodiments, the machine learning model 306 may be trained, for example, offline during an initial training phase. In some embodiments, the machine learning model 306 may continue to be trained on-the-fly or adjusted on-the-fly for continuous improvement based on input data and inferences or patterns drawn from the input data and / or based on comparisons after a certain number of cycles. In some embodiments, the machine learning model 306 may be trained to continuously improve for a certain type, format, or genre of content.

[0046] FIG. 3B shows an illustrative block diagram for generating bitrate-resolution pairs 314 for at least a portion of media asset 106 using a trained machine learning model 312 according to some embodiments of the present disclosure. The trained machine learning model 312 may be used to infer an optimal ABR bitrate-resolution ladder 314 for specific input parameters 310 associated with at least a portion of the captured media asset 106. In the example of FIG. 3B, after the step of capturing at least a portion of media asset 106 and / or after the step of receiving a request to access at least a portion of media asset 106 from a client device, the ABR ladder generator system may determine parameters associated with the media asset or a portion thereof. For example, the ABR ladder generator system may extract one or more parameters 310 from a bitstream associated with the encoding data of at least a portion of media asset 106. For example, the ABR ladder generator system may extract scene and motion statistics such as, for example, quantization parameter (QP), bits per pixel, number of slices or tiles or other regions used when encoding at least a portion of the media asset, number of reference frames used when encoding at least a portion of the media asset, motion vectors used when encoding at least a portion of the media asset, or any other suitable encoding parameter or other parameter, or any combination thereof, in real time. In some embodiments, parameters 310 may be extracted from metadata included within a bitstream corresponding to at least a portion of media asset 106. In some embodiments, parameters 310 may be extracted from metadata in response to capturing at least a portion of media asset 106 or in response to encoding it.In some embodiments, parameter 310 may include an indication of the genre and / or any other suitable characteristics of the media asset or a portion thereof, or such genre or other characteristics may alternatively be input into machine learning model 306 along with parameter 310. In some embodiments, the ABR ladder generator system pre-processes the parameters to be input into the trained machine learning model so as to match the format of the parameters input into the trained model 312 to the formatting of the training data or any other suitable processing, or any combination thereof.

[0047] The trained machine learning model 312 may receive, as input, one or more parameters 310 and may be configured to output an optimal bitrate-resolution ladder 314 for at least a portion of media asset 106 based on real-time processing of such input parameters. For example, such optimal bitrate-resolution ladder 314 may be output for one or more segments of at least a portion of media asset 106 that may be live content while such content is being played back in real time on a client device.

[0048] In some embodiments, the trained model 312 may be implemented by, for example, one or more of the servers 104 and / or edge servers 202 and 204 in the examples of FIGS. 1 and 2A-2B, generate and optimize an ABR ladder for each video, and facilitate the transcoding procedure described above. Thus, the optimal bitrate-resolution ladder 314 output by the trained machine learning model 312 may be provided to the server 104 and / or edge server 202 or 204 for use in transcoding at least a portion of the captured media asset 106 in real time, for example, after and / or in response to each request to view the media asset or a portion thereof, and / or during playback of a portion thereof. For example, such a content optimization ladder 314 may be utilized and included as metadata (e.g., in a bitstream associated with encoding data for at least a portion of the media asset 106) for downstream transcoding at the edge server 202 and / or 204. For example, the content optimization ladder 314 may be applicable to both live and non-live streaming and VOD content. For example, if memory limitations are a concern, transcoding may be performed at the edge, which may be useful for applying the foregoing techniques to non-live content.

[0049] When such a bitrate-resolution pair 314 is optimized, the bitrate-resolution pair 314 can be included in metadata, which can be transmitted, and which can be included within the encoding data for at least a portion of the media asset 106 (e.g., within a video elementary stream, within a multiplexed stream, within a manifest, etc.) and / or within a bitstream associated with at least a portion of the media asset 106. Such metadata, while being a very small payload, can provide a large benefit for downstream transcoding from a high bitrate to a lower target, leveraging such prior knowledge. In other words, the content optimization ladder 314 can thus be made readily available from upstream production. In some embodiments, such metadata can be in a compact form of optimized encoding parameters when it is desired to replicate a lower bitrate bitstream from the step of transcoding a higher bitrate version. In some embodiments, such metadata may be transmitted only once per chunk or segment or other portion of the media asset 106, e.g., the ABR ladder 314 can be extracted from the first I-frame or IDR frame of the target segment that is to undergo transcoding.

[0050] In some embodiments, the foregoing techniques can help alleviate the burden of storing the entire ABR ladder on edge or regional servers, for example, in non-live streaming and / or live streaming use cases. This can particularly apply to content that is not very popular or is likely not to be very popular when viewed, based on the collected user statistics. For example, for such content, it may be feasible to only maintain the high bitrate version and enable responsive transcoding to deliver what the user requests. An optimized table of bitrate-resolution pairs 314 can be utilized in such transcoding, for example, in edge servers 202, 204. The payload of such data can serve as an effective means for ensuring the highest possible picture resolution selected at a given bitrate while consuming minimal computational and / or networking resources.

[0051] The systems and methods described herein can be applied to any suitable type of content. For example, the systems and methods described herein can be used to perform compression on higher data rate volumetric video, attributes or textures in video-based point cloud compression, or any other suitable content, or any combination thereof, in applications with variable network conditions and diverse user requirements.

[0052] FIG. 4 shows a content-dependent resolution-bitrate ladder 400 according to some embodiments of the present disclosure. FIGS. 3A-3B show that the targets of optimization can be intended for bitrate-resolution pairs 308 and 314, but it should be understood that the targets of optimization can also be intended for the resolution-bitrate pair 400. For example, the machine learning model 306 may also be configured and trained to derive the optimal bitrate for each possible resolution. For example, the training of the inference process and the target of the output may thus be a table of resolution-bitrate pairs, as shown in FIG. 4. In some embodiments, once optimized, such a table can be used in combination by the ABR ladder generator system to derive the optimal options for transcoding and the subsequent delivery of the optimal quality for the target user. In some embodiments, assuming that at least a portion of the media asset can receive the user specifications of the expected resolution or the network conditions can be automatically detected, appropriate segments may be delivered to the client device. At least a portion of the transcoded rendition of the media asset 106 or a portion thereof may be stored and / or transmitted via a bitstream using the central server 104 and / or the edge servers 202 and 204 for display on the client device 206. Such a receiving client device 206 may decode the rendition of the media asset 106 or a portion thereof and generate at least a portion of the media asset 106 for display.

[0053] Figures 5-6 illustrate an exemplary device, system, server, and related hardware for transcoding at least a portion of a media asset for an ABR streaming process, according to some embodiments of the present disclosure. In some embodiments, the ABR ladder generator system may comprise each of the network-connected computing servers, devices, and / or databases shown in Figures 5-6, or any combination thereof, and / or any other suitable computing server or device or database may be included as part of the ABR ladder generator system. Figure 5 shows, for example, a generalized embodiment of exemplary user equipment devices 500 and 501 that may correspond to the client device 206 of Figure 2. For example, the user equipment device 500 may be a smartphone device, a tablet, or any other suitable device that communicates with servers 102, 104, 202, 204, 302 to request and obtain one or more portions of a media asset via a network, or interfaces otherwise with the ABR ladder generator system described herein. In another example, the user equipment device 501 may be a user television equipment system or device. The user equipment device 501 may include a set-top box 515. The set-top box 515 may be communicatively connected to a microphone 516, an audio output device (e.g., speakers or headphones 514), and a display 512. In some embodiments, the microphone 516 may receive audio corresponding to a user's voice, e.g., a voice input or a voice command. In some embodiments, the display 512 may be a television display or a computer display. In some embodiments, the set-top box 515 may be communicatively connected to a user input interface 510. In some embodiments, the user input interface 510 may be a remote control device. The set-top box 515 may include one or more circuit boards.In some embodiments, the circuit board may include a control circuit network, a processing circuit network, and a memory device (e.g., RAM, ROM, hard disk, removable disk, etc.). In some embodiments, the circuit board may include an input / output path. A more specific implementation of the device is discussed below in connection with FIG. 6. In some embodiments, the device 500 may include any suitable number of sensors (e.g., gyroscope or gyro-meter, or accelerometer, etc.) for determining the location of the user equipment device 500, and a GPS module (e.g., communicating with one or more servers and / or cell relay towers and / or satellites), or any other suitable location identification technique. In some embodiments, the user equipment device 500 includes a rechargeable battery configured to provide power to the components of the device.

[0054] One of each of user equipment devices 500 and 501 may receive content and data via an input / output (I / O) path 502. The I / O path 502 may provide content (e.g., broadcast programming, on-demand programming, Internet content, content available via a local area network (LAN) or wide area network (WAN), and / or other content) and data to a control circuit network 504 that may include a processing circuit network 506 and a storage device 508. The control circuit network 504 may be used to transmit and receive commands, requests, and other suitable data using the I / O path 502, which may include an I / O circuit network. The I / O path 502 may connect the control circuit network 504 (specifically, the processing circuit network 506) to one or more communication paths (described below). The I / O functionality may be provided by one or more of these communication paths, but is shown as a single path in FIG. 5 to avoid overcomplicating the drawing. A set-top box 515 is shown in FIG. 5 for illustrative purposes, but any suitable computing device having a processing circuit network, a control circuit network, and a storage device may be used in accordance with the present disclosure. For example, the set-top box 515 may be replaced or supplemented by a personal computer (e.g., notebook, laptop, desktop), smartphone (e.g., user equipment device 500), tablet, network-based server hosting a user-accessible client device, non-user-owned device, any other suitable device, or any combination thereof.

[0055] The control circuit network 504 may be based on any suitable control circuit network such as the processing circuit network 506. As referred to herein, a control circuit network is to be understood to mean a circuit network based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or a supercomputer. In some embodiments, the control circuit network may be distributed across a plurality of distinct processors or processing units, e.g., a plurality of the same type of processing units (e.g., two Intel Core i7 processors) or a plurality of different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, the control circuit network 504 executes instructions regarding the ABR ladder generator system stored in a memory (e.g., the storage device 508). Specifically, the control circuit network 504 may be instructed by the ABR ladder generator system to perform the functions discussed above and below. In some implementations, the processing or actions performed by the control circuit network 504 may be based on instructions received from the ABR ladder generator system.

[0056] In a client / server-based embodiment, control circuitry 504 may include a communication circuitry suitable for communicating with a server or other network or server. The ABR ladder generator system may be a stand-alone application implemented on a device or server. The ABR ladder generator system may be implemented as a set of software or executable instructions. Instructions for implementing any of the embodiments discussed herein of the ABR ladder generator system may be encoded on a non-transitory computer-readable medium (e.g., a hard drive, random access memory on a DRAM integrated circuit, read-only memory on a BLU-RAY™ disk, etc.). For example, in FIG. 5, the instructions may be stored within storage device 508 and executed by control circuitry 504 of user equipment device 500.

[0057] In some embodiments, the ABR ladder generator system may be a client / server application where only the client application resides on the user device 500, and the server application resides on an external server (e.g., server 604 of FIG. 6, and / or media content source 602 of FIG. 6, and / or edge server 616 of FIG. 6, which may correspond to server 102, server 104, and edge servers 202, 204 of FIGS. 1-3, respectively). In some embodiments, the database 605 may be a network database that stores an encoded video dataset for training the machine learning model 306, and / or a media asset (or a portion thereof), and associated data (e.g., one or more parameters 304 of FIG. 3), and / or an optimized bitrate resolution ladder output by the trained machine learning model 312. For example, the ABR ladder generator system may be partially implemented as a client application on the control circuitry 504 of the user device 500 and partially implemented as a server application that runs on a control circuitry 611 on an external server (e.g., server 604 of FIG. 6 and / or media content source 602 of FIG. 6 and / or edge server 616 of FIG. 6). Such an external server may be part of a local area network with one or more of the user devices 500, 501, or may be part of a cloud computing environment accessed via the Internet. In a cloud computing environment, various types of computing services that perform searches on the Internet or an information database, provide storage (e.g., for a database), or analyze data are provided by a collection of network-accessible computing and storage resources (e.g., server 604 of FIG. 6 and / or media content source 602 of FIG. 6 and / or edge server 616 of FIG. 6) referred to as a "cloud".The user equipment device 500 may be a cloud client that determines whether processing should be offloaded and relies on cloud computing capabilities from the cloud to facilitate such offloading. When executed by the control circuitry 504 or 611, the ABR ladder generator system may command the control circuitry 504 or 611 to select specific devices and / or networks and perform processing tasks to obtain those specific media assets or segments. The client application may command the control circuitry 504 to select specific devices and / or networks and perform processing tasks to obtain those specific media assets or segments.

[0058] The control circuitry 504 may include a communication circuitry suitable for communicating with a server, a social network service, a table or database server, or other network or server. Instructions for performing the functionality described above may be stored on a server (described in more detail in connection with FIG. 6). The communication circuitry may include a cable modem, an integrated services digital network (ISDN) modem, a digital subscriber line (DSL) modem, a telephone modem, an Ethernet card, or a wireless modem for communication with other devices, or any other suitable communication circuitry. Such communication may involve the Internet or any other suitable communication network or path (described in more detail in connection with FIG. 6). Additionally, the communication circuitry may include circuitry that enables peer-to-peer communication of devices or communication of devices at remotely located from each other (described in more detail below).

[0059] The memory may be an electronic storage device provided as a storage device 508 that is part of the control circuitry 504. As referred to herein, the phrases “electronic storage device” or “storage device” refer to random access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY (registered trademark) disc (BD) recorders, BLU-RAY (registered trademark) 3D disc recorders, digital video recorders (sometimes called personal video recorders, i.e., DVRs, PVRs), solid state devices, quantum memory devices, game consoles, game media, or any other suitable fixed or removable storage device, and / or any combination thereof, etc., any device for storing electronic data, computer software, or firmware. The storage device 508 may be used to store various types of content described herein and the ABR ladder generator system data described above and below. Non-volatile memory may also be used (e.g., to boot up startup routines and other instructions). A cloud-based storage device may be used to complement or instead of the storage device 508.

[0060] The control circuit network 504 may include a video generation circuit network such as one or more analog tuners, one or more MPEG-2 decoders or other digital decoding circuit networks, a high-definition tuner, or any other suitable tuning or video circuit, or a combination of such circuits, and a tuning circuit network. An encoding circuit network (e.g., for converting terrestrial, analog, or digital signals to MPEG signals for storage) may also be provided. The control circuit network 504 may also include a scaler circuit network for up-converting and down-converting content to a preferred output format of the user equipment device 500. The control circuit network 504 may also include a digital / analog converter circuit network and an analog / digital converter circuit network for converting between digital and analog signals. The tuning and encoding circuit networks may be used by the user equipment devices 500, 501 for receiving, displaying, playing, or recording content. The tuning and encoding circuit networks may also be used for receiving media consumption data. For example, the circuit networks described herein, including tuning, video generation, encoding, decoding, encryption, decryption, scaler, and analog / digital circuit networks, may be implemented using software that runs on one or more general-purpose or specialized processors. A plurality of tuners may be provided to handle simultaneous tuning functions (e.g., viewing and recording functions, picture-in-picture (PIP) functions, multi-tuner recording, etc.). If the storage device 508 is provided as a device separate from the user equipment device 500, the tuning and encoding circuit networks (including multiple tuners) may be associated with the storage device 508.

[0061] The control circuit network 504 may receive commands from a user using the user input interface 510. The user input interface 510 may be any suitable user interface such as a remote control device, a mouse, a trackball, a keypad, a keyboard, a touch screen, a touch pad, a stylus input, a joystick, a voice recognition interface, or other user input interfaces. The display 512 may be provided as a stand-alone device or integrated with one of the other elements of each of the user equipment devices 500 and 501. For example, the display 512 may be a touch screen or a touch-sensitive display. In such a situation, the user input interface 510 may be integrated with or combined with the display 512. In some embodiments, the user input interface 510 includes a remote control device having one or more microphones, buttons, a keypad, any other component configured to receive user input, or combinations thereof. For example, the user input interface 510 may include a handheld remote control device having an alphanumeric keypad and selection buttons. In a further example, the user input interface 510 may include a handheld remote control device having a microphone and a control circuit network configured to receive and identify voice commands and transmit information to the set-top box 515.

[0062] The audio output device 514 may be integrated with or combined with the display 512. The display 512 may be a monitor, a television, a liquid crystal display (LCD) for a mobile device, an amorphous silicon display, a low temperature polysilicon display, an electronic ink display, an electrophoretic display, an active matrix display, an electro-wetting display, an electro-fluid display, a cathode ray tube display, a light emitting diode display, an electroluminescent display, a plasma display panel, a high performance addressable display, a thin film transistor display, an organic light emitting diode display, a surface conduction electron emitter display (SED), a laser television, a carbon nanotube, a quantum dot display, an interferometric modulator display, or any other suitable device or more than one of them for displaying visual images. A video card or a graphics card may generate the output to the display 512. The audio output device 514 may be provided as integrated with one of the other elements of each of the user device 500 and the user device 501, or may be a stand-alone unit. The audio components of the video and other content displayed on the display 512 may be reproduced through the speakers (or headphones) of the audio output device 514. In some embodiments, the audio may be distributed to a receiver (not shown), which processes the audio and outputs it via the speakers of the audio output device 514. In some embodiments, for example, the control circuitry 504 is configured to use the speakers of the audio output device 514 to provide an audio queue to the user or other audio feedback to the user. A separate microphone 516 may be present, or the audio output device 514 may include a microphone configured to receive audio inputs such as voice commands or utterances. For example, the user may utter characters, terms, phrases, alphanumeric characters, words, etc. that are received by the microphone and converted to text by the control circuitry 504.In a further embodiment, the user may use voice commands received by the microphone and recognized by the control circuitry 504. The camera 518 may be any suitable camera that is integrated with or externally connected to the device and is capable of capturing still and moving images. In some embodiments, the camera 518 may be a digital camera comprising a charge-coupled device (CCD) and / or a complementary metal-oxide semiconductor (CMOS) image sensor. In some embodiments, the camera 518 may be an analog camera that converts to digital images via a video card.

[0063] The ABR ladder generator system may be implemented using any suitable architecture. For example, this may be a stand-alone application that is implemented entirely on one of each of devices 500 and 501. In such an approach, the instructions of the application may be stored locally (e.g., within storage device 508), and data for use by the application may be downloaded periodically (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach). Control circuitry 504 may read the instructions of the application from storage device 508, process the instructions, and provide the functionality of the ABR ladder generator system discussed herein. Based on the processed instructions, control circuitry 504 may determine the actions to be performed when an input is received from user input interface 510. For example, the up / down movement of a cursor on a display may be indicated by the processed instructions when user input interface 510 indicates that an up / down button has been selected. An application and / or any instructions for implementing any of the embodiments discussed herein may be encoded on a computer-readable medium. The computer-readable medium includes any medium capable of storing data. The computer-readable medium may be non-transitory and include, but is not limited to, volatile and non-volatile computer memories or storage devices such as hard disks, floppy disks, USB drives, DVDs, CDs, media cards, register memories, processor caches, random access memory (RAM), etc.

[0064] The control circuit network 504 may enable a user to provide user profile information or may automatically compile user profile information. For example, the control circuit network 504 may access and monitor network data, video data, audio data, processing data, participation data from an ABR ladder generator system. The control circuit network 504 may obtain all or part of other user profiles related to a particular user (e.g., via a social media network) and / or obtain information about the user from other sources that the control circuit network 504 may access. As a result, the user may be provided with a unified experience across different devices of the user.

[0065] In some embodiments, the ABR ladder generator system is a client / server-based application. Data for use by a thick or thin client implemented on one of each of user device devices 500 and 501 can be read on demand by issuing a request to a server remote to one of each of user device devices 500 and 501. For example, the remote server may store instructions regarding the application in a storage device. The remote server may use a circuit network (e.g., control circuit network 504) to process the stored instructions and generate the displays discussed above and below. The client device may receive the display generated by the remote server and may display the display content locally on user device device 500. Thus, the processing of the instructions may be performed remotely by the server, while the resulting display (which may include, for example, text, a keyboard, or other video) is provided locally on user device device 500. User device device 500 may receive input from the user via input interface 510 and transmit those inputs to the remote server to process and generate corresponding displays. For example, user device device 500 may transmit a communication indicating that the up / down button has been selected via input interface 510 to the remote server. The remote server may process the instructions according to the input and generate a display of the application corresponding to the input (e.g., a display that moves the cursor up / down). The generated display may then be transmitted to user device device 500 for presentation to the user.

[0066] In some embodiments, the ABR ladder generator system may be downloaded and interpreted or otherwise launched by an interpreter or virtual machine (launched by control circuitry 504). In some embodiments, the ABR ladder generator system may be encoded in an ETV binary interchange format (EBIF), received by control circuitry 504 as part of a suitable feed, and interpreted by a user agent that launches on control circuitry 504. For example, the ABR ladder generator system may be an EBIF application. In some embodiments, the ABR ladder generator system may be defined by a series of JAVA (registered trademark)-based files that are received and launched by a local virtual machine or other suitable middleware executed by control circuitry 504. In some of such embodiments (e.g., those employing an MPEG-2 or other digital media encoding scheme), the ABR ladder generator system may be encoded and transmitted in an MPEG-2 object carousel, for example, along with program MPEG audio and video packets.

[0067] FIG. 6 is a schematic diagram of an exemplary system 600 according to some embodiments of the present disclosure. System 600 may include one or more networks, such as communication network 609, coupled user device devices 607, 608, 610 (and / or any other suitable number of user device devices), media content source 602, server 604, database 605, edge server 616, and / or any other suitable computing device, or one or more of any combination thereof. In some embodiments, at least some of such devices may correspond to user device device 500 or user device device 501 of FIG. 5, or may include any suitable portion of components that are the same or similar to those described in connection with FIG. 5. Communication network 606 may be one or more networks including the Internet, a mobile phone network, a mobile voice or data network (e.g., a 5G, 4G, or LTE network, or any other suitable network, or any combination thereof), a cable network, a public switched telephone network, or other type of communication network, or a combination of communication networks. Paths (e.g., depicted as arrows connecting individual devices to communication network 606) may include one or more communication paths, separately or together, satellite paths, optical fiber paths, cable paths, paths supporting Internet communication (e.g., IPTV), free space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communication path, or a combination of such paths. Communication with client devices may be provided by one or more of these communication paths, but for the sake of avoiding unduly complicating the drawings, in FIG. 6, it is shown as a single path.

[0068] Communication paths are not drawn between devices, but these devices may communicate directly with each other via communication paths and other short-distance two-point communication paths such as USB cables, IEEE 1394 cables, wireless paths (e.g., Bluetooth (registered trademark), infrared, IEEE 802.11x, etc.), or other short-distance communications via wired or wireless paths. The devices may also communicate directly with each other through an indirect path via communication network 606.

[0069] System 600 may include one or more media content sources 602 and one or more servers 604. In some embodiments, content source 102 may correspond to one or more of media content source 602 or server 604. Communication with media content source 602 and server 604 may be exchanged via one or more communication paths, but for the sake of avoiding overly complicating the drawings, in FIG. 6, it is shown as a single path. Additionally, more than one of each of media content source 602 and server 604 may exist, but for the sake of avoiding overly complicating the drawings, in FIG. 6, only one of each is shown. Optionally, media content source 602 and server 604 may be integrated as one source device. In some embodiments, the ABR ladder generator system may be executed in one or more of control circuitry 611 of server 604 (and / or control circuitry of user equipment devices 607, 608, 610, or control circuitry 618 of edge server 616, or any other suitable device, or any combination thereof). In some embodiments, any suitable data structure or any combination thereof may be maintained in server 604 or otherwise associated therewith, in database 605, and / or in a storage device of one or more of user equipment devices 607, 608, 610 and / or edge server 616, at least one of which may be configured to host or communicate with database 605. User equipment device 610 may be a smart TV, user equipment device 607 may be a user computer device, user equipment device 608 may be a wireless user communication device, and each of them may be configured to include some or all of the features of the ABR ladder generator system described herein. In some embodiments, the ABR ladder generator system may be adjusted according to the capabilities of a particular device.In some embodiments, the ABR ladder generator system may facilitate the reading and presentation of a media asset (and / or a portion thereof), and / or may present a media asset (and / or a portion thereof) to a user, and may operate in conjunction with a media application (e.g., associated with content source 102).

[0070] In some embodiments, server 604 may include a control circuitry 611 and a storage device 614 (e.g., RAM, ROM, hard disk, removable disk, etc.). The storage device 614 may store one or more databases. Server 604 may also include an input / output path 612. The I / O path 612 may provide media consumption data, social networking data, device information, or other data to the control circuitry 611 and the storage device 614, which may include a processing circuitry, via a local area network (LAN) or a wide area network (WAN), and / or may provide other content and data thereto. The control circuitry 611 may be used to transmit and receive commands, requests, and other suitable data using the I / O path 612, which may comprise an I / O circuitry. The I / O path 612 may connect the control circuitry 611 (specifically, the control circuitry) to one or more communication paths. The I / O path 612 may include an I / O circuitry.

[0071] The control circuit network 611 may be based on any suitable control circuit network such as one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or a supercomputer. In some embodiments, the control circuit network 611 may be distributed across a plurality of distinct processors or processing units, e.g., a plurality of the same type of processing units (e.g., two Intel Core i7 processors) or a plurality of different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor). In some embodiments, the control circuit network 611 executes instructions regarding the ABR ladder generator system stored in a memory (e.g., storage device 614). The memory may be an electronic storage device provided as the storage device 614 which is part of the control circuit network 611.

[0072] The user equipment device 610 may be a smart TV, the user equipment device 607 may be a user computer device, the user equipment device 608 may be a wireless user communication device, and each of them may be configured to include some or all of the features of the ABR ladder generator system described herein. The ABR ladder generator system may be adjusted according to the capabilities of a particular device.

[0073] Edge server 616 may include a control circuit network 618, an I / O path 620, and a storage device 622, which may be implemented in a manner similar to the control circuit network 611, the I / O path 612, and the storage device 614 of server 604, respectively. Edge server 616 may correspond to edge server 202 or 204 of FIG. 2. Edge server 616 may be configured to communicate via communication network 609 with one or more of user equipment devices 607, 608, 610, server 604, and media content source 602, and may be configured to perform processing tasks in connection with the ABR ladder generator system. In some embodiments, multiple edge servers may be strategically located in various geographical locations to optimize content delivery. In some embodiments, one or more of the edge servers may be mobile edge servers configured to provide processing support for mobile devices in various geographical regions. Each edge server may be positioned at the edge of a CDN, may cache certain content according to a certain caching policy, and may facilitate the rapid provision of the content requested by client device 206.

[0074] The media content source 602, the server 604, or the edge server 616, or any combination thereof, may include an encoder and / or a transcoder. Such an encoder may be any suitable combination of hardware and / or software configured to process data so as to minimize the impact of the encoding on the quality of the media asset (or one or more portions thereof) being encoded while reducing the storage space required to store the data and / or the bandwidth required to transmit the image data. The transcoder may be any suitable combination of hardware and / or software configured to operate on the digitally compressed and encoded data of the media asset (or one or more portions thereof) to convert such data from a first format (or specification) to a second format (or specification). In some embodiments, the transcoder and the encoder may be combined. For example, such a combination may access at least a portion of the original uncompressed media asset (not the digitally compressed version of the media asset or a portion thereof), encode it, and transcode such an encoded media asset or a portion thereof into various other formats.

[0075] FIG. 7 is a flowchart of an illustrative process 700 detailed for transcoding at least a portion of a media asset for an ABR streaming process, according to some embodiments of the present disclosure. In various embodiments, individual steps of process 700 may be implemented by one or more components of the devices and systems of FIGS. 1-6. Although the present disclosure may describe some steps of process 700 (and other processes described herein) as being implemented by some components of the devices and systems of FIGS. 1-6, this is for illustrative purposes only, and it should be understood that other components of the devices and systems of FIGS. 1-6 may implement those steps instead.

[0076] In 702, the ABR ladder generator system may be configured to capture at least a portion of the media asset from a content source. For example, server 104 may capture at least a portion of the media asset (e.g., at least a portion of media asset 106 in FIG. 1) from a content source (e.g., content source 102 in FIG. 1). In some embodiments, such at least a portion of the media asset may correspond to live content. In some embodiments, at least a portion of the media asset may be non-live media assets, such as VOD content. In some embodiments, at least a portion of the media asset may be encoded at content source 102 and transmitted therefrom to server 104. For example, the ABR ladder generator system may access at least a portion of the media asset from any suitable source (e.g., media content source 602 in FIG. 6) via a network (e.g., communication network 609 in FIG. 6 or any other suitable network) or any suitable wireless or wired communication path. In some embodiments, the ABR ladder generator system may generate at least a portion of the media asset and / or read at least a portion of the media asset from a memory (e.g., memory or storage device 614 or database 605, or any other suitable data storage device, or any combination thereof) and / or receive at least a portion of the media asset via any suitable data interface. Alternatively, the ABR ladder generator system may capture at least a portion of the media asset in an uncompressed format. In some embodiments, steps 704-712 or any suitable number of steps thereof may be performed in response to capturing at least a portion of the media asset and / or encoding at least a portion of the media asset and / or receiving a request regarding at least a portion of the media asset.

[0077] In 704, the ABR ladder generator system may determine at least some parameters of the media asset. For example, if at least a portion of the media asset is captured in a pre-encoded format, the ABR ladder generator system may determine that such at least a portion of the pre-encoded media asset includes bitstream level statistics (e.g., included in metadata), or is otherwise transmitted or associated therewith. The ABR ladder generator system may be configured to extract parameters from such bitstream level statistics, which may be, for example, scene and motion statistics of the media asset or a portion thereof. Alternatively, if the ABR ladder generator system captures at least a portion of the media asset in an uncompressed format, the ABR ladder generator system may be configured to encode the captured media asset or a portion thereof into a specific format. In such cases, based on the encoding of the captured media asset or a portion thereof and / or during the encoding of the media asset or a portion thereof, the ABR ladder generator system may be configured to collect bitstream level statistics and / or parameters. For example, the ABR ladder generator system may derive spatial and temporal complexity or parameters based on performing an encoding process.

[0078] In some embodiments, any suitable parameters may be obtained from the encoding data (as captured by or generated by the ABR ladder generator system). For example, such parameters may include any suitable number and / or type of parameters, such as quantization parameter (QP), bits per pixel, number of slices or tiles or other regions used when encoding at least a portion of the media asset, number of reference frames used when encoding at least a portion of the media asset, motion vectors used when encoding at least a portion of the media asset, or any other suitable encoding parameter or other parameter, or any combination thereof. In some embodiments, the parameters may include an indication of the genre of the media asset or a portion thereof and / or any other suitable characteristic. In some embodiments, the ABR ladder generator system may determine the complexity of the media asset and / or a portion thereof based on the extracted parameters and / or using any suitable technique, such as a parametric model.

[0079] At 706, the ABR ladder generator system may determine a plurality of optimal bitrate-resolution pairs for at least a portion of a media asset or a portion or segment thereof based on the parameters determined at 704. In some embodiments, the ABR ladder generator system may perform the determination at 706 using a machine learning model (e.g., trained machine learning model 312) as described in more detail in FIG. 3B. In some embodiments, the plurality of optimal bitrate-resolution pairs determined (e.g., shown at 314 in FIG. 3) are content-dependent and may be optimized with respect to a particular type of media asset or segment thereof (e.g., genre and / or color and / or amount of motion and / or any other suitable characteristic). Additionally, or alternatively, the ABR ladder generator system may determine a plurality of optimal resolution-bitrate pairs for at least a portion of a media asset or a segment thereof based on the parameters determined at 704.

[0080] In 708, the ABR ladder generator system may determine whether transcoding of at least a portion of a media asset (e.g., at least a portion of media asset 106 in FIG. 1) should be performed at a central server (e.g., central server 104 in FIG. 1 or server 604 in FIG. 6) or at one or more edge servers (e.g., edge servers 202, 204 in FIG. 2, which may correspond to edge server 616 in FIG. 6) based on the bitrate-resolution pair determined at 708. In some embodiments, the determination at 708 of whether to perform transcoding at server 104 (FIG. 2A) or distribute the transcoding operation to edge servers 202 and / or 204 (FIG. 2B) may be based on any suitable factor or combination thereof. For example, such a determination may consider the computing power of central server 104 and / or edge servers 202 or 204, the current processing load of central server 104 and / or edge servers 202 or 204, the current bandwidth or network conditions or demand or capacity associated with central server 104 and / or edge servers 202 or 204, the demand for a particular media asset (or a portion thereof) at a location, or any other suitable factor, or any combination thereof.

[0081] For example, compared to the embodiment of FIG. 2A, in the arrangement of FIG. 2B, edge servers 202 and / or 204 may utilize more computing power and transcoding or encoding farms to perform transcoding. On the other hand, in the arrangement of FIG. 2A, more bandwidth may be consumed to transmit multiple copies of at least a portion of the transcoded media asset to edge servers 202, 204. Thus, the ABR ladder generator system may perform a determination at 708 to most efficiently allocate the computing and network resources available for content transcoding and delivery. In some embodiments, transcoding may be split among server 104 and edge servers 202 and 204 respectively, or transcoding may be performed at server 104 or at one of edge servers 202 and 204.

[0082] In some embodiments, the ABR ladder generator system may determine to transcode live media content in response to user requests regarding the content or while anticipating one or more user requests regarding the content. For example, the ABR ladder generator system may be configured to receive user input or user requests via, for example, the user input interfaces and / or I / O circuitry of user equipment devices 607, 608, or 610 of FIG. 6, and in response, access, and / or perform processing on, information regarding the transcoded media asset (or a portion thereof) and / or an indication of an optimal bitrate-resolution pair, and output or transmit the same.

[0083] At 710, the ABR ladder generator system may cause at least a portion of the media asset to be transcoded at a central server based on a plurality of optimal bitrate-resolution pairs. For example, server 604 may use the bitrate-resolution pairs determined at 706 to induce transcoding operations, such as transcoding the media asset and / or segments thereof to various quality and resolution levels suitable for various types of content and / or various devices that may request such content. The transcoding operations may be used to obtain any suitable number of different formats for various segments of the media asset. In some embodiments, the central server may distribute the transcoded portions of at least a portion of the media asset to edge servers (e.g., edge servers 202 and / or 204 of FIG. 2) for distribution to client devices (e.g., client device 206 of FIG. 2). In some embodiments, the central server (e.g., server 604) may store the bitrate-resolution pairs determined at 706 (e.g., within a memory or storage device 614 of FIG. 6).

[0084] In 712, the ABR ladder generator system may cause at least a portion of the media asset to be transcoded at the edge server. For example, the ABR ladder generator system may cause the central server (e.g., server 604 of FIG. 6) to transmit metadata including an indication of the optimal bitrate-resolution pair determined at 706 to the edge server (e.g., edge server 616 of FIG. 6), and the edge server may be used to induce transcoding of the media asset (or a portion thereof) into various bitrate-resolution formats and / or any other suitable number or type of formats. In some embodiments, the central server may transmit data representing a single high bitrate rendition of the media asset (or a portion thereof) to the edge server in addition to, or otherwise associated with, the metadata. For example, the edge server (or any other suitable computing device) may receive and detect the carriage and use of metadata within a bitstream (e.g., associated with at least a portion of media asset 106). Such metadata may indicate the availability of an optimized bitrate-resolution pair (e.g., as shown at 314 of FIG. 3) to the edge server (or any other suitable computing device) and induce transcoding. Such metadata within the bitstream may form a very small payload that can be used to provide an effective means for improving transcoding at the edge (or other suitable computing device) while optimizing transcoding of at least a portion (and / or one or more portions thereof) of the media asset. In some embodiments, the edge server (e.g., edge server 616) may store the bitrate-resolution pair determined at 706 (e.g., within the memory or storage device 622 of FIG. 6).

[0085] In 714, the ABR ladder generator system may provide the transcoded portions of the media asset to the client device. For example, a client device (e.g., client device 206 of FIG. 2) may utilize the manifest file and request segments of the media asset as appropriate, based on the client device type, device capabilities, network conditions, and attributes of the current segment of the media asset. In some embodiments, multiple bitrate-resolution pairs may be indicated within the manifest file. Based on receiving the request, the ABR ladder generator system may decode the received media asset or a portion thereof, generate it for display, and transmit the requested media asset or a portion thereof to the requesting client device, which may be configured to request additional segments based on the optimized bitrate-resolution pair.

[0086] FIG. 8 is a flowchart of an illustrative process 800 described in detail for training a machine learning model and using the machine learning model to facilitate transcoding of at least a portion of a media asset for an ABR streaming process, according to some embodiments of the present disclosure. In various embodiments, the individual steps of process 800 may be implemented by one or more components of the devices and systems of FIGS. 1-6. Although the present disclosure may describe certain steps of process 800 (and other processes described herein) as being implemented by certain components of the devices and systems of FIGS. 1-6, this is for illustrative purposes only, and it should be understood that other components of the devices and systems of FIGS. 1-6 may implement those steps instead.

[0087] At 802, the ABR ladder generator system may access training data to train a machine learning model. For example, training data from the encoded video dataset database 302 of FIG. 3A may be used to train the machine learning model 306. In some embodiments, the encoded video dataset database 302 may correspond to a media content source 102 and / or a server 104 and / or a database associated with the server 104. In some embodiments, the training data from the encoded video dataset database 302 may be associated with any suitable number of media assets (or portions thereof) in any suitable format, having various characteristics and associated with various genres. In some embodiments, the encoded video dataset database 302 may include comprehensive combinations of various genres, bitrates, and resolutions of training content. In some embodiments, the encoded video dataset database 302 may include data associated with previously streamed live content and / or previously streamed non-live content.

[0088] At 804, the ABR ladder generator system may use the training data accessed at 802 to train a machine learning model. For example, the ABR ladder generator system may use training data (stored in the encoded video dataset database 302 of FIG. 3A, for example) that includes a plurality of parameters (and / or parameters for a media asset as a whole) for at least individual portions of a plurality of media assets (shown at 304 in FIG. 3A, for example) and corresponding bitrate-resolution pairs (shown in the bitrate-resolution ladder 308 of FIG. 3A, for example) to train a machine learning model (the machine learning model 306 of FIG. 3A, for example).

[0089] In some embodiments, the step of training the machine learning model 306 may be performed using supervised learning, and the training data is preferably formatted and / or labeled (e.g., labeled by a human annotator or editor, or otherwise via a computer-implemented process) to indicate that a particular bitrate-resolution ladder 308 corresponding to a particular input training parameter 304 was previously determined to be optimal for such a media asset or segment thereof. In some embodiments, the machine learning model 306 may be trained using unsupervised learning, for example, to recognize and learn patterns based on unlabeled data.

[0090] In some embodiments, the input parameter 304 may include any suitable number and / or type of parameters, such as quantization parameter (QP), bits per pixel, number of slices or tiles or other regions used when encoding at least a portion of the media asset, number of reference frames used when encoding at least a portion of the media asset, motion vectors used when encoding at least a portion of the media asset, or any other suitable encoding parameter or other parameter, or any combination thereof. In some embodiments, the parameters may include an indication of the genre and / or any other suitable characteristic of the media asset or a portion thereof, or such genre or other characteristic may otherwise be input to the machine learning model 306 along with the parameter 304.

[0091] In 806, the ABR ladder generator system uses a machine learning model (e.g., trained machine learning model 312) trained for predicting the optimal resolution per bitrate for each media asset (or a portion thereof) associated with the input to model 312, and may facilitate real-time adaptive bitrate transcoding of such media assets, e.g., one or more than one portion of media asset 106. For example, the trained machine learning model accepts, as input, at least some parameters of the captured live media asset (e.g., one or more than one of the parameters 310 of FIG. 3B), and is configured to output a plurality of optimal bitrate-resolution pairs for at least a portion of the captured live media asset (e.g., media asset 106 of FIG. 3B) based on real-time processing of such input parameters. For example, 706 of FIG. 7 may be implemented in a manner similar to 806 of FIG. 8. Accordingly, the optimal bitrate-resolution ladder 314 output by the trained machine learning model 312 may be provided to server 104 and / or edge server 202 or 204 for use in transcoding at least a portion of media asset 106 captured in real time, e.g., in response to each request to view the media asset and / or during playback of its segments.

[0092] The processes discussed above are illustrative and not intended to be limiting. One of ordinary skill in the art will understand that the steps of the processes discussed herein may be omitted, modified, combined, and / or rearranged, and that any additional steps may be performed without departing from the scope of the invention. More generally, the above disclosure is intended to be illustrative and not limiting. Only the following claims are intended to set the boundaries as to what the invention encompasses. Further, note that any features described within any one embodiment may be applicable to any other embodiment herein, and that a flowchart or example associated with one embodiment may be combined with any other embodiment in a suitable manner, performed in a different order, or performed in parallel. Additionally, the systems and methods described herein may be implemented in real time. Also note that the systems and / or methods described above may be applied to or used in accordance with other systems and / or methods. This specification discloses embodiments including, but not limited to, the following. 1. A computer-implemented method comprising: capturing at least a portion of a live media asset from a media content source; after capturing at least a portion of the live media asset, in real time, determining parameters of at least a portion of the captured live media asset; determining a plurality of optimal bitrate-resolution pairs for at least a portion of the live media asset based on the parameters; transcoding at least a portion of the live media asset based on the plurality of optimal bitrate-resolution pairs; performing; a method comprising. 2. Metadata further comprising generating a bitstream including metadata including a plurality of optimal bitrate-resolution pairs, The step of transcoding at least a portion of the live media asset includes transmitting a bitstream from a central server to one or more edge servers, where the one or more edge servers are configured to transcode at least a portion of the live media asset based on a plurality of optimal bitrate-resolution pairs indicated in the metadata, the method according to item 1. 3. At least a portion of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments, The transmitted bitstream includes a single indication of metadata for each individual segment of the plurality of segments, The method according to item 2. 4. The central server performs an intake of at least a portion of the live media asset from a media content source, The step of transcoding at least a portion of the live media asset based on a plurality of optimal bitrate-resolution pairs includes the central server transcoding at least a portion of the live media asset, and transmitting at least the transcoded portion of the live media asset to one or more edge servers, The method according to item 1. 5. The method further includes training a machine learning model using training data that includes a plurality of parameters for at least individual portions of a plurality of media assets and corresponding bitrate-resolution pairs, where the trained machine learning model is configured to receive, as input, parameters of at least a portion of the captured live media asset and output a plurality of optimal bitrate-resolution pairs for at least a portion of the captured live media asset, the method according to item 1. 6. The method according to item 5, where the parameters of the training data include an indication of the genre for at least individual portions of the plurality of media assets of the training data. 7. The step of determining at least some parameters of the live media asset includes extracting scene and motion statistics from a bitstream corresponding to at least a part of the captured live media asset, the method according to item 1. 8. At least a part of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments, The step of determining at least some parameters of the live media asset includes determining parameters for at least one segment of the plurality of segments, The method according to item 1. 9. The parameters include the genre of at least a part of the live media asset or at least one segment thereof, the method according to item 8. 10. The step of transcoding at least a part of the live media asset based on a plurality of optimal bitrate-resolution pairs is performed in response to receiving a request regarding at least a part of the captured live media asset from a client device, the method according to item 1. 11. At least a part of the live media asset is encoded when being captured, the method according to item 1. 12. At least a part of the live media asset is not encoded when being captured, and the method further includes a step of encoding at least a part of the captured live media asset, wherein the parameters of the live media asset are determined at least in part based on the step of performing the encoding, the method according to item 1. 13. A computer-implemented system, a memory, a control circuitry, capturing at least a part of a live media asset from a media content source, and in real time after the step of capturing at least a part of the live media asset, Determining at least some parameters of the captured live media asset; Based on the parameters, determining a plurality of optimal bitrate-resolution pairs for at least a part of the live media asset; Based on the plurality of optimal bitrate-resolution pairs, transcoding at least a part of the live media asset; Performing; A control circuitry configured to perform; A computer-implemented system comprising. 14. The system further comprises a central server and one or more edge servers, and the control circuitry further Generating a bitstream including metadata including a plurality of optimal bitrate-resolution pairs; Transmitting the bitstream from the central server to one or more edge servers to transcode at least a part of the live media asset, wherein the one or more edge servers are configured to transcode at least a part of the live media asset based on the plurality of optimal bitrate-resolution pairs indicated in the metadata; The system according to item 13, configured to perform. 15. At least a part of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments, The transmitted bitstream includes a single indication of metadata for each individual segment of the plurality of segments. The system according to item 14. 16. The system further comprises a central server, and the central server Performs the capture of at least a part of the live media asset from a media content source, Transcoding at least a portion of a live media asset based on a plurality of optimal bitrate-resolution pairs of assets, Transmitting the transcoded at least a portion of the live media asset to one or more edge servers, The system according to item 13, configured as such. 17. The control circuit network further Is configured to train a machine learning model using training data including a plurality of parameters for at least individual portions of a plurality of media assets and corresponding bitrate-resolution pairs, The trained machine learning model is configured to receive, as input, parameters of at least a portion of the captured live media asset and output a plurality of optimal bitrate-resolution pairs for at least a portion of the captured live media asset. The system according to item 13. 18. The parameters of the training data include an indication of the genre for at least individual portions of the plurality of media assets of the training data. The system according to item 17. 19. The control circuit network is configured to determine parameters of at least a portion of the live media asset by extracting scene and motion statistics from a bitstream corresponding to at least a portion of the live media asset. The system according to item 13. 20. At least a portion of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments, The control circuit network is configured to determine parameters of at least a portion of the live media asset by determining parameters for at least one segment of the plurality of segments. The system according to item 13. 21. The parameters include the genre of at least a portion of the live media asset or at least one segment thereof. The system according to item 20. 22. The control circuit network is configured to transcode at least a portion of the live media asset based on a plurality of optimal bitrate-resolution pairs in response to receiving a request from a client device regarding at least a portion of the live media asset, the system of item 13. 23. At least a portion of the live media asset is encoded when being captured, the system of item 13. 24. At least a portion of the live media asset is not encoded when being captured, and the control circuit network further encodes at least a portion of the captured live media asset, determines parameters of at least a portion of the live media asset based at least in part on the step of performing the encoding, the system of item 13 configured as such.

Claims

1. A computer implementation method, This involves capturing at least a portion of the live media assets from media content sources, After capturing at least a portion of the aforementioned live media assets, in real time, Determining at least some of the parameters of the captured live media asset, Based on the parameters, determine a plurality of optimal bitrate-resolution pairs for at least a portion of the live media asset, This causes at least a portion of the live media asset to be transcoded based on the plurality of optimal bitrate-resolution pairs. To implement and Methods that include...

2. To generate a bitstream containing metadata, wherein the metadata includes the plurality of optimal bitrate-resolution pairs. It further includes, The method according to claim 1, wherein causing the transcoding of the live media asset comprises transmitting the bitstream from a central server to one or more edge servers, the one or more edge servers being configured to transcode the live media asset based on the plurality of optimal bitrate-resolution pairs indicated in the metadata.

3. At least a portion of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments. The transmitted bitstream includes a single indication of the metadata relating to each individual segment of the plurality of segments, The method according to claim 2.

4. The central server performs the capture of at least a portion of the live media assets from the media content source. The fact that at least a portion of the live media assets is transcoded based on the plurality of optimal bitrate-resolution pairs means that the central server transcodes at least a portion of the live media assets, This includes transmitting at least a transcoded portion of the live media asset to one or more edge servers. The method according to claim 1.

5. Training a machine learning model using training data that includes multiple parameters for at least a portion of multiple media assets, and corresponding bitrate-resolution pairs. It further includes, The method according to claim 1, wherein the trained machine learning model is configured to accept the parameters of at least a portion of the captured live media assets as input and to output the plurality of optimal bitrate-resolution pairs for at least a portion of the captured live media assets.

6. The method according to claim 5, wherein the parameters of the training data include genre indications relating to at least one of the individual media assets of the training data.

7. The method according to claim 6, wherein determining the parameters of at least a portion of the live media asset includes extracting scene and motion statistics from a bitstream corresponding to at least a portion of the captured live media asset.

8. At least a portion of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments. The method according to claim 7, wherein determining the parameters of at least some of the live media assets includes determining parameters relating to at least one segment of the plurality of segments.

9. The method according to claim 8, wherein the parameter includes the genre of at least a portion of the live media asset or the at least one segment thereof.

10. The method according to claim 9, wherein causing at least a portion of the live media asset to be transcoded based on the plurality of optimal bitrate-resolution pairs is performed in response to receiving a request from a client device relating to at least a portion of the captured live media asset.

11. The method according to claim 10, wherein at least a portion of the live media asset is encoded when it is captured.

12. At least a portion of the live media assets is not encoded when captured, and the method further Encoding at least a portion of the captured live media asset, wherein the parameters of the live media asset are determined at least in part on performing the encoding. The method according to claim 11, including the method described in claim 11.

13. A computer implementation system, Memory and A control circuit network, This involves capturing at least a portion of the live media assets from media content sources, After capturing at least a portion of the aforementioned live media assets, in real time, Determining at least some of the parameters of the captured live media asset, Based on the parameters, determine a plurality of optimal bitrate-resolution pairs for at least a portion of the live media asset, This causes at least a portion of the live media asset to be transcoded based on the plurality of optimal bitrate-resolution pairs. To implement and A control network configured to perform the following: A system that includes these features.

14. A computer implementation method, This involves capturing at least a portion of the live media assets from media content sources, Determining the parameters of at least a portion of the captured live media assets, at least partially based on analyzing the captured bitstream level statistics using at least a portion of the captured live media assets, The determined parameters are input into the machine learning model, wherein the machine learning model is trained using training data that includes multiple parameters relating to at least a portion of multiple media assets and corresponding bitrate-resolution pairs. Based on the machine learning model, determine a plurality of optimal bitrate-resolution pairs for at least the portion of the live media asset, Determine the processing capacity of at least one edge server, Identifying the at least one edge server as a transcoding site for transcoding at least a portion of the live media assets, based at least in part on the processing capacity of the at least one edge server, This causes at least a portion of the live media asset to be transcoded on at least one edge server based on the plurality of optimal bitrate-resolution pairs. Methods that include...

15. The method according to claim 14, wherein at least a portion of the live media asset includes one or more frames of the live media asset.

16. The determined parameters of at least a portion of the captured live media asset are: The quantization parameters, bits per pixel, number of slices, number of reference frames, or motion vector used when encoding at least a portion of the captured live media assets. The method according to claim 14, comprising at least one of the following.

17. Generating a bitstream including metadata, wherein the metadata includes the plurality of optimal bitrate-resolution pairs. It further includes, The method according to claim 14, wherein causing the transcoding of at least the portion of the live media asset includes transmitting the bitstream from a central server to at least one edge server, the at least one edge server being configured to transcode the at least the portion of the live media asset based on the plurality of optimal bitrate-resolution pairs indicated in the metadata.

18. At least a portion of the live media asset is a segment of the live media asset, and the live media asset includes a plurality of segments. The transmitted bitstream includes a single indication of the metadata relating to each individual segment of the plurality of segments, The method according to claim 17.

19. Causing the transcoding of at least a portion of the live media asset is: Transcoding at least a portion of the live media asset to be compatible with at least one client device, the at least one client device being required to receive the live media asset. The method according to claim 14, further comprising:

20. The method according to claim 19, wherein at least the captured portion of the live media asset is encoded in a first format, and at least the transcoded portion of the live media asset is encoded in a second format, the second format being compatible with at least one client device.

Images