System and method for dynamically managing media production work packages for media production - Patents.com

Abstract

A system for dynamically managing media production work packages is provided, the system includes a media production work package generator that builds a media production work package including media processing functions, and a processing function manager that generates a function tracker that is assigned to each of a plurality of media processing functions and includes a unique identity. The system further includes a function deployment manager that embeds the function tracker in each media processing function and deploys the media processing functions to available resources in a cloud computing network including both physical and software resources. A performance metric collector generates a graphical representation of a respective total cost of each deployed media processing function, and a work package controller generates a data point structure that uniquely identifies the deployed media processing functions and the total cost of each deployed function.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Application No. 63 / 267,464, filed February 2, 2022, entitled “SYSTEM AND METHOD FOR DYNAMICALLY MANAGING A MEDIA PRODUCTION WORK PACKAGE FOR A MEDIA PRODUCTION,” which is assigned to the assignee of the present application and is incorporated by reference in its entirety as if fully set forth herein.

[0002] FIELD OF THE DISCLOSURE Systems and methods disclosed herein relate to media production, and more particularly, to systems and methods for dynamically managing media production work packages for a media production. [Background technology]

[0003] Live media production typically involves capturing media content from one or more live scenes (e.g., sporting venues, news broadcasts, video game platforms, etc.), transmitting the captured content to a remote production facility where a production switcher manages the video and audio signals, adding graphic effects, etc., and encoding the processed signals for transmission over one or more signals to a distribution network, such as a television broadcast network. More recently, broadcast and media production has evolved from the analog domain to the digital domain and in various protocols (e.g., MPEG-2, Internet Protocol (IP), IPTV, Ethernet). With IP technology, there are countless tools, software code bases, etc., with an infinite set of combinations or development efforts, leading through the execution of a media production workflow incorporating these tools to the final "product" (e.g., media production).

[0004] With the continued growth, development, and accessibility of cloud computing platforms and networks such as Amazon Web Services® ("AWS"), many of the processing components involved in a typical media production environment to generate a final product are being moved to the "cloud" and / or widely distributed across several geographic locations. In the cloud, infrastructure as code (IAC) and configuration as code (CAC) provide a highly dynamic infrastructure that is, in effect, software-defined processing power enabling software execution of each desired task at hand. In this case, code modules may be assigned identities, compilers of the code may be assigned identities, IACs and CACs may be assigned identities, and so on, to perform these tasks.

[0005] However, even with the dynamic software-defined processing capabilities that these cloud computing environments offer, the problem remains of how to optimize the resources of the cloud computing environment to efficiently and economically execute processing functions within a media production workflow. More specifically, cloud environments are where these modern factories can be deployed for workflows such as media production work packages. Thus, there is a need to understand the costs of these media production work packages within these modern factories in order to improve efficiency and optimize such resources. Summary of the Invention

[0006] As a general concept, raw materials (e.g., media essence) are fed into a factory to be processed and produce a final product. That is, essence is the input to a media production factory. In the execution of a work package, essence is processed (e.g., mixed, rendered, etc.) to fit the editorial intent and content is generated. Furthermore, to meet modern revenue goals, it is now necessary to generate multiple renditions of a given content output. By targeting multiple psychographic groups, revenue can be increased significantly. The Internet of Everything is valuable because it enables control loops that can be driven to optimize something that includes revenue and usually one or more additional factors. In the case of a media factory, these factors can be latency, which is important for human factors, and then image quality, which may possibly be a multi-factor metric since it depends on bit depth, processing speed, etc. Finally, in a third aspect, the factory may be represented as a graph where the nodes typically indicate the processing points of the work package and the edges are the distribution of intermediate processed essences.

[0007] These modern media productions have multiple final outputs and multiple inputs, so the media production factory is MIMO (multiple input, multiple output). Conceptually, the mathematical factory has m essence inputs E1(t)-Em(t), n program outputs P1(t)-Pn(t), and factory matrix dimensions m×n, with the internal coefficients being the process of each function f(x) at the various nodes. As mentioned above, there is a creative intent P', so that for a given t, the error is P'n(t)-Pn(t), typically over a time interval, e.g., from t to t+T. The error is used to modulate the value of the function, which in practice may be time-varying in nature and then further time-varying due to optimization. The systems and methods described herein are configured to measure performance metrics of various nodes deployed to execute work packages at a granular level and, in some instances, to drive the error towards zero.

[0008] According to an exemplary embodiment, a system for dynamically managing media production work packages for a media production is provided. In an exemplary aspect, the system includes a media production work package generator configured to build a media production work package including a plurality of media processing functions, and a processing function manager configured to generate a plurality of function trackers, each assigned to a respective one of the plurality of media processing functions and including a unique identity, each of the plurality of function trackers configured to dynamically measure a performance metric of the respective media processing function during deployment. The system further includes a function deployment manager configured to embed the plurality of function trackers in each of the media processing functions, and further configured to deploy the plurality of media processing functions onto a plurality of resources available in a cloud computing network, including both physical resources and software resources, the plurality of resources comprising a plurality of processors and an electronic memory accessible by the plurality of processors. Further, the performance metric collector is configured to receive performance metrics measured from the plurality of function trackers during execution of the plurality of media processing functions deployed by the plurality of resources available in the cloud computing network, and is further configured to generate a graphical representation of a total cost of each of the plurality of deployed media processing functions based on the measured performance metrics associated with a respective unique identification of each function tracker. The system further includes a work package controller configured to generate a data point structure uniquely identifying the plurality of deployed media processing functions of a work package and a total cost of each deployed media processing function relative to an overall cost of the media production work package.

[0009] In an exemplary aspect, the function deployment manager is configured to dynamically change a deployment of at least one of the plurality of media processing functions in the cloud computing network based on the generated data point structure. In yet another exemplary aspect, the plurality of processors includes at least one computer processing unit (CPU), at least one graphics processing unit (GPU), and at least one field programmable gate array (FPGA).

[0010] Additionally, in exemplary aspects of the systems described herein, the work package controller is further configured to generate a data point structure to represent the media production workflow, the data point structure including the work packages and identifying an overall cost of the media production workflow. In exemplary aspects, the system may also include a function manager optimizer configured to redeploy a plurality of media processing functions across different resources to reduce an overall cost of the media production workflow.

[0011] The above simplified summary of exemplary embodiments is helpful in providing a basic understanding of the present disclosure. This summary is not an exhaustive overview of all contemplated embodiments, and is not intended to identify key or critical elements of all embodiments or to delineate the scope of any or all embodiments of the present disclosure. Its sole purpose is to present one or more embodiments in a simplified form as a prelude to the more detailed description of the disclosure that follows. To accomplish the above, one or more embodiments of the present disclosure include the features described and illustratively pointed out in the claims. [Brief description of the drawings]

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more exemplary aspects of the disclosure and, together with the detailed description, serve to explain the principles and implementations thereof.

[0013] [Figure 1]1 illustrates a block diagram of a system for dynamically managing media production work packages for a media production in accordance with an illustrative embodiment. [Figure 2A] 1 illustrates a block diagram of a system for dynamically managing media production work packages for a media production in accordance with an illustrative embodiment. [Figure 2B] 1 illustrates a block diagram of a system for dynamically managing media production work packages for a media production in accordance with an illustrative embodiment. [Diagram 3] 1 illustrates a block diagram of a function deployment manager that dynamically manages media production work packages for a media production in accordance with an illustrative embodiment. [Figure 4] FIG. 1 is a block diagram illustrating example requirements according to an example aspect of the present disclosure. [Diagram 5] 1 illustrates a flowchart of a method for dynamically managing media production work packages for a media production, according to an example embodiment. [Figure 6] FIG. 1 is a block diagram illustrating a computer system upon which aspects of the system and method for dynamically managing media production work packages for a media production may be implemented, according to an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Various aspects of the present invention are described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a thorough understanding of one or more aspects of the present invention. However, it may be apparent that in some or all instances, any aspect described below can be practiced without adopting the specific design details described below. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects. The following presents a simplified summary of one or more aspects of the present invention in order to provide a basic understanding of the invention.

[0015] In general, several aspects of a function expansion optimization system are presented herein with reference to various systems and methods that are described in the following detailed description and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system.

[0016] As an example, an element, or any portion of an element, or any combination of elements, may be implemented as a "processing system" including one or more processors. Examples of processors include microprocessors, microcontrollers, graphic processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on chips (SoCs), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gate logic, discrete hardware circuits, and other suitable hardware configured to perform various functions described throughout this disclosure. One or more processors in a processing system may execute software. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, and the like, regardless of whether the term is software, firmware, middleware, microcode, hardware description language, or the like.

[0017] Generally, to manage and optimize a remotely distributed media production, for example in a cloud environment, the systems and methods described herein can measure the performance of each media processing function such that the error in P'(t) can be zeroed out at a known cost. Furthermore, as described in more detail below, each elemental function can be, for example, a CPU, GPU, or FPGA deployed process. Also, each elemental function can have a latency component due to, for example, transport, transit, or file access time, or a combination of all three.

[0018] Furthermore, each function F(x) that is deployed can be mapped to a technology, which has a usage cost and latency to execute / perform the deployed function F(x). In general, when a work package is deployed, the cloud provider's IAC / CAC process can parse and then decompose the package into its components, which are then deployed in the cloud. In one exemplary aspect, these deployed processes can be configured to be in containers. Once deployed, performance can be measured, for example, runtime speed, memory usage, CPU usage, etc. Additionally, transit time of the essence can be measured.

[0019] According to an exemplary embodiment, each deployed function F(x) is assigned its own immutable identity. However, under existing cloud infrastructures, the IAC / CAC deployment process does not provide an audit trail or pointer structure that identifies which processing functions are deployed with which resources and how they are deployed. Thus, according to an exemplary embodiment, a pointer structure is generated and managed to measure performance at a granular (e.g., micro) level of each processing function deployed in the cloud infrastructure based on identification information provided by tags embedded in the cloud infrastructure. Work packages are generated as logical structures (e.g., one or more nodes representing functions connected together in a topology), and then deployed and implemented as physical configurations in the cloud infrastructure that represent the logical structures, with immutable identification providing a mapping between the logical structures / entities (e.g., tasks or functions of the work package) and the physical implementations / entities (e.g., nodes in the cloud infrastructure that execute the stated functions). Using this mapping, the systems and methods disclosed herein are configured to audit the identity of each work package as well as the functions and processes it is composed of by collecting performance metrics during real-time execution of the functions and processes as they are deployed. In an exemplary aspect, the performance metrics collected may be RAM, CPU, network load, GPU, etc. associated with a unique ID of a particular media processing function without interfering with the actual processes running on a particular node of the cloud computing infrastructure.

[0020] As described in more detail below, the processing function manager, which may be a subcomponent or module of the function deployment manager, is configured to insert markers, flags, and / or identification values ​​(commonly referred to as "function trackers") along with or within (e.g., embedded in) the media processes and functions, so that even after the media processes and functions become deployed media processing functions, their respective identities (unique and / or immutable identities) are sent back via an API to an overall controller application (e.g., a work package controller) that has knowledge of all work packages, their dependent processes and dependent functions, and is therefore configured to build a data point structure, i.e., automatically build the complete automation state of the deployed work package, which uniquely identifies each of those processes and functions with their embedded function trackers and identifies (based on collected performance metrics) what it costs and thereby what the total cost of the entire work package is.

[0021] Thus, in an exemplary aspect, the data point structure may be generated as a viewable graph, topology, histogram, or similar data structure that maps the measurement data (i.e., performance metrics) of each function over time. Additionally, the data point structure provides a physical to logical mapping of functions and work packages based on the unique ID of each function tracker. In one aspect, the data point structure may be considered as a dynamic work package performance mapping structure that logically represents the physical deployment. By measuring the performance metric data over time, the data point structure may be constructed as a statistical model of the resource consumption of the function f(x) component of the work package when deployed with cloud resources, and as part of a workflow topology that represents a graph of the work or tasks to be executed as part of the work package. Thus, the measured performance metrics may determine, for example, the different known costs between GPUs, CPUs, and / or FPGAs of cloud computing resources that execute the deployed function f(x).

[0022] In an exemplary aspect, as performance criteria are learned for each deployed function f(x) in each cloud resource, the system can be configured to flag exceptions, for example, if memory accesses take too long or if a calculation takes too long to satisfy the constraints of a particular work package. Based on these flags, the system can be further configured to determine if a cloud node is failing, or perhaps if a node is over-provisioned by the cloud provider to execute the stated function. In a related aspect, the data point structure can also be configured to validate improvements to the function f(x) and / or the work packages. For example, if there is a better algorithm for inverting a matrix or applying a transform with a work package, the system can modify the algorithm based on performance metrics measured over time to optimize the workflow. This validation can be particularly useful in cases where a change in codec transform parameters may result in equivalent quality, but for example, a less computationally expensive CODEC.

[0023] In any event, the data point structure uses the unique ID of the function tracker to map the physical implementation of the work package as it is deployed to the logical structure of the work package topology. The embedded function tracker in each deployed media processing allows the controller to map the logical representation without losing the mapping when it is physically deployed to the cloud environment. In other words, the function tracker allows the controller to restore the logical graph based on immutable identification information. Using the function tracker's identification, the controller can determine which particular cloud node implements which of one or more media processing functions, and can also develop an understanding of the actual performance metrics by that cloud node to understand how both the media processing functions and the node itself can be optimized.

[0024] By measuring individual costs and calculating the total cost of work packages, the exemplary method and system can be configured to take advantage of underutilized resources available in cloud infrastructures that would otherwise be typically abandoned due to the opportunity cost of extracting them. In an exemplary aspect, the function trackers, i.e., markers, flags, identification values, etc., can be configured as environment variables, insofar as the variables are passed along with the deployed function or process. In another exemplary aspect, the function trackers can be implemented as "burn-in code" for a particular function or process that is generated by a controller application (e.g., a work package controller) that is inserted into each process and function prior to deployment to cloud infrastructure resources. In an exemplary aspect, the immutable identification can be loosely modeled as WPID.FID or WPID.PID, and the process function manager can be configured to assign unique WP,F or both WP,PO, as needed, even for identical work packages, functions, and processes. These parameters can be inserted into the code of the media processing functions, thereby ensuring that each media processing function has a unique identity.

[0025] According to an exemplary aspect, as each media processing function is deployed to a node in the cloud (e.g., as part of a work package for generating a media production), the media processing function begins execution and the function tracker is configured to begin measuring an associated performance metric f(x) for that function. In addition, the function tracker can be configured to register its location and respective identity within the cloud infrastructure (e.g., a particular cloud node) with the controller application and begin collecting and providing performance metric data. As described in more detail below, the function deployment manager 101 can be configured as a resource manager that can recover the logical identity of each deployed media processing function based on the embedded logical identity. As described in more detail below, the function deployment manager 101 can be configured to build a data point structure that represents the overall implementation and usage of resources of the deployed work package based on the logical identity.

[0026] That is, according to an exemplary embodiment, the function tracker can be configured to collect performance metric measurements for programmable time intervals and at a programmable sample rate, e.g., one sample per millisecond, one sample per second, etc. Additionally, the processing function manager can be configured to allow an upstream optimizer (e.g., a work package controller) to adjust these intervals and rates so as not to be saturated with all the information reported by all the deployed media processing functions. For example, the processing function manager can control the sampling instants and the overall window of time in which the function tracker can measure a particular function as it is deployed within the cloud infrastructure. The performance metric collector can also be configured to coalesce the deployed media processing functions associated with a given work package to generate a cloud burst of information that accurately reflects the total cost of operations for a given package. Thus, using the immutable ID of the function tracker, the processing function manager can map the logical entities of the media processing work package and make optimization decisions depending on existing and / or future work packages.

[0027] FIG. 1 illustrates a block diagram of a system for dynamically managing the deployment of processing functions in a media creation workflow, according to an exemplary embodiment. In an exemplary aspect, a workflow can be an overall operation (e.g., video or media content production) that is coordinated by the system and driven, for example, by a user of the system. A workflow can be further formed by one or more work packages that drive the tasks of the workflow. A work package, in turn, is composed of one or more atomic functions. As described in more detail below, the location of the functions can be tracked by a unique ID during execution and performance metrics can be fed back to the controller. A topology of these workflows and / or work packages then maps the logical layout to a physical layout based on the unique ID. Additionally, performance metrics can be tracked over time to optimize the atomic functions and their deployment within the work packages as well as the optimization of the workflow as a whole.

[0028] Thus, in an exemplary aspect, the system 100 is configured to dynamically manage functions executed as part of work packages in a media production environment and workflow, measure performance metrics, and optimize. Generally, media content provided for production by the system 100 is referred to as "essence," which refers to media (e.g., video clips, audio clips, and / or auxiliary data such as captions) that may be consumed by a user. As described above, a media production workflow includes one or more work packages, each having one or more atomic functions to perform some type of processing on the essence as part of the workflow to generate a media content production. As described in detail herein, the function deployment management system 100 enables dynamic and optimal deployment of processing functions to the cloud to perform different types of work on media essences in a media creation workflow. Additionally, the system is configured to dynamically collect and evaluate performance metrics in real time to measure and, if necessary, calibrate (e.g., optimize) the execution of an entire work package as the entire work package is deployed across multiple media processing resources.

[0029] According to an exemplary embodiment, performance metrics are collected at a granularity level, such as the number of video frames encoded and transferred, as opposed to simply the number of megabytes transferred. In this example, the process being measured is the recording of a media stream, and thus the controller may determine, for example, that the stream has a 1080P media format. Thus, the controller may establish how many frames are being transferred per second, as opposed to raw megabytes, since the controller has a specific context for the media processing function.

[0030] As shown, system 100 includes a function deployment manager 101, which is a software tool or module (e.g., implemented on one or more computer processing devices) configured to deploy media processing functions within a cloud computing environment according to the algorithms and techniques described herein. It should also be noted that while the exemplary embodiments are described in the context of media production, and specifically live or real-time media production or broadcast, the disclosed systems and methods may be implemented to deploy processing functions as part of a workflow for any number of industries, as a technique for optimizing development technologies (e.g., manufacturing factories). That is, function deployment optimizer 101 may equally be implemented as a factory optimization tool in a variety of software and hardware manufacturing and creation environments.

[0031] 1 shows a block diagram specific to a live media production environment. As described above, function deployment manager 101 is configured to deploy functions (e.g., media processing functions) as components or nodes of a media production work package. Thus, function deployment manager 101 is generally located remotely from all of the other components in the system, and in some embodiments may be coupled to components (that are part of a cloud computing environment / network) to effectively deploy and optimize functions. Thus, the components shown in system 100 are provided as an exemplary system.

[0032] In general, a media processing function can be thought of as a function f(x) required for a specialized work package (also called a workflow) based on an input data set. In an exemplary embodiment, a media production can be represented by two graphs, e.g., a first graph representing the topology of the entire workflow and a second graph showing the decomposition of the constituent work packages of the workflow into one or more individual atomic functions f(x). Thus, the function f(x) can be a directed acyclic graph that can be decomposed into one or more atomic computation functions such that there is no re-entry of data into the function (i.e., the function f(x) is stateless). In other words, the function f(x) can be decomposed into one or more atomic computation functions that can be deployed on one or more processing components / nodes of a cloud network. As described in more detail below, the function deployment manager 101 is configured to receive the performance metrics and, optionally, determine an optimal deployment (e.g., displace machines and / or select different machines) that satisfies the requirements of the function f(x) while optimizing the memory load of the described function. In an exemplary aspect, performance metrics may be collected and analyzed for the entire time that a function is "deployed" within cloud resources. That is, at deployment time, a function tracker as described herein is configured to collect performance metrics and send them back to a central controller. This information can then be analyzed to optimize individual functions f(x) as they are deployed, which in turn can also optimize work packages or sets of work packages that form one or more workflows.

[0033] An exemplary function may include multiviewer generation, where each atomic computation function is, for example, downward image scaling to display multiple video feeds (e.g., input and output feeds) as inputs to a single interface of the multiviewer. Another exemplary processing function for a work package may include, for example, one or more codecs configured to encode and / or decode video signals in a particular compressed format for transmission to meet media request parameters. Generally, such codecs are configured to perform encoding and decoding of video and audio data into data packets for transmission over IP in a media distribution network. In some examples, the codecs may encode video and audio data into uncompressed (e.g., linear pulse code modulation, pulse density modulation, direct stream digital pulse amplitude modulation, etc.), lossless (e.g., free lossless audio codec, optimFROG, wavepack, true audio, etc.), and lossy (e.g., adaptive differential (or delta) pulse code modulation, adaptive transform audio coding, MPEG-1, MPEG-2, MPEG-3, MPEG-4, etc.). One or more codecs are provided to compress media content for transmission over the network. It is reiterated that the deployed media processing function f(x) can be, for example, any type of processing function required to execute a media production workflow.

[0034] In any event, the system 100 further includes multiple content generating devices 102A and 102B, which may be configured for A / V feeds across a link via the network 110. Additionally, while only two devices are shown, it should be noted that the system 100 may be implemented using any number of content generating devices. The multiple content generating devices 102A and 102B may also include remote cameras configured to capture live media content, such as, for example, "talent" (e.g., news announcers, game commentators, etc.). Additionally, the content generating devices 102A and 102B may also include E-Sports (e.g., electronic sports) real-time content, and the like. Generally, the exemplary embodiment employs content generating devices 102A and 102B (which may be located at a live event, for example), but it should be understood that a similar configuration may be used for a remote video server configured to store media content and distribute this content over a media distribution network, for example.

[0035] As further shown, the multiple content generation devices 102A and 102B may also be coupled to a communication network 110, such as the Internet, and / or hardware that interfaces with the Internet Protocol (IP). That is, the system 100 may consist of a network of network servers and network devices configured to transmit and receive video and audio signals (e.g., media essence) of various formats. In an exemplary aspect, the communication network 110 and its processing components may execute as part of a cloud computing environment / network. Moreover, in one aspect, the essence generated by the content generation devices 102A and 102B may be provided as an input data set for a media processing function f(x) deployed in the cloud as described herein.

[0036] Generally speaking, a cloud computing environment or cloud platform is the virtualization and central management of data center resources configured as a software-defined pool. Cloud computing provides the ability to apply abstracted compute, storage, and network resources to work packages provided on several hardware nodes clustered together to form a cloud. Furthermore, the multiple nodes each have their own identities, for example, to run client microservices, storage, and backups. A management software layer for an application platform provided by a cloud will typically be provided on the hardware nodes and will include a virtual environment manager component, which initiates a virtual environment for the platform and may include, for example, microservices and containers. As described in more detail below, the function deployment manager 101 is configured to access metadata for multiple cloud computing resources available for a media production workflow and deploy one or more processing functions f(x) for the media production. Furthermore, the function deployment manager 101 is configured to obtain performance metrics from each function tracker for each deployed media processing function f(x), and, if necessary, optimize the work packages, reallocate functions, etc., based on the collected performance metrics and the overall cost of the work packages when deployed in the cloud environment.

[0037] As further shown, system 100 may include one or more remote distribution nodes 127, one or more processing nodes 128, and one or more remote production switchers 151. As mentioned above, these components may be implemented as hardware components in various geographic locations or, alternatively, as processing components as part of a cloud computing environment. One or more distribution nodes 127 (e.g., electronic devices) are configured to distribute production media content to one or more distribution nodes (e.g., remote media devices), such as receivers 117A and 117B, which may be content consumption devices (e.g., televisions, computing devices, etc.). Furthermore, although only two receivers 117A and 117B are shown, it should be understood that the network may include any number of content consumption devices configured to receive and consume (e.g., play out) media content, and such content consumption devices may be distributed across different countries or different continents. As a result, system 100 may be configured as a media network for real-time generation and broadcast of video and audio content. It should also be understood that each of the nodes / components illustrated in system 100 comprises a node in a media product workflow, as will be appreciated by those skilled in the art.

[0038] Additionally, in this network, distribution node 127 may be further configured to distribute media content throughout the distribution network to one or more processing nodes 128, which may include mix / effects engines, keyers, etc. Examples of processing nodes 128 may include a remote production switch, similar to remote production switcher 151 or a remote signal processor, which in an exemplary embodiment may be included in a cloud computing environment.

[0039] 2A and 2B show additional block diagrams of a system for dynamically managing media production work packages for a media production, according to an exemplary embodiment. Specifically, FIG. 2A shows a more detailed block diagram of system 200A, which should be understood to include function deployment manager 101 and cloud computing network 110 of system 100. That is, system 200A shows a subset of the components of system 100 in an exemplary embodiment. FIG. 2B shows a more detailed deployment of cloud computing network 110, where multiple nodes have function trackers provided to collect and upstream performance metrics during runtime of media processing work packages.

[0040] As mentioned above, the function deployment manager 101 may be implemented on a computer or similar computing device. Additionally, the function manager optimizer 101 may include a performance metric collector 212 and a metric database 220. The metric database 220 may be configured as a media processing resource database that receives performance metric and identification data 215 from individual nodes of the cloud computing network 110. This identification and metric data 215 received by the performance metric collector 212 and stored in the metric database 220 identifies the deployed resources in the cloud computing network 110, which may include both physical and software resources, for example, as described above. Thus, the metric database 220 may include a collection of measured performance metrics over time of a uniquely identified function f(x) as they are deployed and stored to generate a data point structure as described herein.

[0041] According to an exemplary embodiment, the function deployment manager 101 is configured to generate a number of function trackers (e.g., markers, flags, and / or identification values) that can be assigned to and embedded (e.g., as code) into media processing functions. These deployed functions 214 can then be deployed to a number of respective nodes of the cloud computing network 110. In an exemplary aspect, the function deployment manager 101 can be configured to query a cloud infrastructure (e.g., provided by a cloud provider or a database provided by the cloud provided) to determine available resources that can be configured to execute each desired function f(x) of a media processing work package according to defined requirements / criteria, and provide a blueprint for resource allocation and function f(x) deployment for a media production workflow to a client device. Resource identifiers 230 can also be provided from the cloud computing network 110 to the function deployment manager 101, such that the function deployment manager 101 can generate one or more resource allocation topologies for deploying the media processing work package including the requested media processing function f(x) to optimally complete the desired task to satisfy the user requirements. In an exemplary aspect, the resource identifier 230 may include the type of the resource, the manufacturer of the resource, the capabilities of the resource, the usage parameters for the resource, the network location, and other metadata descriptors of the linked resource. In an exemplary aspect, the resource type may be physical (e.g., a computing device, a mobile device, a microchip, or other Internet-connected device, etc.), software-based (e.g., a software application, a software service, a cloud computing platform, etc.), etc. The capabilities of the resource enable the function deployment manager 101 to deploy the function f(x) to meet the requirements. Furthermore, the resource may be associated with one or more linked resources, whether of the same type or different types, that may be optionally invoked by the returned resource.

[0042] According to an exemplary embodiment, the function deployment manager 101 includes a performance metric collection module 212 configured to collect performance metrics from multiple resources in the cloud computing network 110 by a function tracker. It should be noted that while the performance metric collector module 212 and the metric database 220 are shown as components of the function deployment manager 101, they may be standalone components or modules associated in alternative manners. In general, the term "module" as used herein may refer to a real-world device, component, or arrangement of components implemented using hardware, such as, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or may refer to a combination of hardware and software, such as a microprocessor system and a set of instructions for implementing the module's functions that (during execution) transform the microprocessor system into a special-purpose device. A module may be implemented as a combination of the two, with some functions facilitated solely by hardware and other functions facilitated by a combination of hardware and software. In some implementations, at least some, and possibly all, of the modules may be executed on a processor of a general-purpose computer. As such, each module may be implemented in a variety of suitable configurations and should not be limited to any exemplary implementation illustrated herein.

[0043] An exemplary set of resources 213-1-213-3 and 214-1-214-2 are shown in both Figures 2A and 2B and represent exemplary available resources within cloud computing network 110. In one aspect, each resource 213-1-213-3 and 214-1-214-2 may be a device (tangible or virtual) having a resource ID 230 associated with the respective resource and having a unique relationship to the respective metadata. Resource ID 230 may also be stored remotely, for example, in the cloud computing environment (e.g., as indicated by the dashed line at 130) and / or directly by resources 213-1-213-3 and 214-1-214-2 themselves. In either case, a device ID 230 is associated with metadata for each resource 213-1-213-3, 214-1-214-2.

[0044] According to an exemplary embodiment, resources 213-1-213-3 are multiple processing devices configurable to execute a processing function f(x) in a media processing work package, and resources 214-1-214-2 are accessible memory (e.g., RAM, cache, etc.) that can be accessed by the multiple processing devices to execute the requested functions. In a refinement of the exemplary embodiment, resource 213-1 is at least one computer processing unit (CPU), resource 213-2 is at least one graphics processing unit (GPU), and resource 213-3 is at least one field programmable gate array (FPGA).

[0045] As described in more detail below, the function deployment manager 101 may include a processing function manager configured to generate a number of function trackers (e.g., software modules) that may include unique IDs and that may be assigned to individual media processing functions f(x). Figure 2B shows a representation of a cloud computing network 110 in which a number of media processing functions with function trackers have been deployed to individual nodes within the cloud computing network 110.

[0046] As described above, the cloud computing network 110 includes a set of resources 213-1-213-3 and 214-1-214-2. As further illustrated in this example, the function deployment manager 101 deploys the first media processing function 214A to the resource 213-1, the second media processing function 214B to the resource 214-1, and the third media processing function 214C to the resource 213-3. Additionally, each of the first through third media processing functions 214A-214C is associated with or embedded within a function tracker 216A-216C, respectively. During execution of the media processing function associated with each respective node (i.e., each resource 213-1, 214-1, and 213-3), the respective function tracker at each node is configured to upstream performance metrics at a granularity level via a respective API for each function. This can be performed on a channel directed from the particular API to the function deployment manager 101. For example, a function tracker for a deployed function can be configured to instruct the operating system of a particular node to provide metrics about how each process is consuming resources. The workload ID of each cloud can then be used to map the metrics to the logical function they represent, allowing the system to determine the "cost" of each function as it runs on the node. In effect, the collected performance metrics are sent back to the performance metric collector 212 of the function deployment manager 101 along with the unique identification information associated with each function tracker.

[0047] The performance metric collector 212 is then configured to dynamically collect or receive this information of various devices implemented in the cloud node and develop a logical mapping of the deployed work package. That is, the function tracker provides an identification of each available resource (e.g., resources 213-1 to 214-2) to which each resource is linked and shares its unique identity. In one aspect, the unique identity may be an attribute filed in a metadata record. In another exemplary aspect, the unique identity of a resource may consist of several logically grouped identities. The performance metrics are transmitted along with the respective identification information.

[0048] In another exemplary aspect, a list of resource locators is provided to the function deployment manager 101, which is configured to communicate with the resources to receive performance metrics collected by the function tracker and associated with the resource's unique resource ID. Alternatively, in some aspects, the resources may be smart and connected to the Internet (e.g., in a cloud environment) and configured to submit collected performance metrics to a listening server. The collected performance metrics and identification 215 are stored in the metric database 220. Furthermore, when a request is generated to find and deploy a function to one or more resources in the cloud computing network 110, the function deployment manager 101 is configured to retrieve the resource ID 230 to determine the available associated resources for executing the processing function.

[0049] In an exemplary aspect of the present disclosure, the function deployment manager 101 can be configured to establish a unique identity for each media processing function f(x) for which metric metadata is collected, the identity establishing the current metrics, capabilities, and capacity of a given resource. In one example, the unique identity for an IoT or other connected device can be assigned by a process similar to DHCP (Dynamic Host Configuration Protocol). For example, a physical device can provide its MAC address, or alternatively, a virtual device can provide a unique address based on the port MAC and the served IP address. The AMWA IS-05 specification uses LLDP as a method to initiate a connection and create a unique ID for a virtual resource. It should be understood that no loss of generality is made with respect to such methods when considering transient devices (devices that are spun up and then spun down). Thus, the actual deployment and configuration of available resources by the function deployment manager 101 can be performed using these known techniques in an exemplary aspect.

[0050] 3 illustrates a block diagram of a function deployment manager 101 for dynamically managing the deployment of media processing functions in a media production work package, according to an example embodiment. The function deployment manager 101 may be implemented on one or more computing devices communicatively coupled to the network 110 for media production, as described above. Additionally, the function deployment manager 101 may include multiple components and / or modules for executing the algorithms and techniques described herein.

[0051] More specifically, the media production optimizer 101 includes a user interface 305, a work package controller 310, a media production work package generator 315, a processing function manager generator 320, a performance metric collector 325, and a storage 330. Generally, the storage 330 may be implemented as an electronic memory configured to store one or more work package deployment topologies, function deployment criteria, collected performance metrics, unique identification information, etc. Additionally, each of these components may be implemented as a software engine or module configured to execute the algorithms disclosed herein.

[0052] As described above, system 100 is provided to configure one or more "machines" in a cloud computing environment to deploy each media processing function f(x) in a media production workflow including a function tracker for tracking execution metrics during execution of the work packages. Thus, according to an exemplary embodiment, work package controller 310 is configured to generate and / or access a user interface 305 on a screen of a computing device, for example, to enable a user to define a media production work package and / or set criteria, and further select a topology for deploying each function in the cloud computing environment.

[0053] During operation, the media production work package generator 315 is configured to construct a media production workflow including processing functions f(x) that are executed within a work package to create media content. More specifically, a media production work package may typically include multiple nodes therein, each of which is designed and designated to perform a specific task or operation for the workflow (e.g., encoding, multiview, routing and switching of media signals, etc.). The media production work package generator 315 is configured to define and / or customize a media production workflow including designing and configuring one or more functions f(x) that are executed as part of the workflow. In this regard, the media production work package generator 315 is further configured to determine deployment criteria for each of the media processing functions f(x) including an input data set for the processing function and at least one atomic computation function for executing the processing function within the generated media production workflow. In other words, the deployment criteria may define the type of input data (e.g., one frame, multiple lines of pixel data, etc.) and the time required to compute the task (e.g., a time threshold).

[0054] As mentioned above, the processing function manager 320 may be configured to generate and manage multiple function trackers, e.g., function trackers 216A-216C as shown in Figure 2B. In particular, the processing function manager 320 may generate the function trackers, i.e., markers, flags, identification values, etc., as environment variables that are passed along with the deployed media processing functions. Alternatively, the processing manager 320 may implement each function tracker as "burned-in code" for a particular media processing function generated by the media production work package generator 315, which is then inserted into each media processing function prior to deployment to cloud infrastructure resources.

[0055] Based on the deployment criteria, media production work package generator 315 is configured to generate a topology, or more preferably multiple topologies, of the resources available in the cloud computing network based on the determined deployment criteria. In other words, media production work package generator 315 is configured to identify the available resources (e.g., processors 213-1-213-3 and memories 214-1-214-2) in network 110 using resource IDs 230 and metadata, as described above, and develop possible topologies (e.g., different configurations) of these resources for deploying and executing the requested functions of the workflow. Generally, the different topologies provide different results for the user and the workflow, but for performing the same function.

[0056] According to an exemplary aspect, the generated topologies represent different configurations of these physical and software resources for executing the processing function f(x). It should be appreciated that based on the location of the memories 214-1 to 214-2 within the network 110, the configurations have different capabilities and different latency constraints for executing the function f(x). Furthermore, each processor 213-1 to 213-3 (e.g., GPU, CPU, FPGA, etc.) may have different function execution speeds based on the memory structure of the function f(x). Thus, since the media production work package generator 315 generates the multiple topologies based on the determined deployment criteria in terms of how they interact with different computational implementations and available memory, the media production manager 101 is configured to determine how each media processing function f(x) should be executed and deployed within the cloud computing network 110 to optimize execution and performance while balancing user constraints such as time and cost.

[0057] As further shown, the function deployment manager 101 includes a performance metric collector 325 that may correspond to the performance metric collector 212 of FIG. 2A described above. Similarly, the storage 330 may be an implementation of the metric database 220. Thus, during operation, the function deployment manager includes a processing function manager 320 configured to embed a plurality of function trackers in each media processing function. The media production work package generator 315 is further configured to deploy the plurality of media processing functions on a plurality of resources available in a cloud computing network, including both physical and software resources. The performance metric collector 325 is then configured to receive performance metrics measured from the plurality of function trackers during execution of the plurality of media processing functions deployed by the plurality of resources available in the cloud computing network. Additionally, the performance metric collector 325 may be further configured to generate a graphical representation of a total cost of each of the plurality of deployed media processing functions based on the measured performance metrics associated with the respective unique identification information of each function tracker at each cloud node. Further, the work package controller 310 may be coupled to a storage 330 that stores performance metrics and may be configured to generate a data point structure that uniquely identifies a plurality of deployed media processing functions of a work package and a total cost (in terms of performance) of each deployed media processing function relative to the overall cost of media production of one or more work packages of the workflow.

[0058] In an additional aspect, the function deployment manager 101 may be configured to dynamically change the deployment of at least one of the plurality of media processing functions in the cloud computing network based on the generated data point structure. More specifically, the media production manager 101 may further include a deployment optimizer 325 configured to dynamically adjust a deployment topology with the media processing function f(x) in the cloud computing network 110 based on the received performance metrics. In an exemplary aspect, the topology is adjusted to include at least one processor (e.g., one or more of the processors 113-1-113-3) for optimizing the accessibility of electronic memory (e.g., memories 114-1-114-2) for executing the atomic computational functions of the function f(x). Additionally, memory capabilities and location in the network affect the execution of the function f(x). In an exemplary aspect, accessibility can be optimized based on at least one of network type, network bandwidth, network latency, processor type, and processor location within the cloud computing network 110, all of which can be determined based on fine-grained performance metrics collected by the distributed function tracker.

[0059] For example, in the cloud, the deployment optimizer 325 may select a CPU, a GPU, or an FPGA, or a combination thereof, for a given function f(x) to deploy there for a workflow. As mentioned above, different topologies may differ slightly from one configuration to another. For example, different quantization errors may occur based on the available arithmetic options in each of the CPU, GPU, or FPGA, which can be further monitored from the performance metrics. Another difference may be that the computation time may vary significantly, but if the process can wait, the solution may be less expensive to compute. In a third difference, the "cloud" may actually exist on-premise as a collection of computation nodes, rather than in the cloud. This may be done for reasons of transmission bandwidth, security, or both. However, each of these resources may be configured to run the same function f(x) but with different capabilities (e.g., bandwidth, latency, FLOPs) and / or different capacities (e.g., 10 Gbps, 5 ms, 1 TeraFlop). These changes in potential deployments enable the deployment optimizer 325 to optimize the deployment of one or more work packages while balancing cost and time constraints. In other words, the deployment optimizer 325 is configured to balance the total work package costs (e.g., latency, economic, quality, etc.) by executing the function f(x) on different heterogeneous machines in the cloud computing environment. In an exemplary embodiment, each atomic computation function can be "stateless." Thus, any cascade, or combination of atomic functions, receives and propagates information in an acyclically connected manner such that the function f(x) can be estimated to be predictable and stable, which also enables the deployment optimizer 325 to determine these costs with predictability.As a result, the deployment optimizer 325 can manage the topologies for deploying work packages based on the collected performance metrics, and because its inputs are predictable, it can dynamically adjust the topology configurations of various work packages to optimize cost, latency, power requirements, etc. Thus, any operation (e.g., decimation, discrete cosine transform (DCT), etc.) can be performed by any of multiple resources in the network 110.

[0060] In still further aspects, the deployment optimizer 325 may be further configured to use the collected performance metrics of each cloud node performing each media processing function to determine sequence counts and timing for transmitting media essences across the network (e.g., between nodes), as described in U.S. patent application Ser. No. 17 / 233,256, filed Apr. 16, 2021 ("System and Method for Controlling Distribution of Media Content in a Distributed System"), the contents of which are incorporated herein by reference in their entirety. In this access, since the media content is "pulled" from a given source node, the sequence timing is also stably predictable and can be allocated to any set of available resources at a given time or cost. Then, when the performance metrics of the work packages are provided to the computation engine (e.g., the deployment optimizer 325), the media can be distributed in a pulled manner for the processing function f(x) to operate on. Since each node in the media distribution network is provided with its event list, count sequence, and media, the node is free to execute the pull process at any rate that meets the requirements of the media access requests. Knowledge and predictability of these timing sequences allows the deployment optimizer 325 to dynamically change its determination of the optimal topology when this data input is of a second type (e.g., when a user updates a workflow).

[0061] In a further exemplary aspect, once the deployment optimizer 325 dynamically adjusts the topology for deploying the media processing functions f(x) of the work package based on the collected performance metrics, the deployment optimizer 325 may be further configured to decompose one or more of the media processing functions f(x) into one or more atomic functions to be allocated to specific machines in the cloud computing network 110 as described herein.

[0062] In general, it should be noted that the function deployment manager 101 (e.g., the media production work package generator 315) can be configured to deploy each media processing function f(x) along with its associated function tracker to the cloud as part of a work package using a variety of techniques, as will be appreciated by those skilled in the art. For example, in one aspect, IAC and CAC enable porting (or "lift and shift"), which effectively migrates the processing function f(x) to the cloud platform (including the media production processing components described above). The cloud computing platform can then receive requests and dynamically modify and dynamically adjust the configuration for the function f(x) in response to requests from the function deployment manager 101. In an alternative aspect, the function f(x) can be configured as one or more containers in the cloud platform, such that the function deployment manager 101 is configured to generate a new container once an optimal topology is determined, along with updates to the workflow application. This new container can then be provided to the cloud platform 110.

[0063] In an exemplary aspect, the function deployment manager 101 may be configured to deploy each media processing function f(x) based on one or more defined requirements / criteria and collected performance metrics. FIG. 4 is a block diagram illustrating exemplary requirements 400 according to an exemplary aspect of the present disclosure. The requirements 400, defined by the function deployment optimizer 101 or otherwise obtained (e.g., by user input), may be set for a requested function f(x). More specifically, the requirements 400 may be a set of constraints for executing the function f(x) according to certain defined thresholds for the function f(x) within a media processing workflow.

[0064] As specifically shown in FIG. 4, requirements 400 may be defined by time constraints 402, physical constraints 404, technical constraints 406, and / or efficiency constraints 408. For example, if a function f(x) must be executed within a certain time frame (e.g., a time threshold defined by a user via user interface 305) and computing resources are required to complete part or all of the task, then time constraints 402 may be defined accordingly (e.g., complete the task within one hour), which would then define the possible combinations of computing processor requirements required to satisfy the defined time constraints 402. Similar determinations may be made for each of the physical constraints 404, technical constraints 406, and / or efficiency constraints 408. For example, if the requested task is editing live video content, the required bit depth, brightness, color volume, etc. may be determined based on the defined workflow. The required video characteristics may be specified in the technical constraints 406 of requirements 400. Other criteria may be, for example, processor type as discussed above, and / or specific costs associated with the resources.

[0065] Using performance metrics collected during runtime execution of the media processing work packages, the function deployment manager 101 can be configured to deploy and tune the media processing functions f(x) differently for the processes required and the memory allocated accordingly. In other words, the system does not simply swap a CPU-based task (e.g., a first topology) for an FPGA-based task (e.g., a second topology), but the system can be configured to dynamically determine multiple topologies (e.g., one FPGA vs. ten CPUs operating in parallel) and then determine an optimal deployment based, for example, on one or more of the requirements 400, and also based on the current performance metrics of the actual cloud nodes that will execute the stated functions. That is, the function deployment manager 101 can determine an optimal combination (i.e., configuration) of one or more resources for deploying and executing each media processing function f(x) in the work package based on the results obtained by the performance metrics collector 212 for the allocated devices (e.g., resources 213-1 to 214-2). For example, at the micro-scale, the function deployment manager 101 can effectively remap each decomposed atomic function, while at the macro-scale, the system can remap entire subsystems to achieve very different attributes or characteristics of the process.

[0066] 5 shows a flowchart of a method for managing the evolution of a media production workflow, according to an example embodiment. It should be understood that, in general, the method 500 can be performed using one or more of the components described above with respect to Figures 1-3. As shown, the method begins at step 502, when a user, for example, begins building a workflow for media production.

[0067] At step 504, the system is configured to collect or otherwise obtain metadata associated with a plurality of resources available in the cloud computing network from a media processing resource database. These resources may include both physical and software resources, where the resources comprise a plurality of processors and, for example, electronic memory accessible by the plurality of processors. Then, at step 506, the media production workflow generator builds a media production workflow package including the plurality of media processing functions f(x). Further, a plurality of function trackers are generated and / or assigned to each of the plurality of media processing functions f(x), each function tracker having a unique identification.

[0068] It should be noted that steps 504 and 506 can be performed in any order and / or in parallel. During step 506, the media production workflow generator can be further configured to determine evolution criteria for the processing function f(x), including an input data set for the processing function f(x) and at least one atomic computation function for executing the processing function f(x) within the generated media production workflow.

[0069] At step 508, the function tracker is embedded or otherwise assigned to the media processing function f(x). Further, a work package topology is selected to deploy the media processing function with the function tracker in a cloud computing network, for example as shown in Figure 2B. In one aspect, the topology is selected to include at least one processor for optimizing accessibility of electronic memory for executing the atomic computational function, the accessibility being optimized based on at least one of network type, network bandwidth, network latency, processor type, and processor location in the cloud computing network.

[0070] At step 510, the media production work package is executed in the cloud infrastructure, with each media processing function being executed by one or more nodes in the cloud infrastructure. During execution, the function tracker is configured to periodically (or continuously) upstream performance metrics to the function deployment manager 101. A data point structure can then be built that indicates the total cost of the work package in its current deployment in the cloud infrastructure. Knowledge of the performance metrics at a granular level enables the function deployment manager 101 to take some additional deployment actions. In an exemplary embodiment, the function deployment manager 101 can also build a histogram representing the collected performance metrics (e.g., frame transfers), which includes the mean and standard deviation for each such metric. In an exemplary embodiment, the histogram may be used to map any performance metric of the cloud nodes executing the respective functions. In particular, one or more of many different metrics can be recorded, including, for example, CPU usage, RAM load, network and disk IO, GPU usage and parameters, etc. In an exemplary aspect, as work packages become executable, the histogram provides a first-order estimate of the performance level priorities the system can achieve and what the system should try to set as optimizer targets as a function of deployment. Additionally, during optimization, the system can further refine expectations about how work packages will be deployed or, in fact, how the resources on which they are deployed will behave.

[0071] For example, in step 512, once one or more functions f(x) are deployed according to a selected configuration in the cloud computing network, a media production workflow can be implemented to perform real-time video creation, e.g., with an optimized deployment according to total cost. That is, video production is performed, including the deployed function f(x), which can be configured based on, e.g., user- or system-defined criteria, and dynamically adjusted (e.g., reallocated) according to collected metrics. That is, even if the cloud computing platform / network has a fixed set of resources (e.g., processors 213-1 to 213-3 and memories 214-1 to 214-2), the overall machines (e.g., resource combinations) are elastic based on variations in possible deployment topologies, which can be modified based on these metrics. As a result, the optimization deployment system can displace one or more machines on which the function f(x) runs in order to optimize memory load accessibility.

[0072] As an optimization example, at the global computer level, the function deployment manager 101 can measure the amount of computation being used for each particular media processing function. For example, if two media processing functions are deployed on the same cloud node, the first function deployed on the node (e.g., processor 213-1) may take 2% of the CPU, while the second function of the workload may take 20% for the same node. With this information, the function deployment manager 101 can effectively determine how much of the total CPU utilization is utilized by each process. As a result, the function deployment manager 101 may be measuring numerous functions and metrics per process. Being able to determine all of these different metrics for GPU, CPU, network, RAM, etc., the function deployment manager 101 can determine how these resources fit together and how these resources can be added or removed and what the impact is on the total cost of the work package.

[0073] In step 514, the function deployment manager 101 may determine whether the work package is complete, e.g., whether the media production has been generated and can be sent to the content consumption device as described above with respect to Figure 1. If not, the system may execute a feedback loop to step 510 to continue to collect performance metrics and dynamically adjust the deployment and delivery of media processing functions accordingly. Finally, once the work package is complete, the method ends in step 516.

[0074] 6 is a block diagram illustrating a computer system 20 on which aspects of a system and method for dynamically managing media production work packages for a media production may be implemented, according to an exemplary embodiment. It should be noted that computer system 20 may correspond to system 100, or any of the components therein, including, for example, function deployment manager 101. Computer system 20 may be in the form of multiple computing devices or in the form of a single computing device, such as a desktop computer, a notebook computer, a laptop computer, a mobile computing device, a smartphone, a tablet computer, a server, a mainframe, an embedded device, and other forms of computing devices.

[0075] As shown, computer system 20 includes a central processing unit (CPU) 21, a system memory 22, and a system bus 23 that connects various system components including the memory associated with central processing unit 21. System bus 23 may include a bus memory or bus memory controller, a peripheral bus, and a local bus that may interface with any other bus architecture. Examples of buses may include PCI, ISA, PCI-Express, HyperTransport™, InfiniBand™, Serial ATA, I2C, and other suitable interconnects. Central processing unit 21 (also referred to as a processor) may include a single or multiple processor sets having single or multiple cores. Processor 21 may execute one or more computer executable codes that implement the techniques of this disclosure. System memory 22 may be any memory for storing data and / or computer programs executable by processor 21 as used herein. System memory 22 may include volatile memory, such as random access memory (RAM) 25, and non-volatile memory, such as read only memory (ROM) 24, flash memory, or any combination thereof. A basic input / output system (BIOS) 26 may use ROM 24 to store basic instructions for transferring information between elements of computer system 20, such as when loading an operating system.

[0076] Computer system 20 may include one or more storage devices, such as one or more removable storage devices 27, one or more non-removable storage devices 28, or a combination thereof. The one or more removable storage devices 27 and non-removable storage devices 28 are connected to system bus 23 via storage interface 32. In one aspect, the storage devices and corresponding computer-readable storage media are power independent modules for storing computer instructions, data structures, program modules, and other data for computer system 20. System memory 22, removable storage device 27, and non-removable storage device 28 may use a variety of computer-readable storage media. Examples of computer-readable storage media include mechanical memory such as cache, SRAM, DRAM, Zero Capacitor RAM, Twin Transistor RAM, eDRAM, EDO RAM, DDR RAM, EEPROM, NRAM, RRAM, SONOS, PRAM, etc.; flash memory or other memory technologies such as solid-state drives (SSD) or flash drives; magnetic disk storage such as magnetic cassettes, magnetic tapes, and hard disk drives or floppy disks; optical storage such as compact disks (CD-ROMs) or digital versatile disks (DVDs); and any other medium that can be used to store desired data and that can be accessed by computer system 20.

[0077] The system memory 22, removable storage device 27, and non-removable storage device 28 of computer system 20 may be used to store an operating system 35, additional program applications 37, other program modules 38, and program data 39. Computer system 20 may include a peripheral interface 46 for communicating data from input devices 40, such as a keyboard, mouse, stylus, game controller, voice input device, touch input device, or other peripheral device, such as a printer or scanner, via one or more I / O ports, such as a serial port, a parallel port, a universal serial bus (USB), or other peripheral interface. A display device 47, such as one or more monitors, projectors, or integrated displays, may be connected to system bus 23 via an output interface 48, such as a video adapter. In addition to the display device 47, computer system 20 may include other peripheral output devices (not shown), such as loudspeakers and other audiovisual devices.

[0078] The computer system 20 can operate in a network environment using a network connection to one or more remote computers 49. The one(s) remote computers 49 can be a local computer workstation or server including most or all of the elements described above in describing the nature of the computer system 20. The one or more remote computers 49 can represent various cloud resources of the network 101, as described above. Additionally, other devices, such as, but not limited to, routers, network stations, peer devices, or other network nodes, can also be present in the computer network. The computer system 20 can include one or more network interfaces 51 or network adapters for communicating with the remote computers 49 over one or more networks, such as a local area computer network (LAN) 50, a wide area computer network (WAN), an intranet, and the Internet. Examples of network interfaces 51 are, for example, an Ethernet interface, a frame relay interface, a SONET interface, and a wireless interface. In an exemplary embodiment, the one or more remote computers 49 can correspond to a cloud computing network 110, as described above.

[0079] In general, example aspects of the present disclosure may be systems, methods, and / or computer program products that may include computer-readable storage medium(s) having computer-readable program instructions for causing a processor to perform aspects of the present disclosure.

[0080] A computer-readable storage medium may be a tangible device that can hold and store program code in the form of instructions or data structures that can be accessed by a processor of a computing device such as computing system 20. A computer-readable storage medium may be an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. By way of example, such a computer-readable storage medium may comprise a random access memory (RAM), a read-only memory (ROM), an EEPROM, a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a flash memory, a hard disk, a portable computer diskette, a memory stick, a floppy disk, or even a mechanically encoded device such as a punch card or a raised structure having grooves on which instructions are recorded. As used herein, a computer-readable storage medium should not be interpreted as a transient signal itself, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a wave guide or transmission medium, or an electric signal transmitted through a wire.

[0081] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network interface in each computing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing device.

[0082] The computer readable program instructions for carrying out the operations of the present disclosure may be either assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​and traditional procedural programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a LAN or WAN, or the connection may be made to an external computer (e.g., via the Internet). In some aspects, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuit to carry out aspects of the present disclosure.

[0083] For clarity, not all of the routine features of the aspects are disclosed herein. In the development of any actual implementation of the present disclosure, it will be understood that numerous implementation-specific decisions must be made to achieve the developer's particular goals, and that these particular goals will vary for different implementations and different developers. It will be understood that such a development effort may be complex and time-consuming, but would nevertheless be a routine engineering undertaking for those of ordinary skill in the art having the benefit of this disclosure.

[0084] Moreover, it is to be understood that the phrases or terms used herein are for purposes of description and not of limitation, and that the terms or terms herein are to be interpreted by one of ordinary skill in the art in light of the teachings and guidance presented herein, in combination with the knowledge of persons of ordinary skill in the relevant art. Moreover, no term in the specification or claims is intended to be given a non-generic or special meaning unless expressly so stated.

[0085] The various aspects disclosed herein encompass currently known and future known equivalents of the known modules referenced herein by way of example. Moreover, while aspects and applications have been illustrated and described, it will be apparent to one of ordinary skill in the art having the benefit of this disclosure that many more modifications than those described above are possible without departing from the inventive concepts disclosed herein.

Claims

1. 1. A system for dynamically managing media production work packages for a media production, the system comprising: a media production work package generator configured to build a media production work package including a plurality of media processing functions; a processing function manager configured to generate a plurality of function trackers, each of the plurality of function trackers assigned to a respective one of the plurality of media processing functions and including a unique identity, each of the plurality of function trackers configured to dynamically measure at least one performance metric of the respective media processing function during evolution; a function deployment manager configured to embed the plurality of function trackers in the respective media processing functions and further configured to deploy the plurality of media processing functions onto a plurality of resources available in a cloud computing network, including both physical and software resources, the plurality of resources comprising a plurality of processors and electronic memory accessible by the plurality of processors; a performance metric collector configured to receive the at least one performance metric measured from the plurality of function trackers during execution of the plurality of media processing functions deployed by the plurality of resources available in the cloud computing network, and further configured to generate a graphical representation of a total cost of each of the plurality of respectively deployed media processing functions based on the at least one performance metric measured in association with the unique identification of each of each function tracker; a work package controller configured to generate a data point structure uniquely identifying the deployed media processing functions of the work package and a total cost of each deployed media processing function relative to an overall cost of the media production work package; Equipped with the function deployment manager is configured to dynamically modify a deployment of at least one of the plurality of media processing functions within the cloud computing network based on the generated data point structure; The system, wherein the plurality of processors includes at least one computer processing unit (CPU), at least one graphics processing unit (GPU), and at least one field programmable gate array (FPGA).

2. 2. The system of claim 1 , wherein the work package controller is further configured to generate the data point structure to represent a media production workflow, the data point structure including the work packages and identifying an overall cost of the media production workflow.

3. 3. The system of claim 2, further comprising a function manager optimizer configured to redeploy one of the plurality of media processing functions to a different one of the plurality of resources to reduce the overall cost of the media production workflow.

Images