NBMP workflow management method, workflow manager, and medium

Abstract

The present disclosure provides a method, workflow manager, and medium for managing a network-based media processing NBMP workflow. The method includes: a workflow manager acquiring a network-based media processing NBMP workflow, the NBMP workflow comprising a plurality of workflow tasks and a plurality of proximity parameters, the plurality of proximity parameters indicating a plurality of expected distances between the plurality of workflow tasks and at least one of a media source and a media receiver; assigning the plurality of workflow tasks to the media receiver, the media source, and at least one cloud element or network element based on the plurality of expected distances; and managing the NBMP workflow based on the plurality of assigned workflow tasks.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 17 / 213,948, filed on March 26, 2021, which claims priority to U.S. Provisional Patent Application No. 63 / 006,194, filed on April 7, 2020, U.S. Provisional Patent Application No. 63 / 042,477, filed on June 22, 2020, and U.S. Provisional Patent Application No. 63 / 087,735, filed on October 5, 2020. The entire contents of the prior applications are incorporated herein by reference. Technical Field

[0003] Embodiments of the present disclosure relate to Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP), and more particularly, to task proximity in a media workflow. Background Art

[0004] The MPEG Network-Based Media Processing (NBMP) project has developed concepts for processing media in the cloud. However, current NBMP designs do not provide any information about how workflows consisting of various tasks are distributed across different cloud and network resources, taking into account the proximity of these tasks to various sources and sinks. Furthermore, while current NBMP designs provide proximity parameters for running various tasks, they do not provide any logical grouping of tasks or any method for measuring the quality of partitioning workflows between different network entities, media processing entities (MPEs), sources, or sinks.

[0005] The NBMP draft international specification shows great potential for increased media processing efficiency, faster and lower-cost deployment of media services, and the ability to provide large-scale deployment by leveraging public, private, or hybrid cloud services.

[0006] The current NBMP specification defines the geographic placement of tasks to data centers. However, there is no signaling of the relative distances of tasks when there are multiple sources and sinks, no logical grouping of tasks to run on the same hardware or cloud nodes or network cluster, or a way to measure the efficiency of assigning tasks to different network entities. Summary of the Invention

[0007] According to one or more embodiments, a method performed by at least one processor is provided, the method comprising: a workflow manager obtaining a network-based media processing (NBMP) workflow, the NBMP workflow comprising a plurality of workflow tasks and a plurality of proximity parameters, the plurality of proximity parameters indicating a plurality of expected distances between the plurality of workflow tasks and at least one of a media source and a media receiver; allocating the plurality of workflow tasks to the media receiver, the media source and at least one cloud element or network element based on the plurality of expected distances; and managing the NBMP workflow according to the plurality of allocated workflow tasks.

[0008] According to one or more embodiments, a workflow manager for a media system is provided. The workflow manager includes at least one processor; and a memory containing computer code. The computer code includes: an acquisition code for causing the at least one processor to acquire a network-based media processing (NBMP) workflow, wherein the NBMP workflow includes a plurality of workflow tasks and a plurality of proximity parameters, wherein the plurality of proximity parameters indicate a plurality of expected distances between the plurality of workflow tasks and at least one of a media source and a media receiver; an allocation code for causing the at least one processor to allocate the plurality of workflow tasks to the media receiver, the media source, and at least one cloud element or network element based on the plurality of expected distances; and a management code for causing the at least one processor to manage the NBMP workflow according to the plurality of allocated workflow tasks.

[0009] According to one or more embodiments, a non-transitory computer-readable medium storing computer code is provided. The computer code, when executed by at least one processor implementing a workflow manager of a media system, causes the at least one processor to: obtain a network-based media processing (NBMP) workflow, the NBMP workflow comprising a plurality of workflow tasks and a plurality of proximity parameters, the plurality of proximity parameters indicating a plurality of expected distances between the plurality of workflow tasks and at least one of a media source and a media sink; assign the plurality of workflow tasks to the media sink, the media source, and at least one cloud element or network element based on the plurality of expected distances; and manage the NBMP workflow based on the plurality of assigned workflow tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Further features, nature, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0011] Figure 1 is a schematic diagram of an environment in which the methods, apparatuses, and systems described herein may be implemented, according to an embodiment.

[0012] Figure 2 yes Figure 1 A block diagram of exemplary components of one or more devices in FIG.

[0013] Figure 3 is a block diagram of an NBMP system according to an embodiment.

[0014] Figure 4 is a block diagram of an example of a workflow management process according to an embodiment.

[0015] Figure 5 is a block diagram of computer code according to an embodiment. DETAILED DESCRIPTION

[0016] Figure 1 FIG. 1 is a schematic diagram of an environment 100 in which the methods, apparatuses, and systems described herein may be implemented, according to an embodiment. Figure 1 As shown, environment 100 may include user device 110, platform 120, and network 130. The devices of environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0017] User device 110 includes one or more devices that can receive, generate, store, process, and / or provide information related to platform 120. For example, user device 110 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smartphone, a wireless phone, etc.), a wearable device (e.g., smart glasses or a smart watch), or similar devices. In some embodiments, user device 110 can receive information from platform 120 and / or send information to platform 120.

[0018] The platform 120 includes one or more devices as described elsewhere herein. In some embodiments, the platform 120 may include a cloud server or a group of cloud servers. In some embodiments, the platform 120 may be designed to be modular so that software components can be swapped in or out based on specific needs. In this way, the platform 120 can be easily and / or quickly reconfigured for different uses.

[0019] In some embodiments, as shown, the platform 120 can be hosted in a cloud computing environment 122. It is worth noting that although the embodiments described herein describe the platform 120 as being hosted in a cloud computing environment 122, in some embodiments, the platform 120 is not cloud-based (i.e., can be implemented outside of a cloud computing environment) or can be partially cloud-based.

[0020] Cloud computing environment 122 includes an environment that hosts platform 120. Cloud computing environment 122 can provide computing, software, data access, storage, and other services without requiring end users (e.g., user devices 110) to be aware of the physical location and configuration of the systems and / or devices hosting platform 120. As shown, cloud computing environment 122 can include a set of computing resources 124 (collectively, "computing resources 124" and individually, "computing resource 124").

[0021] Computing resources 124 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some embodiments, computing resources 124 may host platform 120. Cloud resources may include computing instances executed on computing resources 124, storage devices provided on computing resources 124, data transmission devices provided by computing resources 124, and the like. In some embodiments, computing resources 124 may communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0022] Further Figure 1 As shown, the computing resources 124 include a set of cloud resources, such as one or more application programs ("APP") 124-1, one or more virtual machines ("VM") 124-2, virtualized storage ("VS") 124-3, one or more hypervisors ("HYP") 124-4, etc.

[0023] Applications 124-1 include one or more software applications that can be provided to or accessed by user device 110 and / or platform 120. Applications 124-1 do not require software applications to be installed and executed on user device 110. For example, applications 124-1 may include software associated with platform 120 and / or any other software that can be provided via cloud computing environment 122. In some embodiments, one application 124-1 can send / receive information to / from one or more other applications 124-1 via virtual machine 124-2.

[0024] The virtual machine 124-2 comprises a software implementation of a machine (e.g., a computer) that executes programs, similar to a physical machine. The virtual machine 124-2 can be a system virtual machine or a process virtual machine, depending on the use and correspondence of the virtual machine 124-2 to any real machine. A system virtual machine can provide a complete system platform that supports the execution of a complete operating system ("OS"). A process virtual machine can execute a single program and can support a single process. In some embodiments, the virtual machine 124-2 can execute on behalf of a user (e.g., user device 110) and can manage the infrastructure of the cloud computing environment 122, such as data management, synchronization, or long-term data transfer.

[0025] Virtualized storage 124-3 includes one or more storage systems and / or one or more devices that use virtualization technology within the storage system or device of the computing resource 124. In some embodiments, within the context of the storage system, the types of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage so that the storage system can be accessed without regard to physical storage or heterogeneous structures. Separation may allow administrators of the storage system to flexibly manage storage for end users. File virtualization may eliminate the dependency between data accessed at the file level and the location of the physical storage file. This may optimize the performance of storage usage, server consolidation, and / or non-disruptive file migration.

[0026] Hypervisor 124-4 can provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to execute simultaneously on a host computer such as computing resource 124. Hypervisor 124-4 can provide a virtual operating platform to the guest operating systems and can manage the execution of the guest operating systems. Multiple instances of various operating systems can share virtualized hardware resources.

[0027] The network 130 includes one or more wired and / or wireless networks. For example, the network 130 may include a cellular network (e.g., a fifth generation (5G) network, a Long-Term Evolution (LTE) network, a third generation (3G) network, a Code Division Multiple Access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-optic-based network, etc., and / or a combination of these or other types of networks.

[0028] Figure 1 The number and arrangement of devices and networks shown are provided as examples. Figure 1 There may be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of environment 100 may perform one or more functions described as being performed by another set of devices of environment 100.

[0029] Figure 2 yes Figure 1 1. The device 200 may correspond to the user device 110 and / or the platform 120. Figure 2 As shown, device 200 may include a bus 210 , a processor 220 , a memory 230 , a storage component 240 , an input component 250 , an output component 260 , and a communication interface 270 .

[0030] The bus 210 includes components that allow communication between components of the device 200. The processor 220 is implemented in hardware, firmware, or a combination of hardware and software. The processor 220 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some embodiments, the processor 220 includes one or more processors that can be programmed to perform functions. The memory 230 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 220.

[0031] Storage component 240 stores information and / or software related to the operation and use of device 200. For example, storage component 240 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of non-volatile computer-readable medium, and corresponding drives.

[0032] Input components 250 include components that allow device 200 to receive information, such as through user input, such as a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone. Additionally or alternatively, input components 250 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 260 include components that provide output information from device 200, such as a display, a speaker, and / or one or more light emitting diodes (LEDs).

[0033] The communication interface 270 includes a transceiver-like component (e.g., a transceiver and / or a separate receiver and transmitter) that enables the device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 270 can allow the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 270 can include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0034] Device 200 can perform one or more processes described herein. Device 200 can perform these processes in response to processor 220 executing software instructions stored by non-volatile computer-readable media (e.g., memory 230 and / or storage component 240). Computer-readable media is defined herein as non-volatile memory devices. Memory devices include storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0035] The software instructions may be read into the memory 230 and / or storage component 240 from another computer-readable medium or from another device via the communication interface 270. When executed, the software instructions stored in the memory 230 and / or storage component 240 may cause the processor 220 to perform one or more of the processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more of the processes described herein. Accordingly, the embodiments described herein are not limited to any specific combination of hardware circuitry and software.

[0036] Figure 2 The number and arrangement of components shown are provided as examples. Figure 2 The device 200 may include more components, fewer components, different components, or components arranged differently than those shown. Additionally or alternatively, one or more components of the device 200 may perform one or more functions described as being performed by another group of components of the device 200.

[0037] In an embodiment of the present disclosure, a NBMP system 300 is provided. Figure 3 , the NBMP system 300 includes a NBMP source 310 , a NBMP workflow manager 320 , a function repository 330 , one or more media processing entities (MPEs) 350 , a media source 360 , and a media receiver 370 .

[0038] NBMP Source 310 can receive instructions from third-party devices, communicate with NBMP Workflow Manager 320 via NBMP Workflow API 392, and communicate with Function Repository 330 via Capability Discovery API 391. For example, NBMP Source 310 can send a Workflow Description Document (WDD) to NBMP Workflow Manager 320 and read functional descriptions of multiple functions stored in Function Repository 330. These functions are media processing functions stored in the memory of Function Repository 330. Examples include functions for media decoding, feature point extraction, camera parameter extraction, projection methods, seam information extraction, blending, post-processing, and encoding. NBMP Source 310 can include a processor and memory storing code, or be implemented by at least one processor and memory storing code, where the code is configured to cause the at least one processor to execute the functions of NBMP Source 310.

[0039] The NBMP source 310 may request the NBMP workflow manager 320 to create a workflow including tasks 352 by sending a workflow descriptor document WDD to the NBMP workflow manager 320, wherein the tasks 352 are to be executed by one or more media processing entities 350 by sending a workflow descriptor document WDD. The workflow descriptor document WDD may include several descriptors, each of which may include several parameters.

[0040] For example, NBMP source 310 may select a function stored in function repository 330 and send a workflow descriptor document (WDD) to NBMP workflow manager 320. The workflow descriptor document (WDD) includes various descriptors describing details such as input and output data, required functions, and workflow requirements. The workflow descriptor document (WDD) may include a set of task descriptions and a connection mapping of the inputs and outputs of tasks 352 to be executed by one or more media processing entities 350. When NBMP workflow manager 320 receives this information from NBMP source 310, it may instantiate tasks 352 based on the function names and connect the tasks 352 according to the connection mapping to create a workflow.

[0041] Alternatively or additionally, the NBMP source 310 may request the NBMP workflow manager 320 to create a workflow using a set of keywords. For example, the NBMP source 310 may send a workflow descriptor document (WDD) including the set of keywords to the NBMP workflow manager 320. The NBMP workflow manager 320 may then use the set of keywords to search for appropriate functions stored in the function repository 330. Upon receiving this information from the NBMP source 310, the NBMP workflow manager 320 may use the keywords specified in the process descriptor of the workflow descriptor document (WDD) to search for appropriate functions, and use other descriptors in the workflow descriptor document (WDD) to provide tasks and connect tasks, thereby creating a workflow.

[0042] The NBMP workflow manager 320 can communicate with the function repository 330 via a capability discovery API 393, wherein the capability discovery API 393 can be the same as or different from the function discovery API 391. Furthermore, the NBMP workflow manager 320 can communicate with one or more media processing entities 350 via an API 394 (e.g., an NBMP task API). The NBMP workflow manager 320 can include or be implemented by at least one processor and a memory storing code, wherein the code is configured to cause the at least one processor to perform the functions of the NBMP workflow manager 320.

[0043] The NBMP workflow manager 320 can use API 394 to set up, configure, manage, and monitor one or more tasks 352 of a workflow, wherein the one or more tasks 352 of the workflow can be executed by one or more media processing entities 350. In one embodiment, the NBMP workflow manager 320 can use API 394 to update and destroy tasks 352. To configure, manage, and monitor the tasks 352 of a workflow, the NBMP workflow manager 320 can send messages, such as requests, to one or more media processing entities 350, where each message can have several descriptors, each of which can include several parameters. Each task 352 can include a media processing function 354 and a configuration 353 for the media processing function 354.

[0044] In one embodiment, after receiving a workflow descriptor document from NBMP source 310 that does not include a task list (e.g., includes a keyword list instead of a task list), NBMP workflow manager 320 may select a task based on the description of the task in the workflow descriptor document, searching function repository 330 via capability discovery API 393 to locate an appropriate function to run as task 352 of the current workflow. For example, NBMP workflow manager 320 may select a task based on keywords provided in the workflow descriptor document. After identifying an appropriate function using keywords or a set of task descriptions provided by NBMP source 310, NBMP workflow manager 320 may use API 394 to configure the selected task in the workflow. For example, NBMP workflow manager 320 may extract configuration data from the information received from the NBMP source and configure task 352 based on the configuration data.

[0045] One or more media processing entities 350 may be configured to receive media content from a media source 360, process the media content according to a workflow including tasks 352 and created by the NBMP workflow manager 320, and output the processed media content to a media receiver 370. The one or more media processing entities 350 may each include or be implemented by at least one processor and a memory storing code, the code being configured to cause the at least one processor to perform the functions of the media processing entity 350.

[0046] Media source 360 may include a memory for storing media and may be integrated with or separate from NBMP source 310. In one embodiment, NBMP workflow manager 320 notifies NBMP source 310 when a workflow is ready, and media source 360 may transmit the media content to one or more media processing entities 350 based on the notification that the workflow is ready.

[0047] The media receiver 370 may include or be implemented by at least one processor and at least one display configured to display media processed by the one or more media processing entities 350 .

[0048] As discussed above, messages from NBMP source 310 to NBMP workflow manager 320 (e.g., requesting the creation of a workflow descriptor document for a workflow), and messages from NBMP workflow manager 320 to one or more media processing entities 350 (e.g., messages causing a workflow to be executed), may include several descriptors, each of which may include several parameters. In some cases, communications between any components of NBMP system 300 using an API may include several descriptors, each of which may include several parameters.

[0049] In an embodiment, each workflow or task may provide a proximity parameter for each source or sink to indicate the desired / required proximity to the source, for example, as shown in Table 1.

[0050] Table 1 - Relative distances of tasks or workflows from various sources / sinks

[0051]

[0052] Distance can be defined as a number representing the relative distance of a workflow or task to each source and sink, as shown in Table 2:

[0053] Table 2 - Proximity Parameters

[0054]

[0055] Therefore, if the task's distance to S1 is N and the task's distance to S2 is 2N, then the task's distance to S1 is twice as great as its distance to S1. A distance of 0 means there is no distance between the task and the sink / source.

[0056] To indicate the distance of each workflow / task, a new object array, proximity, can be added to the General Descriptor, as shown in Table 3:

[0057] Table 3 - General Descriptors

[0058]

[0059] In Table 3 above, and in other tables shown herein, added elements are shown in italics.

[0060] This parameter can be added to the general descriptor in the form of a JSON object array. The JSON object may have two parameters: sink / source identifier (id) and distance, as shown in Table 4.

[0061] Table 4 - Generic Descriptors with Increased Proximity Array

[0062]

[0063]

[0064]

[0065]

[0066]

[0067] The NBMP source 310 can assign relative distances to each source and / or sink in a given workflow description. By examining the distance array, it can determine whether the entire workflow or a portion of it should be implemented on the cloud platform, or whether sources or sinks with smaller distance values in the workflow description should be closer to network elements. The exact assignment depends on the availability of cloud or network resources. Optimization can be performed by the workflow manager 320 and the cloud manager.

[0068] In an embodiment where the workflow is provided by an NBMP source 310, the workflow description document (WDD) may include a connection map. Each function instance may have a functional constraint using a general descriptor. The proximity object in this descriptor may be used to describe the required distance of the function instance from the source and sink.

[0069] As the workflow manager 320 instantiates each task for each function instance, it uses a proximity array to allocate the best cloud / network resources based on the desired distances indicated by those tasks.

[0070] In an embodiment, a workflow can be exported by a workflow manager 320. The workflow manager 320 can provide the WDD, which includes a connectivity graph and task constraints, to the NBMP source 310. The NBMP source 310 can update the WDD by adding proximity objects to each task constraint. Subsequently, with the assistance of the cloud manager, the workflow manager 320 may want to reassign tasks to various cloud / network resources that meet the proximity requirements described by the updated WDD. Finally, the workflow manager 320 can return the updated WDD to each source / sink, indicating the relative distances between the tasks and the actual updates.

[0071] Embodiments may provide a method for describing relative distances of a workflow or task from a plurality of sources, or sinks and sources, comprising assigning a number to a corresponding source or sink in the plurality of sources or sinks, wherein the number represents a relative distance from the workflow or task to the corresponding source or sink compared to distances from the workflow or task to other sources or sinks in the plurality of sources or sinks.

[0072] Embodiments may provide a method for signaling target proximity of a workflow or task to multiple sources and sinks using assigned numbers via a network-based media processing NBMP sink.

[0073] Embodiments may provide a method for determining proximity information of cloud or network resources assigned to each task based on the number assigned to the assignment number; the NBMP workflow manager 320 uses the proximity information of the assigned cloud or network resources to adapt the target proximity and provide updates to the NBMP receiver.

[0074] In an embodiment, the update may include how to implement the workflow or task based on target proximity.

[0075] Embodiments may involve a new logical entity called a Task Group (TG). A task group can be a collection of tasks or function instances that are expected to run on the same cloud node / cluster. A unique identifier that is unique between the task group and the task can be used to identify the task group.

[0076] A workflow description with a collection of functions or tasks can have a table defining task groups, as shown in Table 5:

[0077] surface 5—Task Grouping

[0078]

[0079]

[0080] In Table 5 above:

[0081] K is the number of task groups;

[0082] ·m i It's G i the number of tasks in the group;

[0083] ·G i is the identifier of task group i, and

[0084] ·Id ij is the identifier of task or function instance j in task group i.

[0085] The current NBMP TuC defines the distance table between tasks, MPEs, sources and sinks as shown in Table 6 below:

[0086] Table 6 - Tasks or Workflows and Various Sources / Receiver / MPE / appoint Relative distance of services

[0087]

[0088] An embodiment may involve an extension of Table 6 to include a task group as shown in Table 7 below:

[0089] Table 7 - Adding a task group to the task distance table

[0090]

[0091] Note that you can define a distance table for each task, or extend a distance table for task groups. Furthermore, the distance table for each task group can include multiple columns for one or more task groups.

[0092] In an embodiment, if a task group is included in the above table, each member of the task group that is not explicitly included in the table inherits the distance of the task group.

[0093] In an embodiment, if a task group has the above table, each member of the task group inherits multiple entries of the task group table unless it has an explicit column for task or task group.

[0094] In an embodiment, in a distance table of a task, if there is a distance to the task group to which the task belongs, the distance indicates the distance from the task to all other tasks in the task group.

[0095] In an embodiment, in a distance table of a task, if there is a distance of any task belonging to the same task group as the task, the distance of the task group in the same table is replaced by the distance of the task group.

[0096] Task grouping can be implemented using JSON, as shown in Table 8 below:

[0097] Table 8 - Generic Descriptors with Increased Proximity Array

[0098]

[0099]

[0100]

[0101]

[0102]

[0103] An embodiment may provide a method for describing a task group or a group of function instances, the method being capable of defining a logical group of tasks or function instances to be implemented together, wherein the distances of a group of tasks or function instances from a source, a receiver, an MPE, other tasks or other task groups are described, wherein the distances of the task groups are defined together and the distances of each task or function instance within the task group from other tasks / function instances of the same group are defined, thereby showing a detailed description of the distances as well as the logical grouping of multiple functions.

[0104] In an embodiment, each workflow or task may provide a distance to a source, or a sink, or an MPE, or any other network element to indicate the desired / required proximity to the source, as shown in Table 9:

[0105] Table 9 - Relative distances of tasks or workflows from various sources / sinks

[0106]

[0107] Distance can be defined as a number showing the relative distance of a workflow or task to each source and sink, as shown in Table 10:

[0108] Table 10 - Distance Definitions

[0109]

[0110] Thus, if a task's distance to S1 is N, and its distance to S2 is 2N, then the task's distance is twice its distance to S1. A distance of 0 means there is no distance between the task and the sink / source.

[0111] To signal that a task cannot be run on a network entity or source or sink, an embodiment may define an infinite distance.

[0112] If a resource (source / sink / MPE) cannot run a task, a maximum unsigned integer (INF) can be assigned to the distance of the task to the resource. Therefore, when the workflow manager 320 wants to split a task between different resources, it should not assign the task to a resource with an INF value.

[0113] Difference schemes can be used to split a workflow between multiple sources, sinks, MPEs, and other network entities. The best split rendering scheme is the one that reduces the average distance of all tasks in the workflow.

[0114] For example, the average p-norm distance can be defined as shown in Equation 1 below:

[0115]

[0116] For example, for n tasks in a workflow, d_i is the distance of task i to the assigned sink / source / MPE.

[0117] If the task proximity parameter is given by the NBMP source 310 , the NBMP source 310 may also provide a distance function so that the workflow manager 320 can optimize the workflow splitting based on the distance function.

[0118] To signal this metric, an embodiment may add objects to the general descriptor, as shown in Tables 11-12 below:

[0119] Table 11 — General Descriptors

[0120]

[0121]

[0122] Table 12 — Proximity metric object parameters

[0123]

[0124] The above parameters may be optional, but at least one of them may be present in the proximity metric object. In the above object, the default equation may be the 2-norm.

[0125] Embodiments may provide a method for describing an infinite distance between a task and a device or network entity, where the infinite distance means that the task cannot be executed on the device or network entity.

[0126] An embodiment may provide a method for describing splitting efficiency by introducing the average distance of all tasks of a workflow to the entities to which they are assigned to run, wherein a smaller average distance indicates a more effective splitting of the workflow, wherein, in a specific case, the p-norm distance is used to calculate the splitting efficiency.

[0127] Embodiments may provide a method for signaling an equation for calculating splitting efficiency in a workflow description, wherein information may be exchanged between the NBMP source 310 and the workflow manager 320 / cloud platform, and any equation may be implemented, in particular the p-norm distance.

[0128] Figure 4 is a flow chart illustrating an exemplary process 400 for managing NBMP workflows. In some implementations, Figure 4 One or more process blocks of may be performed by, for example, the workflow manager 320 .

[0129] like Figure 4 As shown, process 400 may include obtaining, by a workflow manager, a network-based media processing NBMP workflow, the NBMP workflow including a plurality of workflow tasks and a plurality of proximity parameters indicating a plurality of desired distances between the plurality of workflow tasks and at least one of a media source and a media sink (block 410).

[0130] like Figure 4 As further shown in FIG. 4 , process 400 may include assigning a plurality of workflow tasks to the media receiver, the media source, and the at least one cloud element or network element based on the plurality of desired distances (block 420 ).

[0131] like Figure 4 As further shown in FIG. 4 , process 400 may include managing the NBMP workflow according to the assigned plurality of workflow tasks (block 430 ).

[0132] In an embodiment, the NBMP workflow may be provided by at least one of a workflow manager or a NBMP source.

[0133] In an embodiment, a proximity parameter of the plurality of proximity parameters may include a number, wherein the number indicates a desired distance between the one of the plurality of workflow tasks and at least one of the media source and the media sink.

[0134] In an embodiment, based on the expected distance being 0, the workflow task may be intended to be performed by at least one of the media source or the media sink.

[0135] In an embodiment, the workflow task may not be executable by at least one of the media source or the media sink based on the expected distance being infinite.

[0136] In an embodiment, the NBMP workflow may correspond to a workflow description document WDD, and at least one general descriptor in the WDD may include multiple proximity parameters.

[0137] In an embodiment, a NBMP workflow may include a task group including at least one of a plurality of workflow tasks.

[0138] In an embodiment, a proximity parameter of the plurality of proximity parameters may indicate a desired distance between the task group and at least one of the media source and the media sink.

[0139] In an embodiment, at least one workflow task among the plurality of workflow tasks included in the task group may inherit the desired distance.

[0140] In an embodiment, the average of multiple expected distances may be used to determine the efficiency of the NBMP workflow.

[0141] In the examples, reference Figure 5 , computer code 500 may be implemented in NBMP system 300. For example, the computer code may be stored in a memory of NBMP workflow manager 320 and may be executed by at least one processor of NBMP workflow manager 320. The computer code may include, for example, acquisition code 510, allocation code 520, and management code 530.

[0142] The acquisition code 510, the allocation code 520, and the management code 530 may be configured to cause at least one processor of the NBMP workflow manager 320 to respectively execute the above-referenced Figure 4 Describe various aspects of the process.

[0143] The acquisition code 510 can be configured to cause at least one processor to acquire a network-based media processing NBMP workflow, which includes multiple workflow tasks and multiple proximity parameters, which indicate multiple expected distances between the multiple workflow tasks and at least one of a media source and a media sink.

[0144] The allocation code 520 may be configured to cause the at least one processor to allocate a plurality of workflow tasks to the media receiver, the media source, and the at least one cloud element or network element based on a plurality of desired distances.

[0145] The management code 530 may be configured to enable at least one processor to manage the NBMP workflow according to the assigned plurality of workflow tasks.

[0146] The embodiments of the present disclosure may be used alone or in any combination. In addition, each embodiment (and method thereof) may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0147] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0148] As used herein, the term component is intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0149] Although combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly dependent on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0150] The elements, actions or instructions used herein should not be interpreted as critical or necessary unless explicitly described as such. In addition, as used herein, the articles "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and can be used interchangeably with "one or more". When it is intended to refer to only one item, the term "one" or similar language is used. In addition, as used herein, the terms "has", "have", "having" etc. are intended to be open terms. In addition, the phrase "based on" is intended to mean "based at least in part on", unless otherwise explicitly stated.

Claims

1. A network-based media processing NBMP workflow management method, characterized in that: The method comprises: The workflow manager obtains a network-based media processing (NBMP) workflow, the NBMP workflow comprising a plurality of workflow tasks and a plurality of proximity parameters, the plurality of proximity parameters indicating a plurality of desired distances between the plurality of workflow tasks and at least one of a media source and a media sink; a proximity parameter of the plurality of proximity parameters comprising a number, wherein the number indicates a desired distance between one of the plurality of workflow tasks and at least one of the media source and the media sink; assigning the plurality of workflow tasks to the media receiver, the media source, and at least one cloud element or network element based on the plurality of expected distances; and The NBMP workflow is managed according to the multiple assigned workflow tasks.

2. The method according to claim 1, characterized in that The NBMP workflow is provided by at least one of the workflow manager or a NBMP source.

3. The method according to claim 1, characterized in that Based on the expected distance being 0, the workflow task is intended to be performed by the at least one of the media source or the media sink.

4. The method according to claim 1, wherein Based on the expected distance being infinite, the workflow task cannot be performed by the at least one of the media source or the media sink.

5. The method according to claim 1, wherein The NBMP workflow corresponds to a workflow description document WDD, and the plurality of proximity parameters are included in at least one general descriptor of the workflow description document WDD.

6. The method according to claim 1, characterized in that The NBMP workflow includes a task group, and the task group includes at least one workflow task among the multiple workflow tasks.

7. The method according to claim 6, characterized in that A proximity parameter of the plurality of proximity parameters indicates a desired distance between the task group and the at least one of the media source and the media sink.

8. The method according to claim 7, characterized in that The method further includes inheriting the desired distance at least one workflow task among the plurality of workflow tasks included in the task group.

9. The method according to any one of claims 1 to 8, characterized in that The method further includes determining an efficiency of the NBMP workflow using an average of the plurality of expected distances.

10. A workflow manager for a media system, characterized in that: The workflow manager includes: an acquisition module configured to acquire a network-based media processing (NBMP) workflow, the NBMP workflow comprising a plurality of workflow tasks and a plurality of proximity parameters, the plurality of proximity parameters indicating a plurality of expected distances between the plurality of workflow tasks and at least one of a media source and a media receiver; wherein one proximity parameter of the plurality of proximity parameters comprises a number, wherein the number indicates an expected distance between one of the plurality of workflow tasks and at least one of the media source and the media receiver; an allocation module for allocating the plurality of workflow tasks to the media receiver, the media source, and at least one cloud element or network element based on the plurality of expected distances; and The management module is used to manage the NBMP workflow according to the multiple assigned workflow tasks. The workflow manager according to claim 10 , wherein: The NBMP workflow is provided by at least one of the workflow manager or a NBMP source.

12. The workflow manager according to claim 10, wherein: Based on the expected distance being 0, the workflow task is intended to be performed by the at least one of the media source or the media sink.

13. The workflow manager according to claim 10, wherein: Based on the expected distance being infinite, the workflow task cannot be performed by the at least one of the media source or the media sink. The workflow manager according to claim 10 , wherein: The NBMP workflow corresponds to a workflow description document WDD, and the plurality of proximity parameters are included in at least one general descriptor of the workflow description document WDD.

15. The workflow manager according to claim 10, wherein: The NBMP workflow includes a task group, and the task group includes at least one workflow task among the multiple workflow tasks.

16. The workflow manager according to claim 15, wherein: A proximity parameter of the plurality of proximity parameters indicates a desired distance between the task group and the at least one of the media source and the media sink, and at least one workflow task of the plurality of workflow tasks included in the task group inherits the desired distance.

17. The workflow manager according to any one of claims 10 to 16, characterized in that: The workflow manager further includes a determination module for determining the efficiency of the NBMP workflow using an average value of the plurality of expected distances.

18. A workflow manager for a media system, characterized in that: include: one or more computer-readable non-volatile storage media for storing computer program code; as well as, One or more computer processors, configured to access the computer program code and operate according to instructions of the computer program code to execute the method according to any one of claims 1 to 9.

19. A non-volatile computer-readable medium storing computer code, characterized in that: The computer code is configured to, when executed by at least one processor implementing a workflow manager of a media system, cause the at least one processor to perform the method of any one of claims 1 to 9.

Images