Method, workflow manager, computer device and computer readable medium for managing network-based media processing workflow

Abstract

A system and method for managing a network based media processing (NBMP) workflow is provided. A method is provided including a method performed by at least one processor. The method includes: exporting a NBMP workflow; obtaining at least one first syntax element, the first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is non-essential; determining a plurality of necessary tasks based on the at least one first syntax element; and assigning the plurality of necessary tasks to at least one of a media receiver, a media source, and a media processing entity included in the NBMP workflow.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 042,480 filed on June 22, 2020, U.S. Provisional Application No. 63 / 087,746 filed on October 5, 2020, U.S. Provisional Application No. 63 / 087,748 filed on October 5, 2020, U.S. Provisional Application No. 63 / 087,751 filed on October 5, 2020, and U.S. Application No. 17 / 352,616 filed on June 21, 2021, the disclosures of which are incorporated by reference in their entirety into this application. Technical Field

[0003] Embodiments of the present disclosure relate to Moving Picture Experts Group (MPEG) Network Based Media Processing (NBMP), and more particularly, to managing NBMP workflows. Background Art

[0004] MPEG's Network-Based Media Processing (NBMP) project has developed the concept of processing media on the cloud. The draft NBMP international specification shows great potential in improving media processing efficiency, faster and lower-cost deployment of media services, and the ability to provide large-scale deployment by leveraging public, private or hybrid cloud services.

[0005] However, the current NBMP specification does not distinguish between essential and non-essential tasks of a workflow.

[0006] Furthermore, while segmenting a media stream with segments of the same duration is useful for many applications, in some applications the duration of the segments may become different for different reasons. Therefore, in these cases, fixed duration segments do not work.

[0007] In addition, the NBMP standard defines a step descriptor for processing media in independent or parallel time segments. Although the segmentation of media streams in time is the first level of parallelization, for large videos or volumetric data, segmentation in other dimensions can further provide more parallelization or independent processing of media data and enable the use of more parallel tasks / microservices to process data. Such multi-dimensional parallelism is not fully addressed in the NBMP standard.

[0008] In addition, the NBMP standard defines workflow descriptions to define the required processing. However, the current design does not allow for the exclusion of certain functional instances. Although the NBMP workflow description provides detailed requirements for running a workflow, it does not allow for the exclusion of certain functional instances from the description that applies to the entire workflow. Summary of the invention

[0009] The NBMP draft international specification shows great potential in improving media processing efficiency, faster and lower-cost deployment of media services, and the ability to provide large-scale deployment by leveraging public, private or hybrid cloud services.

[0010] Embodiments of the present disclosure provide a mechanism for implementing various changes in the NBMP standard.

[0011] In an embodiment, a signaling mechanism is provided to identify non-essential inputs, outputs and tasks, and to derive the necessity of inputs and outputs from the necessity of tasks.

[0012] In an embodiment, a method for independently processing media segments that may have different durations or dimensions in the cloud is provided, which may involve extending the NBMP standard to support independent processing of segments with variable durations / dimensions.

[0013] In an embodiment, an extension of the step descriptor of the NBMP standard is provided to define multi-dimensional segments.

[0014] In an embodiment, a method is provided for excluding certain function instances from a description that applies to an entire workflow.

[0015] According to one or more embodiments, a method performed by at least one processor is provided. The method includes: deriving a network-based media processing (NBMP) workflow; obtaining at least one first syntax element, the first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is non-essential; determining a plurality of necessary tasks based on the at least one first syntax element; and assigning the plurality of necessary tasks to at least one of a media receiver, a media source, and a media processing entity included in the NBMP workflow.

[0016] According to one or more embodiments, a workflow manager of a media system is provided. The workflow manager includes: at least one processor and a memory including computer code. The computer code includes: a workflow export code configured to cause at least one processor to export a network-based media processing (NBMP) workflow; a first acquisition code configured to cause at least one processor to acquire at least one first syntax element, the first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is non-essential; a determination code configured to cause at least one processor to determine a plurality of necessary tasks based on the at least one first syntax element; and an allocation code configured to cause at least one processor to allocate a plurality of necessary tasks to at least one of a media receiver, a media source, and a media processing entity included in the NBMP workflow.

[0017] According to one or more embodiments, a non-transitory computer-readable medium storing computer code is provided. The computer code is configured to, when executed by at least one processor of a workflow manager implementing a media system, cause the at least one processor to perform: exporting a network-based media processing (NBMP) workflow; obtaining at least one first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is non-essential; determining a plurality of necessary tasks based on the at least one first syntax element; and assigning the plurality of necessary tasks to at least one of a media receiver, a media source, and a media processing entity included in the NBMP workflow. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0019] Figure 1 is a diagram of an environment in which the methods, apparatuses, and systems described herein may be implemented, according to an embodiment.

[0020] Figure 2 yes Figure 1 A block diagram of exemplary components of one or more devices in.

[0021] Figure 3 is a block diagram of a NBMP system according to an embodiment.

[0022] Figure 4 is a schematic diagram of an example of a workflow with essential and non-essential tasks according to an embodiment.

[0023] Figure 5is a schematic diagram of an example of a workflow with essential and non-essential tasks according to an embodiment.

[0024] Figure 6 is a schematic diagram of a process for identifying necessary tasks in a workflow according to an embodiment.

[0025] Figure 7 is a schematic diagram of a process for independently processing media segments that may have different durations or sizes, according to an embodiment.

[0026] Figure 8 is a block diagram of an example of a workflow management process according to an embodiment.

[0027] Fig. 9 is a block diagram of an example of computer code according to an embodiment. DETAILED DESCRIPTION

[0028] Figure 1 1 is a diagram of an environment 100 in which the methods, apparatuses, and systems described herein may be implemented, according to an embodiment. Figure 1 As shown, environment 100 may include user device 110, platform 120, and network 130. The devices of environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0029] The user device 110 includes one or more devices that are capable of receiving, generating, storing, processing, and / or providing information associated with the platform 120. For example, the user device 110 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smart phone, a wireless phone, etc.), a wearable device (e.g., smart glasses or a smart watch), or a similar device. In some embodiments, the user device 110 can receive information from the platform 120 and / or send information to the platform 120.

[0030] The platform 120 includes one or more devices as described elsewhere in this disclosure. In some embodiments, the platform 120 may include a cloud server or a group of cloud servers. In some embodiments, the platform 120 may be designed to be modular so that software components can be swapped in or out according to specific needs. Therefore, the platform 120 can be easily and / or quickly reconfigured for different uses.

[0031] In some embodiments, as shown, the platform 120 can be hosted in a cloud computing environment 122. It is worth noting that although the embodiments described herein describe the platform 120 as being hosted in a cloud computing environment 122, in some embodiments, the platform 120 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0032] Cloud computing environment 122 includes an environment hosting platform 120. Cloud computing environment 122 may provide end users (e.g., user devices 110) with computing, software, data access, storage, etc. services that do not require knowledge of the physical location and configuration of the system(s) and / or device(s) hosting platform 120. As shown, cloud computing environment 122 may include a set of computing resources 124 (collectively, “computing resources 124” and individually, “computing resource 124”).

[0033] The computing resources 124 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some embodiments, the computing resources 124 can host the platform 120. Cloud resources can include computing instances executed in the computing resources 124, storage devices provided in the computing resources 124, data transmission devices provided by the computing resources 124, etc. In some embodiments, the computing resources 124 can communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0034] like Figure 1 As further shown, the computing resources 124 include a set of cloud resources, such as one or more applications ("APPs") 124-1, one or more virtual machines ("VMs") 124-2, virtualized storage ("VSs") 124-3, or one or more hypervisors ("HYPs") 124-4.

[0035] The applications 124-1 include one or more software applications that can be provided to or accessed by the user device 110 and / or the platform 120. The applications 124-1 can eliminate the need to install and execute software applications on the user device 110. For example, the applications 124-1 can include software associated with the platform 120 and / or any other software that can be provided via the cloud computing environment 122. In some embodiments, one application 124-1 can send information to / receive information from one or more other applications 124-1 via the virtual machine 124-2.

[0036] Virtual machine 124-2 includes a software implementation of a machine (e.g., a computer) that executes programs similar to a physical machine. Virtual machine 124-2 can be a system virtual machine or a process virtual machine, depending on the use of virtual machine 124-2 and its correspondence to any real machine. System virtual machines can provide a complete system platform that supports the execution of a complete operating system ("OS"). Process virtual machines can execute a single program and can support a single process. In some embodiments, virtual machine 124-2 can execute on behalf of a user (e.g., user device 110) and can manage the infrastructure of cloud computing environment 122, such as data management, synchronization, or long-term data transfer.

[0037] Virtualized memory 124-3 includes one or more storage systems and / or one or more devices using virtualization technology within the storage system or device of computing resource 124. In some embodiments, in the context of a storage system, the types of virtualization may include block-level virtualization and file virtualization. Block-level virtualization may refer to abstracting (or separating) logical storage from physical storage so that the storage system can be accessed without considering physical storage or heterogeneous structures. Separation may allow administrators of storage systems flexibility in how administrators manage storage for end users. File virtualization may eliminate the dependency between data accessed at the file level and the location of the physical storage file. This may enable optimization of storage usage, server consolidation, and / or performance of non-disruptive file migration.

[0038] Hypervisor 124-4 may provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to execute simultaneously on a host computer (e.g., computing resource 124). Hypervisor 124-4 may present a virtual operating platform to the guest operating systems and may manage the execution of the guest operating systems. Multiple instances of various operating systems may share virtualized hardware resources.

[0039] The network 130 includes one or more wired and / or wireless networks. For example, the network 130 may include a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-based network, etc., and / or a combination of these networks or other types of networks.

[0040] supply Figure 1 The number and arrangement of devices and networks shown are examples. In practice, Figure 1There may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown. Figure 1 Two or more of the devices shown in may be implemented in a single device, or Figure 1 A single device shown in the environment 100 may be implemented as multiple distributed devices. Additionally or alternatively, a set of devices (eg, one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of devices of the environment 100.

[0041] Figure 2 yes Figure 1 Device 200 may correspond to user device 110 and / or platform 120. Figure 2 As shown, device 200 may include a bus 210 , a processor 220 , a memory 230 , a storage component 240 , an input component 250 , an output component 260 , and a communication interface 270 .

[0042] The bus 210 includes components that allow communication between components of the device 200. The processor 220 is implemented in hardware, firmware, or a combination of hardware and software. The processor 220 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other types of processing components. In some embodiments, the processor 220 includes one or more processors that can be programmed to perform functions. The memory 230 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 220.

[0043] The memory component 240 stores information and / or software related to the operation and use of the device 200. For example, the memory component 240 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or other types of non-transitory computer-readable media, and corresponding drives.

[0044] Input components 250 include components that allow device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keyboard, a mouse, buttons, switches, and / or a microphone). In addition, or alternatively, input components 250 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 260 include components that provide output information from device 200 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0045] The communication interface 270 includes transceiver-like components (e.g., a transceiver and / or a separate receiver and transmitter) that enable the device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. The communication interface 270 can allow the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 270 can include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0046] Device 200 may perform one or more processes described in the present disclosure. Device 200 may perform these processes in response to processor 220 executing software instructions stored by non-transitory computer-readable media (e.g., memory 230 and / or memory component 240). Computer-readable media is defined herein as non-transitory memory devices. Memory devices include memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0047] The software instructions may be read into the memory 230 and / or the memory component 240 from another computer-readable medium or from another device via the communication interface 270. When the software instructions stored in the memory 230 and / or the memory component 240 are executed, the software instructions may cause the processor 220 to perform one or more processes described in the present disclosure. In addition, or alternatively, hard-wired circuitry may be used instead of or in combination with software instructions to perform one or more processes described in the present disclosure. Therefore, the embodiments described herein are not limited to any specific combination of hardware circuitry and software.

[0048] supply Figure 2 The number and arrangement of components shown are provided as examples. In practice, Figure 2 Compared to the illustrated, the device 200 may include additional components, fewer components, different components, or components arranged differently. In addition, or alternatively, a group of components (e.g., one or more components) of the device 200 may perform one or more functions described as being performed by another group of components of the device 200.

[0049] In an embodiment of the present disclosure, a NBMP system 300 is provided. Figure 3 , the NBMP system 300 includes a NBMP source 310, a NBMP workflow manager 320, a function repository 330, one or more media processing entities 350, a media source 360, and a media receiver 370.

[0050] The NBMP source 310 may receive instructions from a third-party entity, may communicate with the NBMP workflow manager 320 via the NBMP workflow API 392, and may communicate with the function repository 330 via the function discovery API 391. For example, the NBMP source 310 may send a workflow description document (Workflow Description Document(s), WDD) to the NBMP workflow manager 320, and may read a function description of a function stored in the function repository 330, which is a media processing function stored in the memory of the function repository 330, such as, for example, media decoding, feature point extraction, camera parameter extraction, projection method, seam information extraction, mixing, post-processing, and encoding. The NBMP source 310 may include at least one processor and a memory, or be implemented by at least one processor and a memory, the memory storing code configured to cause at least one processor to perform the functions of the NBMP source 310.

[0051] NBMP source 310 may request NBMP workflow manager 320 to create a workflow including tasks 352 to be performed by one or more media processing entities 350 by sending a workflow description document, which may include several descriptors, each of which may have several parameters.

[0052] For example, the NBMP source 310 may select a function stored in the function repository 330 and send a workflow description document to the NBMP workflow manager 320, the workflow description document including various descriptors for describing details such as input and output data, required functions, and requirements of the workflow. The workflow description document may include a set of task descriptions and connection mappings of inputs and outputs of tasks 352 to be performed by one or more media processing entities 350. When the NBMP workflow manager 320 receives such information from the NBMP source 310, the NBMP workflow manager 320 may create a workflow by instantiating tasks based on the function names and connecting the tasks according to the connection mappings.

[0053] Additionally or alternatively, the NBMP source 310 may request the NBMP workflow manager 320 to create a workflow by using a set of keywords. For example, the NBMP source 310 may send a workflow description document to the NBMP workflow manager 320, which may include a set of keywords that the NBMP workflow manager 320 may use to find appropriate functions stored in the function repository 330. When the NBMP workflow manager 320 receives such information from the NBMP source 310, the NBMP workflow manager 320 may create a workflow by searching for appropriate functions using the keywords specified in the processing descriptor of the workflow description document, and use other descriptors in the workflow description document to provide tasks and connect them to create a workflow.

[0054] The NBMP workflow manager 320 may communicate with the function repository 330 via a function discovery API 393, which may be the same or different API from the function discovery API 391, and may communicate with one or more media processing entities 350 via an API 394 (e.g., an NBMP task API). The NBMP workflow manager 320 may include or be implemented by at least one processor and a memory storing code configured to cause the at least one processor to perform the functions of the NBMP workflow manager 320.

[0055] The NBMP workflow manager 320 can use the API 394 to set, configure, manage, and detect one or more tasks 352 of a workflow that can be executed by one or more media processing entities 350. In one embodiment, the NBMP workflow manager 320 can use the API 394 to update and destroy tasks 352. In order to configure, manage, and detect the tasks 352 of a workflow, the NBMP workflow manager 320 can send messages such as requests to one or more media processing entities 350, where each message can have several descriptors, and each descriptor has several parameters. Each task 352 can include a media processing function 354 and a configuration 353 for the media processing function 354.

[0056] In one embodiment, after receiving a workflow description document that does not include a task list (e.g., includes a keyword list instead of a task list) from a NBMP source 310, the NBMP workflow manager 320 can select a task based on the task description in the workflow description document to search the function repository 330 via the function discovery API 393 to find an appropriate function to run as a task 352 of the current workflow. For example, the NBMP workflow manager 320 can select a task based on a keyword provided in the workflow description document. After identifying an appropriate function by using a keyword or a task description group provided by the NBMP source 310, the NBMP workflow manager 320 can configure the selected task in the workflow by using the API 394. For example, the NBMP workflow manager 320 can extract configuration data from the information received from the NBMP source and configure the task 352 based on the configuration data.

[0057] The one or more media processing entities 350 may be configured to receive media content from a media source 360, process the media content according to a workflow including tasks 352 created by the NBMP workflow manager 320, and output the processed media content to a media receiver 370. The one or more media processing entities 350 may each include or be implemented by at least one processor and a memory, the memory storing code configured to cause the at least one processor to perform the functions of the media processing entity 350.

[0058] Media source 360 ​​may include a memory that stores media and may be integrated or separate from NBMP source 310. In one embodiment, NBMP workflow manager 320 may notify NBMP source 310 when preparing a workflow, and media source 360 ​​may send media content to one or more media processing entities 350 based on the notification of preparing the workflow.

[0059] The media receiver 370 may include or be implemented by at least one processor and at least one display configured to display media processed by the one or more media processing entities 350 .

[0060] As described above, messages from NBMP source 310 to NBMP workflow manager 320 (e.g., workflow description documents for requesting the creation of a workflow), and messages from NBMP workflow manager 320 to one or more media processing entities 350 (e.g., messages for executing a workflow) may include several descriptors, each of which may have several parameters. In this case, communications between any components of NBMP system 300 using the API may include several descriptors, each of which may have several parameters.

[0061] [Nonessential input, output, and task signaling in workflows]

[0062] Embodiments may relate to methods for identifying and signaling non-essential inputs, outputs, and tasks in a workflow running on a cloud platform.

[0063] The necessary output of a workflow may be the output of a workflow that must produce data in order for the workflow to be considered to be running correctly. The necessary input of a workflow may be the input that must be processed by the workflow to create its necessary output. A workflow that runs correctly may be a workflow that processes all of its necessary inputs and produces all of its necessary outputs. The necessary tasks of a workflow may be tasks that are necessary to run correctly and process the data required to run the workflow correctly. For example, a necessary task may be a task that processes necessary inputs and / or produces necessary outputs. In an embodiment, the necessary input may be the input required to run the necessary task, and the necessary output may be the output required as the necessary input of the necessary task, or the output required as the output of the entire workflow. Non-necessary inputs may be inputs that are not required for a workflow to produce the necessary outputs of the workflow. For example, if all necessary inputs are provided, the workflow may produce all necessary outputs even if no non-necessary inputs are provided. Non-necessary tasks may be tasks that are included in the workflow but are not necessary tasks. For example, non-necessary tasks may process non-necessary inputs and produce non-necessary outputs.

[0064] Figure 4 An example of a workflow 400 with essential and non-essential tasks is shown according to an embodiment.

[0065] In workflow 400, non-essential tasks are shown as grey shaded circles. Figure 4 As shown, Task 1, Task 3, Task 5, and Task 8 are necessary. In addition, Figure 5 In the example shown, Task 2, Task 4, Task 6, and Task 7 are optional.

[0066] Accordingly, in workflow 400, input 1 can be classified as a required input, while input 2 can be classified as a non-required input. Similarly, output 2 can be classified as a required output, while output 1 and output 3 can be classified as non-required outputs.

[0067] [Necessity of signaling in workflow]

[0068] In an embodiment, a flag may be used to signal the necessity of a task in a given workflow. By defining all necessary and non-essential tasks of a workflow, necessary and non-essential inputs and outputs may also be defined or determined.

[0069] Table 1 shows an example of adding non-essential flags in the general descriptor of a task. In Table 1 and throughout this disclosure, the added non-essential flags are shown in italics.

[0070] Table 1 - Generic descriptor for a task with added non-essential flags

[0071] Parameter name type Cardinality id P 1 name P 1 description P 1 rank P 0-1 mpeg-compatibility P 0-1 published-time P 0-1 priority P 0-1 execution-time P 0-1 input-ports Array of objects 1 output-ports Array of objects 1 is-group P 0-1 nonessential P 0-1 state P 1

[0072] Added parameters are shown in italics. Examples of non-essential parameters are defined in Table 2.

[0073] Table 2- Non-essential parameter definitions

[0074]

[0075] [Necessity and non-necessity of exporting workflow inputs and outputs]

[0076] If a workflow is defined by a graph of tasks for the workflow, and if the tasks are identified as essential and non-essential, then any input or output of the workflow may be identified as essential or non-essential.

[0077] For a given workflow as a graph of essential and non-essential tasks, for any pair of inputs and outputs, if the input can be connected to the output through at least one essential path, then both the input and the output are essential. Any other inputs and outputs that do not meet the above conditions are non-essential.

[0078] A path in a workflow may be a subset of the workflow that has only one input and one output, and the input may be connected to the output by a set of arrows connecting the tasks in the path, where the direction of each connection between two tasks is the same as the direction of the arrow directly connecting the input to the output. A necessary path of a workflow may be a path where all tasks included in the path are necessary tasks.

[0079] Figure 5 A necessary path in the workflow 400 is shown with dashed arrow lines.

[0080] [Necessity of using signals to represent input and output]

[0081] If a workflow diagram is defined with the necessity of its tasks defined, as described above, the necessity of the inputs and outputs of the workflow can be derived.

[0082] However, in some cases, in the workflow description, only its input and output can be defined, and the precise workflow is left to the workflow manager to derive. In this case, the workflow description can include or indicate the necessity of the input and output of the workflow.

[0083] In an embodiment, non-essential parameters may be defined for each output in an output descriptor as shown in Table 3, Table 4, Table 5, and Table 6. For example, a non-essential parameter may indicate whether an input, output, task, or other object is essential or not.

[0084] Table 3 - Input Media-Parameter Object

[0085] name type Cardinality stream-id P 1 Name P 1 Keywords P 1 mime-type P 1 video-format P 0-1 audio-format P 0-1 image-format P 0-1 codec-type P 0-1 Protocol P 1 Throughput P 0-1 Buffersize P 0-1 caching-server-url P 1 nonessential P 0-1

[0086] Table 4 - Input Metadata - Parameter Object

[0087]

[0088]

[0089] Table 5 - Output Media-Parameters Object

[0090] name type Cardinality stream-id P 1 name P 1 keywords P 1 mime-type P 1 video-format P 0-1 audio-format P 0-1 image-format P 0-1 codec-type P 0-1 protocol P 1 throughput P 0-1 buffersize P 0-1 caching-server-url P 1 nonessential P 0-1

[0091] Table 6: Output metadata parameter object

[0092] name type Cardinality stream-id P 1 name P 1 keywords P 1 mime-type P 1 codec-type P 0-1 protocol P 0-1 max-size P 0-1 min-interval P 0-1 caching-server-url P 1 scheme-uri P 0-1 nonessential P 0-1

[0093] In an embodiment, if both the media and metadata parameters are optional, then the output may be optional.

[0094] [Derive the necessity and non-necessity of tasks from the input and output of the workflow]

[0095] If the necessity of a workflow input and output is defined, the necessity of the tasks (and their inputs) of that workflow can be derived.

[0096] For a given workflow as a task graph and identified necessary outputs, the following process can be used to identify the necessary tasks:

[0097] First, for each non-essential input, if the inputs of all other corresponding tasks are also non-essential, the output of that task can be marked as non-essential and the task can be removed. This process can be applied to all new non-essential inputs until there are no non-essential inputs left to apply to the process.

[0098] • Second, for each non-essential output, if the outputs of all other corresponding tasks are also non-essential, the input of the task can be marked as non-essential and the task can be removed. This process can be applied until there are no non-essential outputs left to apply to the process.

[0099] Figure 6 An example of such a process is shown. For example, Figure 6 A workflow 600A is shown in which only the necessity of an input is known. Figure 6 As shown, input 1 is known to be essential, input 2 is known to be non-essential, output 2 is known to be essential, and output 1 and output 3 are known to be non-essential.

[0100] Figure 6 Also shown is workflow 600B, which is the result of executing operation 610 on workflow 600A. Operation 610 may correspond to the first part of the above process. For example, since input 2 is not essential, and since task 2 has no other inputs, the output of task 2 (shown as input 2.1) is also marked as not essential, and task 2 is removed.

[0101] Figure 6 Also shown is workflow 600C, which is the result of performing operation 620 on workflow 600B. Operation 620 may correspond to the second part of the above process. For example, since output 1 is not necessary, and since task 6 has no other outputs, the input of task 6 (shown as output 1.2) is marked as not necessary, and task 6 is removed. Similarly, since task 4 has no necessary output, the input of task 6 (shown as output 1.1) is also marked as not necessary, and task 6 is removed. In addition, since output 3 is not necessary, and since task 7 has no other outputs, the input of task 7 (shown as output 3) is also marked as not necessary, and task 7 is removed.

[0102] Therefore, as a result of operations 610 and 620, Task 2, Task 4, Task 6, and Task 7 are removed, and thus these tasks can be determined as non-essential tasks. Similarly, since Task 1, Task 3, Task 5, and Task 8 are still retained, these tasks can be determined as essential tasks.

[0103] Embodiments may relate to a method for defining the necessity of inputs, outputs, and tasks in a workflow, wherein an input, output, or task may be defined as non-essential to the rest of the workflow, and if the entity is removed, the workflow still functions normally.

[0104] Embodiments may relate to a method of signaling necessity of a task, wherein a flag is used in its description to signal non-necessity of the task.

[0105] Embodiments may relate to a method of signaling the necessity of an input or an output, wherein a flag is used in the description thereof to signal the non-necessity of the input or output.

[0106] Embodiments may be directed to a method of deriving necessity of inputs and outputs of a workflow when identifying necessity of tasks of the workflow, wherein necessary paths of the workflow and corresponding necessary input-output pairs are identified.

[0107] Embodiments may be directed to a method for deriving necessity of tasks of a workflow when necessity of inputs and / or outputs of the workflow is identified, wherein the method is used to simplify a workflow graph until all non-essential tasks are identified.

[0108] [Independent processing of media segments with variable duration on the cloud]

[0109] Embodiments may relate to a method for independently processing media segments that may have different durations or sizes in a cloud.

[0110] Figure 7 An example of a process 700 for processing segments in parallel using split / merge is shown. Figure 7 In the example, task T is converted into n instances of task T that run in parallel (i.e., task T0, task T1, ..., task T N-1 ).

[0111] exist Figure 7 In the task T, the media stream is continuous. However, the 1:N split function converts the media stream into N media substreams. Each substream is processed by an instance of T, and then the substreams are interleaved to generate an output equivalent to the output stream of task T.

[0112] The 1:N split and N:1 merge functions work on segment boundaries. Each segment has a duration equal to one or more T's Task Segment Duration (TSD), so segments and substreams are independent of each other in terms of being processed by task T. Note that tasks T0, ..., T N-1 There is no need to process the fragments simultaneously. Since the fragments and sub-streams are independent, each instance of the task can run at its own speed.

[0113] Since each segment in each substream carries the start time, duration and length of the segment, the segments can be multiplexed together in the correct order.

[0114] NBMP amendments define splitter and merge function templates for the above operations.

[0115] The NBMP standard includes step descriptors, as shown in Table 7, which are used to describe the processing mode and time segment duration of each function.

[0116] Table 7: NBMP step descriptor

[0117]

[0118] As shown in Table 7, the segment duration is the same for all segments.

[0119] [Extended Step Descriptor]

[0120] In practice, the segment duration of each segment may be different. Therefore, instead of segment duration as an operating parameter, an embodiment may include a segment marker for separating segments. In this case, it is useful to have a maximum segment duration. An embodiment may also signal whether variable segment duration is allowed.

[0121] Table 8: Step descriptors with variable segment duration

[0122]

[0123] [Extended Splitter and Merger Reference Function Templates]

[0124] In order to convert a media stream with variable segment duration into multiple streams using a segmenter function and then combine the streams after processing using a merge function, embodiments may extend the NBMP segmenter and merge reference function templates as follows.

[0125] [Splitters]

[0126] [NBMP tag name]

[0127] urn:mpeg:mpegi:nbmp:2020:split

[0128] [describe]

[0129] These requirements assume that the input media stream is continuous without any gaps. In this clause, [a, b[ refers to a time range that is equal to or greater than a and less than b.

[0130] A splitter function can have the following requirements:

[0131] One input and N output FIFO buffers, where N is a configuration parameter for the number of partitions.

[0132] Each time it runs one input fragment and generates N output fragments

[0133] Each input fragment should meet the following requirements:

[0134] oContinuous sample sets

[0135] o Maximum duration N*D in time scale T, where D and T are configuration parameters

[0136] oIncludes the following metadata and constraints:

[0137] ■Start time s (milliseconds)

[0138] ■Time scale t

[0139] ■ Maximum duration N*d, where d=D*T / t is an integer, i.e. D*T is divisible by t

[0140] ■Length l (bytes)

[0141] ■ Segment markers when variable duration = 'True', where the segment markers indicate the start time of each segment

[0142] oSamples that do not overlap with other input fragments

[0143] The set of input fragments should cover the entire duration of the input, i.e. no input samples are missed in the total duration

[0144] Operate on the input segments in increasing order, i.e. segments covering earlier time periods should be processed before any segments covering later time periods

[0145] At any time, the media stream at each output buffer consists of zero or more output fragments. Each output fragment shall meet the following requirements:

[0146] oThe continuous duration of the sample

[0147] o The maximum duration of d in time scale t

[0148] o Includes the following metadata:

[0149] ■Start time s (milliseconds)

[0150] ■Time scale t

[0151] ■The maximum duration d in time scale t

[0152] ■Length l (bytes)

[0153] The set of all output segments of all output buffers should cover the entire duration of the input, i.e. no input samples are missed in the set of output segments.

[0154] The splitter should run on each input segment and split it into N output segments in the following way:

[0155] oEach output buffer receives one output fragment.

[0156] o Each input segment is split into N output segments in the order s0, s1, …, s N-1 , where fragment s i Covers the following segments i-1 and i+1 The duration of the media that follows.

[0157] oOutput buffers are ordered as O0, O1, ..., O N-1 .

[0158] oOutput fragments i Place in buffer O i middle.

[0159] [Function description template]

[0160] Table 9: Splitter function description template

[0161]

[0162] [Configuration parameters]

[0163] [parameter]

[0164] Table 10: Slicer configuration parameters

[0165]

[0166]

[0167] Table 11: MIFIFO variable parameters

[0168]

[0169] [JSON Schema]

[0170] [Configuration]

[0171] Table 12

[0172]

[0173]

[0174]

[0175] [variable]

[0176] Table 13

[0177]

[0178] [Event JSON]

[0179] Table 14

[0180]

[0181]

[0182] [merge]

[0183] [NBMP tag name]

[0184] urn:mpeg:mpegi:nbmp:2020:merge

[0185] [describe]

[0186] These requirements assume that the input media stream is continuous without any gaps. In this clause, [a, b[ refers to a time range that is equal to or greater than a and less than b.

[0187] The merge function should have the following requirements:

[0188] N input and one output FIFO buffers, where N is a configuration parameter for the number of partitions.

[0189] Run one input fragment at a time from each input buffer, for a total of N fragments, and generate one output fragment

[0190] Each input fragment should meet the following requirements:

[0191] oContinuous sample sets

[0192] o Maximum duration D in time scale T, where D and T are configuration parameters

[0193] o Includes the following metadata:

[0194] ■Start time s (milliseconds)

[0195] ■Time scale t

[0196] ■ The maximum duration d in the t scale, where d = D*T / t is an integer, i.e. D*T is divisible by t

[0197] ■Length l (bytes)

[0198] ■

[0199] oSamples that do not overlap with other input fragments

[0200] The set of input fragments of N buffers should cover the entire duration of the input, i.e. no input samples are missed in the total duration.

[0201] • Operate on the input segments in increasing order, i.e., segments covering earlier time periods should be processed before any segments covering later time periods.

[0202] The media stream at the output buffer at any time consists of zero or more output fragments. Each output fragment shall meet the following requirements:

[0203] oThe continuous duration of the sample

[0204] o duration of N*d in scale t

[0205] o Includes the following metadata:

[0206] ■Start time s (milliseconds)

[0207] ■Time scale t

[0208] ■Duration N*d

[0209] ■Length l (bytes)

[0210] ■ Segment markers when variable duration = 'True', where the segment markers indicate the start time of each segment

[0211] The set of output segments of all output buffers should cover the entire duration of the input, i.e. no input samples are missed in the set of output segments.

[0212] The splitter should run on each input fragment from each input buffer and merge them into one output fragment in the following way:

[0213] o respectively process the data from the input buffers I0, I1, ..., I N-1 A fragment s0, s1, ..., s of each input buffer in N-1 . Fragments i Including exactly between s i-1 and i+1 The time interval between samples.

[0214] o segments s0, s1, ..., s N-1 The output segments are concatenated in the order shown, so that the output segments include segments s0, s1, ..., s N-1 All samples of without duplicate samples and with the initial representation of EPT.

[0215] [Function description template]

[0216] Table 15: Combined functional description template

[0217]

[0218] [Configuration parameters]

[0219] [parameter]

[0220] Table 16: Merge configuration parameters

[0221]

[0222]

[0223] Table 17: Merge variable parameters

[0224]

[0225] [JSON Schema]

[0226] [Configuration]

[0227] Table 18

[0228]

[0229]

[0230]

[0231] [variable]

[0232] Table 19

[0233]

[0234] [Event JSON]

[0235] Table 20

[0236]

[0237]

[0238] Embodiments may be directed to a segmentation method for describing media having variable duration segments, wherein the media is processed by independent segments having variable durations, wherein a step descriptor is extended to signal functional support for variable segment durations.

[0239] Embodiments may relate to a method for segmenting a media stream into multiple media substreams and merging multiple media substreams into a media stream, wherein the segmentation and merging functions are performed on segments having variable durations, wherein the boundaries of the segments are defined and the segmentation and merging processes are performed at these boundaries.

[0240] [Processing multi-dimensional media clips on the cloud]

[0241] Embodiments may be directed to a multi-dimensional media segment method for independently processing media segments in a cloud. Embodiments may extend the step descriptor of the NBMP standard to define multi-dimensional segments.

[0242] The NBMP standard includes step descriptors, which are used to describe the processing mode and time segment duration of each function as shown in Table 21.

[0243] Table 21: NBMP step descriptor

[0244]

[0245] [General concept]

[0246] An embodiment may extend the segment dimension to other dimensions of the time axis. Table 22 shows an example of this extension.

[0247] Table 22: Multidimensional step descriptors

[0248]

[0249]

[0250] Example 1: For example, dividing a video frame into tiles for step processing may be achieved by the following configuration:

[0251] o Higher Dimension - Size = 2

[0252] o Higher dimensions - description = [width, height]

[0253] o Higher fragments - size = [w, h], where w and h represent the width and height of each individual fragment.

[0254] Example 2: For a video with 3 color components:

[0255] o Higher Dimensions - Size = 1

[0256] oHigher Dimension - Description = [RGB]

[0257] o Higher fragment-size = [c], where c represents color component video.

[0258] The parameter overlap_size indicates the amount of overlap in each dimension. For example, the overlap_size [16 16] in Example 1 means that 16 samples overlap in the width and height of adjacent tiles.

[0259] [JSON implementation]

[0260] The JSON implementation of the above concepts is shown in Table 23.

[0261] Table 23

[0262]

[0263]

[0264] Embodiments may relate to a segmentation method for describing a multi-dimensional media signal into multi-dimensional segments, by which each segment can be processed independently, wherein the dimensions of the media segments and their properties are described in a descriptor, including the size in each dimension and the amount of overlap in each dimension and the number of processed segments, wherein the descriptor can be used in a function description to describe the properties of the function, or in a workflow or task description to describe the processing operations of the function in running the workflow, thereby allowing independent or parallel processing of these multi-dimensional segments.

[0265] [Exclude function instances and tasks from workflow descriptor configuration and policies]

[0266] Embodiments may be directed to a method for excluding function instances and tasks from one or more descriptors defined in a workflow description.

[0267] NBMP Amendment 1 includes the following methods for exclusion functionality:

[0268] [Range Descriptor]

[0269] As shown in Table 24, a descriptor as a parameter range of a descriptor defined in a WDD should be included to indicate a task whose descriptor parameters should be ignored.

[0270] Table 24: Range Descriptors

[0271] parameter type Cardinality blacklist String Array 0.1

[0272] Table 25: JSON schema

[0273]

[0274]

[0275] [Range descriptor parameters]

[0276] Table 26 defines the parameters used in the range descriptor.

[0277] Table 26: Range parameters

[0278]

[0279] There are many problems with the above implementation. For example, the scope descriptor is used to exclude specific descriptors for a subset of functions in the WDD. The scope descriptor is used inside the descriptor and is not an independent descriptor. The scope descriptor applies to some descriptors but not all descriptors. The descriptors that may be used are security, requirements, reporting, and notification. It is not clear whether the function list is the name or identifier of these functions.

[0280] Therefore, there is no clear list of descriptors defined to which a blackout list can be applied. It is not clear whether a blackout is applied to a function, function instance / task in a workflow. A workflow description document (WDD) may or may not include a workflow. If the WDD includes a workflow, the WDD has a connection map of objects and an array of possible function restrictions. If the WDD does not include a workflow, the descriptor is applied to the WDD and functions and tasks are not yet defined. The workflow manager can select different functions from one or more repositories to implement the workflow. Blackout lists are binary flags, which means that if a function is listed in a blackout list of a descriptor, the descriptor should not be applied to the tasks that implement that function.

[0281] Because function restrictions list the restrictions for each function in the workflow, embodiments may involve adding a blacklist to this element.

[0282] For example, the following changes can be made:

[0283] Functional limitations

[0284] a) Other descriptors missing from the above table may be added regardless of the blacklist features.

[0285] b) Blacklist An array can be added to list the descriptors that are excluded from the capability instance.

[0286] Table 27: Functional Restriction Array Elements

[0287]

[0288]

[0289] An example JSON implementation is shown in Table 28.

[0290] Table 28

[0291]

[0292]

[0293] As shown in Table 29, an embodiment may add a simple array of lists of descriptors to exclude for each function instance.

[0294] Table 29: Processing parameters

[0295]

[0296]

[0297] This modification can provide several benefits. For example:

[0298] 1. Act on function instances instead of functions. In a workflow, a function instance can be blacklisted instead of all instances of the function being blacklisted.

[0299] 2. Function restrictions have been listed for different descriptors. If you want a descriptor to interrupt a function instance, you can mark it in this array.

[0300] 3. Since the interrupt is a single parameter, no separate descriptor is required to define it.

[0301] 4. If the WDD does not have the workflow, the WDD can export the workflow and return it to the NBMP in the updated WDD. The NBMP source can then add the interrupt flag to the required function instance / task.

[0302] Embodiments may relate to a method for excluding any descriptor in a workflow description document (WDD) from a function instance, wherein the function instance does not have to implement a specific descriptor, wherein each function instance may have its own exclusion list, and other function instances of the same function may or may not have the same exclusions, wherein an NBMP source may provide the exclusion list in the WDD when providing a workflow, or, if no workflow is provided, the workflow manager may export the workflow and provide it to the NBMP source, which then updates the WDD with the exclusion list for each function instance / task and returns it to the workflow manager for implementation.

[0303] [Exemplary process]

[0304] refer to Figure 3 and Figure 8 , the following describes a process 800 for segmented presentation of a workflow performed by the NBMP workflow manager 320.

[0305] Figure 8 is a flow chart of an example process 800 for reconstructing a current Network Abstraction Layer (NAL) unit for video decoding.

[0306] like Figure 8 As shown, process 800 may include exporting a network-based media processing (NBMP) workflow (block 810).

[0307] like Figure 8 As further shown, process 800 may include obtaining at least one first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is optional (block 820).

[0308] like Figure 8 As further shown, process 800 may include determining a plurality of necessary tasks based on the at least one first syntax element (block 830).

[0309] like Figure 8 As further shown, process 800 may include assigning a plurality of necessary tasks to at least one of a media receiver, a media source, and a media processing entity included in the NBMP workflow (block 840).

[0310] In an embodiment, the at least one first syntax element may include a first flag included in a general descriptor of the at least one task, and the first flag may indicate that the at least one task is non-essential. In an embodiment, the first flag may correspond to the non-essential flag of Table 1.

[0311] In an embodiment, a plurality of necessary tasks may be determined based on the first flag.

[0312] In an embodiment, the at least one first syntax element may include a first parameter included in an output descriptor of the at least one output, and the first parameter may indicate that the at least one output is optional. In an embodiment, the first parameter may correspond to the optional parameter of Table 3.

[0313] In an embodiment, a plurality of necessary tasks may be determined based on the first parameter.

[0314] In an embodiment, process 800 may further include: obtaining a step descriptor including at least one second syntax element indicating parallel processing information of at least one media segment associated with the NBMP workflow; and allocating a plurality of necessary tasks based on the parallel processing information.

[0315] In an embodiment, at least one second syntax element may include a second flag indicating that the duration of the media segment is variable, and a second parameter indicating a maximum duration in the duration. In an embodiment, the second flag may correspond to the above-mentioned element "variable duration", and the second parameter may correspond to the above-mentioned element "segment duration".

[0316] In an embodiment, at least one second syntax element may include a third parameter, the third parameter indicating at least one of: a temporal overlap of at least one media segment, a number of dimensions of at least one media segment, a description of a dimension of at least one media segment, and an overlap of dimensions of at least one media segment. In an embodiment, the third parameter may correspond to at least one parameter shown in Table 21.

[0317] In an embodiment, process 800 may further include obtaining a function restriction list corresponding to an instance of a function included in the NBMP workflow. In an embodiment, the function restriction list may indicate restrictions on the corresponding function. In an embodiment, the function restriction list may include a blacklist array that specifies descriptors that are excluded for instances of functions in a workflow description document corresponding to the NBMP workflow.

[0318] In an embodiment, the function restriction list may be a first function restriction list, and the instance of the function may be a first instance of the function, and a second function restriction list corresponding to the second instance of the function may not include a blacklist array.

[0319] although Figure 8 Example blocks of process 800 are shown, but in some implementations, process 800 may include Figure 8 Additional blocks, fewer blocks, different blocks, or differently arranged blocks than those shown in . Additionally or alternatively, two or more blocks of process 800 may be performed in parallel.

[0320] In addition, the proposed method can be implemented by a processing circuit (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium to perform one or more proposed methods.

[0321] [Exemplary Computer Code]

[0322] According to an embodiment of the present disclosure, at least one processor having a memory storing computer code may be provided. When executed by at least one processor, the computer code may be configured to perform any number of aspects of the present disclosure.

[0323] For example, refer to Fig. 9 , the computer code 900 may be implemented in the NBMP system 300. For example, the computer code may be stored in a memory of the NBMP workflow manager 320 and may be executed by at least one processor of the NBMP workflow manager 320. The computer code may include, for example, a workflow export code 910, a first acquisition code 920, a determination code 930, and an allocation code 940.

[0324] The workflow export code 910, the first acquisition code 920, the determination code 930 and the allocation code 940 may be configured to enable the NBMP workflow manager 320 to respectively execute the above-mentioned reference code 910. Figure 8 Describe various aspects of the process.

[0325] The embodiments of the present disclosure may be used alone or in combination in any order. In addition, each embodiment (and method thereof) may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0326] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the embodiments.

[0327] As used herein, the term component is intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0328] Even though particular combinations of features are set forth in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features may be combined in ways not specifically set forth in the claims and / or disclosed in the specification. Although each dependent claim listed below may only directly depend on one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0329] Unless explicitly described, any element, action or instruction used in the present disclosure should not be understood as critical or necessary. In addition, as used in the present disclosure, the term "one" is intended to include one or more items and can be used interchangeably with "one or more". In addition, as used in the present disclosure, the term "set" is intended to include one or more items (e.g., related items, non-related items, a combination of related and non-related items, etc.), and can be used interchangeably with "one or more". In the case of only one item, the term "one" or similar language is used. In addition, as used in the present disclosure, the term "has" or similar terms are open terms. In addition, the term "based on" means "based at least in part on", unless otherwise explicitly stated.

Claims

1. A method for managing a network-based media processing workflow, the method comprising: Export network-based media processing NBMP workflow; Retrieve at least one first syntax element, the first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is optional; determining a plurality of necessary tasks based on the at least one first grammatical element; Acquire a step descriptor, wherein the step descriptor includes at least one second syntax element, wherein the second syntax element indicates parallel processing information of at least one media segment associated with the NBMP workflow, wherein the parallel processing information includes a continuous execution mode, a state retention mode, and a stateless mode; as well as Based on the parallel processing information, the multiple necessary tasks are assigned to at least one of the media receiver, media source and media processing entities included in the NBMP workflow, wherein the at least one second syntax element includes a second flag indicating that the duration of the media segment is variable and a second parameter indicating a maximum duration among the durations.

2. The method according to claim 1, wherein: The at least one first syntax element comprises a first flag included in a general descriptor of the at least one task, and The first flag indicates that the at least one task is not necessary.

3. The method according to claim 2, wherein: The plurality of necessary tasks are determined based on the first flag.

4. The method according to claim 1, wherein: The at least one first syntax element comprises a first parameter included in an output descriptor of the at least one output, and The first parameter indicates that the at least one output is optional.

5. The method according to claim 4, wherein: The plurality of necessary tasks are determined based on the first parameter.

6. The method according to claim 1, wherein: The at least one second syntax element comprises a third parameter indicating at least one of: a temporal overlap of the at least one media segment, a number of dimensions of the at least one media segment, a description of a dimension of the at least one media segment, and an overlap of dimensions of the at least one media segment.

7. The method according to any one of claims 1 to 6, wherein: The method further includes: obtaining a function restriction list corresponding to the instance of the function included in the NBMP workflow, The function restriction list includes a blacklist array, and the blacklist array specifies descriptors corresponding to instance exclusions of the function in the workflow description document of the NBMP workflow.

8. The method according to claim 7, wherein: The function restriction list is a first function restriction list, the instance of the function is a first instance of the function, and The second function restriction list corresponding to the second instance of the function does not include the blacklist array.

9. A workflow manager for managing a network-based media processing workflow, the workflow manager comprising: at least one processor; as well as A memory comprising computer code comprising: Workflow export code configured to cause the at least one processor to export a network-based media processing NBMP workflow; first acquisition code configured to cause the at least one processor to acquire at least one first syntax element, the first syntax element indicating that at least one task included in the NBMP workflow, at least one input received by the at least one task, or at least one output generated by the at least one task is optional; determining code configured to cause the at least one processor to determine a plurality of necessary tasks based on the at least one first syntax element; and Allocating code configured to cause the at least one processor to obtain a step descriptor, the step descriptor comprising at least one second syntax element, the second syntax element indicating parallel processing information of at least one media segment associated with the NBMP workflow, the parallel processing information comprising a continuous execution mode, a state retention mode, and a stateless mode; and Based on the parallel processing information, the multiple necessary tasks are assigned to at least one of the media receiver, media source and media processing entities included in the NBMP workflow, wherein the at least one second syntax element includes a second flag indicating that the duration of the media segment is variable and a second parameter indicating a maximum duration among the durations.

10. The workflow manager according to claim 9, wherein: The at least one first syntax element comprises a first flag included in a general descriptor of the at least one task, and The first flag indicates that the at least one task is not necessary.

11. The workflow manager according to claim 10, wherein: The determination code is further configured to cause the at least one processor to determine the plurality of necessary tasks based on the first flag.

12. The workflow manager according to claim 9, wherein: The at least one first syntax element comprises a first parameter included in an output descriptor of the at least one output, and The first parameter indicates that the at least one output is optional.

13. The workflow manager according to claim 12, wherein: The determining code is further configured to cause the at least one processor to determine the plurality of necessary tasks based on the first parameter.

14. The workflow manager according to claim 9, wherein: The at least one second syntax element comprises a third parameter indicating at least one of: a temporal overlap of the at least one media segment, a number of dimensions of the at least one media segment, a description of a dimension of the at least one media segment, and an overlap of dimensions of the at least one media segment.

15. The workflow manager according to any one of claims 9 to 14, wherein: The computer code further includes a third acquisition code configured to cause the at least one processor to acquire a first function restriction list corresponding to a first instance of a function included in the NBMP workflow, wherein the first function restriction list includes a blacklist array, the blacklist array specifies a descriptor corresponding to a first instance of the function in a workflow description document of the NBMP workflow that is excluded, and The second function restriction list corresponding to the second instance of the function does not include the blacklist array.

16. A non-transitory computer-readable medium storing computer code, wherein the computer code is configured to, when executed by at least one processor of a workflow manager for managing a network-based media processing workflow, cause the at least one processor to perform the method according to any one of claims 1 to 8.

17. A computer device comprising: a memory configured to store computer program code; as well as At least one processor is configured to access the computer program code and operate according to instructions of the computer program code to perform the method according to any one of claims 1 to 8.

Images