Method and apparatus for stepwise workflow

Abstract

Aspects of the disclosure provide methods, apparatuses, computer systems, and storage media for processing a workflow. In one aspect, a method for processing a workflow includes receiving workflow information of a workflow. The workflow is generated based on the workflow information. The workflow includes a first buffer task and a plurality of processing tasks. The plurality of processing tasks includes a first processing task and a second processing task. The first processing task is brought into a running state, in which the first processing task processes a first subset of input data and outputs the processed first subset as first processed subset data to the first buffer task and stores the first processed subset data in the first buffer task. The first processing task is transitioned from the running state to a non-running state based on an amount of the first processed subset data stored in the first buffer task being equal to a first threshold. Subsequently, the second processing task is brought into the running state, in which the first processed subset data stored in the first buffer task is processed by the second processing task.

Description

[0001] Incorporation by Reference

[0002] This application claims priority to U.S. Patent Application No. 17 / 024,193, “Method and Apparatus for a Step-Enabled Workflow,” filed September 17, 2020, which claims priority to U.S. Provisional Application No. 62 / 907,627, “Step-Enabled Stateless Workflows for Cloud Computing,” filed September 28, 2019, and U.S. Provisional Application No. 62 / 907,625, “Indexed FIFO Functions for Segmented Processing in Cloud,” filed September 28, 2019. The entire disclosures of these prior applications are hereby incorporated by reference in their entirety. TECHNICAL FIELD

[0003] Embodiments described herein generally relate to data processing, including processing of one or more workflows. BACKGROUND

[0004] The background description provided herein is intended to present the context of the present application. As such, work performed by the inventors in their respective fields before the filing of the present application, as well as their contributions to the art, are not presented as prior art against the present application. Work performed by the inventors in their respective fields before the filing of the present application, as well as their contributions to the art, are not presented as prior art against the present application.

[0005] Workflows can be used for data processing to process data, such as media data, in a data processing system. In some examples, a workflow includes a large number of processing tasks. When a large number of tasks are run in parallel, a data processing system needs to invest a large amount of resources. SUMMARY

[0006] Aspects of the disclosure provide a method and apparatus for processing one or more workflows. The apparatus includes processing circuitry. The processing circuitry is configured to receive workflow information of a workflow. The processing circuitry generates the workflow based on the workflow information to process input data. The workflow includes a plurality of processing tasks and a first buffering task. The plurality of processing tasks includes a first processing task and a second processing task. The processing circuitry causes the first processing task to enter a running state in which a subset of the input data is processed by the first processing task and output as first processed subset data to the first buffering task. The processing circuitry causes the first processing task to transition from the running state to a non-running state based on an amount of the first processed subset data stored in the first buffering task being equal to a first threshold. After the first processing task transitions to the non-running state, the processing circuitry causes the second processing task to enter a running state in which the first processed subset data stored in the first buffering task is processed by the second processing task.

[0007] In one embodiment, the second processing task is configured to output the first processed subset data as second processed subset data to a second buffering task in the running state. The processing circuitry causes the second processing task to transition from the running state to a non-running state based on one of (i) an amount of the second processed subset data stored in the second buffering task being equal to a second threshold, and (ii) the amount of the first processed subset data stored in the first buffering task being provided to the second processing task.

[0008] In one embodiment, the plurality of processing tasks are configured with respective task segment durations (TSDs). Each of the plurality of processing tasks is configured to process data in an amount equal to the respective TSD independently of data outside the TSD. The processing circuitry determines a workflow segment duration (WSD) as a least common multiple of the TSDs. The first threshold is based on (i) the TSDs, (ii) the WSD, and (iii) one of the WSD and a parameter S indicating a number of steps in which the first processing task is configured to run consecutively.

[0009] In one embodiment, the first buffering task is an indexed first-in-first-out (iFIFO) task configured with an iFIFO buffer. The iFIFO task is configured to store buffered segments of the input data. Each buffered segment is indexed with a start time, a duration, and a length. An iFIFO buffer size is based on the length and a number of buffered segments in the iFIFO buffer. The iFIFO task is configured to generate an event indicating that an amount of the first processed subset data stored in the iFIFO task is equal to the first threshold. The iFIFO task is configured to generate another event indicating that the amount of the first processed subset data stored in the iFIFO task is provided to the second processing task.

[0010] In one embodiment, the first buffering task is a multiple-indexed first-in-first-out (MiFIFO) task configured with a plurality of iFIFO buffers. The MiFIFO task is configured to provide one or more outputs for the first processing task. Each of the plurality of iFIFO buffers is configured to store buffered segments of the input data. The buffered segments are from a respective one of the one or more outputs. Each buffered segment is indexed with a start time, a duration, and a length. A buffer size of each iFIFO buffer is based on the length and a number of buffered segments in the respective iFIFO buffer. The MiFIFO task is configured to generate an event indicating that an amount of the first processed subset data stored in the MiFIFO task is equal to the first threshold. The MiFIFO task is configured to generate another event indicating that the amount of the first processed subset data stored in the MiFIFO task is provided to the second processing task.

[0011] In one embodiment, when the first processing task is in the non-running state, state information of the first processing task is not stored, the non-running state including one of an idle state and a destroyed state.

[0012] In one embodiment, the workflow includes a plurality of buffering tasks, the plurality of buffering tasks including the iFIFO task. Each of the plurality of processing tasks inputs from the input data and one or more of the plurality of buffering tasks. Only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

[0013] In one embodiment, the workflow includes a plurality of buffering tasks. The plurality of buffering tasks includes the MiFIFO task. Each of the plurality of processing tasks inputs from one of the input data and one or more of the plurality of buffering tasks. Only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

[0014] In one embodiment, the input data includes the subset of the input data and another subset of the input data. The workflow includes a plurality of buffering tasks, the plurality of buffering tasks including the MiFIFO task. Each of the plurality of processing tasks inputs from one of the input data and one or more of the plurality of buffering tasks. Processing circuitry sequentially processes the subset of the input data through the plurality of processing tasks. After the subset of the input data is processed by the plurality of processing tasks, processing circuitry sequentially processes the another subset of the input data through the plurality of processing tasks, the remaining ones of the plurality of processing tasks being in a non-running state. Only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

[0015] The processing circuitry is configured to receive workflow information of the plurality of workflows. The processing circuitry generates the plurality of workflows based on the workflow information to process input data. At least one of the plurality of workflows includes a plurality of processing tasks, the plurality of workflows including a first workflow and a second workflow. The processing circuitry causes the first workflow to enter a running state, where in the running state, a subset of the input data is processed by the first workflow and output as first processed subset data to a buffer task. The processing circuitry causes the first workflow to transition from the running state to a non-running state based on an amount of the first processed subset data stored in the buffer task being equal to a threshold value. After the first workflow transitions to the non-running state, the processing circuitry causes the second workflow to enter a running state, where in the running state, the first processed subset data stored in the buffer task is processed by the second workflow.

[0016] In one embodiment, the non-running state of the first workflow includes one of an idle state and a destroyed state. When the first workflow is in the non-running state, state information of the first workflow is not stored. The first workflow is configured to process the subset of the input data independently of a remaining subset of the input data other than the first subset.

[0017] In one embodiment, the buffer task is an iFIFO task configured with an iFIFO buffer. The iFIFO task is configured to store buffered segments of the input data. Each buffered segment is indexed with a start time, a duration, and a length. An iFIFO buffer size is based on the length and a number of buffered segments in the iFIFO buffer. The iFIFO task is configured to generate an event indicating that the amount of the first processed subset data stored in the iFIFO task is equal to the threshold value. The iFIFO task is configured to generate another event. The other event indicates that the amount of the first processed subset data stored in the iFIFO task is provided to the second workflow.

[0018] In one embodiment, the buffering task is a multiple-index first-in-first-out (MiFIFO) task configured with a plurality of iFIFO buffers. The MiFIFO task is configured to provide one or more outputs for the first workflow. Each of the plurality of iFIFO buffers is configured to store a buffered segment of the input data. The buffered segment is from a respective one of the one or more outputs. Each buffered segment is indexed with a start time, a duration, and a length. A buffer size of each iFIFO buffer is based on the length and a number of buffered segments in the respective iFIFO buffer. The MiFIFO task is configured to generate an event. The event indicates that an amount of the first processed subset data stored in the MiFIFO task is equal to the threshold. The MiFIFO task is configured to generate another event. The other event indicates that the amount of the first processed subset data stored in the MiFIFO task is provided to the second workflow.

[0019] Various aspects of the present disclosure also provide a non-transitory computer readable medium storing instructions that, when executed by a computer, cause the computer to perform any of the methods for processing one or more workflows. BRIEF DESCRIPTION OF DRAWINGS

[0020] Other features, properties, and various advantages of the disclosed subject matter will become further apparent during the following detailed description, the appended claims and the drawings, in which:

[0021] Figure 1 An exemplary data processing system is shown in accordance with an embodiment of the present disclosure;

[0022] Figures 2A-2C An example of a workflow is shown in accordance with an embodiment of the present disclosure;

[0023] Figure 2D An example of a workflow is shown in accordance with an embodiment of the present disclosure;

[0024] Figure 3 An example of a task segment duration (TSD) for a task is shown in accordance with an embodiment of the present disclosure;

[0025] Figure 4A An exemplary task life cycle of a processing task is shown in accordance with an embodiment of the present disclosure;

[0026] Figure 4B An exemplary task life cycle of a workflow is shown in accordance with an embodiment of the present disclosure;

[0027] Figure 5 An example of a workflow is shown in accordance with an embodiment of the present disclosure;

[0028] Figure 6A flowchart outlining a method according to embodiments of the disclosure is shown;

[0029] Figure 7 A flowchart outlining a method according to embodiments of the disclosure is shown;

[0030] Figure 8 A flowchart outlining a method according to embodiments of the disclosure is shown;

[0031] Figure 9 A schematic diagram of a computer system according to embodiments is shown. DETAILED DESCRIPTION

[0032] Aspects of the disclosure relate to data processing (e.g., media processing in a network). A data processing source (e.g., a network-based media processing (NBMP) source) can describe a requested data processing (e.g., a requested media processing) and provide control information for data to be processed (e.g., media data). Thus, a workflow manager (e.g., a NBMP workflow manager) can establish a workflow (e.g., a media processing workflow, a NBMP workflow), and can inform the data processing source (e.g., the NBMP source) that the workflow is ready and, thus, the data processing can begin. One or more data sources (e.g., one or more media sources) can then begin transmitting data to, for example, a network for processing.

[0033] Generally, a workflow can include processing tasks or tasks (e.g., media processing tasks). A connection graph can represent a workflow by using connections to represent input / output relationships between processing tasks. Thus, a workflow can be considered as a connection graph of processing tasks. Each task can perform an operation (e.g., a media processing operation), such as video decoding, video stitching, video encoding, etc. A workflow manager can ensure correct operation of a workflow by configuring and monitoring each task and workflow output. The workflow manager can be configured to select processing functions (e.g., media processing functions) and instantiate the processing functions (e.g., media processing functions) as corresponding tasks based on, for example, a workflow description received from a data processing source (e.g., a NBMP source).

[0034] Processing entities (e.g., media processing entities) configured to run processing tasks are established, loaded, instantiated, and monitored. Interfaces, such as application programming interfaces (APIs), can be defined between data processing sources and a workflow manager, the workflow manager and one or more processing tasks. APIs for discovering one or more appropriate functions can also be defined. Generally, a data processing system or platform (e.g., an NBMP platform) can be data format (e.g., media format) and protocol agnostic. The data processing system can identify and signal data (e.g., media data), metadata, and auxiliary information formats for data exchanged between data sources (e.g., media sources), a workflow manager, and tasks.

[0035] A data processing framework (e.g., an NBMP framework) can define interfaces including both data formats and APIs between various entities for data processing connected through a digital network. Users can remotely access and configure user operations for efficient and intelligent processing. For example, data can be uploaded to a network, processing tasks can be instantiated, and processing tasks can be further configured. The data processing framework can enable dynamic creation of data processing pipelines (e.g., media processing pipelines) and real-time or delayed access to processed data and metadata. Data and metadata formats can be specified in a data processing pipeline. Data and metadata formats are used between data sources, a workflow manager, and data processing entities.

[0036] A data processing framework can enable clients (e.g., creators, service providers, and consumers of digital media) to describe data processing operations to be performed by data processing entities, for example, in a network. A workflow can be described by composing a set of data processing functions accessible through an interface (e.g., an NBMP API). Data processing entities (e.g., media processing entities (MPEs)) can run processing tasks or other tasks that apply to data and related metadata received from one or more data sources. Data processing entities (e.g., MPEs) can provide capabilities for configuring, managing, and monitoring processing tasks. A data processing task can be a process that applies to data and one or more metadata inputs, produces data outputs and related metadata outputs for use by a data sink (e.g., a media sink) or other data processing tasks (e.g., media processing tasks).

[0037] A data processing framework can support various delivery methods, such as streaming, file delivery, push-based progressive download, hybrid delivery, multi-path, and heterogeneous network environments.

[0038] Figure 1An exemplary data processing system (e.g., NBMP system, NBMP reference architecture, NBMP architecture) (100) is shown in accordance with embodiments of the present disclosure. The data processing system (100) can include a plurality of entities, such as a data source (e.g., NBMP source, NBMP source entity) (101), a workflow manager (e.g., NBMP workflow manager) (103), a function repository (105), a data source or data source entity (e.g., media source, media source entity) (111), a data processing entity (e.g., MPE) (113), a data sink or data sink entity (e.g., media sink or media sink entity) (115), a third party entity, etc. The data processing system (100) can include one or more additional data sources, one or more data sinks, and / or data processing entities. The data processing system (100) can process data across one or more processing entities in a network. Information, such as various data and control information (or control data) for the data, can be communicated between the plurality of entities in the data processing system (100).

[0039] To provide context for discussion purposes, the data processing system (100) is described below as an NBMP system (100). The description can be adapted as appropriate for any data processing system.

[0040] The NBMP source (101) can describe or otherwise indicate media processing in a network. The function repository (105) can include NBMP function descriptions of various NBMP functions. The NBMP source (101) and the workflow manager (103) can retrieve NBMP function descriptions or functions from the function repository (105). An NBMP function can refer to an implementation of an independent and self-contained media processing operation and / or a corresponding description of the operation.

[0041] A processing task or task can refer to a runtime instance of an NBMP function executed by an MPE (113). An NBMP workflow or workflow can be represented by a graph (e.g., directed acyclic graph (DAG)) of one or more connected tasks. The one or more connected tasks implement a requested media processing. The workflow manager (103) can provide one or more tasks and connect one or more tasks to create, control, manage, and monitor a workflow based on, for example, a workflow description document (WDD).

[0042] The media source (111) can provide media content (e.g., media data, supplemental information) to be processed by a workflow. The supplemental information can include metadata or auxiliary information related to the media data. The media source (111) can provide input to a workflow. The media sink (115) can consume output of a workflow. The MPE (113) can run one or more media processing tasks to process the media content.

[0043] Different entities in the NBMP system (100) (e.g., the NBMP source (101), the workflow manager (103), and the MPE (113)) can use APIs to invoke and respond to media service requests. The APIs can include an NBMP workflow API or workflow API, a function discovery API, and a task API. The workflow API can provide an interface between the NBMP source (101) and the workflow manager (103). The task API can provide an interface between the workflow manager (103) and media processing tasks. The function discovery API can provide an interface between the workflow manager (103) / NBMP source (101) and the function repository (105).

[0044] The above-described NBMP interfaces can be used to create and control media processing workflows in a network. The NBMP system (100) can be split into a control plane and a media plane (or media data plane). The control plane can include the workflow API, the function discovery API, and the task API.

[0045] The NBMP source (101) can use the workflow API to create and control media processing workflows. The NBMP source (101) can use the workflow API to communicate with the workflow manager (103) to configure and control media processing in a network. When the NBMP source (101) sends a request to the workflow manager (103) by including a workflow resource (WR) in an operation of the workflow API, the workflow manager (103) can parse the WR, the included WDD, and the corresponding descriptor, and take appropriate action according to the requested operation. The workflow manager (103) can then acknowledge the request with a response. Workflow API operations can include creating a workflow (e.g., CreateWorkflow), updating a workflow (e.g., UpdateWorkflow), deleting a workflow (e.g., DeleteWorkflow), retrieving a workflow (e.g., RetrieveWorkflow), and the like.

[0046] The function discovery API can provide the workflow manager (103) and / or the NBMP source (101) with a way to discover media processing functions that can be loaded as part of a media processing workflow.

[0047] The workflow manager (103) can use a task API to configure and monitor one or more tasks (e.g., task 1 and task 2 run by the MPE (113)) at runtime. For example, after allocating resources for the tasks in the MPE (113), the task API can define one or more interfaces for configuring the media processing tasks by the workflow manager (103). The task API operations can include creating a task (e.g., CreateTask), updating a task (e.g., UpdateTask), getting a task (e.g., GetTask), deleting a task (e.g., DeleteTask), and the like.

[0048] On the media plane, the media format, metadata, and supplemental information format between the NBMP source (111) and one or more tasks and between the tasks can be defined.

[0049] A workflow description (WD) can be passed from the NBMP source (101) to the workflow manager (103). The WD can describe information such as input data and output data, functions, and other requirements of the workflow.

[0050] The workflow manager (103) can receive the WDD from the NBMP source (101) and can construct a workflow for the requested media processing. During the workflow process, the media processing functions can be selected, for example, from the function repository (105), and then the corresponding media processing tasks can be configured and distributed to a set of MPEs (e.g., including the MPE (113)) containing one or more MPEs.

[0051] The function repository (105) can provide a set of functions that can be read by the NBMP source (101) and the workflow manager (103). In an embodiment, the NBMP source (101) uses a set of functions in the function repository (105) to request creation of a workflow. Thus, the NBMP source (101) is configured to select functions for the workflow. The NBMP source (101) can request creation of a workflow as described below. The NBMP source (101) can use a description for the media processing tasks to be created for the workflow and can specify a connection graph to define connections of inputs and outputs of the media processing tasks. When the workflow manager (103) receives the above information from the NBMP source (101), the workflow manager (103) can instantiate the media processing tasks based on the corresponding function names and can connect the media processing tasks according to the connection graph.

[0052] Optionally, the NBMP source (101) can use a set of keywords to request creation of a workflow, which the workflow manager (103) can use to construct the workflow. Thus, the NBMP source (101) can not know the set of functions to be inserted into the workflow. The NBMP source (101) can request creation of a workflow as described below. The NBMP source (101) can use the set of keywords for the workflow manager (103) to find appropriate functions, and can use an appropriate workflow description to specify the requirements of the workflow.

[0053] When the workflow manager (103) receives the above information (e.g., the set of keywords) from the NBMP source (101), the workflow manager (103) can create the workflow by searching for appropriate functions using the keywords. The keywords are specified, for example, in processing descriptors. The workflow manager (103) can then use other descriptors in the workflow description to provide media processing tasks and connect the media processing tasks to create the final workflow.

[0054] The processing model of the workflow manager (103) can be described as follows.

[0055] The workflow manager (103) can discover available media processing functions as follows. The NBMP function repository (105) can provide a function discovery interface (or API) to allow external entities to query for media processing functions that can implement a requested processing. The workflow manager (103) can access a directory service that provides a searchable list of media processing functions. The workflow manager (103) can use the descriptions of the media processing tasks in the workflow description to find appropriate functions for the workflow.

[0056] Selection of media processing tasks for a workflow can be described below. When a request for media processing is received from the NBMP source (101), the workflow manager (103) can search the function repository (105) to find a list of all available functions that can implement the workflow. Using the workflow description from the NBMP source (101), the workflow manager (103) can find functions from the function library (105) to implement the workflow, which can depend on the information for media processing from the NBMP source (101). The information for media processing can include the input description and output description, a description of the requested processing, and information in other descriptors of functions in the function directory (105). Mapping the source request to appropriate media processing tasks to be included in the workflow can be part of the implementation of NBMP in a network. To reference and link input sources with input port names and output port names at task creation, input ports and output ports can be used to reference input streams.

[0057] The search for appropriate functions to be instantiated as tasks can be performed by the workflow manager (103) using the function discovery API. Optionally, the workflow manager (103) can use the function discovery API to retrieve detailed information of certain appropriate functions or detailed information of all appropriate functions in the function repository (105). The workflow manager (103) can then compare the information for media processing from the NBMP source (101) with different descriptors of each function.

[0058] The selected media processing tasks can be configured in the workflow. When the functions to be included in the workflow are identified, the NBMP workflow manager (103) can instantiate the functions as corresponding tasks and configure the tasks in order to add the tasks to the workflow. The NBMP workflow manager (103) can extract configuration data from the media processing information received from the NBMP source (101) and configure the corresponding tasks. The configuration of the tasks can be performed using a task API (e.g., the NBMP task API).

[0059] The task configuration and allocation can be described as follows. The workflow manager (103) can use the workflow to perform processing deployment and configure the media processing entities. In an example, for a compute-intensive media processing request, the workflow manager (103) can establish multiple compute instances and allocate workload among the multiple compute instances. Thus, the workflow manager (103) can connect and configure multiple compute instances as needed. In an example, the workflow manager (103) allocates the same task to multiple instances and provides a load balancer to allocate workload among the multiple instances using a selected scheduling mechanism. In an optional example, the workflow manager (103) allocates different operations of the same task to different instances (e.g., parallel operations). In both of the above examples, the workflow manager (103) can establish a workflow path among the instances and thus can successfully achieve suitable workload. The workflow manager (103) can configure the tasks to push the processed media data / streams (or make them available through a pull mechanism) to the next task in the workflow graph.

[0060] When the workflow manager (103) receives the WDD from the NBMP source (101), the workflow manager (103) can perform selection of media processing functions to be inserted in the workflow. When the list of tasks to be included in the workflow is compiled, the workflow manager (103) can then connect the tasks to prepare the workflow.

[0061] The workflow manager (103) can generate a workflow represented, for example, by a graph (e.g., a DAG) from the WDD. Figure 2AAn example of a graph (e.g., a DAG) (200A) is shown in accordance with embodiments of the present disclosure. The DAG (200A) can include a plurality of nodes (T1) through (T6) and a plurality of links (or connections) (202) through (208). In an example, the DAG (200A) represents a workflow (200A).

[0062] Each node of the DAG (200A) can represent a media processing task in the workflow (200A). In other embodiments, various types of processing tasks can be implemented that are not limited to media processing. A link (e.g., link (202)) connecting a first node (e.g., node (T1)) to a second node (e.g., node (T2)) in the DAG (200A) can represent a transfer of an output of the first node (e.g., node (T1)) as an input of the second node (e.g., node (T2)).

[0063] Generally, a workflow can include any suitable number of one or more inputs (or one or more workflow inputs) and any suitable number of one or more outputs (or one or more workflow outputs). The one or more workflow inputs can be connected to a media source (111), one or more other workflows, and the like, and the one or more workflow outputs can be connected to a media sink (115), one or more other workflows, and the like. The workflow (200A) has an input (201) and outputs (209) and (210). In some embodiments, the workflow (200A) can have one or more outputs from intermediate nodes.

[0064] Figure 4A An example task life cycle (400) for a media processing task is shown in accordance with embodiments of the present disclosure. The task life cycle (400) can be applied to other types of processing tasks. A media processing task can transition among different states at different points in the execution of the media processing task. Different states in the task life cycle (400) can include an instantiation state (401), an idle state (402), a running state (403), an error state (404), a destruction state (405), and the like.

[0065] Instantiation State (401): When the workflow manager (103) instantiates a media processing task, e.g., by using a service of the infrastructure manager as represented by the onlnstantiation transition, the media processing task is in the instantiation state (401). When the media processing task is configured, e.g., as represented by the onTaskConfiguration transition, the instantiation state (401) can transition (or change) to the idle state (402). Optionally, if the media processing task terminates, e.g., as represented by the onTermination transition, the media processing task can transition to the destruction state (405).

[0066] Idle State (402): When the media processing task is in the instantiation state (401) and the workflow manager (103) can perform task configuration, e.g., as represented by the onTaskConfiguration transition, the media processing task can transition to the idle state (402). In the idle state (402), the media processing task is configured with the appropriate processing. When the media processing task starts, e.g., as represented by the onStart transition, the media processing task can transition to the running state (403). Optionally, in the idle state (402), the media processing task is reconfigured and remains in the idle state (402). In the idle state (402), if the media processing task terminates, e.g., as represented by the onTermination transition, the media processing task can transition to the destruction state (405). In the idle state (402), if the media processing task is reset, e.g., as represented by the onReset transition, the media processing task can transition to the instantiation state (401).

[0067] Running state (403): When the media processing task is in the idle state (402) and the media processing task is started (e.g., using an onStart transition), the media processing task can transition from the idle state (402) to the running state (403). In the running state (403), the media processing task assigned to the MPE (113) can process data (e.g., media data from the media source (111)) received by the media processing task from a previous media processing task in the workflow or the NBMP source (101). Optionally, in the running state (403), the media processing task can remain in the running state (403) if the workflow manager (103) can perform a reconfiguration on the media processing task (e.g., as represented using an onTaskConfiguration transition) and if the reconfiguration results in a processing reconfiguration on the current media / metadata stream of the media processing task. In the running state (403), the media processing task can transition to the idle state (402) if the media processing task is stopped (e.g., as represented using an onStop transition) or completed (e.g., as represented using an onCompletion transition). In the running state (403), the media processing task can transition to the error state (404) if the media processing task encounters an error (e.g., as represented using an onError transition). In the running state (403), the media processing task can transition to the destroyed state (405) if the media processing task is terminated (e.g., as represented using an onTermination transition).

[0068] Error state (404): When the media processing task encounters an error and is unable to process media data or metadata, the media processing task can be in the error state (404). In processing the error (e.g., as represented using an onErrorHandling transition), the media processing task can move back to the idle state (402). Optionally, when in the error state (404), the media processing task can be reset (e.g., as represented using an onReset transition) such that the media processing task can transition to the instantiated state (401). In the error state (404), the media processing task can transition to the destroyed state (405) if the media processing task is terminated.

[0069] Destroyed state (405): When the workflow manager (103) terminates the media processing task, the media processing task can be in the destroyed state (405). The media processing task can be discarded and cannot be reused.

[0070] The status of a media processing task can be reflected in the'status' parameter of a generic descriptor. In some examples, each of the above transitions, except for the onError transition, can occur as a result of a task operation initiated by the workflow manager (103). The onError transition can occur as a result of an internal state change of a media processing task.

[0071] Figure 4B An exemplary workflow lifecycle (410) of a workflow is shown in accordance with embodiments of the present disclosure. The workflow can transition among different states at different points of execution of the workflow. The different states in the workflow lifecycle (410) can include an instantiation state (411), an idle state (412), a running state (413), an error state (414), a destruction state (415), and the like.

[0072] Instantiation state (411): The workflow manager (103) can place the workflow in the instantiation state (411) (e.g., as represented using an onInstantiation transition), for example, through a service of the infrastructure manager. When the workflow is configured (e.g., as represented using an onWorkflowConfig transition), the workflow can transition to the idle state (412). Optionally, if the workflow is terminated while in the instantiation state (e.g., as represented using an onTermination transition) (411), the workflow can transition to the destruction state (415).

[0073] Idle state (412): When the workflow is in the instantiation state (412) and the workflow manager (103) executes the workflow configuration (e.g., as represented using an onWorkflowConfig transition), the workflow can transition to the idle state (412). In the idle state (412), one or more MPEs are set up (e.g., via the infrastructure manager) and tasks running in the MPEs are provided and configured. When the workflow starts (e.g., as represented using an onStart transition), the workflow can transition to the running state (413). Optionally, in the idle state (412), the workflow can be reconfigured. The workflow can remain in the idle state (412) waiting for media data or metadata to arrive. In the idle state (412), if the workflow is terminated (e.g., as represented using an onTermination transition), the workflow can transition to the destruction state (415). In the idle state (412), if the workflow is reset (e.g., as represented using an onReset transition), the workflow can transition to the instantiation state (411).

[0074] Running state (413): When the workflow is in the idle state (412) and the workflow starts (e.g., using an onStart transition), the workflow can transition from the idle state (412) to the running state (413). In the running state (413), data from the NBMP source (101) can be processed by the MPEs in the workflow. Optionally, in the running state (413), if the workflow manager (103) performs reconfiguration on the workflow (e.g., as represented using an onWorkflowConfig transition) and if the reconfiguration results in performing processing reconfiguration on the current media / metadata stream, the workflow can remain in the running state (413). In the running state (413), if the workflow stops (e.g., as represented using an onStop transition) or the processing completes (e.g., as represented using an onComplete transition), the workflow can transition to the idle state (412). In the running state (413), if the workflow encounters an error (e.g., as represented using an onError transition), the workflow can transition to the error state (414). In the running state (413), if the workflow terminates (e.g., as represented using an onTermination transition), the workflow can transition to the destroyed state (415).

[0075] Error state (414): When the workflow encounters an error and cannot continue the workflow processing, the workflow is in the error state (414). In processing the error (e.g., as represented using an onErrorHandling transition), the workflow can move back to the idle state (412). Optionally, while in the error state (414), the workflow can be reset (e.g., as represented using an onReset transition) such that the workflow can transition to the instantiated state (411). In the error state (414), if the workflow terminates, the workflow can transition to the destroyed state (415).

[0076] Destroyed state (415): When the workflow manager (103) terminates the workflow, the workflow is in the destroyed state (415). The workflow can be instantiated for the workflow to be used again.

[0077] In some examples, each of the above transitions, except for the onError transition, can occur as a result of a workflow operation initiated by the NBMP source (101). The onError transition can occur as a result of an internal state change of the workflow.

[0078] In examples, the concept of processing media on the cloud was developed by the NBMP project. However, the NBMP design does not provide a single method for monitoring, reporting, and notification. In some examples, a message format for reporting or notification is not defined.

[0079] In some examples, a portion of the NBMP workflow is not allowed to run, so all tasks in the workflow will be instantiated for workflow operation, and all tasks will work at the same time. Thus, the underlying platform allocates resources (e.g., compute resources, memory resources, bandwidth) to all tasks in the workflow at the same time. The large allocation of resources can not be available, or can be available but at a high cost.

[0080] To improve media processing efficiency, deploy media services faster and at a lower cost, and enable large-scale deployment by leveraging public, private, or hybrid cloud services, aspects of the present disclosure provide methods and apparatuses that allow tasks in a workflow to operate or run one step at a time and / or one group of steps at a time. The above-mentioned functional mode or step-by-step mode is advantageous, for example, when there are not enough resources (e.g., memory resources, compute resources) to run the entire workflow at the same time. The methods and embodiments disclosed in the present disclosure can be used for any suitable workflow implementation on a cloud computing or any suitable platform utilizing one or more workflows.

[0081] According to aspects of the present disclosure, a step-by-step mode function, such as a buffering function (e.g., a first-in-first-out function, a variant of the first-in-first-out function, as described below), can be added to a workflow. The step-by-step mode function can enable step-based operation or step-by-step operation for tasks in one or more workflows of a plurality of workflows. Thus, data is processed using multiple steps. Using the step-by-step mode, a workflow can run tasks individually, and thus a subset of tasks in the workflow can run at the same time while the remaining task or tasks do not run. In an example, a workflow includes six tasks. In an example, the subset of tasks includes one task, and thus one task can process data in a running state while the other five tasks do not run. The other five tasks can be in a non-running state (e.g., an idle state and / or a destroyed state, as described below) and do not process data. In another example, the subset of tasks includes two tasks, and thus the two tasks can run at the same time while the other four tasks do not run.

[0082] In another example, a plurality of workflows includes a first workflow and a second workflow. When the first workflow is running to process data, the second workflow is in a non-running state (e.g., an idle state or a destroyed state, as described below) and does not process data.

[0083] In an embodiment, media data to be processed by tasks in a workflow can include a plurality of subsets of media data. According to aspects of the present disclosure, the media data can be processed in one or more of the plurality of subsets, and the media data can be processed in a step-by-step mode (or step-by-step mode). A step can be one or more of the plurality of subsets.

[0084] The maximum step size can be the entire media data, and the media data is processed in a single step. In an example, the media data includes 100 frames, and includes 10 subsets. Each subset includes 10 frames. The media data can be processed in a single step with a step size of 100 frames. Alternatively, the media data can be processed in 10 steps with a step size of 10 frames. In some embodiments, the step size does not need to be fixed. For example, the step size of the last step can be smaller than other steps in the workflow.

[0085] Each task can run in one or more steps, and one of the subsets of the media data can be processed in one step. Certain tasks can run for multiple steps before other steps can process or start running.

[0086] In each step of the task execution, a portion of the media data (e.g., one of the subsets, such as 10 frames) can be processed, and an output (also referred to as a task output) of the task with a finite duration (or finite length, finite size) can be generated. The task can then stop (e.g., the task transitions to a non-running state, such as an idle state or a destroyed state as described below). According to various aspects of the present disclosure, a task is allowed to run in a stateless mode, where in the stateless mode, no state information (e.g., internal state and data) of the task in the non-running state is stored. For example, a task in the non-running state can be terminated or placed in an idle state. In an example, the non-running state is an idle state or a destroyed state.

[0087] In some examples, the resources allocated for the task can be released. The next task can then transition from the non-running state to the running state to process data at the input of the next task and generate an output with a finite duration. Subsequently, the next task can transition to a non-running state, where no state information (e.g., internal state and data) of the next task is stored, and the flow continues to execute another task.

[0088] The benefits of the step-wise (or step-based) processing of a workflow can be described below. The workflow can be executed in a resource-limited or resource-reduced situation as only one or a few tasks in the workflow are operating (or running) at the same time and other tasks in the workflow are stopped. By stopping the workflow during high traffic time of using a cloud service and / or expensive daytime hours, the operational cost can be reduced. For example, the workflow can be stopped during high traffic time, such as during high traffic on the Internet. The workflow can be fully serialized, e.g., an initial task is executed to process the entire media data (e.g., a movie) while other tasks in the workflow are not running, and moving to the next task to process the entire media data while other tasks in the workflow are not running. For example, the size of a step (or the amount of data to be processed in one step) can be the entire media data. Thus, the initial task (T1) can run or process the entire movie in one step. Tasks with various computational complexities can be executed on the same hardware. The steps of simpler tasks can be executed faster while the execution of steps of more complex tasks can take more time. The use of stateless mode can be advantageous in certain computing platforms, e.g., cloud platforms. Wherein in a cloud platform, a task can be put in a non-running state without storing state information (e.g., internal state) of the task.

[0089] Aspects of the present disclosure provide a segment processing mode of functioning and running tasks based on the step model described above. The functions or corresponding tasks can be designed to support the segment processing mode, e.g., to process a certain amount of data.

[0090] Aspects of the present disclosure describe a buffering function (e.g., an index first-in-first-out (FIFO) (iFIFO) function). A buffer can be configured for the buffering function, e.g., an iFIFO buffer for an iFIFO function. In an embodiment, the buffering function (e.g., the iFIFO function) can operate on segments (also referred to as buffered segments), thus enabling the segment processing mode. According to the present disclosure, the iFIFO function can store index buffered segments (also referred to as entries) of data such as a bitstream. The iFIFO function can have the following properties: (i) a FIFO buffer, wherein the first entry stored in the FIFO buffer is the first entry read out of the FIFO buffer, and (ii) the unit of operation in the FIFO is an entry. Each entry has the following attributes: a start time s i (e.g., in milliseconds (ms)), a duration d i (e.g., in ms), and a length l i (e.g., in bytes), where integer i indicates the position of the entry in the iFIFO buffer. In an example, each entry (or each buffered segment) is tagged with the corresponding start time s i , duration d i , and length li Index is (s i , d i , l i ). Configuration parameters of an iFIFO task can include a total size of the FIFO buffer (e.g., in number of entries) and a maximum size of each entry (e.g., in bytes). In an example, a maximum amount of data that can be stored in the iFIFO is equal to the total size of the FIFO buffer multiplied by the maximum size of each entry, and this maximum amount of data can be referred to as the buffer size.

[0091] In an example, an iFIFO function or iFIFO task is configured to generate one event (e.g., a full event or a partial full event) indicating that an amount of data stored in the iFIFO task is equal to a threshold. The iFIFO function or iFIFO task is configured to generate another event (e.g., an empty event) indicating that the amount of data stored in the iFIFO task is empty or the amount of data is provided to a processing task.

[0092] At the input or output of an iFIFO buffer, an index input or index output of the iFIFO buffer can provide a start time, a duration, and a length of a next buffer segment, thus indexing each buffer segment.

[0093] According to the present disclosure, a buffering function can be, for example, a multiple-index first-in-first-out (MiFIFO) function for processing multiple code streams. The MiFIFO function can include multiple FIFO buffers. Each of the multiple FIFO buffers can be the same as or similar to the iFIFO buffer described above. In the time domain, corresponding buffer segments across the multiple FIFO buffers can be aligned. In an example, a MiFIFO includes a first iFIFO and a second iFIFO. The first iFIFO includes first buffer segments 0-4 with respective first start times and first durations. The second iFIFO includes second buffer segments 0-4 with respective second start times and second durations. In an example, the first start times of the first buffer segments 0-4 are respectively the same as the second start times of the second buffer segments 0-4, and the first durations of the first buffer segments 0-4 are respectively the same as the second durations of the second buffer segments 0-4. The MiFIFO function can be used when a task has multiple inputs or multiple outputs.

[0094] In an example, a MiFIFO function or MiFIFO task is configured to generate one event (e.g., a full event or a partial full event) indicating that an amount of data stored in the MiFIFO task is equal to a threshold. The MiFIFO function or MiFIFO task is configured to generate another event (e.g., an empty event) indicating that the amount of data stored in the MiFIFO task is empty or the amount of data is provided to a processing task.

[0095] Buffering functions (e.g., iFIFO functions, MiFIFO functions) can support events such as empty events and full events as described below. An empty event can be generated when a buffer (e.g., an iFIFO buffer) is empty or a plurality of buffers (e.g., iFIFO buffers in a MiFIFO function) are empty (e.g., data stored in the buffers is read or provided to corresponding tasks). A full event can be generated when a buffer (e.g., an iFIFO buffer) is full or each of the plurality of buffers is full. In some examples, a buffering function can support X% (or %X) events. X can be a configuration parameter such as a value greater than 0 and less than or equal to 100. In an example, X is less than 100 and an X% event can be referred to as a partial full event. An X% event can be generated when each iFIFO buffer is X% full. An event (or X% event) can be generated when each iFIFO buffer is X% full. One or more values of X can be set as configuration parameters. Buffering tasks (e.g., iFIFO tasks, MiFIFO tasks) can be instantiated to implement buffering functions (e.g., iFIFO functions, MiFIFO functions).

[0096] Benefits of buffering functions can be described below. Buffering functions can be simple functions implemented in any suitable platform. The platform provides buffers and includes cache memory, random access memory, long term memory such as solid state and hard disks. Events (e.g., empty events, full events) can be generated based on emptiness, fullness of buffers. Thus, a buffering task can effectively inform a workflow manager (103) of a status of the buffering task.

[0097] According to aspects of the present disclosure, a workflow can be converted to a step-wise workflow (or step-based workflow, step-by-step workflow, steppable workflow) by adding one or more buffering tasks (e.g., iFIFO functions, MiFIFO functions) in the workflow. A step-wise workflow can operate in a step-wise mode. A buffering task can be provided between two tasks. The two tasks can include task A and task B, where an output of task A is provided to an input of task B.

[0098] Reference Figures 2A-2BWorkflow (200A) can be referred to as a regular workflow (200A). In the example, the regular workflow (200A) is not a step-based workflow. The regular workflow (200A) can be converted into a step-based workflow (e.g., workflow (200B)) by inserting at least one buffer task into it. Typically, tasks in a step-based workflow can have a first number of input buffer tasks and a second number of output buffer tasks, where the first number of input buffer tasks provides the task with the data to be processed, and the second number of output buffer tasks stores the data processed by the task. The first number can be any suitable non-negative integer, such as 0, 1, 2, etc. The second number can be any suitable non-negative integer, such as 0, 1, 2, etc. Figure 2B In some of the examples shown, no buffered task is provided for each workflow input (e.g., (201)) of the step-based workflow. In some examples, at least one buffered task is provided for one or more workflow inputs of the step-based workflow. In examples such as Figure 2B In some of the examples shown, no buffering task is provided for each workflow output from the step-based workflow (e.g., (209) or (210)). In some examples, at least one buffering task is provided for one or more workflow outputs from the step-based workflow.

[0099] exist Figure 2B In the example shown, buffer tasks (e.g., iFIFO tasks) (231) to (237) are provided for each of the links (202) to (208). For example, task (T1) is connected to the workflow input and not to an input buffer task. Task (T1) is connected to an output buffer task (231). Task (T2) is connected to an input buffer task (231) and two output buffer tasks (232) to (233). Task (T5) is connected to two input buffer tasks (233) and (235) and to an output buffer task (237).

[0100] Each link (e.g., link (202)) is a connection between two tasks (e.g., tasks (T1) to (T2)) in a workflow (200B). A link (e.g., (202)) may represent a data transfer from one of the two tasks (e.g., (T1)) to the other of the two tasks (e.g., (T2)). Buffer tasks (231) to (237) may be instantiated from the buffering function described above. In the example, buffer tasks (231) to (237) include iFIFO tasks instantiated from the iFIFO function.

[0101] Generally, a task pair (or two tasks) connected to a buffering task can include a "connect-from" task of the buffering task and a "connect-to" task of the buffering task. With reference to a buffering task (231) and a task pair (T1) to (T2) connected by the buffering task (231), task (T1) is referred to as the "connect-from" task and task (T2) is referred to as the "connect-to" task of the buffering task (231). Workflow (200B) includes seven task pairs corresponding to seven buffering tasks (231) to (237).

[0102] Adding a buffering task between a task pair (including a "connect-from" task and a "connect-to" task) can allow the "connect-from" task to run, generate output, and fill the buffer configured for the buffering task, while the "connect-to" task does not run. The "connect-from" task can run until the buffer (or each buffer) is full or partially full (e.g., X% full), and the "connect-from" task can have enough data for processing from one or more buffers connected to one or more inputs of the "connect-from" task.

[0103] The amount of time a task (e.g., (T1)) runs independently of other tasks (e.g., (T2) to (T6)) in a workflow (200B) can be defined by the buffering tasks connected to the inputs and outputs of the task. One or more buffering tasks connected to one or more task inputs can be referred to as input buffering tasks (or input buffers), and one or more buffering tasks connected to one or more task outputs can be referred to as one or more output buffering tasks (or one or more output buffers). Generally, the size of the buffer and the fullness level (such as described by X%) can define the maximum size (or length) of steps (or maximum step size) that a task can run.

[0104] In an example, for a task (e.g., (T2)) in a workflow (200B), the task (e.g., (T2)) can run independently of other tasks (e.g., (T1) and (T3) to (T6)) in the workflow (200B) when none of the one or more input buffers (e.g., (231)) of the task are empty and none of the one or more output buffers (e.g., (232) to (233)) of the task are full. In an example, the task (e.g., (T2)) can run independently of other tasks (e.g., (T1) and (T3) to (T6)) in the workflow (200B) when none of the one or more input buffers of the task are empty and none of the one or more output buffers of the task are X% full (where X is a preconfigured value less than 100).

[0105] In an example, one or more tasks in the step-by-step workflow can run to process data while the remaining tasks in the step-by-step workflow do not run. Referring to Figure 2B If the buffer task (231) is omitted, tasks (T1) to (T2) can run to process data while tasks (T3) to (T6) are not running and in a non-running state (e.g., idle state and / or destroyed state). For example, task (T1) runs to process next data and task (T2) runs to further process data processed by task (T1). In some examples, when the buffer task (231) is omitted, task (T1) and task (T2) do not run at the same time, for example, when task (T2) has no input data or task (T2) is not in a running state.

[0106] The advantages of converting a regular (non-step-based) workflow to a step-based workflow are described as follows. The conversion is relatively simple and does not depend on the topology or complexity of the workflow, and thus can be performed in an unsupervised manner.

[0107] A processing task can be configured with a parameter task segment duration (TSD) (also referred to as task processing segment duration). The TSD of a processing task can refer to a duration in time domain or a data length such that output data at one or more outputs of the processing task can be independent of a duration of any of the inputs of the processing task or data outside of the data length. For example, input data of a processing task has a first subset, a second subset, and a third subset, each subset having a duration of TSD. The processing task can process the first subset, the second subset, and the third subset independently of each other. When the processing task processes the second subset, the corresponding output data only depends on the second subset of data and is independent of the first subset and the third subset. For TSD, any suitable unit can be used, such as time (e.g., ms), bytes, bits, etc. The output data generated at one or more outputs of the processing task is within a duration in time domain or a data length.

[0108] Figure 3 An example of TSD configured for a task (301) is shown in accordance with an embodiment of the disclosure. The task (301) can be instantiated from a corresponding function, such as in a function repository (105). The task (301) can implement any processing method, such as video stitching or other media processing method. For example, based on metadata, the task (301) can stitch video 1 from a first camera and video 2 from a second camera. The task (301) can output video 3 and video 4 as stitched videos. Video 4 can correspond to a different operation, such as indexing. In some examples, video 4 is the same as video 3. In accordance with the disclosure, the task (301) can run for a duration of TSD (e.g.,Figure 3 a time window having a duration equal to the TSD. The task (301) runs and processes input data from one or more inputs to generate output data at one or more outputs within the time window having a duration equal to the TSD. The one or more outputs (having a duration setting of the TSD) can depend only on input data within the TSD. The one or more outputs can be independent of input data outside the time window. Subsequently, the task (301) can be stopped or destroyed at the end of each TSD and can be instantiated or started at the beginning of the next TSD.

[0109] In Figure 3 In the example shown, the TSD of the task (301) is T. Video 1 includes multiple subsets (302) of first media data. Each subset of first media data can have a first size, where three subsets (302) of first media data are within the TSD T. Video 2 includes multiple subsets (303) of second media data. Each subset (303) of second media data can have a second size, i.e., the TSD T. Metadata includes multiple subsets (304) of metadata. Each subset (304) of metadata can have a metadata size, where six subsets (304) of metadata are within the TSD T.

[0110] For example, the input data includes the three subsets (302) of first media data and the subsets (303) of second media data. The input data can be processed by the task (301) to generate output, e.g., based on the six subsets (304) of metadata. The output includes a subset (305) of third media data for video 3 and two subsets (306) of fourth media data for video 4. The output (e.g., video 3 and video 4) of the task (301) can be generated independently within a time window having a duration equal to the TSD, e.g., T. Subsequently, the task (301) can be stopped or destroyed at the end of the time window to switch to a non-running state. The non-running state can be an idle state (402) or a destroyed state (405). According to the present disclosure, when the task (301) is in the non-running state, there is no need to store state information of the task because the task (301) can process the input data without using previous data when processing the input data within a time window having a duration of the TSD. Therefore, previous processing does not need memory and previous state information. The task (301) can be instantiated or started at the beginning of the next time window having a duration equal to the TSD. In Figure 3 In the example shown, the video 4 output can operate in independent segments (also referred to as task segments) having a duration of T / 2.

[0111] Reference can be made to Figure 2Bthe above description with appropriate modifications to provide one or more MiFIFO tasks in a non-step-based workflow to convert the non-step-based workflow to a step-based workflow. Referring to Figure 2A and Figure 5 Workflow (200A) can be converted to a step-based workflow (e.g., workflow) (500) by inserting one or more MiFIFO tasks into workflow (200A). In an example, a MiFIFO task is used when a task includes multiple outputs.

[0112] Referring to Figure 5 , the buffering tasks can include iFIFO tasks (531), (536), and (537), and two MiFIFO tasks (e.g., (MiFIFO1) and (MiFIFO2)). For example, iFIFO task (531) is provided between task (T1) and task (T2). Task (T4) is connected to iFIFO task (536). Task (T5) is connected to iFIFO task (537). iFIFO tasks (531), (536), and (537) can be instantiated from the iFIFO buffering functionality described above.

[0113] Task (T2) has two outputs, and thus can be connected to MiFIFO task (MiFIFO1). (MiFIFO1) task can be configured with two iFIFO buffers (MiFIFO1-1) and (MiFIFO1-2) corresponding to links (203) to (204), respectively. Task (T3) has two outputs, and thus can be connected to MiFIFO task (MiFIFO2). (MiFIFO2) task can be configured with two iFIFO buffers (MiFIFO2-1) and (MiFIFO2-2) corresponding to links (205) to (206), respectively. MiFIFO tasks (MiFIFO1) and (MiFIFO2) can be instantiated from the MiFIFO buffering functionality described above.

[0114] Adding a MiFIFO task (e.g., (MiFIFO1)) can allow a “connect from...” task (e.g., (T2)) to run, generate outputs, and fill the iFIFO buffers (e.g., (MiFIFO1-1) and (MiFIFO1-2)) while “connect to...” tasks (e.g., (T3) and (T5)) do not run. The iFIFO buffers are configured for the MiFIFO task (e.g., (MiFIFO1)). The “connect from...” task can continue to run until the iFIFO buffers are full, and the “connect from...” task can have enough data to process the inputs connected to the “connect from...” task from the buffers.

[0115] In some examples, the iFIFO tasks (531), (536), and (537) can be replaced by MiFIFO tasks, and thus the workflow (500) is configured with MiFIFO tasks.

[0116] In examples, when a task includes multiple inputs, an MiFIFO task is provided at the input, and the above description can be modified as appropriate.

[0117] The workflows (200B) and (500) can be configured to operate in a stateless mode by using processing tasks configured with TSDs. Referring to Figure 2B When the iFIFO task duration (or the maximum amount of data that can be stored in the iFIFO task) of each iFIFO task is set to at least reach the TSD size of a processing task (or a "connect to..." task) to which the iFIFO task is connected, the "connect to..." task can operate and generate an output for one TSD. As described above, the "connect to..." task does not need to carry or store state information for generating the next task segment at the output, and thus no state information is stored. A processing task (e.g., a "connect to..." task) can be referred to as a stateless step processing task. A workflow (200B) including a stateless step processing task can be configured to operate in a stateless mode, and can be referred to as a stateless step workflow. The above description can be adjusted as appropriate to apply to MiFIFO tasks and workflows (e.g., the workflow (500)) including one or more MiFIFO tasks.

[0118] Although the data (e.g., a task segment) generated in one TSD is independent of the data (e.g., other task segments) outside the TSD, the beginning and end of the data can be important. For example, if an encoder starts to encode, the beginning of the code stream generated by the encoder (e.g., due to the header) is different from other segments generated thereafter. Thus, the iFIFO buffer can be configured to store additional information, e.g., start time, duration, and length, as described above.

[0119] Figure 6A flowchart illustrating a method (600) summarizing embodiments in accordance with the present disclosure is shown. The method (600) can be applied to the operation of a step-by-step workflow having any suitable number of one or more workflow inputs and any suitable number of one or more workflow outputs. The step-by-step workflow can include any suitable number of processing tasks and any suitable links (or connections) connecting two tasks. The step-by-step workflow can include one or more buffering tasks (e.g., one or more iFIFO tasks, one or more MiFIFO tasks, or a combination of iFIFO tasks and MiFIFO tasks), and can provide the one or more buffering tasks at the respective links. The step-by-step workflow can be the workflow (200B) or (500).

[0120] In an example, the step-by-step workflow is Figure 5 the workflow (500). The workflow (500) can include MiFIFO tasks, where the buffering tasks (531), (536), and (537) are MiFIFO tasks. Each of the tasks (T1) through (T6) can have the task life cycle (400), different states (401) through (405), and the corresponding transitions described with reference to Figure 4A The step-by-step (or step-based, step-by) operation of the workflow (200B) can be implemented as follows. Any suitable data processing system (e.g., an NBMP system) can implement the step-by-step operation of the workflow. The description below is given with reference to the data processing system (100), and can be adapted to any suitable data processing system (e.g., an NBMP system) as appropriate.

[0121] At (S605), the workflow can be initialized as described below. Let TSD(k) denote the TSD of a processing task T(k) in an initial workflow, where k is a non-negative integer from 0 to L-1, and L is the number of processing tasks in the initial workflow. The initial workflow can be a non-step-by-step workflow.

[0122] The workflow segment duration (WSD) can be determined based on the TSDs. For example, the WSD is computed based on the least common multiple (LCM) of the TSDs. In an example, the WSD is computed using the following equation (1).

[0123] WSD = LCM(TSD(0),...,TSD(L-1)) Equation (1)

[0124] A workflow (or step-by-step workflow) can be generated by providing a MiFIFO task at one or more outputs of each processing task in which data from the one or more outputs is one or more inputs of one or more other tasks in the workflow. For example, with reference to Figure 5For two outputs of the task (T2), a MiFIFO task (MiFIFO1) is provided. The MiFIFO task can have a length (or size) based on parameters S and WSD, where parameter S is an integer (e.g., 1 or 2, etc.) indicating a step size of the stepped workflow. In an example, the step size can be S times WSD. In an example, parameter S indicates a number of steps configured for running the processing task (e.g., (T2)) consecutively.

[0125] Each MiFIFO task can be reset to empty with a corresponding sequence number of 0.

[0126] At (S610), one or more initial tasks of the processing tasks in the workflow are identified, e.g., by the workflow manager (103). One or more inputs of each of the one or more initial tasks can be one or more workflow inputs. For example, all inputs of the one or more initial tasks are connected to a data source (e.g., a media source) (111). The one or more initial tasks can be labeled as Ti0, Ti1, …, TiN, where N is a non-negative integer. Referring to Figure 5 The one or more initial tasks identified in the workflow (500) include the task (T1), where Ti0 is (T1) and N is 0.

[0127] At (S620), the identified one or more initial tasks Ti0, Ti1, …, and TiN can be run in sequence. Referring to Figure 5 In the illustrated example, the initial tasks include (T1). The following description can apply to each of the identified initial tasks.

[0128] The workflow manager (103) can send a request to the initial task (T1) and transition the initial task (T1) to a running state (403).

[0129] The initial task (T1) can run S (e.g., 1 or 2, etc.) steps, and at each step, process a portion of the input data. For example, the portion of the input data has a duration or length determined based on WSD. In an example, the duration (or length) is equal to WSD in each step.

[0130] When the amount of data output by the initial task (T1) to the MiFIFO task reaches or approaches a first threshold (e.g., a total duration of SxWSD (or a total size)), the MiFIFO task (e.g., MiFIFO task (531)) is full (or partially filled to X% full, where X is less than 100). The first threshold can be any suitable duration or size and can be expressed using any suitable unit. The first threshold can be based on TSD, WSD, and / or parameter S.

[0131] Accordingly, the MiFIFO task can send a full event (or a partial full event if, for example, the total duration of S x WSD is X% of the time required to fill) to the workflow manager (103). For example, S is 2, when the amount data output by the task (T1) reaches 2WSD, the MiFIFO task (531) can generate a full event (or a partial full event) and send a notification to the workflow manager (103).

[0132] The workflow manager (103) can send a request to the task (T1) to transition the task (T1) from the running state (403) to a non-running state, such as the idle state (402) or the destroyed state (405). In the non-running state (e.g., the idle state (402) or the destroyed state (405)), the state information (e.g., the state and values of the internal functions) of the task (T1) does not need to be maintained or stored, and thus in some examples, the state information is not stored in the non-running state, and the task (T1) can run in a stateless mode.

[0133] The execution (S620) can be repeated for each of one or more initial tasks. The above description of the task (T1) can be modified appropriately for the other one or more initial tasks. In an example, the one or more initial tasks have different threshold values including the first threshold value. Alternatively, the one or more initial tasks have the same first threshold value.

[0134] At (S630), the workflow manager (103) can identify one or more subsequent tasks, where all input of the one or more subsequent tasks are connected to one or more non-empty buffer tasks (e.g., iFIFO tasks, MiFIFO tasks). The one or more subsequent tasks can be labeled as Tj0, Tj1, …, TjM, where M is a non-negative number. Referring to Figure 5 The one or more non-empty buffer tasks include the buffer task (531) that stores the amount data processed by the task (T1). The one or more subsequent tasks identified at (S630) include the task (T2), and thus Tj0= (T2) and M is 0.

[0135] At (S640), the identified one or more subsequent tasks Tj0, Tj1, …, TjM can be run in sequence. Referring to Figure 5 The subsequent tasks include (T2) in the example shown. The following description can be applied to each of the identified subsequent tasks.

[0136] The workflow manager (103) can send a request to the subsequent task (T2) and transition the task (T2) to the running state (403).

[0137] The subsequent task (T2) can run S (e.g., 1 or 2, etc.) steps and process data stored in the buffer task (531), e.g., having a duration or length determined based on the WSD in each step. In an example, the duration (or length) is equal to the WSD in each step.

[0138] When the amount of data output by the subsequent task (T2) reaches or approaches the second threshold (e.g., the total duration (or total size) of S x WSD), the MiFIFO task can be filled (or partially filled to X% full, where X is less than 100). Thus, the MiFIFO task (e.g., the MiFIFO task (MiFIFO-1)) can send a full event (or a partial full event) to the workflow manager (103). For example, S is 2, when the amount of data output by the task (T2) reaches 2 WSD, the MiFIFO task (MiFIFO-1) can generate a full event (or a partial full event if, for example, the total duration of S x WSD is X% full) and send a notification to the workflow manager (103). The second threshold can be any suitable duration or size and can be expressed using any suitable unit. The second threshold can be based on the TSD, the WSD, and / or the parameter S.

[0139] The workflow manager (103) can send a request to the task (T2) to transition the task (T2) from the running state (403) to a non-running state, e.g., the idle state (402) or the destroyed state (405). In the non-running state (e.g., the idle state (402) or the destroyed state (405)), state information (e.g., states and values of internal functions) of the task (T2) does not need to be maintained or stored, and thus in some examples, the state information is not stored in the non-running state, and the task (T2) can run in a stateless mode.

[0140] The repeating (S640) can be repeated for each of one or more subsequent tasks. The above description of (T2) can be modified as appropriate for the other one or more subsequent tasks. In an example, the one or more subsequent tasks have different thresholds including the second threshold. Alternatively, the one or more subsequent tasks have the same second threshold. In an example, multiple processing tasks in a workflow have the same threshold, e.g., S x WSD.

[0141] At (S650), it is determined whether the remaining tasks to be processed are subsequent tasks. As described above, all inputs of a subsequent task are non-empty buffer tasks (e.g., MiFIFO tasks). When it is determined that the remaining tasks are subsequent tasks, the method (600) returns to (S630), and thus repeats (S630) and (S640) for the newly identified one or more subsequent tasks. When no remaining tasks are determined to be subsequent tasks, the method (600) proceeds to perform (S660).

[0142] For example, task (T3) is a newly identified subsequent task, and (S640) is repeated for task (T3). Subsequently, (S650) and (S630) are repeated, and thus tasks (T4) through (T5) are identified as new subsequent tasks, and (S640) is repeated for (T4) through (T5). Finally, (T6) is identified, and (S640) is repeated for (T6).

[0143] Generally, (S630) and (S640) are repeated for the remaining subsequent tasks having inputs connected to one or more non-empty buffer tasks until all tasks in the workflow (500) are run to process data. Thus, when no remaining tasks are determined to be subsequent tasks, the method (600) proceeds to perform (S660).

[0144] At (S660), it is determined whether additional input data remains to be processed. When it is determined that no additional input data is to be processed, the process (600) proceeds to (S699) and terminates. When it is determined that additional input data is to be processed, the method (600) returns to (S610), and (S610), (S620), (S630), (S640), and (S650) are repeated for another subset of input data (e.g., having a duration of SxWSD).

[0145] In one example, the processing tasks in the workflow are run in a stateless mode, and thus the workflow can be run in a stateless mode, and no state information of a processing task in a non-running state is saved when the processing task is in the non-running state.

[0146] The method (600) can be adjusted as appropriate. One or more steps in the process (600) can be modified and / or omitted. Additional step(s) can be added. Any suitable order of implementation can be used. In the above example, other tasks in the workflow (500) are not run while one task is run in one step. In another example, one or more tasks can be run simultaneously in one step while other tasks in the workflow are not run.

[0147] In an example, the input data to be processed by the step-wise workflow is divided into a plurality of buffered segments, where each buffered segment is indexed. Thus, the input data can comprise a plurality of subsets of the input data, where each subset comprises a plurality of indexed buffered segments. The size or length of each subset can be determined based on the TSD and / or parameter S of the processing task. In an example, the length of each subset is S x WSD (e.g., WSD, 2WSD, 3WSD), and each processing task can process a subset of input data having a length of S x WSD before switching to a non-running state. In an example, the TSD is the same, and thus WSD is equal to the TSD of each processing task, and thus the length of each subset is S x TSD, e.g., TSD, 2TSD, or 3TSD, etc.

[0148] The method (600) is described using the workflow (500) with MiFIFO tasks. Appropriate modifications can be made to the method (600) for a step-wise workflow with iFIFO tasks, e.g., the workflow (200B). Appropriate modifications can be made to the process for a step-wise workflow with one or more iFIFO tasks and one or more MiFIFO tasks.

[0149] The benefits of the method (600) and / or the step-wise workflow (500) can be described as follows. The logic is simple for a data processing system (e.g., an NBMP system) (100) (e.g., a workflow manager (103)) to perform the method (600). The propagation model can start from one or more workflow inputs and perform one step of each task followed by a task connected to the output of the task. The method (600) and / or the workflow (500) can generate one or more outputs of the workflow in a plurality of steps. The plurality of steps can be triggered by events, thus the workflow management can be very efficient. The step size S can be selected to be any suitable number.

[0150] According to the present disclosure, to implement a stateless step-wise processing task, the corresponding function is configured with configuration parameters and input parameters. The corresponding function can be instantiated to generate the processing task. The configuration parameters can include a TSD parameter as described above. The TSD can determine a duration of a task segment. Wherein, the task segment can be processed by the processing task, which is independent of other task segments in the data to be processed. In certain examples, the TSD parameter is dependent on other configuration parameters of the processing task. The configuration parameters can include an operation sample, which indicates a number of TSDs that the processing task can operate in a stateless mode. For example, an output is generated based on processing Nl input samples, where Nl can be an integer associated with the TSD. The input parameters can include additional inputs, e.g., metadata. The additional inputs can indicate a start time, a duration, etc. of an input task segment in the input data. The duration of the input task segment can be the TSD.

[0151] In one example, the input data to be processed is divided into a plurality of buffer segments, each of which can be indexed to provide a start time, a duration, and a length of the buffer segment.

[0152] The above design of step-based workflow can be applied to the entire workflow, e.g., at the workflow input and / or the workflow output, implementing a buffer task (e.g., an iFIFO task, an MiFIFO task) to perform the workflow step-by-step. The benefit of the above approach is that although the individual tasks of the workflow can not be step-wise, the workflow can be processed step-by-step. Thus, the workflow can be performed step-by-step. Therefore, when the platform resources are not continuous to be used to perform the entire session in one step, one or more workflows can run while the rest of the workflows are not running (e.g., in an idle state or a destroyed state). When the workflow is part of a multi-workflow setup and each workflow can be performed step-by-step, not all workflows are running at the same time.

[0153] Figure 2C An example of a workflow (200C) is shown in accordance with an embodiment of the disclosure. The workflow (200C) includes a plurality of processing tasks (or tasks) (T4) through (T9). In an example, no buffer task is inserted between two of the processing tasks (T4) through (T9), and the processing tasks (T4) through (T9) can run simultaneously to process data and are not step-wise. One or more buffer tasks can be inserted in one or more workflow inputs and / or one or more workflow outputs, thus the workflow (200C) can run in a step-wise mode or one step at a time. In an example, a buffer task (271) is provided at the workflow input (255), and a buffer task (272) is provided at the workflow output (263). In an example, the workflow (200C) runs to process data until one of the following conditions is met: (i) the data stored in the buffer task (271) is provided to the workflow (200C) (e.g., the buffer task (271) is empty), and (ii) the buffer task (272) is full. The above description of the workflow (200C) can be adjusted as appropriate when the workflow includes one or more step-wise tasks. The buffer task (271) can be an iFIFO task, an MiFIFO task, etc. The buffer task (272) can be an iFIFO task, an MiFIFO task, etc. In one example, an MiFIFO task is provided for the link 263-264.

[0154] Figure 2D An example of a plurality of workflows (280) is shown in accordance with an embodiment of the disclosure. The plurality of workflows (280) can include a concatenation of workflows having the workflow (200C) and the workflow (200D). The workflow (200C) is in a step-wise mode and the workflow (200D) is not in a step-wise mode. The workflow (200C) is running and the workflow (200D) is not running. The workflow (200C) can be running in a step-wise mode and the workflow (200D) can be running in a non-step-wise mode. The workflow (200C) can be running in a step-wise mode and the workflow (200D) can be running in a step-wise mode. The workflow (200C) can be running in a non-step-wise mode and the workflow (200D) can be running in a non-step-wise mode. Figure 2CThe workflow (200D) includes processing tasks (T1) through (T3), where the workflow input (251) provides input data to task (T1) and the workflow output (255) provides input data to workflow (200C). Thus, the workflow output (255) of workflow (200D) is the workflow input (255) of workflow (200C).

[0155] The buffer task (271) is inserted between workflows (200C) and (200D), thus the multiple workflows (280) run in a step-by-step mode, where workflow (200D) runs to process data and workflow (200C) is in a non-running state and does not run to process data. The buffer task (271) can be an iFIFO task, an MiFIFO task, etc.

[0156] When a buffer task is provided at a workflow input or a workflow output, the workflow can run in a step-by-step mode.

[0157] According to aspects of the present disclosure, by using a simple buffer function (or buffer task), a conventional workflow (or non-step workflow) that includes non-step tasks can be converted to a step workflow that includes one or more step tasks. The buffer function (e.g., iFIFO function, MiFIFO function) or the corresponding task (e.g., iFIFO task, MiFIFO task) can be efficiently implemented in all suitable platforms. Step processing driven by events (e.g., empty event, full event, partial full event) can simplify design and efficiently manage the workflow. The workflow algorithm for propagating the workflow by executing one step per task is simple. One or more outputs of the workflow can be generated step-by-step. The step size can be based on the parameters S and TSD of the processing tasks in the workflow, thus the step size can be configured by changing the parameter S. When a processing task in the step workflow transitions from a running state to a non-running state, there is no need to store state information of the processing task. Thus, in various examples, the stateless processing task is a stateless step processing task and implements a stateless task operation. The stateless task operation is advantageous in certain computing platforms, such as cloud platforms.

[0158] According to aspects of the present disclosure, when a buffer task is provided for a workflow input, a workflow output, or between two tasks in a workflow, the workflow can run in a step-by-step mode. The workflow can run step-by-step, thus when data is input to the workflow, the workflow does not continuously run to process the data (e.g., media data). Instead, the workflow can process a first subset of data in a first step. After outputting the processed first subset of data, the workflow can process a second subset of data in a second step.

[0159] Any suitable method can be used to implement the step-by-step workflow. In an example, the workflow can include one or more buffer tasks (e.g., iFIFO tasks, MiFIFO tasks) provided between tasks in the workflow, as shown in Figure 2B and Figure 5 One or more tasks in the workflow can be running in a running state at the same time as other tasks are in a non-running state, thus reducing the resources (e.g., compute resources, memory resources) used in running all tasks at the same time.

[0160] Figure 7 A flowchart outlining a method (700) according to embodiments of the disclosure is shown. In an example, the method (700) is implemented using a data processing system (e.g., an NBMP system) (100) to process input data, such as media data. In some embodiments, the method (700) is implemented in software instructions, thus when a processing circuit executes the software instructions, the processing circuit performs the method (700). The method starts at (S701) and proceeds to (S710).

[0161] At (S710), the processing circuit can receive workflow information of a workflow. In an example, the workflow information is received by a workflow manager (e.g., workflow manager (103)) from a data processing source (e.g., NBMP source (101)).

[0162] At (S720), based on the workflow information, a workflow to process the input data is generated. The workflow includes a plurality of processing tasks and a first buffer task. In an example, the plurality of processing tasks are configured with respective TSDs. Each of the plurality of processing tasks can be configured to process an amount of data equal to the respective TSD. The processing is independent of data outside the TSD. Thus, each processing task can independently process data within a time window having a TSD duration, and the output generated by the processing task within the time window is independent of other data outside the time window. The plurality of processing tasks can include a first processing task and a second processing task. The workflow is a step-by-step workflow, as shown in Figure 2B or Figure 5 The first buffer task can be instantiated from a respective buffer function as described above. The first buffer task can be an iFIFO task, a MiFIFO task, or the like. In an example, the input data includes a plurality of subsets of input data having a subset.

[0163] At (S730), the first processing task is brought into a running state. Wherein, a subset of the input data is processed by the first processing task and the subset is outputted as first processed subset data to the first buffering task. For example, when the first processing task is in the running state to process the subset of the input data, the second processing task is in a non-running state (e.g., an idle state or a destroyed state), as described above. The first buffering task can be configured to store the first processed subset data and provide the first processed subset data to the second processing task.

[0164] At (S740), based on the amount of the first processed subset data stored in the first buffering task being equal to a first threshold, the first processing task is transitioned from the running state to a non-running state (e.g., an idle state or a destroyed state). In an example, in the non-running state, the state information of the first processing task is not stored. The first threshold can be based on (i) the TSD of the plurality of processing tasks, (ii) the WSD of the workflow, or (iii) the WSD and a parameter S, where the parameter S indicates a number of steps for which the first processing task is configured to run continuously. In one example, the first threshold is S x WSD.

[0165] In an example, when the amount of the first processed subset data stored in the first buffering task is equal to the first threshold, an event (e.g., a FULL event, as described above) is generated, for example, by the first buffering task, and a notification can be sent to the workflow manager. Subsequently, the workflow manager can send a request to the first processing task and transition the first processing task from the running state to the non-running state.

[0166] In an example, the first processing task is transitioned from the running state to the non-running state when a buffering task (referred to as an input buffering task) provides input to the first processing task and provides data stored in the buffering task to the first processing task. For example, when the data stored in the buffering task is provided to the first processing task, the buffering task can generate an empty event and send a notification to the workflow manager. Subsequently, the workflow manager can send a request to the first processing task and transition the first processing task from the running state to the non-running state.

[0167] In an example, the first processing task is transitioned from the running state to the non-running state when the data stored in the input buffering task is provided to the first processing task or the amount of the first processed subset data stored in the first buffering task is equal to the first threshold.

[0168] At (S750), after the first processing task transitions to the non-running state, the second processing task is caused to enter a running state, where in the running state, the first processed subset data stored in the first buffer task is processed by the second processing task. In an example, the first processing task is in the non-running state, while the second processing task is running to process the first processed subset data. Thus, the first processing task and the second processing task do not run concurrently, but rather sequentially.

[0169] The method (700) can be adapted as appropriate. Steps in the method (700) can be modified and / or omitted. Other steps can be added. Any suitable order of implementation can be used.

[0170] In an example, the second processing task in the running state outputs the processed first processed subset data as a second processed subset data to a second buffer task. The second buffer task can be configured to store the second processed subset data and provide the second processed subset data to a third processing task of the plurality of processing tasks. After (S750), the second processing task is caused to transition from the running state to the non-running state based on one of (i) an amount of the second processed subset data stored in the second buffer task is equal to a second threshold value, and (ii) an amount of the first processed subset data stored in the first buffer task is provided to the second processing task (e.g., a buffer configured for the first buffer task is empty). In an example, in the non-running state, state information of the second processing task is not stored. The second threshold value can be based on (i) a TSD of the plurality of processing tasks, (ii) a WSD of the workflow, or (iii) the WSD and a parameter S, where the parameter S indicates a number of steps for which the first processing task is configured to run consecutively. In one example, the second threshold value is S x WSD. The first threshold value and the second threshold value can be the same or different.

[0171] In one example, the workflow further includes a plurality of buffer tasks having a first buffer task (e.g., an iFIFO task or a MiFIFO). Each of the plurality of processing tasks has an input from one of the input data and one or more of the plurality of buffer tasks, and only one of the plurality of processing tasks is allowed to be in a running state at any point in time. Each of the inputs is from

[0172] In one example, the input data includes the subset of the input data and another subset of the input data. The workflow includes a plurality of buffering tasks, where the plurality of buffering tasks includes a first buffering task (e.g., iFIFO task, MiFIFO task). Each input of the plurality of processing tasks is from one of the input data and one or more of the plurality of buffering tasks. The subset of the input data can be processed in sequence by the plurality of processing tasks. After the subset of the input data is processed by the plurality of processing tasks, another subset of the input data can be processed in sequence by the plurality of processing tasks while remaining processing tasks of the plurality of processing tasks are in a non-running state. Only one of the plurality of processing tasks is allowed to be in the running state at any point in time. Figure 8 A flowchart illustrating a method (800) according to embodiments of the disclosure is shown. In an example, the method (800) is implemented using a data processing system (e.g., an NBMP system) (100) to process input data, such as media data. In some embodiments, the method (800) is implemented in software instructions, thus when a processing circuit executes the software instructions, the processing circuit performs the method (800). The method starts at (S801) and proceeds to (S810).

[0173] At (S810), workflow information of a plurality of workflows is received by the processing circuit. In an example, the workflow information is received by a workflow manager (e.g., workflow manager (103)) from a data processing source (e.g., NBMP source (101)).

[0174] At (S820), based on the workflow information, a plurality of workflows for processing the input data is generated. At least one of the plurality of workflows can include a plurality of processing tasks. The plurality of workflows can include a first workflow and a second workflow. At least one of the first workflow and the second workflow has an input buffering task or an output buffering task, as described above in Figure 2C .

[0175] At (S830), the first workflow is brought into a running state, where the first workflow processes a subset of the input data and outputs the subset as first processed subset data to a buffering task. For example, when the first workflow is in the running state to process the subset of the input data, the second workflow is in a non-running state (e.g., idle state or destroyed state), as described above. In an example, the buffering task can be configured to store the first processed subset data and provide the first processed subset data to the second workflow. The buffering task can be instantiated from a corresponding buffering function, as described above. The buffering task can be an iFIFO task, a MiFIFO task, etc.

[0176] At (S840), based on an amount of the first processed subset data stored in the buffering task being equal to a threshold value, the first workflow is transitioned from the running state to the non-running state.

[0177] In an example, when the amount of first processed subset data stored in the buffer task equals the threshold, e.g., a buffer task generates an event (e.g., a FULL event as described above), and sends a notification to the workflow manager. Subsequently, the workflow manager can send a request to the first workflow and cause the first workflow to transition from the running state to the non-running state.

[0178] In an example, when a buffer task (referred to as an input buffer task) provides input to the first workflow and provides data stored in the input buffer task to the first workflow, the first workflow transitions from the running state to the non-running state. For example, when the data stored in the input buffer task is provided to the first workflow, the input buffer task generates an empty event and sends a notification to the workflow manager. Subsequently, the workflow manager can send a request to the first workflow and cause the first workflow to transition from the running state to the non-running state.

[0179] In an example, the first workflow transitions from the running state to the non-running state when the data stored in the input buffer task is provided to the first workflow or the amount of first processed subset data stored in the buffer task (or an output buffer task) equals the threshold.

[0180] At (S850), after the first workflow transitions to the non-running state, the second workflow enters the running state in which the first processed subset data stored in the buffer task is processed by the second workflow. In an example, the first workflow is in the non-running state while the second workflow is running to process the first processed subset data. Thus, the first workflow and the second workflow do not run concurrently but in sequence.

[0181] The method (800) can be adapted as appropriate. Steps in the method (800) can be modified and / or omitted. Other steps can be added. Any suitable order of implementation can be used. In one example, the non-running state of the first workflow includes an idle state or a destroyed state. When the first workflow is in the non-running state, no state information of the first workflow is stored. The first workflow is configured to process a subset of input data. The processing is independent of a remaining subset of the input data outside the first subset.

[0182] The techniques described above, can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. The methods and embodiments of the disclosure can be implemented using one or more processing circuits (for example, one or more processors or one or more integrated circuits). In one example, the one or more processors execute program code stored in a non-transitory computer-readable medium.

[0183] The techniques described above, can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 9 A computer system (900) is shown suitable for implementing certain embodiments of the disclosed subject matter.

[0184] The computer software can be coded using any suitable machine code or computer language, that can be subject to assembly, compilation, linking, or like mechanisms to create code that can be executed directly by the computer CPU(s), GPU(s), or the like. In one example, the computer software includes one or more of an operating system (OS), a database management system (DBMS), one or more applications (apps), or the like.

[0185] The instructions can be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, internet of things devices, and the like.

[0186] Figure 9 The components shown in the computer system (900) are exemplary and not limiting, and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of a computer system (900).

[0187] The computer system (900) can include certain human interface input devices. Such a human interface input device can be a keyboard, touchscreen, or voice recognition device, among others. The computer system (900) can also include certain human interface output devices, such as a display, speakers, or printer, among others. The computer system (900) can also include certain storage devices, such as a hard drive or an optical drive, among others. The computer system (900) can also include certain communication devices, such as a modem, a network interface card, a wireless transmitter / receiver, or the like.

[0188] The human interface input devices can include one or more of: a keyboard (901), mouse (902), trackpad (903), touch screen (910), data glove (not shown), joystick (905), microphone (906), scanner (907), camera (908).

[0189] Computer system (900) can also include certain human interface output devices. Such human interface output devices can be stimulating one or more of the human senses, including but not limited to sight, touch, hearing, taste, and smell. Such human interface output devices can include a display (910) (which can be virtual or virtualized), a haptic feedback device (e.g., a tactile feedback interface (not shown) of a data glove (not shown), a tactile feedback interface (not shown) of a joystick (905), haptic feedback

[0190] Computer system (900) can also include human accessible storage devices and their associated media and interfaces, such as, for example, an optical disk drive (920) with

[0191] Those skilled in the art will realize that the term "computer readable medium" as used in connection with the disclosed subject matter does not encompass transitory

[0192] Computer system (900) can also include an interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks, such as Ethernet, wireless LANs, cellular networks (e.g., 3G, 4G, 5G, LTE, etc.), TV wireline or wireless wide area

[0193] The aforementioned human interface devices, human-accessible storage devices, and network interfaces can be connected to the core (940) of the computer system (900).

[0194] The core (940) can include one or more Central Processing Units (CPU) (941), Graphics Processing Units (GPU) (942), specialized programmable processing units in the form of Field Programmable Gate Arrays (FPGA) (943), hardware accelerators for certain tasks (944), and so on. These devices, along with Read-only memory (ROM) (945), Random-access memory (RAM) (946), internal mass storage such as internal non-user accessible hard drives, solid state drives, or the like (947), can be connected through a system bus (948). In some computer systems, the system bus (948) can be accessible in the form of one or more physical plugs to enable extensions of the system bus (948) through addition of one or more physical plugs (949). The peripheral devices can be attached either directly to the core’s system bus (948), or through a peripheral bus (949). Architectures for a peripheral bus include Peripheral Component Interconnect (PCI), USB, and so on.

[0195] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (945) or RAM (946). Transient data can also be stored in RAM (946), while permanent data can be stored, for example, in internal mass storage (947). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage (947), ROM (945), RAM (946), etc.

[0196] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this application, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0197] By way of example and not limitation, a computer system having an architecture (900), particularly a core (940), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the core (940) that is non-volatile, such as internal mass storage (947) or ROM (945). Software implementing various embodiments of this application can be stored in such a device and executed by the core (940). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (940), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (946) and modifying such data structures according to software-defined processes. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (944)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0198] While several exemplary embodiments have been described, various alterations, permutations and equivalents thereof will be apparent to those skilled in the art. Therefore, the foregoing detailed description of the embodiments does not limit the scope of the application, and the scope of the application is defined by the claims and their equivalents.

Claims

1. A method for processing a workflow, characterized by, The method comprises: receiving workflow information for the workflow; generating the workflow based on the workflow information to process input data, the workflow comprising a plurality of processing tasks and a first buffering task, the plurality of processing tasks comprising a first processing task and a second processing task; causing the first processing task to enter a running state in which, in the running state, a first subset of the input data is processed by the first processing task and output as first processed subset data to the first buffering task and stored in the first buffering task; causing the first processing task to transition from the running state to a non-running state based on an amount of the first processed subset data stored in the first buffering task being equal to a first threshold, the first buffering task being an index first-in-first-out (iFIFO) task configured with an iFIFO buffer or a multi-index first-in-first-out (MiFIFO) task configured with a plurality of iFIFO buffers; and causing the second processing task to enter a running state after the first processing task transitions to the non-running state, in which, in the running state, the first processed subset data stored in the first buffering task is processed by the second processing task; wherein the iFIFO task is configured to: store buffer segments of data input into the iFIFO task, each buffer segment being indexed with a start time, a duration, and a length, an iFIFO buffer size being based on the length and a number of buffer segments in the iFIFO buffer; generate a first event indicating that the amount of the first processed subset data stored in the iFIFO task is equal to the first threshold; and generate a second event indicating that the amount of the first processed subset data stored in the iFIFO task is provided to the second processing task.

2. The method of claim 1, wherein: the second processing task is configured to output, in the running state, the first processed subset data as second processed subset data to a second buffering task and store the second processed subset data in the second buffering task; and the method further comprises: causing the second processing task to transition from the running state to a non-running state based on one of (i) an amount of the second processed subset data stored in the second buffering task being equal to a second threshold and (ii) the amount of the first processed subset data stored in the first buffering task being provided to the second processing task.

3. The method of claim 1, wherein: the plurality of processing tasks are configured with respective task segment durations (TSDs); each of the plurality of processing tasks is configured to process an amount of data equal to the respective TSD, the processing being independent of data outside of the TSD; and the first processing task is configured to process the first subset of the input data in the running state based on a first processing rate (PR) and the second processing task is configured to process the first processed subset data in the running state based on a second PR that is different from the first PR. The method further includes determining a workflow segment duration (WSD) as a least common multiple of the TSDs, the first threshold being based on one of (i) the TSDs, (ii) the WSD, and (iii) the WSD and a parameter S, the parameter S indicating a number of steps the first processing task is configured to run continuously.

4. The method of any one of claims 1-3, wherein, the MiFIFO task is configured to: receive one or more outputs of the first processing task, each of the plurality of iFIFO buffers is configured to store a buffer segment of data input into each of the iFIFO buffers, the buffer segment being from a respective one of the one or more outputs, each buffer segment is indexed with a start time, a duration, and a length, a buffer size of each iFIFO buffer is based on the length and a number of buffer segments in the respective iFIFO buffer; generate a first event indicating that an amount of the first processed subset data stored in the MiFIFO task is equal to the first threshold; and generate a second event indicating that the amount of the first processed subset data stored in the MiFIFO task is provided to the second processing task.

5. The method of any one of claims 1-3, wherein, when the first processing task is in the non-running state, no state information of the first processing task is stored, the non-running state including one of an idle state and a destroyed state.

6. The method of claim 1, wherein, the workflow includes a plurality of buffering tasks, the plurality of buffering tasks including the iFIFO task, each input of the plurality of processing tasks is from one of the input data and one or more of the plurality of buffering tasks, and only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

7. The method of claim 1, wherein, the input data includes a first subset of the input data and a second subset of the input data; the workflow includes a plurality of buffering tasks, the plurality of buffering tasks including the iFIFO task; each input of the plurality of processing tasks is from one of the input data and one or more of the plurality of buffering tasks; the first subset of the input data is processed sequentially by the plurality of processing tasks; after the first subset of the input data is processed by the plurality of processing tasks, the second subset of the input data is processed sequentially by the plurality of processing tasks, the remaining processing tasks of the plurality of processing tasks other than the processing task in the running state are in a non-running state; and only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

8. The method of claim 4, wherein, the workflow includes a plurality of buffering tasks, the plurality of buffering tasks including the MiFIFO task, each input of the plurality of processing tasks is from one of the input data and one or more of the plurality of buffering tasks, and only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

9. The method of claim 4, wherein, the input data comprises a first subset of the input data and a second subset of the input data; the workflow comprises a plurality of buffering tasks, the plurality of buffering tasks comprising the MiFIFO task; each input of the plurality of processing tasks is from one of: the input data and one or more of the plurality of buffering tasks; the first subset of the input data is processed in order by the plurality of processing tasks; after the first subset of the input data is processed by the plurality of processing tasks, the second subset of the input data is processed in order by the plurality of processing tasks, the remaining processing tasks of the plurality of processing tasks other than the processing task in the running state being in a non-running state; and only one of the plurality of processing tasks is allowed to be in the running state at any point in time.

10. A method for processing a plurality of workflows, characterized by, The method comprises: receiving workflow information of the plurality of workflows; generating the plurality of workflows based on the workflow information to process input data, at least one of the plurality of workflows comprising a plurality of processing tasks, the plurality of workflows comprising a first workflow and a second workflow; entering the first workflow into a running state, wherein in the running state, a first subset of the input data is processed by the first workflow and the processed first subset is outputted as first processed subset data to a buffering task and stored in the buffering task; based on an amount of the first processed subset data stored in the buffering task being equal to a threshold value, transitioning the first workflow from the running state to a non-running state, the buffering task being an index first-in-first-out iFIFO task configured with an iFIFO buffer or a multiple index first-in-first-out MiFIFO task configured with a plurality of iFIFO buffers; and after the first workflow is transitioned to the non-running state, entering the second workflow into a running state, wherein in the running state, the first processed subset data stored in the buffering task is processed by the second workflow; the iFIFO task is configured to: store buffer segments of data inputted into the iFIFO task, each buffer segment being indexed with a start time, a duration, and a length, an iFIFO buffer size being based on the length and a number of buffer segments in the iFIFO buffer; generate a first event indicating that the amount of the first processed subset data stored in the iFIFO task is equal to the threshold value; and generate a second event indicating that the amount of the first processed subset data stored in the iFIFO task is provided to the second workflow.

11. The method of claim 10, wherein, the non-running state of the first workflow comprises one of an idle state and a destroy state; when the first workflow is in the non-running state, not storing state information of the first workflow; and the first workflow is configured to process the first subset of the input data, the processing being independent of remaining subsets of the input data other than the first subset.

12. The method of claim 10 or 11, wherein, the MiFIFO task is configured to: receive one or more outputs of the first workflow, each of the plurality of iFIFO buffers being configured to store a buffered segment of data input into each of the iFIFO buffers, the buffered segment being from a respective one of the one or more outputs, each buffered segment being indexed with a start time, a duration, and a length, a buffer size of each iFIFO buffer being based on the length and a number of buffered segments in the respective iFIFO buffer; generate a first event, the first event indicating that an amount of the first processed subset data stored in the MiFIFO task is equal to the threshold value; and generate a second event, the second event indicating that the amount of the first processed subset data stored in the MiFIFO task is provided to the second workflow.

13. An apparatus for processing a workflow, the apparatus comprising: the apparatus comprising processing circuitry configured to perform the method of any of claims 1-12.

14. A non-transitory computer readable storage medium storing instructions, the medium comprising: instructions that, when executed by a computer, cause the computer to perform the method of any of claims 1-12.

15. A computer system, characterized by the computer system comprising: one or more computer-readable non-transitory storage media configured to store computer program code; and one or more computer processors configured to access said computer program code and operate as instructed by said computer program code to perform the method of any of claims 1-12.

Images