Improve task execution and resource management

Abstract

This document describes techniques for improving task execution and resource management. Multiple tasks can be received. These tasks can be associated with multiple workflows. Each workflow includes multiple nodes. Multiple management components can be created. Each management component corresponds to a specific node. Based on the corresponding workflow, a first node and a second node are determined for executing a task among the multiple tasks. The execution of the first part of a task can be managed by the first management component corresponding to the first node. In response to determining that the first part of a task has been completed by the first node, the task is transferred to the second management component corresponding to the second node to execute the second part of the task.

Description

Background Technology

[0001] Performing complex computational tasks can be costly and time-consuming. Furthermore, the computational resources available for performing such tasks may be limited. Therefore, improved techniques for task execution are needed. Attached Figure Description

[0002] The following detailed description will be better understood when read in conjunction with the accompanying drawings. For illustrative purposes, exemplary embodiments of various aspects of this disclosure are shown in the drawings; however, the invention is not limited to the specific methods and means disclosed.

[0003] Figure 1 An example system for implementing asynchronous task execution and resource management according to this disclosure is shown.

[0004] Figure 2 An example system for implementing asynchronous task execution and resource management according to this disclosure is shown.

[0005] Figure 3 An example workflow according to this disclosure is shown.

[0006] Figures 4A to 4E An example dedicated queue according to this disclosure is shown.

[0007] Figure 5 An example process for implementing asynchronous task execution and resource management according to this disclosure is shown.

[0008] Figure 6 An example process for creating a dedicated queue according to this disclosure is shown.

[0009] Figure 7 An example process for implementing asynchronous task execution according to this disclosure is shown.

[0010] Figure 8 An example process for implementing asynchronous task execution according to this disclosure is shown.

[0011] Figure 9 An example process for implementing asynchronous task execution and resource management according to this disclosure is shown.

[0012] Figure 10 An example process for implementing asynchronous tasks and resource management according to this disclosure is shown.

[0013] Figure 11 An example process for implementing resource management according to this disclosure is shown.

[0014] Figure 12 An example computing device is shown that can be used to perform any of the techniques disclosed herein. Detailed Implementation

[0015] Complex computational tasks, such as image stylization, are computationally expensive. These tasks may require significant amounts of graphics processing units (GPUs) and GPU hours. This can be particularly problematic given the limited availability of GPUs. Therefore, techniques for improving task execution and resource management are needed. This paper describes techniques for improving task execution by taking into account task priority, retries, and time constraints. Furthermore, this paper describes techniques for improving resource management to facilitate automatic resource scaling based on task load.

[0016] Figure 1 An example system 100 for asynchronous task execution and resource management is illustrated. System 100 may include user devices(s) 102, server 103, multiple node management components 104a to 104n (collectively referred to as 104), and resource layer 106. Resource layer 106 may include multiple processing resources 108a to 108n (collectively referred to as 108). Each of the multiple processing resources 108a to 108n may include a central processing unit (CPU) or a GPU.

[0017] Multiple user devices 102, server 103, node management component 104, and resource layer 106 can communicate with each other via one or more networks. These networks include various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, and proxy devices. The networks may include physical links such as coaxial cable links, twisted-pair cable links, fiber optic links, and combinations thereof. The networks may include wireless links such as cellular links, satellite links, and Wi-Fi links.

[0018] System 100 can receive tasks from various user devices (including multiple user devices 102). Each task may include a computational task, such as creating AI-generated content (AIGC) effects. Each task may correspond to a specific workflow among multiple workflows. Each workflow may be associated with a specific business line, product, or AIGC effect. Each workflow may include multiple nodes. Each node in a workflow may include executable code and / or multiple machine learning models that take input from the previous node and send its output to the next node until it reaches the final node in the workflow. The output of the final node in the workflow may include the final output (e.g., the result of the task, such as the AIGC effect).

[0019] Each workflow can be used to accomplish multiple tasks. For example, a workflow can be used to accomplish two different tasks associated with the same business line, product, or AIGC effect. The two different tasks can each involve a different subset of the nodes included in that particular workflow. For example, a workflow can be used to accomplish both a first task associated with an AIGC effect and a second different task associated with the same AIGC effect. A workflow can include five nodes: node A, node B, node C, node D, and node E. The execution of the first task can involve nodes A, B, and E, while the execution of the second task can involve nodes C, D, and E.

[0020] Each node in the multiple workflows can correspond to a specific node management component among the multiple node management components 104a to 104n. Each of the multiple node management components 104a to 104n can be configured to manage and implement partial execution of multiple tasks on a specific node. Referring back to the example described above, the execution of the first task associated with the AIGC effect can involve three nodes: node A, node B, and node E. Each of nodes A, B, and E can correspond to a different node management component among the multiple node management components 104a to 104n. For example, node A can correspond to node management component 104a, node B can correspond to node management component 104b, and node E can correspond to node management component 104e. The first task can be input into node A for processing, where node management component 104a manages and implements the partial execution of the first task on node A. Then, the output from node A can be input into node B for processing, where node management component 104b manages and implements the partial execution of the first task on node B. Finally, the output from node E can be input into node E for processing, where the node management component 104e manages and implements part of the execution of the first task on node E.

[0021] In this embodiment, server 103 receives a task from user equipment 102. In response to receiving the task, server 103 can determine the workflow associated with the task. Server 103 can determine a path through the workflow for executing the task. For example, the path may include a first node followed by a second node. Server 103 can send the task to a first node management component corresponding to the first node among a plurality of node management components 104a to 104n. The first node management component can manage and implement the execution of a first portion of the task (e.g., the portion of the task associated with the first node). For example, the first node management component can manage and implement the first portion of the task executed by resource layer 106. In response to resource layer 106 completing the execution of the first portion of the task, resource layer 106 can send a completion indication (e.g., a message) to server 103.

[0022] In response to receiving a completion indication, server 103 can determine the next node in the path based on the workflow. For example, server 103 can determine that a second node is the next node in the path based on the workflow. Server 103 can transfer the task to a second management component corresponding to the second node among multiple node management components 104a to 104n. The second node management component can manage and implement the execution of a second part of the task (e.g., the part of the task associated with the second node). For example, the second node management component can manage and implement the second part of the task executed by resource layer 106.

[0023] If resource layer 106 completes the execution of the second part of the task, it can send a completion indication (e.g., a message) to server 103. If, based on the workflow, the second node is the final node in the path, then in response to receiving the completion indication, server 103 can, for example, transmit a message to user device 102 indicating that the task has been successfully executed. Conversely, if resource layer 106 has not completed the execution of the second part of the task, it can send an indication (e.g., a message) to server 103 that the task execution has not yet been completed. Server 103 can, for example, transmit a message to user device 102 indicating that the task has failed.

[0024] Figure 2 An example system 200 for asynchronous task execution and resource management is shown. System 200 may include user equipment 102, server 103, multiple node management components 104a to 104n (collectively referred to as 104), resource layer 106 including multiple processing resources 108a to 108n (collectively referred to as 108), reporting component 230, and resource scaler 220.

[0025] As mentioned above Figure 1 Each node in the multiple workflows can correspond to a specific node management component among the multiple node management components 104a to 104n. Each node management component 104a to 104n can be configured to manage and implement partial execution of multiple tasks on the specific node. Each node management component 104a to 104n can include a dedicated queue 202 corresponding to the specific node. The manager 210 of the corresponding node can use the dedicated queue 202 to manage and implement partial execution of tasks on the specific node based on a dedicated code that implements the timestamp, priority, and number of execution attempts associated with each task. Each dedicated queue 202 can include a first task list 204a and a second task list 204b. The first task list can include tasks waiting to be processed on the specific node. The second task list 204b can include tasks currently being processed on the specific node. The following is about... Figures 4A to 4E Dedicated queue 202 will be discussed in more detail.

[0026] In an embodiment, each of the multiple node management components 104a to 104n may include at least one queue of unlimited size, such as queues 205a and 205b. For example, queue 205a may be configured to store online tasks (e.g., tasks requiring near real-time processing), while queue 205b may be configured to store offline tasks (e.g., tasks not requiring near real-time processing). Tasks received by the node management components may be categorized into one of the queues of unlimited size before being sent to dedicated queue 202. For example, all offline tasks received by server 103 may be sent to queue 205b instead of directly to dedicated queue 202. Because offline tasks are less urgent, they may be sent to queue 205b to prioritize more urgent tasks. Offline tasks may remain in queue 205b until more urgent tasks have been completed.

[0027] Dedicated queue 202 can be associated with a maximum capacity (e.g., the maximum number of tasks that can be stored in dedicated queue 202). If dedicated queue 202 is not full, online tasks can be sent directly from server 103 to manager 210 and dedicated queue 202 to enable partial execution of online tasks on a specific node. If dedicated queue 202 associated with a specific node management component is full (e.g., at maximum capacity), all tasks sent from server 103 to that node management component can be sent to one of the queues of unlimited size. For example, if the task is an online task, it can be sent to queue 205a. Conversely, if the task is an offline task, it can be sent to queue 205b. Tasks can remain in the queue of unlimited size until dedicated queue 202 is no longer full.

[0028] If server 103 receives a task from user equipment 102, server 103 can determine the workflow associated with the task. For example, refer to... Figure 3 In example system 300, server 103 can determine that a task (e.g., task 1) corresponds to workflow 302. Server 103 can determine the path used to execute the task through workflow 302. The path may include three nodes: node 0, node 1, and node 3. Node 0 may be a source (e.g., a non-functional) node. It should be understood that server 103 may receive other tasks corresponding to workflow 302 from other user devices (e.g., task 2, task 3, task 4, etc.), and those other tasks may be associated with different paths through workflow 302.

[0029] Return to reference Figure 2 Server 103 can send tasks to the first node management component corresponding to the first node (e.g., node 1, since node 0 is the source node) among multiple node management components 104a to 104n. The first node management component can manage and implement the execution of the first part of the task (e.g., the part of the task associated with the first node). For example, the first node management component can send the task to a dedicated queue 202 (or one of a queues of infinite size, if applicable) corresponding to the first node. The manager 210 corresponding to the first node can utilize the dedicated queue 202 to manage and implement the execution of the first part of the task by utilizing multiple processing resources in resource layer 106. In response to resource layer 106 completing the execution of the first part of the task, resource layer 106 can send a completion indication (e.g., a message) to server 103.

[0030] Server 103 can determine the next node in the workflow path based on the received completion indication. For example, server 103 can determine that a second node (e.g., node 3) is the next node in the workflow path. Server 103 can transfer the task to a second management component corresponding to the second node (e.g., node 3) among multiple node management components 104a to 104n. The second node management component can manage and implement the execution of a second part of the task (e.g., the part of the task associated with the second node). For example, the second node management component can send the task to a dedicated queue 202 (or an infinitely large queue among infinitely large queues, if applicable) corresponding to the second node. The manager 210 corresponding to the second node can utilize the dedicated queue 202 to manage and implement the execution of the second part of the task by utilizing multiple processing resources in resource layer 106.

[0031] In this embodiment, the reporting component 230 is configured to monitor metrics for each processing resource 108 associated with each node in the nodes. The reporting component 230 can monitor the metrics for each processing resource 108 to determine how many processing resources 108 are focusing on each workflow node, the number of available processing resources 108, and / or the number of tasks waiting to be processed by each node in the nodes. The reporting component 230 can send data to the resource scaler 220 indicating how many processing resources 108 are focusing on each workflow node, the number of available processing resources 108, and / or the number of tasks waiting to be processed by each node in the nodes. The resource scaler 220 can use this data to scale the processing resources 108 up or down for each node.

[0032] Figures 4A to 4EAn example dedicated queue 202 is shown. As described above, the manager 210 of a specific node can use the dedicated queue 202 to manage and implement partial execution of tasks on the specific node based on dedicated coding that associates the timestamp, priority, and number of execution attempts with each task in the task. Each dedicated queue 202 may include a first task list 204a and a second task list 204b. The first list 204a may include tasks waiting to be processed by the specific node. The second list 204b may include tasks currently being processed by the specific node.

[0033] like Figure 4A As shown, three tasks are listed in the first task list 204a: Task 0, Task 1, and Task 2. Each of the three tasks includes a timestamp and a three-digit code. For example, Task 0 has a timestamp "1717529194042" and a three-digit code "000", Task 1 has a timestamp "1559744564042" and a three-digit code "500", and Task 2 has a timestamp "1465073786042" and a three-digit code "800". The first digit in the three-digit code indicates the task's priority. The second digit in the three-digit code indicates the number of execution attempts associated with each of the tasks (e.g., the number of times resource layer 106 has attempted to execute the task). For example, if all three tasks are submitted simultaneously with different priorities, Task 0 can use the current time as its timestamp, and "000" can be appended to the timestamp to indicate...<priority=0><retry=0> <0> Task 1 and Task 2 can be decremented by the priority years from their timestamps, and the priority can be encoded in hundreds of digits.

[0034] When processing resource 108 becomes available to execute (multiple) tasks, (multiple) tasks in the first task list 204a) are moved to the second task list 204b based on priority. For example, as Figure 4B As shown, in response to the availability of processing resource 108, two tasks can be moved from the first list 204a to the second list 204b. The two tasks moved are task 1 and task 2 because task 1 and task 2 have higher priorities than task 0 (priority "0"). Task 1 has priorities of "5" and "8" respectively. Once a task is started (in... Figure 4B In the example where two tasks have already been started, the tasks receive a new timestamp corresponding to the time they were started. For example, Task 1 and Task 2 now both have the timestamp "1717531217548". This new timestamp is used to ensure that nodes do not spend more than a predetermined amount of time (e.g., a timeout window) processing the tasks. The three-digit code for the tasks can remain the same.

[0035] It can be determined whether the execution of Task 1 and Task 2 has been completed within the scheduled time. If a transient error occurs (e.g., a network error) and / or if processing resource 108 goes offline, the tasks may not complete within the scheduled time. Figure 4C As shown, if the execution of Task 1 has not yet been completed within the predetermined time, and if the second digit in the three-digit code of Task 1 indicates that the number of execution attempts associated with Task 1 is no greater than the retry limit (e.g., the retry limit is 1, meaning each task can be retried once), then Task 1 can be moved back from the second task list 204b to the first task list 204a. The number of execution attempts for Task 1 can be increased by 1, so that the three-digit code of Task 1 is "510" instead of "500". Moving Task 1 back to the first task list 204a can cause Task 1 to lose its timestamp and can make the three-digit code negative. This is because tasks in the first task list 204a that are to be retried should be retried before a new task is first attempted, and higher-priority tasks need to have lower values ​​so that they are pulled from the first task list 204a to the second task list 204b first.

[0036] Similarly, if the execution of Task 2 has not yet been completed within the predetermined time, and if the second digit in the three-digit code of Task 2 indicates that the number of execution attempts associated with Task 2 is not greater than the retry limit, then Task 2 can be moved back from the second task list 204b to the first task list 204a. The number of execution attempts for Task 2 can be increased by 1, thus making the three-digit code of Task 2 "810" instead of "800". Moving Task 2 back to the first task list 204a can cause Task 2 to lose its timestamp and can make the three-digit code negative.

[0037] In this embodiment, the user can discard any task while it is waiting in the dedicated queue 202. For example, the user can discard task 1 while it is waiting to be retried in the first task list 204a. Figure 4D As shown in the example, if the user discards task 1, task 1 can be removed from dedicated queue 202. In response to two processing resources 108 becoming available, two tasks can be moved from the first task list 204a to the second task list 204b. The two tasks moved are task 0 and task 2, as they are the only remaining tasks in the first task list 204a. Once task 0 and task 2 are started, both task 0 and task 2 receive a new timestamp corresponding to the time the task was started. For example, task 0 and task 2 now both have the timestamp "17175365075". This new timestamp is used to ensure that nodes do not spend more than a predetermined amount of time (e.g., a timeout window) processing the task. The three-bit code for each task can remain the same.

[0038] It can be determined whether the execution of tasks 0 and 2 has been completed within the predetermined time. If a transient error occurs (e.g., a network error) and / or if processing resource 108 goes offline, the tasks may not complete within the predetermined time. If the execution of task 0 has not yet completed within the predetermined time, and if the second digit in the three-digit code of task 0 indicates that the number of execution attempts associated with task 0 is not greater than the retry limit, then task 0 can be moved back from the second task list 204b to the first task list 204a. The number of execution attempts for task 0 can be increased by 1, thus making the three-digit code of task 0 “010” instead of “000”. Moving task 0 back to the first task list 204a can cause task 0 to lose its timestamp and can make the three-digit code negative. On the other hand, as... Figure 4E As shown, if the retry execution attempt of task 2 has not been completed within the predetermined time and the retry limit is 1, task 2 can be discarded instead of being moved back to the first task list 204a, and task 2 can be marked as failed.

[0039] Figure 5 An example process 500 for asynchronous task execution and resource management according to this disclosure is shown. Although in Figure 5 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0040] At point 502, multiple tasks can be received (e.g., from various user devices). Each task may include a computational task, such as creating an AIGC effect. Multiple tasks may correspond to multiple workflows. Each task may correspond to a specific workflow within the multiple workflows. Each workflow may be associated with a specific business line, product, or AIGC effect. Each workflow may include multiple nodes. For example, each workflow may be represented by a directed acyclic graph, and each node (e.g., a vertex) may include executable code and / or a machine learning model that takes input from the previous node and sends its output to the next node until it reaches the final node in the workflow. The output of the final node may include the final output (e.g., the result of the task, such as an AIGC effect). Each workflow may be used to implement multiple tasks. For example, a workflow may be used to implement two different tasks associated with the same business line, product, or AIGC effect. The two different tasks may each involve a different subset of the nodes included in that particular workflow.

[0041] At position 504, multiple management components can be created (e.g., multiple node management components 104a to 104n). Each of the multiple management components can correspond to a specific node in multiple workflow nodes. Each of the multiple management components can be configured to manage and implement partial execution of multiple tasks on a specific node.

[0042] At point 506, a first node and a second node can be determined for executing a task among multiple tasks. The first and second nodes can be determined based on a workflow corresponding to the task among multiple workflows (e.g., workflow 302). The task can be sent to a first management component corresponding to the first node among multiple management components. At point 508, the execution of a first portion of the task (e.g., the portion of the task associated with the first node) can be managed by the first management component. For example, the execution of the first portion of the task by a resource layer (e.g., resource layer 106) can be managed and implemented by the first management component. In response to determining that the first portion of the task has been completed on the first node, the task can be transferred to a second management component corresponding to the second node among multiple management components. At point 510, the task can be transferred to the second management component to execute a second portion of the task (e.g., the portion of the task associated with the second node).

[0043] Figure 6 An example process 600 for creating a dedicated queue is shown according to this disclosure. Although in Figure 6 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0044] At point 602, multiple tasks can be received (e.g., from various user devices). Each of the tasks may include a task that performs a computational task, such as creating an AIGC effect. The multiple tasks may correspond to multiple workflows. Each task may correspond to a specific workflow among the multiple workflows. Each workflow among the multiple workflows may include multiple nodes. For example, each workflow among the multiple workflows may be represented by a directed acyclic graph, and each node (e.g., a vertex) may include executable code and / or(multiple) machine learning models that take input from(multiple) previous nodes and send output to the next node until it reaches the final node in the workflow. The output of the final node in the graph may include the final output (e.g., the result of the task, such as an AIGC effect).

[0045] Multiple management components (e.g., multiple node management components 104a to 104n) can be created. Each of the multiple management components can correspond to a specific node among the nodes of multiple workflows. Each of the multiple management components can be configured to manage and implement the partial execution of multiple tasks on the specific node. Each of the multiple node management components can include a dedicated queue corresponding to the specific node (e.g., dedicated queue 202). At 604, multiple dedicated queues can be created based on dedicated encoding that implements the timestamps, priorities, and number of execution attempts associated with each of the multiple tasks. Each dedicated queue includes a first list of tasks waiting to be processed by the specific node (e.g., first task list 204a) and a second list of tasks being processed by the specific node (e.g., first task list 204b).

[0046] Figure 7 An example process 700 for asynchronous task execution according to this disclosure is shown. Although in Figure 7 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0047] Multiple management components (e.g., multiple node management components 104a to 104n) can be created. Each of the multiple management components can correspond to a specific node among the nodes of multiple workflows. Each of the multiple management components can be configured to manage and implement partial execution of multiple tasks on the specific node. Each of the multiple node management components can include a dedicated queue corresponding to the specific node (e.g., dedicated queue 202). At 702, multiple dedicated queues can be created based on dedicated coding that implements the timestamp, priority, and number of execution attempts associated with each of the multiple tasks. Each dedicated queue includes a first list of tasks waiting to be processed by the specific node (e.g., first task list 204a) and a second list of tasks being processed by the specific node (e.g., first task list 204b). At 704, based on the priority of at least one task, at least one task can be moved from the first list to the second list for processing by the specific node. For example, based on determining that at least one task has the highest priority compared to other tasks in the first list, at least one task can be moved from the first list to the second list for processing.

[0048] At point 706, it can be determined whether the execution of at least one task has been completed within the predetermined time. At point 708, in response to determining that the execution of at least one task has not been completed within the predetermined time, at least one task can be moved back to the first list. At point 710, the number of execution attempts for at least one task can be increased. In response to determining that at least one task has not successfully completed, the number of execution attempts for at least one task can be increased.

[0049] Figure 8 An example process 800 for asynchronous task execution according to this disclosure is shown. Although in Figure 8 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0050] Multiple management components (e.g., multiple node management components 104a to 104n) can be created. Each of the multiple management components can correspond to a specific node in multiple workflow nodes. Each of the multiple management components can be configured to manage and implement partial execution of multiple tasks on the specific node. Each node management component can include a dedicated queue (e.g., dedicated queue 202) corresponding to the specific node. Each dedicated queue can include a first task list (e.g., first task list 204a) and a second task list (e.g., second task list 204b). The first task list can include tasks waiting to be processed on the specific node. The second task list can include tasks currently being processed on the specific node. When a processing resource (e.g., processing resource 108) becomes available to execute (multiple) tasks, (multiple) tasks in the first task list are moved to the second task list based on priority. For example, in response to at least one processing resource becoming available, at least one task can be moved from the first list to the second list. Once at least one task is initiated for processing, at least one task obtains a new timestamp corresponding to the time when the task was initiated for processing. This new timestamp can be used to ensure that a particular node does not spend more than a predetermined amount of time (e.g., a timeout window) processing a task.

[0051] At point 802, it can be determined whether the execution of at least one task has been completed on the specific node within a predetermined time period. If a transient error occurs (e.g., a network error) and / or if processing resources are offline, at least one task may not be completed within the predetermined time period. At point 804, in response to determining that the execution of at least one task has not been completed within the predetermined time period, the number of execution attempts for at least one task may be increased. At point 806, it can be determined whether the number of execution attempts exceeds a retry limit (e.g., if the retry limit is 1, each task can be retried once). The execution of at least one task can be retried equal to the number of retry attempts. At point 808, at least one task can be dropped. For example, at least one task can be dropped from the management component corresponding to the specific node. In response to determining that the number of execution attempts for at least one task exceeds the retry limit, at least one task can be dropped, and at least one task can be marked as failed.

[0052] Figure 9 An example process 900 for asynchronous task execution and resource management according to this disclosure is shown. Although in Figure 9 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0053] At point 902, multiple tasks can be received (e.g., from various user devices). Each task may include a task that performs computational tasks, such as creating an AIGC effect. Multiple tasks may correspond to multiple workflows. Each task may correspond to a specific workflow within the multiple workflows. Each workflow may include multiple nodes. For example, each workflow may be represented by a directed acyclic graph, and each node (e.g., a vertex) may include executable code and / or (multiple) machine learning models that take input from (multiple) previous nodes and send output to the next node until it reaches the final node in the graph. The output of the final node in the graph may include the final output (e.g., the result of the task, such as an AIGC effect).

[0054] At position 904, multiple management components can be created (e.g., multiple node management components 104a to 104n). Each of the multiple management components can correspond to a specific node in multiple workflow nodes. Each of the multiple management components can be configured to manage and implement partial execution of multiple tasks on the specific node.

[0055] At 906, a first node and a second node can be determined for executing a task among multiple tasks. The first and second nodes can be determined based on a workflow corresponding to the task among multiple workflows (e.g., workflow 302). The task can be sent to a first management component corresponding to the first node among multiple management components. At 908, the execution of a first portion of the task (e.g., the portion of the task associated with the first node) can be managed and implemented by the first management component. For example, the execution of the first portion of the task by a resource layer (e.g., resource layer 106) can be managed and implemented by the first management component. In response to determining that the first portion of the task has been completed by the first node, the task can be transferred to a second management component corresponding to the second node among multiple management components. At 910, the task can be transferred to the second management component to manage the execution of a second portion of the task (e.g., the portion of the task associated with the second node).

[0056] At point 912, a message indicating that the task has been successfully executed can be sent. This message can be sent, for example, to the user equipment that requested the task. In response to determining that the second part of the task has been successfully completed by the second node, a message indicating that the task has been successfully executed can be sent. Alternatively, at point 914, a message indicating that the task has failed can be sent. This message can be sent, for example, to the user equipment that requested the task. In response to determining that the second part of the task has not yet been successfully completed by the second node, a message indicating that the task has failed can be sent.

[0057] Figure 10 An example process 1000 for resource management according to this disclosure is shown. Although in Figure 10 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0058] At point 1002, multiple tasks can be received (e.g., from various user devices). Each task may include a task that performs computational tasks, such as creating an AIGC effect. Multiple tasks may correspond to multiple workflows. Each task may correspond to a specific workflow within the multiple workflows. Each workflow may include multiple nodes. For example, each workflow may be represented by a directed acyclic graph, and each node (e.g., a vertex) may include executable code and / or a machine learning model that takes input from the previous node and sends its output to the next node until it reaches the final node in the graph. The output of the final node in the graph may include the final output (e.g., the result of the task, such as an AIGC effect).

[0059] At point 1004, multiple management components (e.g., multiple node management components 104a to 104n) can be created. Each of the multiple management components can correspond to a specific node among the nodes of multiple workflows. Each of the multiple management components can be configured to manage the partial execution of multiple tasks on a specific node. Each of the multiple management components can include at least one queue of unlimited size (e.g., queue 205a and / or queue 205b). At point 1006, it can be determined whether a task among the multiple tasks is an online task (e.g., a task that needs to be processed in near real-time) or an offline task (e.g., a task that does not need to be processed in near real-time). At point 1008, based on the determination that a task is an offline task, the task can be inserted into at least one queue of unlimited size.

[0060] Figure 11 An example process 1100 for resource management according to this disclosure is shown. Although in Figure 11 The operations are depicted as a sequence of operations, but those skilled in the art will understand that various embodiments may add, remove, reorder, or modify the depicted operations.

[0061] At point 1102, multiple tasks can be received (e.g., from various user devices). Each task may include a computational task, such as creating an AIGC effect. Multiple tasks may correspond to multiple workflows. Each task may correspond to a specific workflow within the multiple workflows. Each workflow may include multiple nodes. For example, each workflow may be represented by a directed acyclic graph, and each node (e.g., a vertex) may include executable code and / or a machine learning model that takes input from the previous node and sends its output to the next node until it reaches the final node in the graph. The output of the final node in the graph may include the final output (e.g., the result of the task, such as the AIGC effect).

[0062] At 1104, metrics for each processing resource (e.g., processing resource 108) associated with each node in multiple workflows can be monitored. These metrics can indicate how much processing resource is being monitored at each node, the amount of available processing resources, and / or the number of tasks waiting to be processed by each node in the multiple workflows. At 1106, based on the determined available processing resources and / or the number of tasks waiting to be processed by each node in the multiple workflows, the processing resources can be scaled up or down.

[0063] Figure 12 It shows that it can be used in, for example Figure 1 The computing devices used in various aspects of the models, components, and / or devices depicted in Figure 4. Regarding Figure 1As shown in Figure 4, any or all components can be freely separated. Figure 12 This is implemented using one or more instances of the computing device 1200. Figure 12 The computer architecture shown illustrates a conventional server computer, workstation, desktop computer, laptop computer, tablet computer, network device, PDA, e-reader, digital cellular phone, or other computing node, and can be used to perform any aspect of the computer described herein, such as to implement the methods described herein.

[0064] The computing device 1200 may include a substrate or “motherboard,” which is a printed circuit board to which multiple components or devices can be connected via a system bus or other electrical communication path. One or more central processing units (CPUs) 1204 may operate in conjunction with a chipset 1206. The CPUs 1204 may be standard programmable processors that perform the arithmetic and logic operations required to perform the operations of the computing device 1200.

[0065] Multiple CPUs 1204 can perform necessary operations by manipulating switching elements that distinguish and change these states, transitioning from one discrete physical state to the next. Switching elements typically include electronic circuitry, such as flip-flops, that maintains one of two binary states, and electronic circuitry that provides an output state based on a logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements can be combined to create more complex logic circuits, including registers, adder-subtractor units, arithmetic logic units, floating-point units, etc.

[0066] The (multiple) CPUs 1204 can be expanded or replaced by other processing units such as (multiple) GPUs 1205. The (multiple) GPUs 1205 may include processing units specifically designed for, but not necessarily limited to, highly parallel computing, such as graphics and other visualization-related processing.

[0067] Chipset 1206 can provide an interface between CPU(s) 1204 and the remaining components and devices on the substrate. Chipset 1206 can also provide an interface for random access memory (RAM) 1208, which serves as the main memory in computing device 1200. Chipset 1206 can also provide an interface for computer-readable storage media such as read-only memory (ROM) 1220 or non-volatile RAM (NVRAM) (not shown) to store basic routines that can help boot computing device 1200 and transfer information between various components and devices. ROM 1220 or NVRAM can also store other software components required for the operation of computing device 1200 according to the aspects described herein.

[0068] Computing device 1200 can operate in a networked environment using a logical connection to remote computing nodes and computer systems via a local area network (LAN). Chipset 1206 may include functionality for providing network connectivity via a network interface controller (NIC) 1222 (such as a Gigabit Ethernet adapter). NIC 1222 can connect computing device 1200 to other computing nodes via network 1218. It should be understood that multiple NICs 1222 may exist in computing device 1200, connecting the computing device to other types of networks and remote computer systems.

[0069] Computing device 1200 can be connected to mass storage device 1228, which provides non-volatile storage for the computer. Mass storage device 1228 can store system programs, application programs, other program modules, and data, as described in more detail herein. Mass storage device 1228 can be connected to computing device 1200 via storage controller 1224 connected via chipset 1206. Mass storage device 1228 may include one or more physical storage units. Mass storage device 1228 may include management component 1210. Storage controller 1224 can interface with physical storage units via Serial Attached SCSI (SAS) interface, Serial Advanced Technology Attached (SATA) interface, Fibre Channel (FC) interface, or other types of interfaces used for physically connecting and transferring data between the computer and physical storage units.

[0070] The computing device 1200 can store data on the mass storage device 1228 by changing the physical state of the physical storage units to reflect that information is being stored. The specific changes in physical state can depend on various factors and the different implementations described herein. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage units and whether the mass storage device 1228 is characterized as a primary storage device or a secondary storage device.

[0071] For example, computing device 1200 can store information in mass storage device 1228 by issuing instructions via storage controller 1224 to change the magnetic properties of a specific location within a disk drive unit, the reflection or refraction properties of a specific location in an optical storage unit, or the electrical properties of a specific capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of the physical medium are also possible without departing from the scope and spirit of this specification; the foregoing examples are provided merely for ease of description. Computing device 1200 can also read information from mass storage device 1228 by detecting the physical state or characteristics of one or more specific locations within a physical storage unit.

[0072] In addition to the aforementioned high-capacity storage device 1228, the computing device 1200 can access other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. Those skilled in the art will understand that a computer-readable storage medium can be any available medium that provides storage for non-transitory data and can be accessed by the computing device 1200.

[0073] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, transient and non-transitory computer-readable storage media implemented in any method or technology, as well as removable and non-removable media. Computer-readable storage media include, but are not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technologies, compact disc ROM (“CD-ROM”), digital versatile disc (“DVD”), high-definition DVD (“HD-DVD”), Blu-ray or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices, other magnetic storage devices, or any other medium that can be used to store desired information in a non-transitory manner.

[0074] Such as Figure 12 The mass storage device 1228 shown can store the operating system used to control the operation of the computing device 1200. The operating system may include a version of the Linux operating system. The operating system may include a version of the Windows Server operating system from Microsoft. Depending on another aspect, the operating system may include a version of the UNIX operating system. Various mobile phone operating systems, such as iOS and Android, may also be used. It should be understood that other operating systems may also be used. The mass storage device 12228 can store other systems, applications, and data used by the computing device 1200.

[0075] Mass storage device 1228 or other computer-readable storage medium may also be encoded with computer-executable instructions that, when loaded into computing device 1200, transform the computing device from a general-purpose computing system into a special-purpose computer capable of implementing the aspects described herein. As described above, these computer-executable instructions transform computing device 1200 by specifying how CPU(s) 1204 transition between states. Computing device 1200 can access the computer-readable storage medium storing the computer-executable instructions, which, when executed by computing device 1200, can perform the methods described herein.

[0076] Such as Figure 12The computing device 1200 shown may further include an input / output controller 1232 for receiving and processing input from multiple input devices such as a keyboard, mouse, touchpad, touchscreen, electronic pen, or other types of input devices. Similarly, the input / output controller 1232 may provide output to a display such as a computer monitor, flat panel display, digital projector, printer, plotter, or other types of output devices. It should be understood that the computing device 1200 may not include... Figure 12 All components shown may include Figure 12 Other components not explicitly shown in the document, or those that can be utilized with Figure 12 The architecture shown is completely different.

[0077] As described in this article, a computing device can be a physical computing device, such as... Figure 12 The computing device 1200. A computing node may also include virtual machine host processes and one or more virtual machine instances. Computer-executable instructions may be indirectly executed by the physical hardware of the computing device by interpreting and / or executing instructions stored and executed in the context of the virtual machine.

[0078] It should be understood that the methods and systems are not limited to specific methods, specific components, or particular implementations. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0079] Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” as used in the specification and appended claims include plural indicators. A range may be expressed herein as from “about” a particular value and / or to “about” another particular value. When expressing such a range, another embodiment includes from one particular value and / or to another particular value. Similarly, when a value is expressed as an approximation using the antecedent “about,” it should be understood that the particular value forms another embodiment. It should also be understood that the endpoints of each range are significant both relative to and independent of the other endpoint.

[0080] "Optional" or "optionally" means that the event or situation described below may or may not occur, and the description includes instances where the event or situation occurs and instances where it does not occur.

[0081] Throughout the description and claims of this specification, the word “comprising” and variations thereof, such as “comprising” and “including,” mean “including but not limited to” and are not intended to exclude, for example, other components, integers, or steps. “Exemplary” means “an example of…” and is not intended to convey indications of preferred or ideal embodiments. “Like” is not used in a limiting sense but for illustrative purposes.

[0082] Components that can be used to perform the described methods and systems are described. When describing combinations, subsets, interactions, groups, etc., of these components, it should be understood that although specific references to each of the various individual and collective combinations and arrangements of these components may not be explicitly described, each is specifically considered and described herein for all methods and systems. This applies to all aspects of this application, including but not limited to operations in the described methods. Therefore, if various additional operations exist that can be performed, it should be understood that each of these additional operations can be performed using any particular embodiment or combination of embodiments of the described methods.

[0083] The methods and systems of the present invention can be more readily understood by referring to the following detailed description of preferred embodiments and examples included therein, as well as the accompanying drawings and their descriptions.

[0084] Those skilled in the art will understand that the methods and systems may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the methods and systems may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied within the storage medium. More specifically, the methods and systems may take the form of computer software implemented on the web. Any suitable computer-readable storage medium may be used, including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices.

[0085] The following description of embodiments of methods and systems is based on block diagrams and flowcharts of methods, systems, apparatuses, and computer program products. It should be understood that each block in the block diagrams and flowcharts, as well as combinations of blocks in the block diagrams and flowcharts, can be implemented by computer program instructions. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, create apparatus for implementing the functions specified in one or more flowchart blocks.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can instruct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including computer-readable instructions for implementing the functions specified in one or more flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowchart blocks.

[0087] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some implementations. The methods and processes described herein are not limited to any particular order and may be executed in other suitable orders with respect to their associated blocks or states. For example, described blocks or states may be executed in an order different from that specifically described, or multiple blocks or states may be combined in a single block or state. Example blocks or states may be executed serially, in parallel, or in some other manner. Blocks or states may be added to or removed from the described example embodiments. The example systems and components described herein may be configured differently from those described. For example, elements may be added to, removed from, or rearranged from the described example embodiments compared to the described exemplary embodiments.

[0088] It should also be understood that the items illustrated are stored in memory or on a storage device when in use, and these items or portions thereof may be transferred between memory and other storage devices for memory management and data integrity purposes. Alternatively, in other embodiments, some or all of the software modules and / or system may execute in memory on another device and communicate with the illustrated computing system via inter-computer communication. Furthermore, in some embodiments, some or all of the system and / or modules may be implemented or provided in other ways, such as at least in part as firmware and / or hardware, including but not limited to one or more application-specific integrated circuits (“ASICs”), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field-programmable gate arrays (“FPGAs”), complex programmable logic devices (“CPLDs”), etc. Some or all of the modules, systems, and data structures may also be stored (e.g., as software instructions or structured data) on computer-readable media, such as hard disks, memory, networks, or portable media articles, for retrieval by appropriate devices or via appropriate connections. The systems, modules, and data structures can also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagation signal) over various computer-readable transmission media, including wireless and wired / cable-based media, and can take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). In other embodiments, such computer program products can also take other forms. Therefore, the present invention can be implemented using other computer system configurations.

[0089] While methods and systems have been described in conjunction with preferred embodiments and specific examples, they are not intended to limit the scope to the particular embodiments illustrated, as the embodiments herein are intended in all respects to be illustrative rather than restrictive.

[0090] Unless otherwise expressly stated, it is not intended that any method described herein require its operations to be performed in a particular order. Therefore, no order is intended to be inferred in any way where the method claims do not actually describe the order of their operations or where the claims or description do not otherwise specify that the operations will be limited to a particular order. This applies to any possible non-expressive basis of interpretation, including: logical questions concerning the arrangement of steps or flow of operations; general meanings derived from grammatical organization or punctuation; and the number or type of embodiments described in the description.

[0091] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit of this disclosure. Other embodiments will be apparent to those skilled in the art in light of the description and practice described herein. The description and example figures are to be considered exemplary only, and their true scope and spirit are indicated by the appended claims.

Claims

1. A method for task execution, comprising: Receive multiple tasks, wherein the multiple tasks correspond to multiple workflows, and wherein each of the multiple workflows includes multiple nodes; Multiple management components are created, each of which corresponds to a specific node in the multiple workflow nodes, wherein each of the multiple management components is configured to manage and implement a portion of the execution of the multiple tasks on the specific node; Based on the workflows in the plurality of workflows, determine the first node and the second node for executing the corresponding tasks in the plurality of tasks; The execution of the first part of the task is managed and implemented by utilizing a first management component corresponding to the first node, wherein the first management component is among the plurality of management components; as well as In response to determining that the first part of the task has been completed by the first node, the task is transferred to a second management component corresponding to the second node for the execution of the second part of the task, wherein the second management component is among the plurality of management components.

2. The method according to claim 1, wherein creating a plurality of management components, each management component corresponding to a specific node, further comprises: Based on dedicated coding that associates the timestamp, priority, and number of execution attempts with each of the plurality of tasks, a plurality of dedicated queues are created, each of the plurality of dedicated queues corresponding to the specific node.

3. The method of claim 2, wherein each of the dedicated queues includes a first list of tasks waiting to be processed by the specific node, and each of the dedicated queues also includes a second list of tasks being processed by the specific node.

4. The method according to claim 3, further comprising: Based on the priority of the at least one task, move the at least one task from the first list to the second list for processing by the specific node; as well as Determine whether the execution of the at least one task has been completed within a predetermined time period.

5. The method according to claim 4, further comprising: In response to determining that the execution of the at least one task has not been completed within the predetermined time amount, the at least one task is moved to the first list; as well as Increase the number of execution attempts for the at least one task.

6. The method of claim 5, further comprising: Determine whether the number of execution attempts exceeds the retry limit; as well as In response to determining that the number of execution attempts for the at least one task exceeds the retry limit, the at least one task is discarded.

7. The method according to claim 1, further comprising: In response to determining that the second part of the task has been completed by the second node, a message indicating that the task has been successfully executed is transmitted; or In response to determining that the second part of the task has not yet been completed by the second node, a message indicating that the task has failed is transmitted.

8. The method of claim 1, wherein each of the plurality of management components comprises at least one queue of infinite size, and wherein the method further comprises: Determine whether the task is online or offline; as well as Based on the determination that the task is an offline task, the task is inserted into the at least one queue of infinite size.

9. The method according to claim 1, further comprising: Monitor the metrics of each processing resource associated with each of the nodes in the plurality of workflows; as well as Processing resources are scaled up or down based on the available processing resources and the number of tasks waiting to be processed by each of the nodes in the plurality of workflows.

10. A task execution system, comprising: At least one processor; as well as At least one memory, communicatively coupled to the at least one processor, and including computer-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations, the operations including: Receive multiple tasks, wherein the multiple tasks correspond to multiple workflows, and wherein each of the multiple workflows includes multiple nodes; Multiple management components are created, each of which corresponds to a specific node in the multiple workflow nodes, wherein each of the multiple management components is configured to manage and implement a portion of the execution of the multiple tasks on the specific node; Based on the workflows in the plurality of workflows, determine the first node and the second node for executing the corresponding tasks in the plurality of tasks; The execution of the first part of the task is managed and implemented by utilizing a first management component corresponding to the first node, wherein the first management component is among the plurality of management components; and In response to determining that the first part of the task has been completed by the first node, the task is transferred to a second management component corresponding to the second node for the execution of the second part of the task, wherein the second management component is among the plurality of management components.

11. The system of claim 10, wherein creating a plurality of management components, each management component corresponding to a specific node, further comprises: Multiple dedicated queues are created based on dedicated codes that associate the timestamp, priority, and number of execution attempts with each of the multiple tasks. Each of the multiple dedicated queues corresponds to the specific node. Each of the dedicated queues includes a first list of tasks waiting to be processed by the specific node, and each of the dedicated queues also includes a second list of tasks being processed by the specific node.

12. The system of claim 11, further comprising: Based on the priority of the at least one task, move the at least one task from the first list to the second list for processing by the specific node; as well as Determine whether the execution of the at least one task has been completed within a predetermined time period.

13. The system of claim 12, further comprising: In response to determining that the execution of the at least one task has not been completed within the predetermined time amount, the at least one task is moved to the first list; as well as Increase the number of execution attempts for the at least one task.

14. The system of claim 13, further comprising: Determine whether the number of execution attempts exceeds the retry limit; as well as In response to determining that the number of execution attempts for the at least one task exceeds the retry limit, the at least one task is discarded.

15. The system of claim 10, further comprising: Monitor the metrics of each processing resource associated with each of the nodes in the plurality of workflows; as well as Processing resources are scaled up or down based on the available processing resources and the number of tasks waiting to be processed by each of the nodes in the plurality of workflows.

16. A non-transient computer-readable storage medium storing computer-readable instructions that, when executed by a processor, cause the processor to perform operations, the operations including: Receive multiple tasks, wherein the multiple tasks correspond to multiple workflows, and wherein each of the multiple workflows includes multiple nodes; Multiple management components are created, each of which corresponds to a specific node in the multiple workflow nodes, wherein each of the multiple management components is configured to manage and implement a portion of the execution of the multiple tasks on the specific node; Based on the workflows in the plurality of workflows, determine the first node and the second node for executing the corresponding tasks in the plurality of tasks; The execution of the first part of the task is managed and implemented by utilizing a first management component corresponding to the first node, wherein the first management component is among the plurality of management components; as well as In response to determining that the first part of the task has been completed by the first node, the task is transferred to a second management component corresponding to the second node for the execution of the second part of the task, wherein the second management component is among the plurality of management components.

17. The non-transient computer-readable storage medium of claim 16, wherein the creation of a plurality of management components, each management component corresponding to a specific node, further comprises: Based on dedicated coding that associates the timestamp, priority, and number of execution attempts with each of the plurality of tasks, a plurality of dedicated queues are created, each of the plurality of dedicated queues corresponding to the specific node, wherein each of the dedicated queues includes a first list of tasks waiting to be processed by the specific node, and each of the dedicated queues also includes a second list of tasks being processed by the specific node.

18. The non-transient computer-readable storage medium of claim 17, further comprising: Based on the priority of the at least one task, move the at least one task from the first list to the second list for processing by the specific node; as well as Determine whether the execution of the at least one task has been completed within a predetermined time period.

19. The non-transient computer-readable storage medium of claim 18, further comprising: In response to determining that the execution of the at least one task has not been completed within the predetermined time amount, the at least one task is moved to the first list; as well as Increase the number of execution attempts for the at least one task.

20. The non-transient computer-readable storage medium of claim 19, further comprising: Determine whether the number of execution attempts exceeds the retry limit; as well as In response to determining that the number of execution attempts for the at least one task exceeds the retry limit, the at least one task is discarded.

Images