Workflow task processing method, device and system, storage medium and product

Abstract

The invention discloses a workflow task processing method, device and system, a storage medium and a product, and the method comprises the steps: obtaining the current saturation of a resource on which a to-be-executed task depends and a temporary storage threshold value of the to-be-executed task relative to the resource when the to-be-executed task corresponding to the workflow is triggered and the to-be-executed task is not distributed; on the basis of the current saturation of the resource on which the to-be-executed task depends and the temporary storage threshold value corresponding to the to-be-executed task, when it is judged that the pause condition of the to-be-executed task is met, the distribution of the to-be-executed task is paused, and in the process of pausing the distribution of the to-be-executed task, the to-be-executed task is executed. Monitoring a current saturation degree of a resource on which the to-be-executed task depends and a current swap-in threshold value corresponding to the to-be-executed task; the current swap-in threshold value is related to the duration of pausing distribution of the to-be-executed task; and based on the current saturation of the resource on which the to-be-executed task depends and the current swap-in threshold value corresponding to the to-be-executed task, under the condition that it is judged that the wake-up condition of the to-be-executed task is met, the to-be-executed task is recovered to be distributed.

Description

Technical Field

[0001] The present application relates to the field of software technology, and in particular to a workflow task processing method and device, system, storage medium, and product. Background Art

[0002] A distributed workflow engine is a software system used to manage and execute distributed tasks, usually for large-scale data processing and analysis tasks. Currently, there are many workflow engine products used to process multi-task work, such as Diffy, DophinScheduler, elsa-workflows, etc. However, these workflow engine products generally do not consider the temporary storage and recovery functions during the task execution process. Once a resource that a task relies on is unavailable among the many tasks in a workflow, it will directly cause the execution of the workflow to fail, or even the crash of the entire workflow engine system.

[0003] In this regard, the multi-task scheduling system proposes the function of temporarily storing tasks during execution and restoring them according to priority. However, in actual work scenarios, high-priority tasks are continuously executed, and low-priority tasks will never be executed. Even when resources fully meet the execution conditions of each task, all tasks cannot use resources fairly, resulting in low task processing efficiency in the workflow.

[0004] How to fully utilize resources while ensuring the stable execution of workflows is a technical problem that needs to be solved at present. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a workflow task processing method and device, system, readable storage medium, and computer program product to solve the problem of low efficiency of workflow task processing.

[0006] In order to solve the above technical problems, this specification is implemented as follows: In a first aspect, a workflow task processing method is provided, comprising: When a task to be executed corresponding to a workflow is triggered and the task to be executed has not yet been dispatched, obtaining a current saturation of a resource on which the task to be executed depends and a temporary storage threshold of the task to be executed relative to the resource; Based on the current saturation of the resources on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, if it is determined that the suspension condition of the task to be executed is met, the dispatch of the task to be executed is suspended, and in the process of suspending the dispatch of the task to be executed, the current saturation of the resources on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored; the current swap-in threshold is related to the duration of the suspension of the dispatch of the task to be executed; Based on the current saturation of the resource on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed, if it is determined that the wake-up condition of the task to be executed is met, the task to be executed is resumed and dispatched.

[0007] Optionally, during the process of suspending the dispatching of the to-be-executed task, monitoring the current saturation of the resource on which the to-be-executed task depends and the current swap-in threshold corresponding to the to-be-executed task includes: In the process of suspending the dispatch of the to-be-executed tasks, the current swap-in threshold corresponding to the to-be-executed tasks at the current moment is calculated based on a user anxiety level simulation algorithm; the user anxiety level simulation algorithm is an exponential function that changes over time by simulating the user's anxiety level for task execution based on user feedback and survey data; Calculate the current saturation of the resource on which the task to be executed depends at the current moment according to the ratio of the occupied capacity of the resource on which the task to be executed depends at the current moment to the total capacity of the resource on which the task to be executed depends; By comparing the current saturation of the resource on which the task to be executed depends at the current moment with the current swap-in threshold corresponding to the task to be executed, it is determined whether the wake-up condition of the task to be executed is met.

[0008] Optionally, the user anxiety level simulation algorithm is expressed as follows: , Wherein y represents the current swap-in threshold, a is the fitting coefficient, e is the natural constant, t represents the duration between the moment when the task to be executed is suspended and the current moment, and b represents the preset initial swap-in threshold.

[0009] Optionally, the temporary storage threshold is related to the importance and occupied resources of the task to be executed, wherein the higher the importance of the task to be executed, the larger the corresponding temporary storage threshold; the fewer the occupied resources of the task to be executed, the larger the corresponding temporary storage threshold.

[0010] Optionally, the suspension condition of the task to be executed includes: the current saturation of the resource on which the task to be executed depends is higher than the temporary storage threshold corresponding to the task to be executed; The wake-up conditions of the task to be executed include: the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed; or the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed, and the current saturation of the resources on which the task to be executed depends is not higher than the preset maximum swap-in threshold corresponding to the task to be executed.

[0011] Optionally, the method further comprises: In the case of suspending the dispatch of the task to be executed, the parameter information and execution context information of the task to be executed are temporarily stored in the target database.

[0012] Optionally, the resuming and dispatching the to-be-executed task includes: Acquire parameter information and execution context information of the task to be executed from the target database; Determine a corresponding executor according to the task type of the task to be executed; The acquired parameter information and execution context information of the task to be executed are sent to the executor to call and execute the task to be executed.

[0013] Optionally, the tasks to be executed corresponding to the triggered workflow include: In response to triggering of a target event, initiating the workflow, the workflow comprising a plurality of task nodes; Each task node included in the workflow is traversed to determine the to-be-executed task currently to be dispatched.

[0014] In a second aspect, a workflow task processing system is provided, comprising: The scheduler is used to traverse each task node included in the initiated workflow to determine the tasks to be executed currently to be dispatched; A resource manager, used to calculate the current saturation of the resources on which the task to be executed depends; A dispatcher, for, when a to-be-executed task corresponding to a triggered workflow has not yet been dispatched, judging whether to suspend dispatching the to-be-executed task based on a temporary storage threshold of the to-be-executed task relative to the resource and a current saturation of the resource on which the to-be-executed task depends as queried from the resource manager, and for, during the process of suspending dispatching the to-be-executed task, judging whether to resume dispatching the to-be-executed task based on a current swap-in threshold of the to-be-executed task and a current saturation of the resource on which the to-be-executed task depends as queried from the resource manager, wherein the current swap-in threshold is related to the duration of suspending dispatching of the to-be-executed task; The executor is used to schedule and execute the dispatched tasks to be executed.

[0015] In a third aspect, a workflow task processing device is provided, comprising a processor and a memory, wherein the memory stores a program or instruction that can be executed on the processor, and when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0016] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] In a fifth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the steps of the method described in the first aspect.

[0018] In an embodiment of the present application, when a task to be executed corresponding to a triggered workflow is not yet dispatched, the current saturation of a resource on which the task to be executed depends and a temporary storage threshold of the task to be executed relative to the resource are obtained; based on the current saturation of the resource on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, when it is determined that the suspension condition of the task to be executed is met, the dispatch of the task to be executed is suspended, and in the process of suspending the dispatch of the task to be executed, the current saturation of the resource on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored; the current swap-in threshold is related to the duration of the suspension of the dispatch of the task to be executed; based on the current saturation of the resource on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, the dispatch of the task to be executed is suspended. The current swap-in threshold corresponding to the task is determined to meet the wake-up conditions of the task to be executed, and the dispatch of the task to be executed is resumed. In this way, the task execution order can be dynamically adjusted in combination with system resources and the priority of tasks relative to resources, so as to effectively perform task execution management, make full use of system resources and improve system throughput, realize full utilization of resources while ensuring stable execution of workflows, and dynamically calculate the priority of corresponding tasks in combination with the length of time that users can wait for task dispatch to be suspended, effectively balance system resources with user needs and expectations, achieve maximum throughput and stability of the system while meeting user needs and feelings, ensure that high-priority tasks can be executed in time, and avoid low-priority tasks being ignored for a long time, improve user experience, and improve task processing efficiency of workflows. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a flowchart of a task processing method of a workflow in an embodiment of the present application.

[0020] Figure 2 It is a structural block diagram of a task processing system of a workflow in an embodiment of the present application.

[0021] Figure 3 It is a structural block diagram of a task processing device of a workflow in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. The numbering of the drawings in this application is only used to distinguish the various steps in the scheme, and is not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0023] In order to solve the problems existing in the prior art, the present application embodiment provides a workflow task processing method, such as Figure 1 As shown, the following steps 102 to 106 are included.

[0024] Step 102: when a task to be executed corresponding to a workflow is triggered and has not yet been dispatched, obtain the current saturation of a resource on which the task to be executed depends and a temporary storage threshold of the task to be executed relative to the resource.

[0025] A workflow is a directed circular graph that includes multiple task nodes. These task nodes have a corresponding execution order. Each task node corresponds to a task to be executed. Each task to be executed will occupy resources when executed, that is, it has corresponding dependent resources. The resources that different tasks to be executed depend on may be different or the same.

[0026] The to-be-executed tasks corresponding to the triggered workflow include: initiating the workflow in response to the triggering of a target event, the workflow including a plurality of task nodes; traversing the task nodes included in the workflow to determine the to-be-executed tasks currently to be dispatched.

[0027] The distributed workflow engine system can process multiple workflows. The workflows are triggered by corresponding events. Events can include timed events set by users. For example, if a user sets a timed automatic post or video, when the set time arrives, the timed event occurs, triggering the workflow of automatically posting posts or videos. Events can also include Kafka messages, which carry relevant data about the workflow corresponding to the task that the user needs to perform. Alternatively, it also includes external triggering of corresponding events by calling the Application Programming Interface (API) provided by the workflow engine.

[0028] When the user triggers a target event, the corresponding workflow will be initiated, and the tasks to be executed on each task node in the workflow will be traversed and dispatched and executed in the corresponding order. If the traversal determines that the target task to be executed needs to be dispatched, the current saturation of the resources on which the target task to be executed depends and the temporary storage threshold of the target task to be executed relative to the dependent resources are obtained before dispatching the target task to be executed.

[0029] The temporary storage threshold can be predetermined based on a combination of one or more of the importance of the target task to be executed and the amount of resources occupied, and is a fixed value. Optionally, the temporary storage threshold is related to the importance of the task to be executed and the occupied resources, wherein the higher the importance of the task to be executed, the larger the corresponding temporary storage threshold; the fewer the occupied resources of the task to be executed, the larger the corresponding temporary storage threshold. For example, the more important the target task to be executed, the larger the temporary storage threshold; the fewer the resources occupied by the target task to be executed, the larger the temporary storage threshold.

[0030] The current saturation of the resources that the target task to be executed depends on indicates the current occupation of the corresponding resources. The resources that the target task to be executed depends on may be occupied by multiple tasks being executed at the same time. The more resources are occupied, the higher the saturation of the resources.

[0031] Step 104, based on the current saturation of the resources on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, it is determined that the suspension condition of the task to be executed is met, and the dispatch of the task to be executed is suspended. In the process of suspending the dispatch of the task to be executed, the current saturation of the resources on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored; the current swap-in threshold is related to the duration of the suspension of the dispatch of the task to be executed.

[0032] Specifically, by comparing the current saturation of the resource on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, it can be determined whether the task to be executed is currently being dispatched normally or needs to be suspended. In one embodiment, the suspension condition of the task to be executed includes: the current saturation of the resource on which the task to be executed depends is higher than the temporary storage threshold corresponding to the task to be executed.

[0033] That is to say, if the current saturation of the resources that the tasks to be executed depend on is higher, the tasks to be executed are more likely to be suspended. If the current saturation is higher than the preset temporary storage threshold of the tasks to be executed, the tasks to be executed need to be temporarily stored and stopped from being dispatched, so as to avoid too many tasks occupying the corresponding resources, resulting in overuse of resources and crash and termination, so as to ensure the smooth operation of the workflow engine system.

[0034] The method further comprises: in the case of suspending the dispatching of the to-be-executed task, temporarily storing the parameter information and execution context information of the to-be-executed task in a target database.

[0035] By dynamically adjusting the task execution queue, when resources cannot meet the needs of multiple tasks being executed concurrently, tasks that meet the suspension conditions will be removed from the execution queue and suspended, and the information or data required for the execution of the suspended tasks will be saved in the database, waiting for the opportunity to be put back into the execution queue.

[0036] Accordingly, if the current saturation of the dependent resources is not higher than the preset temporary storage threshold of the pending tasks, the pending tasks can be normally dispatched and executed. When the resources fully meet the execution conditions of each task, all tasks can use the resources fairly.

[0037] In the case where dispatching needs to be suspended, the current saturation of the resources on which the tasks to be executed depend is monitored in real time while dispatching the tasks to be executed is suspended. As mentioned above, the current saturation reflects the current occupancy of the resources on which the tasks to be executed depend. As time goes by, some of the multiple tasks that originally occupied the resources may have been completed, which will release the corresponding occupied resources. Other tasks that occupy more resources may also be added. Therefore, the current saturation of the resources on which the tasks to be executed depend may change over time and is not a fixed value. By monitoring the current saturation of the resources on which the tasks to be executed depend, this parameter is one of the indicators used to determine whether to resume dispatching the tasks to be executed.

[0038] In addition, the current swap-in threshold of the pending task is monitored at the same time. This parameter is one of the indicators used to determine whether to resume the dispatch of the pending task in the future, and is related to the length of time the user can wait for the pending task to be suspended. After the pending task is suspended, the corresponding user is in a waiting state. The longer the user waits for the pending task to be suspended, the higher the user's anxiety level. The swap-in threshold is used to indicate the user's anxiety level about the pending task. The swap-in threshold of the currently suspended task can be calculated by an algorithm that simulates the user's anxiety level.

[0039] Specifically, in the process of suspending the dispatch of the task to be executed, the current saturation of the resources on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored, including: in the process of suspending the dispatch of the task to be executed, the current swap-in threshold corresponding to the task to be executed at the current moment is calculated based on a user anxiety level simulation algorithm; the user anxiety level simulation algorithm is an exponential function that changes with time and is obtained by simulating the user's anxiety level for task execution based on user feedback and survey data; the current saturation of the resources on which the task to be executed depends at the current moment is calculated based on the ratio of the occupied capacity of the resources on which the task to be executed depends at the current moment to the total capacity of the resources on which it depends; and whether the wake-up conditions of the task to be executed are met by comparing the current saturation of the resources on which the task to be executed depends at the current moment and the current swap-in threshold corresponding to the task to be executed.

[0040] The real-time calculation of the swap-in threshold is to achieve a balance between resource utilization and user experience by simulating the user's anxiety level about task execution. Through a large amount of previous user feedback and survey data, the user's anxiety level for each task can be evaluated when waiting for different durations. For example, the horizontal axis is the waiting duration of the target task, and the vertical axis is the total number of users waiting for the target task to be executed corresponding to each duration. The longer the waiting duration, the fewer the total number of users, indicating that as the duration increases, more and more users cannot bear to continue waiting for the execution of the target task and exit or close the execution of the target task, and the anxiety level of users waiting for the execution of the target task increases. Based on this, a function curve is fitted as the user anxiety curve.

[0041] Specifically, after mining and investigating a large amount of user feedback and survey data, it was found that the user's anxiety level about task execution follows an exponential relationship that changes over time. Therefore, the task priority can be reflected by the user's anxiety level and defined as a percentage value, which is related to the length of time the user can wait for the corresponding task to be suspended.

[0042] Optionally, the user anxiety level simulation algorithm is expressed as follows: , Wherein y represents the current swap-in threshold, a is the fitting coefficient, e is the natural constant, t represents the duration between the moment when the task to be executed is suspended and the current moment, and b represents the preset initial swap-in threshold.

[0043] That is, the user's anxiety level changes with the length of time it takes to resume the task after it is suspended, which is consistent with the formula . Among them, e is a natural constant; t is the length of time that the task to be executed has stopped being dispatched, which is the time interval between the current moment and the moment when the dispatch is suspended; y is the current swap-in threshold to be calculated, which changes with time; a is the fitting coefficient obtained after calculation through the above fitting process. b is the initially set swap-in threshold. Optionally, the initial swap-in threshold is less than the temporary storage threshold corresponding to the task to be executed. For example, the initial swap-in threshold is obtained by subtracting 10% from the temporary storage threshold corresponding to the task to be executed. Through the calculation of the above y function and variable t, the current swap-in threshold of the task to be dispatched can be calculated.

[0044] The above algorithm for simulating user anxiety level can simulate the user's anxiety level about task execution based on user data, and obtain an exponential function that changes with time as a basis for evaluating task priority. This is a novel task priority evaluation method that can better reflect the user's real needs and feelings about waiting for the target task, and can be used as a priority indicator for pausing or resuming the execution of the corresponding task.

[0045] The swap-in threshold is related to the user's anxiety level about how long they can wait for different tasks. The user's anxiety level for different tasks is different, and the corresponding swap-in threshold is also different. The larger the swap-in threshold, the higher the priority of the corresponding task to be restored.

[0046] The current saturation of the resources that the task to be executed depends on can be calculated based on the ratio of the currently occupied capacity of the dependent resources to the total capacity. The total capacity of the resources is fixed, the currently occupied capacity may be variable, and the larger the currently occupied capacity, the greater the current saturation of the resources that the task to be executed depends on.

[0047] By comparing the swap-in threshold of the task to be executed at the current time with the current saturation of the resource on which it depends, it is monitored whether the current saturation of the resource on which the task to be executed depends and the current swap-in threshold meet the wake-up condition of the task to be executed.

[0048] Step 106 : based on the current saturation of the resource on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed, if it is determined that the wake-up condition of the task to be executed is met, the task to be executed is resumed for dispatch.

[0049] In one embodiment, the wake-up condition of the task to be executed includes: a current saturation of a resource on which the task to be executed depends is lower than a current swap-in threshold corresponding to the task to be executed.

[0050] That is to say, at the current moment, the calculated swap-in threshold and the resource saturation are compared. If the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed, it is determined that the wake-up conditions for the task to be executed are met, indicating that it is necessary to give priority to the user's anxiety level about the task, and give priority to executing the task to ensure that high-priority tasks can be executed in a timely manner, meet user needs and feelings, and improve user experience.

[0051] Assume that the temporary storage threshold set for the dependent resource of the task to be executed is 60%, and the initial swap-in threshold is 40%. If the current saturation of the resource is 75% before the task is dispatched, and the current saturation exceeds the temporary storage threshold, the dispatch of the task will be suspended. At the same time, the current saturation of the resource is queried regularly, and the real-time swap-in threshold of the resource corresponding to the task is calculated. Assume that at a certain moment, the current saturation of the resource is 55%, and the calculated real-time swap-in threshold is 78%. When the swap-in threshold exceeds the current saturation, the task is resumed for execution.

[0052] Correspondingly, if the current saturation of the resource on which the task to be executed depends is higher than the current swap-in threshold of the task to be executed, the wake-up condition is not met, and the task to be executed continues to be suspended from being dispatched.

[0053] To ensure the stability of the entire workflow engine system, in another embodiment, the wake-up conditions of the task to be executed include: the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed, and the current saturation of the resources on which the task to be executed depends is not higher than the preset maximum swap-in threshold corresponding to the task to be executed.

[0054] In this embodiment, a maximum swap-in threshold is also set. If the calculated current swap-in threshold corresponding to the task to be executed exceeds the maximum swap-in threshold, the maximum swap-in threshold is used as a basis for determining whether the wake-up condition is met, thereby serving as a means of protecting the workflow engine system. For example, the set maximum swap-in threshold is 92%, and the calculated current swap-in threshold is 91%. When the current saturation of the resource reaches 90%, the current swap-in threshold of 91% is higher than the current saturation of the resource of 90%, and the current saturation of 90% is not higher than the maximum swap-in threshold of 92%, then the task is resumed for execution.

[0055] Specifically, the recovery and dispatching of the task to be executed includes: obtaining parameter information and execution context information of the task to be executed from the target database; determining the corresponding executor according to the task type of the task to be executed; and sending the obtained parameter information and execution context information of the task to be executed to the executor to call and execute the task to be executed.

[0056] Accordingly, if the current saturation of the resources on which the task to be executed depends is higher than the current swap-in threshold corresponding to the task to be executed, or the current saturation of the resources on which the task to be executed depends is higher than the preset maximum swap-in threshold of the task to be executed, the wake-up condition is not met and the task to be executed continues to be suspended.

[0057] For example, the maximum swap-in threshold is set to 90%, and the calculated current swap-in threshold is 92%. When the current saturation of the resource reaches 91%, even if the current swap-in threshold of 92% calculated based on the user anxiety level curve exceeds the current saturation of 91%, the resource will not be restored or swapped in because the current saturation of 91% of the resource is higher than the maximum swap-in threshold of 92%, thereby avoiding the resource from reaching the critical value and ensuring the smooth operation of the system.

[0058] By simulating the user's anxiety level and dynamically calculating the task swap threshold, we can achieve dynamic protection of resources. The entire workflow engine system can avoid crashes and terminations caused by excessive use of dependent resources. At the same time, we can also resume the execution of temporarily stored tasks in time before the user becomes overly anxious, thereby improving the user experience.

[0059] In an embodiment of the present application, when a task to be executed corresponding to a triggered workflow is not yet dispatched, the current saturation of a resource on which the task to be executed depends and a temporary storage threshold of the task to be executed relative to the resource are obtained; based on the current saturation of the resource on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, when it is determined that the suspension condition of the task to be executed is met, the dispatch of the task to be executed is suspended, and in the process of suspending the dispatch of the task to be executed, the current saturation of the resource on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored; the current swap-in threshold is related to the duration of the suspension of the dispatch of the task to be executed; based on the current saturation of the resource on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, the dispatch of the task to be executed is suspended. The current swap-in threshold corresponding to the task is determined to meet the wake-up conditions of the task to be executed, and the dispatch of the task to be executed is resumed. In this way, the task execution order can be dynamically adjusted in combination with system resources and the priority of tasks relative to resources, so as to effectively perform task execution management, make full use of system resources and improve system throughput, realize full utilization of resources while ensuring stable execution of workflows, and dynamically calculate the priority of corresponding tasks in combination with the length of time that users can wait for task dispatch to be suspended, effectively balance system resources with user needs and expectations, achieve maximum throughput and stability of the system while meeting user needs and feelings, ensure that high-priority tasks can be executed in time, and avoid low-priority tasks being ignored for a long time, improve user experience, and improve task processing efficiency of workflows.

[0060] Alternatively, if Figure 2As shown, the embodiment of the present application also provides a workflow task processing system, including: The scheduler 1400 is used to traverse each task node included in the initiated workflow to determine the tasks to be executed that are currently to be dispatched; The resource manager 1800 is used to calculate the current saturation of the resources on which the task to be executed depends; The dispatcher 1600 is used to determine whether to suspend the dispatch of the to-be-executed task based on the temporary storage threshold of the to-be-executed task relative to the resource and the current saturation of the resource on which the to-be-executed task depends queried from the resource manager when the to-be-executed task corresponding to the triggered workflow has not been dispatched yet; in the process of suspending the dispatch of the to-be-executed task, determine whether to resume the dispatch of the to-be-executed task based on the current swap-in threshold of the to-be-executed task and the current saturation of the resource on which the to-be-executed task depends queried from the resource manager, wherein the current swap-in threshold is related to the length of time a user can wait for the to-be-executed task to be suspended from dispatching; The executor 1900 is used to schedule and execute the dispatched tasks to be executed.

[0061] The initiated workflow can be initiated by the trigger 1200 in response to the triggering of the target event. The workflow includes multiple task nodes. The target event 1100 can include a timed event, a kafka message or an API. The executor 1900 can include a FaaS executor, a large model executor, an Http executor, a user-defined executor, etc. The storage service 1700 is used to temporarily store the parameter information and execution context information of the suspended tasks to be executed.

[0062] The workflow task processing system here can correspond to the workflow engine system mentioned above. The workflow of the entire system is described as follows.

[0063] The workflow engine consists of several parts, such as trigger 1200, scheduler 1400, dispatcher 1600, executor 1900, and resource manager 1800. When the target event 1100 occurs, the trigger 1200 will be triggered to execute, submit and initiate a workflow, and the scheduler 1400 is responsible for traversing each task node in the flowchart corresponding to the workflow and passing the task to be executed to the dispatcher 1600. After the dispatcher 1600 receives the task to be executed, it determines the executor 1900 responsible for executing the task according to the type of the task, and dispatches the task to the specific executor 1900 for execution. For example, a task of the type of Http protocol is sent to the Http executor for calling and execution, and a task of the type of artificial intelligence (AI) large model is dispatched to the large model executor, etc. The executor 1900 is responsible for the specific execution of the task, such as calling the Http interface, executing the AI ​​large model task, etc., and feeds back the execution result status and the result data of the task execution to the dispatcher 1600, thereby completing the execution of this task.

[0064] The resource manager 1800 can calculate the current saturation of the resources that the task to be executed depends on, and the dispatcher 1600 can calculate the current swap-in threshold of the task to be executed, and determine the suspension condition and wake-up condition of the task to be executed. Specifically, the dispatcher 1600 can implement Figure 1 To avoid repetition, the various processes implemented by the workflow task processing method embodiment are not described here.

[0065] Alternatively, if Figure 3 As shown, an embodiment of the present application also provides a workflow task processing device 2000, including a processor 2400 and a memory 2200, and the memory 2200 stores a program or instruction that can be run on the processor 2400. When the program or instruction is executed by the processor 2400, the various steps of the above-mentioned workflow task processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0066] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the task processing method embodiment of any of the above-mentioned workflows is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The readable storage medium includes a computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0067] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program can be operated to enable a computer to implement the various processes of the task processing method embodiment of any of the above-mentioned workflows when executed, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0068] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0069] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0070] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A workflow task processing method, characterized in that: include: When a task to be executed corresponding to a workflow is triggered and the task to be executed has not yet been dispatched, obtaining a current saturation of a resource on which the task to be executed depends and a temporary storage threshold of the task to be executed relative to the resource; Based on the current saturation of the resources on which the task to be executed depends and the temporary storage threshold corresponding to the task to be executed, if it is determined that the suspension condition of the task to be executed is met, the dispatch of the task to be executed is suspended, and in the process of suspending the dispatch of the task to be executed, the current saturation of the resources on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed are monitored; the current swap-in threshold is related to the duration of the suspension of the dispatch of the task to be executed; Based on the current saturation of the resource on which the task to be executed depends and the current swap-in threshold corresponding to the task to be executed, if it is determined that the wake-up condition of the task to be executed is met, the task to be executed is resumed and dispatched.

2. The method according to claim 1, characterized in that In the process of suspending the dispatching of the to-be-executed task, monitoring the current saturation of the resource on which the to-be-executed task depends and the current swap-in threshold corresponding to the to-be-executed task includes: In the process of suspending the dispatch of the to-be-executed tasks, the current swap-in threshold corresponding to the to-be-executed tasks at the current moment is calculated based on a user anxiety level simulation algorithm; the user anxiety level simulation algorithm is an exponential function that changes over time by simulating the user's anxiety level for task execution based on user feedback and survey data; Calculate the current saturation of the resource on which the task to be executed depends at the current moment according to the ratio of the occupied capacity of the resource on which the task to be executed depends at the current moment to the total capacity of the resource on which the task to be executed depends; By comparing the current saturation of the resource on which the task to be executed depends at the current moment with the current swap-in threshold corresponding to the task to be executed, it is determined whether the wake-up condition of the task to be executed is met.

3. The method according to claim 2, characterized in that The user anxiety simulation algorithm is expressed as follows: ， Wherein y represents the current swap-in threshold, a is the fitting coefficient, e is the natural constant, t represents the duration between the moment when the task to be executed is suspended and the current moment, and b represents the preset initial swap-in threshold.

4. The method according to claim 1, characterized in that The temporary storage threshold is related to the importance and occupied resources of the task to be executed, wherein the higher the importance of the task to be executed, the larger the corresponding temporary storage threshold; the fewer occupied resources of the task to be executed, the larger the corresponding temporary storage threshold.

5. The method according to claim 1, characterized in that The suspension condition of the task to be executed includes: the current saturation of the resource on which the task to be executed depends is higher than the temporary storage threshold corresponding to the task to be executed; The wake-up conditions of the task to be executed include: the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed; or the current saturation of the resources on which the task to be executed depends is lower than the current swap-in threshold corresponding to the task to be executed, and the current saturation of the resources on which the task to be executed depends is not higher than the preset maximum swap-in threshold corresponding to the task to be executed.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: In the case of suspending the dispatch of the task to be executed, the parameter information and execution context information of the task to be executed are temporarily stored in the target database.

7. The method according to claim 6, characterized in that The resuming and dispatching of the to-be-executed task includes: Acquire parameter information and execution context information of the task to be executed from the target database; Determine a corresponding executor according to the task type of the task to be executed; The acquired parameter information and execution context information of the task to be executed are sent to the executor to call and execute the task to be executed.

8. The method according to claim 1, characterized in that The tasks to be executed corresponding to the triggered workflow include: In response to triggering of a target event, initiating the workflow, wherein the workflow includes a plurality of task nodes; Each task node included in the workflow is traversed to determine the to-be-executed task currently to be dispatched.

9. A workflow task processing system, characterized in that: include: The scheduler is used to traverse each task node included in the initiated workflow to determine the tasks to be executed currently to be dispatched; A resource manager, used to calculate the current saturation of the resources on which the task to be executed depends; a dispatcher, configured to determine whether to suspend dispatching of a to-be-executed task based on a temporary storage threshold of the to-be-executed task relative to the resource and a current saturation of the resource on which the to-be-executed task depends queried from the resource manager when a to-be-executed task corresponding to the triggered workflow has not been dispatched; In the process of suspending the dispatch of the task to be executed, judging whether to resume dispatching the task to be executed based on the current swap-in threshold of the task to be executed and the current saturation of the resource on which the task to be executed depends queried from the resource manager, wherein the current swap-in threshold is related to the duration of suspending the dispatch of the task to be executed; The executor is used to schedule and execute the dispatched tasks to be executed.

10. A workflow task processing device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images