A task scheduling method, device, server, and storage medium

Abstract

The embodiment of the present invention discloses a task scheduling method, device, server, and storage medium, wherein the method comprises: calling a scheduler to topologically sort a pre-built directed acyclic graph operation graph to determine a task node to be executed from at least one task node included in the directed acyclic graph operation graph; if the task node to be executed is not a termination node, then executing a business sub-process in a target business corresponding to the task node to be executed by calling a processing interface, and when receiving the business sub-process execution result returned by the processing interface, triggering the scheduler to perform a step of topologically sorting the pre-built directed acyclic graph operation graph to determine the task node to be executed from at least one task node included in the directed acyclic graph operation graph. By adopting the embodiment of the present invention, not only can the utilization rate of resources be improved, but also the execution relationship between each business sub-process can be intuitively understood, which is conducive to reducing the maintenance cost of task scheduling.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, server, and storage medium. Background Art

[0002] At present, the task scheduling model includes synchronous blocking model and asynchronous non-blocking model. The synchronous blocking model usually limits the execution of the actual process to synchronous serial execution, which limits the processing capacity of the program and thus limits the utilization of resources. The asynchronous non-blocking model, such as the asynchronous programming model promise / future model, where future represents an asynchronous task result that may not have been actually completed, and promise represents an asynchronous abstract processing object. The promise / future model realizes the asynchronous execution process by separating the task executor and the task result receiver. The task executor holds the promise, and the result receiver holds the future representing the result. When the task executor completes the task, the future takes effect and the receiver can obtain the result. However, this model separates the relationship between the execution processes in the target business, and it is impossible to intuitively understand the relationship between the business sub-processes and the execution process. Since the business scenarios that task scheduling needs to support are relatively complex, the asynchronous programming model is used for task scheduling, and the maintenance cost is relatively high.

[0003] Therefore, how to improve resource utilization while reducing model maintenance costs has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present invention provide a task scheduling method, device, server, and storage medium, which execute each business sub-process in the target business based on the topological sorting corresponding to the directed acyclic graph operation graph, which can not only improve the utilization of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling.

[0005] On the one hand, an embodiment of the present invention provides a task scheduling method, including:

[0006] Calling a scheduler to topologically sort a pre-constructed directed acyclic graph operation graph to determine a task node to be executed from at least one task node included in the directed acyclic graph operation graph, wherein the directed acyclic graph operation graph is constructed based on directed acyclic graph description information;

[0007] If the task node to be executed is not a termination node, executing the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface;

[0008] When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to execute the step of topologically sorting the pre-built directed acyclic graph operation graph to determine the task node to be executed from at least one task node included in the directed acyclic graph operation graph.

[0009] On the other hand, an embodiment of the present invention provides a task scheduling device, including:

[0010] A sorting module, used for calling a scheduler to perform topological sorting on a pre-built directed acyclic graph (DAG) operation graph, so as to determine a task node to be executed from at least one task node included in the DAG operation graph, wherein the DAG operation graph is constructed based on the DAG description information;

[0011] A processing module, configured to execute a business sub-process in a target business corresponding to the task node to be executed by calling a processing interface if it is detected that the task node to be executed is not a termination node;

[0012] The sorting module is also used to trigger the scheduler to execute the step of topologically sorting the pre-built directed acyclic graph operation graph when receiving the business sub-process execution result returned by the processing interface, so as to determine the task node to be executed from at least one task node included in the directed acyclic graph operation graph.

[0013] Correspondingly, an embodiment of the present invention also provides a server, comprising: a processor and a storage device; the storage device is used to store program instructions; the processor calls the program instructions to execute: calling a scheduler to perform topological sorting on a pre-built directed acyclic graph operation graph to determine a task node to be executed from at least one task node included in the directed acyclic graph operation graph, wherein the directed acyclic graph operation graph is constructed based on directed acyclic graph description information; if the task node to be executed is not a termination node, executing a business sub-process in a target business corresponding to the task node to be executed by calling a processing interface; when receiving the business sub-process execution result returned by the processing interface, triggering the scheduler to execute the step of topologically sorting the pre-built directed acyclic graph operation graph to determine the task node to be executed from at least one task node included in the directed acyclic graph operation graph.

[0014] Correspondingly, an embodiment of the present invention further provides a computer storage medium, in which program instructions are stored. When the program instructions are executed, they are used to implement the above methods.

[0015] In an embodiment of the present invention, a scheduler can be called to perform topological sorting on a pre-constructed directed acyclic graph operation graph to determine a task node to be executed from at least one task node included in the directed acyclic graph operation graph. Furthermore, if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling a processing interface. When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to perform a topological sorting on the pre-constructed directed acyclic graph operation graph to determine the step of determining the task node to be executed from at least one task node included in the directed acyclic graph operation graph. By adopting an embodiment of the present invention, each business sub-process in the target business is executed based on the topological sorting corresponding to the directed acyclic graph operation graph, which can not only improve the utilization rate of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flowchart of a task scheduling method according to an embodiment of the present invention;

[0018] Figure 2 It is a schematic diagram of a directed acyclic graph operation diagram corresponding to a target service in an embodiment of the present invention;

[0019] Figure 3 It is a schematic diagram of topologically sorting a directed acyclic graph operation graph according to an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of the relationship between a business sub-process and a scheduler according to an embodiment of the present invention;

[0021] Figure 5 is a flowchart of another task scheduling method according to an embodiment of the present invention;

[0022] Figure 6 is a flowchart of another task scheduling method according to an embodiment of the present invention;

[0023] Figure 7 is a structural schematic diagram of a task scheduling device according to an embodiment of the present invention;

[0024] Figure 8 It is a structural diagram of a server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Common task scheduling models include synchronous blocking execution model and asynchronous non-blocking execution model. Among them, the synchronous blocking model usually limits the execution of the actual process to synchronous serial execution, which limits the processing capacity of the program, thereby limiting the utilization of resources and failing to describe the essential relationship between processes.

[0026] A common implementation mechanism of the asynchronous non-blocking model is asynchronous callback. The typical model is the asynchronous programming model promise / future. Promise represents an asynchronous abstract processing object, and future represents the result of an asynchronous task that may not have been actually completed. A callback function can be added to this result so that corresponding operations can be performed after the task execution succeeds or fails. The promise is handed over to the task executor, and the task executor can mark the task completion or failure through the promise. This model implements the asynchronous execution process by separating the task executor and the task result receiver. The task executor holds the promise, and the result receiver holds the future representing the result (which may not be completed). When the task executor completes the task, the future takes effect and the receiver can obtain the result.

[0027] Expanding on this basis, the result receiver can add a callback to the future, which will be executed when the future takes effect, thus achieving the effect of asynchronous execution. Using the promise / future model through the serial / parallel combination of futures, the execution process of the service flow can be fully described, which is consistent with the expected execution process.

[0028] Among them, although the promise / future model can describe the service process execution process and is consistent with the expected execution process, the logic implementation is relatively cumbersome when describing the process execution process, the code readability is low, and when the process is changed, the context logic changes involved are relatively large, and the maintenance cost is high.

[0029] In addition, the coroutine model that has been popular in recent years is also a synchronous blocking model in theory. Concurrent processing between different coroutines can be implemented on it. The essential relationship between processes can be characterized through proper organization. However, the implementation method of organizing concurrent execution between coroutines has poor readability and is not conducive to program maintenance.

[0030] It can be seen that both the coroutine model and the asynchronous non-blocking model separate the relationship between the target business and the corresponding business sub-process execution process, and it is impossible to intuitively understand the relationship and execution process between the target business and the corresponding business sub-process.

[0031] In order to solve the above problems, the present invention proposes a task scheduling method, which can call a scheduler to perform topological sorting on a pre-built directed acyclic graph (DAG) operation diagram to determine the task node to be executed from at least one task node included in the DAG operation diagram. Further, if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling a processing interface, and when the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to perform topological sorting on the pre-built DAG operation diagram to determine the step of the task node to be executed from at least one task node included in the DAG operation diagram. Using the embodiment of the present invention, the topological sorting corresponding to the DAG operation diagram is used to execute each business sub-process in the target business, which can not only improve the utilization rate of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling. Wherein, the above-mentioned directed acyclic graph DAG refers to a graph in which any one edge has a direction and there is no loop.

[0032] In one embodiment, the above-mentioned task scheduling method may be applied to a server, and the server may be a server or a server group for performing task scheduling, which is not specifically limited in the present invention.

[0033] In one embodiment, Figure 1 A flowchart of task scheduling is shown. First, the target business can be divided. In one embodiment, the processing process of the target business can be decomposed into multiple business sub-processes according to the business logic, and each business sub-process is marked as a task node in the directed acyclic graph DAG. For example, the target business is a search business, and the multiple business sub-processes corresponding to the target may include: obtaining search keywords, querying target data matching the search keywords from the database, and outputting target data matching the search keywords to the front end. Furthermore, the processing logic corresponding to each processing sub-process can be pre-packaged.

[0034] Furthermore, the directed acyclic graph description information can be obtained, and a DAG operation diagram corresponding to the target business can be constructed based on the directed acyclic graph description information. Among them, the directed acyclic graph description information characterizes the dependency relationship between the business sub-processes corresponding to the target business. The directed acyclic graph description information is a collection of relationships between business sub-processes, and the business sub-processes are referenced in the form of process aliases. Exemplarily, the description method corresponding to the directed acyclic graph description information is A->B, indicating that there is an edge between B and A, and business sub-process B depends on the execution of business sub-process A, and can only be executed after A is executed. If a business sub-process has no dependency on other business sub-processes, it is described using an isolated node method, such as C.

[0035] For example, assuming that the directed acyclic graph description information includes A->B, B->C, A->D, D->E, C->F and E->F, then the DAG operation diagram corresponding to the target business constructed based on the directed acyclic graph description information is as follows: Figure 2 shown.

[0036] It can be seen that the information described in the directed acyclic graph is an intuitive description of the execution relationship between the target business and the corresponding business sub-processes, which reduces the process binding code and reduces the cost of changes when the target business changes.

[0037] After building the DAG operation diagram corresponding to the target business, each task node can be bound to a pre-installed corresponding processing logic, that is, the processing logic corresponding to each pre-packaged business sub-process, and introduce a unique virtual start node and end node to facilitate the management of the execution of the target business.

[0038] The above processing logic can be a conditional expression or function corresponding to the business sub-process corresponding to each task node. All task nodes corresponding to the target business can be strung together to realize the complete business process of the target business. In this way, developers no longer need to write programs through hard code. They only need to control the input source, write each executor or C-like expression, and then adjust the business flexibly by adjusting the task nodes, without the need to repeatedly write code, compile and publish, which is conducive to improving the iterative efficiency of task scheduling.

[0039] Furthermore, the scheduler can be called to execute a scheduling function to perform a topological sorting on the pre-built DAG operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph. If the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface, that is, the processing logic pre-bound to the task node to be executed is executed. When the execution result of the business sub-process returned by the processing interface is received, the scheduler is triggered to perform a topological sorting on the pre-built DAG operation graph to determine the next task node to be executed from at least one task node included in the DAG operation graph, and so on, until the termination node is detected. When the termination node is detected, the termination process corresponding to the termination node is executed by calling the processing interface. When the termination process execution result returned by the processing interface is received, the statistical information corresponding to the execution of the target business is output.

[0040] In one embodiment, the statistical information includes at least one of the following: business sub-process execution results, time-consuming statistical information corresponding to executing the target business, and the number of executions of the business sub-process corresponding to the task node to be executed. Developers can locate problems and conduct business analysis based on the statistical information.

[0041] In one embodiment, during the DAG topology sorting process, the scheduler may determine a task node with an in-degree of 0 as a task node to be executed. When the in-degree is 0, it means that the point in the directed graph is not the end point of any edge, that is, the edges connected to this point all take this point as the starting point, for example Figure 2 Point A in .

[0042] For example, see Figure 3 , assuming the pre-built DAG operation diagram is as follows Figure 3 As shown in S30 in the figure, the scheduler can topologically sort the DAG operation graph corresponding to S30, first detect that the task node with in-degree 0 in the DAG operation graph is A, then A can be determined as the node to be executed. Further, if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface. When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to take A out of the DAG operation graph corresponding to S30, obtain the DAG operation graph corresponding to S31, and topologically sort the DAG operation graph corresponding to S31, detect that the task nodes with in-degree 0 in the current DAG operation graph are B and D, and both B and D are determined as the task nodes to be executed. If the task nodes to be executed B and D are not termination nodes, the processing interface is called to concurrently execute the business sub-processes in the target business corresponding to the task nodes to be executed B and D. Furthermore, when the execution results of the business sub-processes corresponding to the task nodes B and D to be executed are received from the processing interface, the scheduler is triggered again to take out B and D from the DAG operation diagram corresponding to S31, obtain the DAG operation diagram corresponding to S32, and topologically sort the DAG operation diagram corresponding to S32, and detect that the task nodes with in-degree 0 in the current DAG operation diagram are C and E, and both C and E are determined as task nodes to be executed. If the task nodes C and E to be executed are not termination nodes, the processing interface is called to concurrently execute the business sub-processes in the target business corresponding to the task nodes C and E to be executed, and so on, until the termination node is detected, and the termination process corresponding to the termination node is executed by calling the processing interface. When the execution result of the termination process returned by the processing interface is received, the statistical information corresponding to the execution of the target business is output.

[0043] in, Figure 3 The task nodes B and D to be executed, and the task nodes C and E to be executed belong to the same level. In one embodiment, in the case where there are multiple task nodes to be executed at the same level, the server can concurrently execute the business sub-processes in the target business corresponding to each task node to be executed by calling the processing interface, thereby improving the execution efficiency of the business sub-processes and further improving the efficiency of task scheduling.

[0044] In one embodiment, the relationship between each business sub-process corresponding to the target business and the scheduler can be seen in Figure 4 As shown. Figure 4 It can be seen that the server can input the directed acyclic graph description information into the scheduler. The directed acyclic graph description information represents the dependency relationship between the business sub-processes corresponding to the target business, and each task node corresponds to a business sub-process. After the directed acyclic graph description information is input into the scheduler, the scheduler can build a DAG operation graph corresponding to the target business based on the directed acyclic graph description information. The node name of each task node in the DAG operation graph can find the mapped business sub-process. Furthermore, the scheduler topologically sorts the DAG operation graph corresponding to the target business, selects the task node to be executed, and calls the processing interface to execute the business sub-process corresponding to the task node to be executed, and returns the execution result to the scheduler through the callback function when the business sub-process corresponding to the task node to be executed is executed.

[0045] It can be seen that the task scheduling method proposed in the embodiment of the present invention can naturally describe the execution relationship between the business sub-processes corresponding to the target business, and adopts the method of thread pool or coroutine pool to achieve complete parallelism for the to-be-executed task nodes at the same level of topological sorting. Its scheduling process achieves the natural serial / parallel execution of each business sub-process without the developer's perception. Compared with the above-mentioned synchronous blocking model, coroutine and asynchronous non-blocking model, on the one hand, it is not limited to synchronous serial or asynchronous execution, but is based on topological sorting to execute each business sub-process in series / parallel, which is conducive to improving resource utilization; on the other hand, it can save a lot of process binding code and reduce the cost of modification.

[0046] See also Figure 5 , is a flowchart of a task scheduling process of an embodiment of the present invention, and the method of the embodiment of the present invention can be executed by a server or a server group. The method of the embodiment of the present invention includes the following steps.

[0047] S501: calling a scheduler to topologically sort a pre-built directed acyclic graph (DAG) operation graph to determine a task node to be executed from at least one task node included in the DAG operation graph, where the DAG operation graph is built based on directed acyclic graph description information.

[0048] In one embodiment, developers can decompose the processing of the target business into multiple business sub-processes according to the business logic, and each business sub-process is marked as a task node in the directed acyclic graph DAG. Each business sub-process does not limit the specific execution mode, which can be synchronous execution or asynchronous execution.

[0049] Furthermore, in one embodiment, after the target business is decomposed into steps, the developer can configure the execution relationship between each business sub-process corresponding to the target business in the development corresponding interface. After the developer completes the configuration, the server can automatically generate the directed acyclic graph description information corresponding to the target business, and the directed acyclic graph description information represents the dependency relationship between each business sub-process corresponding to the target business, and each task node in at least one task node corresponds to a business sub-process in the target business. In this way, the developer can flexibly adjust the execution relationship between each business sub-process through visual configuration, without the need to repeatedly write code, compile and publish, thereby improving the iterative efficiency of task scheduling.

[0050] Further, in one embodiment, the server can obtain the directed acyclic graph description information, and can construct a DAG operation diagram corresponding to the target business based on the directed acyclic graph description information, wherein each task node in the DAG operation diagram corresponds to a business sub-process of the target business. The specific implementation method of constructing the DAG operation diagram corresponding to the target business based on the directed acyclic graph description information can be referred to in the above embodiment for Figure 2 Related description.

[0051] S502: If the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling a processing interface.

[0052] In one embodiment, after the server constructs the DAG operation diagram, a unique virtual start node and end node can be introduced into the DAG operation diagram, and a corresponding relationship between each task node in at least one task node and each business sub-process in the target business is established, and then each task node is associated and stored with each business sub-process based on the corresponding relationship. Further, when it is detected that the task node to be executed is not a termination node, the target business sub-process associated and stored with the task node to be executed is determined based on the above corresponding relationship, and the target business sub-process is executed by calling the processing interface.

[0053] In one embodiment, the above-mentioned establishment of a correspondence between each task node in at least one task node and each business sub-process in the target business can be to pre-establish a correspondence between each task node and each conditional expression or function function, and the conditional expression or function function is used to implement the business sub-process corresponding to each task node.

[0054] In one embodiment, if the server performs step S501 to determine that the task nodes to be executed include at least two task nodes, and checks that each of the at least two task nodes is not a termination node, then the business sub-process in the target business corresponding to each of the at least two task nodes can be obtained, and then the business sub-process in the target business corresponding to each task node can be concurrently executed by calling the processing interface. This method can improve the execution efficiency of task scheduling.

[0055] Furthermore, in one embodiment, after the server concurrently executes the business sub-processes in the target business corresponding to each task node by calling the processing interface, if the business sub-process of one of the task nodes is completed, S503 can be directly triggered.

[0056] For example, see Figure 3 , task nodes B and D belong to the same level, task nodes B and D belong to the same level, the server performs step S501 to determine that the task nodes to be executed include task nodes B and D belonging to the same level, and task nodes B and D are not termination nodes. For this situation, after the server executes the business sub-flows corresponding to task nodes B and D respectively by calling the processing interface concurrently, if any task node is detected, such as the business sub-flow corresponding to task node B is completed, then S503 can be directly triggered to determine two nodes to be executed at the next level: task node C and task node E, if task node C and task node E are not termination nodes, then the business sub-flows corresponding to task nodes C and E are directly executed by calling the processing interface concurrently. It can be seen that the concurrent execution of the embodiment of the present invention not only includes the concurrency of multiple task nodes to be executed at the same level, but also includes the concurrency of task nodes to be executed between different levels. Adopting such a concurrent mode between levels can further improve the efficiency of task scheduling.

[0057] S503: When receiving the business sub-process execution result returned by the processing interface, the scheduler is triggered to perform a topological sorting of the pre-built DAG operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph. The specific implementation of step S503 can be found in the above embodiment for Figure 3 The detailed description will not be repeated here.

[0058] In one embodiment, after the server constructs the DAG operation diagram corresponding to the target business based on the directed acyclic graph description information, it can output the DAG operation diagram corresponding to the target business, and the DAG operation diagram is used to indicate the execution relationship of each business sub-process of the target business. In this case, the developer can sort out the business logic of the target business based on the DAG operation diagram corresponding to the target business. If it is confirmed to be correct, a confirmation instruction can be output for the DAG operation diagram corresponding to the output target business. Further, after the server detects the confirmation instruction, step S501 is executed.

[0059] If the user finds that the execution relationship between each business sub-process is incorrect after viewing the DAG operation diagram corresponding to the target business, the user can reconfigure the directed acyclic graph description information between the incorrect business sub-processes to update the DAG operation diagram corresponding to the target business.

[0060] Alternatively, in one embodiment, if the user views the DAG operation diagram corresponding to the target business and finds that the execution relationship between the various business sub-processes is incorrect, the user can directly adjust the execution relationship between the incorrect business sub-processes by dragging and dropping on the display page, that is, the user can directly adjust the connection relationship between the incorrect task nodes by dragging and dropping on the display page.

[0061] In one implementation, the server may output the DAG operation diagram corresponding to the target business in a preset format, such as ASCII and png, so that non-professionals can quickly become familiar with the business logic through the DAG operation diagram, thereby improving development efficiency.

[0062] In an embodiment of the present invention, a scheduler can be called to topologically sort a pre-built directed acyclic graph (DAG) operation graph, so as to determine a task node to be executed from at least one task node included in the DAG operation graph. Furthermore, if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling a processing interface. When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to perform a topological sorting of the pre-built DAG operation graph, so as to determine the step of the task node to be executed from at least one task node included in the DAG operation graph. By adopting an embodiment of the present invention, each business sub-process in the target business is executed based on the topological sorting corresponding to the DAG operation graph, which can not only improve the utilization rate of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling.

[0063] See also Figure 6 , is another flow chart of task scheduling according to an embodiment of the present invention. The method according to the embodiment of the present invention can be executed by a server or a server group. The method according to the embodiment of the present invention includes the following steps.

[0064] S601: calling a scheduler to topologically sort a pre-built directed acyclic graph (DAG) operation graph to determine a task node to be executed from at least one task node included in the DAG operation graph, where the DAG operation graph is built based on directed acyclic graph description information.

[0065] S602: If the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface. When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to perform a topological sorting of the pre-built DAG operation graph to determine the step of the task node to be executed from at least one task node included in the DAG operation graph. The specific implementation of steps S601 to S602 can refer to the relevant description of steps S501 to S503 in the above embodiment, which will not be repeated here.

[0066] In one embodiment, the server can configure a corresponding control strategy for a target task node in at least one task node, and the control strategy includes at least one of a flow control strategy, a timeout control strategy, and a retry control strategy, and the target task node is any one of the at least one task node. In this case, if the task node to be executed is the target task node, the control strategy corresponding to the target task node can be obtained, and based on the control strategy corresponding to the target task node, the processing interface is called to execute the business sub-process in the target business corresponding to the task node to be executed.

[0067] In one embodiment, assuming that the control strategy corresponding to the target task node is a flow control strategy, when the server calls the processing interface based on the flow control strategy corresponding to the target task node to be executed, if it is detected that the flow occupied by the business sub-process is greater than or equal to the first preset flow threshold, the execution of the business sub-process corresponding to the target task node is limited, and the flow is limited to the second preset flow threshold. Among them, the first preset flow threshold is greater than the second preset flow threshold, and the specific data of both are pre-set by the developer based on experimental data, and can be adjusted later according to specific design requirements.

[0068] In one embodiment, assuming that the control strategy corresponding to the target task node is a timeout control strategy, when the server calls the processing interface based on the flow control strategy corresponding to the target task node to execute the business sub-process in the target business corresponding to the task node to be executed, the timing can be started. If the timed length is greater than or equal to the preset time threshold, and the business sub-process execution result returned by the processing interface has not been received, the preset execution result can be determined as the execution result of the business sub-process corresponding to the target task node. The preset execution result is used to characterize the timeout of the execution time.

[0069] In one embodiment, assuming that the control strategy corresponding to the target task node is a retry control strategy, when the server receives the business sub-process execution result corresponding to the target task node returned by the processing interface, if the execution result is detected as an execution exception or execution failure, the processing interface can be called again to execute the business sub-process in the target business corresponding to the target task node. When the execution result of the business sub-process corresponding to the target task node returned by the processing interface is received, the execution result can be judged again as an execution exception or execution failure. If the execution result is an execution exception or execution failure, the processing interface is called again to execute the business sub-process in the target business corresponding to the target task node. And so on, until the number of times the processing interface is called again to execute the business sub-process in the target business corresponding to the target task node is greater than or equal to the number threshold, or when the execution result is detected as successful execution, the retry is stopped.

[0070] It can be seen that the embodiment of the present invention can separately configure control strategies such as flow control strategy, timeout control strategy and retry control strategy in the business sub-process corresponding to the task node, without the need for developers to implement redundant control codes one by one.

[0071] S603: If the task node to be executed is a termination node, the termination process corresponding to the termination node is executed by calling the processing interface. When the termination process execution result returned by the processing interface is received, the statistical information corresponding to the execution target business is output, and the statistical information includes at least one of the following: the execution result of the business sub-process, the time-consuming statistical information corresponding to the execution target business, and the execution times of the business sub-process corresponding to the task node to be executed. In this way, the statistical information of the global execution target business can be output, which provides an effective analysis basis for developers to locate problems or analyze business processes.

[0072] In one embodiment, each task node in the DAG operation diagram corresponds to a node identifier, such as a node name, and the termination node introduced in the DAG operation diagram also corresponds to a unique termination node identifier. When the server determines a task node to be executed from at least one task node included in the DAG operation diagram, it can first detect the similarity between the node identifier of the task to be executed and the termination node identifier. If the similarity is greater than or equal to a preset similarity threshold (e.g., 95%), it can be determined that the task node to be executed is a termination node; conversely, if the similarity is less than a preset similarity threshold (e.g., 95%), it can be determined that the task node to be executed is not a termination node.

[0073] In one embodiment, each time the server executes the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface, the execution result of the business sub-process corresponding to the current task node to be executed and / or the number of executions of the business sub-process corresponding to the current task node to be executed can be recorded, and the execution result can include success, failure or exception. In view of this situation, when the server receives the termination process execution result returned by the processing interface, it can generate statistical information corresponding to the target business based on the pre-recorded execution results of the business sub-process corresponding to each task node to be executed and / or the number of executions of the business sub-process corresponding to each task node to be executed, and output the statistical information.

[0074] In one embodiment, the server can start a timer to start timing when executing the business sub-process in the target business corresponding to the first task node to be executed by calling the processing interface, and when receiving the termination process execution result returned by the processing interface, obtain the duration of the timer, and determine the duration of the timer as the time-consuming statistical information corresponding to the execution of the target business, and then output statistical information including the time-consuming statistical information. The time-consuming statistical information represents the total duration corresponding to the execution of the target business.

[0075] It can be seen that in an embodiment of the present invention, after calling the scheduler to perform topological sorting on the pre-built directed acyclic graph DAG operation graph and determining the task node to be executed from at least one task node included in the DAG operation graph, when receiving the business sub-process execution result returned by the processing interface, the scheduler will be triggered again to perform topological sorting on the pre-built DAG operation graph again to determine the task node to be executed from at least one task node included in the DAG operation graph until all business sub-processes corresponding to the target business are executed.

[0076] Among them, the completion of the execution of all business sub-processes corresponding to the target business can be called a complete scheduling process of the scheduler. In one embodiment, the scheduler in the embodiment of the present invention can be non-volatile, the scheduler is pre-bound to a single request, and the internal state is stateful. All the schedulers of the two complete scheduling processes cannot be reused, that is, the two complete scheduling processes will not use the same scheduler. Exemplarily, when the server receives a processing request for processing the target business G1, it can copy a new scheduler from the pre-built scheduler, and trigger the new scheduler to schedule the target business G1, thereby entering a complete scheduling process corresponding to the target business G1. Furthermore, when the processing of the target business G1 is not completed, the server receives a processing request for the target business G2. In this case, the server can copy a new scheduler from the pre-built scheduler again, and trigger the new scheduler to schedule the target business G2, thereby entering a complete scheduling process corresponding to the target business G2.

[0077] In another embodiment, the scheduler in the embodiment of the present invention may also be a stateful independent scheduler based on an event message loop. In this case, the same scheduler may be used for the two complete scheduling processes.

[0078] In an embodiment of the present invention, a scheduler can be called to topologically sort a pre-built directed acyclic graph (DAG) operation diagram to determine a task node to be executed from at least one task node included in the DAG operation diagram. Furthermore, if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling a processing interface, and when the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to perform a step of topologically sorting the pre-built DAG operation diagram to determine the task node to be executed from at least one task node included in the DAG operation diagram. If the task node to be executed is a termination node, the termination process corresponding to the termination node is executed by calling the processing interface, and when the termination process execution result returned by the processing interface is received, the statistical information corresponding to the execution target business is output. In this way, the statistical information of the global execution target business can be output, providing an effective analysis basis for developers to locate problems or analyze business processes.

[0079] An embodiment of the present invention further provides a computer storage medium, in which program instructions are stored. When the program instructions are executed, they are used to implement the corresponding methods described in the above embodiments.

[0080] See also Figure 7 , is a structural diagram of a task scheduling device according to an embodiment of the present invention. The task scheduling device according to the embodiment of the present invention can be set in a server.

[0081] In one implementation of the device according to an embodiment of the present invention, the device includes the following structure.

[0082] A sorting module 70 is used to call a scheduler to perform topological sorting on a pre-constructed directed acyclic graph DAG operation graph, so as to determine a task node to be executed from at least one task node included in the DAG operation graph, wherein the DAG operation graph is constructed based on the directed acyclic graph description information;

[0083] The processing module 71 is used for executing the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface if it is detected that the task node to be executed is not a termination node;

[0084] The sorting module 70 is also used to trigger the scheduler to execute the step of topologically sorting the pre-built DAG operation graph when receiving the business sub-process execution result returned by the processing interface, so as to determine the task node to be executed from at least one task node included in the DAG operation graph.

[0085] In one embodiment, the device also includes a DAG operation diagram module 72, which is used to obtain directed acyclic graph description information, wherein the directed acyclic graph description information represents the dependency relationship between the business sub-processes corresponding to the target business; based on the directed acyclic graph description information, a DAG operation diagram corresponding to the target business is constructed, wherein each task node in the DAG operation diagram corresponds to a business sub-process of the target business.

[0086] In one embodiment, the device further includes an output module 73, which is used to output a DAG operation diagram corresponding to the target business, wherein the DAG operation diagram is used to indicate the execution relationship between each business sub-process of the target business.

[0087] In one embodiment, the processing module 71 is also used to establish a correspondence between each task node in the at least one task node and each business sub-process in the target business; store the each task node in association with the each business sub-process based on the correspondence; determine the target business sub-process stored in association with the task node to be executed based on the correspondence, and execute the target business sub-process by calling the processing interface.

[0088] In one embodiment, the task nodes to be executed include at least two task nodes, and the processing module 71 is specifically used to obtain the business sub-process in the target business corresponding to each of the at least two task nodes; and concurrently execute the business sub-process in the target business corresponding to each of the task nodes by calling the processing interface.

[0089] In one embodiment, the processing module 71 is also used to execute the termination process corresponding to the termination node by calling the processing interface if the task node to be executed is a termination node; when the termination process execution result returned by the processing interface is received, output statistical information corresponding to the execution of the target business, and the statistical information includes at least one of the following: the execution result of the business sub-process, the time-consuming statistical information corresponding to the execution of the target business, and the number of executions of the business sub-process corresponding to the task node to be executed.

[0090] In one embodiment, the processing module 71 is also used to configure a corresponding control strategy for a target task node among the at least one task node, wherein the control strategy includes at least one of a flow control strategy, a timeout control strategy and a retry control strategy, and the target task node is any one of the at least one task node; if the task node to be executed is the target task node, then the control strategy corresponding to the target task node is obtained; based on the control strategy corresponding to the target task node, a processing interface is called to execute a business sub-process in the target business corresponding to the task node to be executed.

[0091] In the embodiment of the present invention, the specific implementation of each of the above modules can refer to the description of the relevant contents in the embodiments corresponding to the above-mentioned figures.

[0092] In an embodiment of the present invention, the sorting module 70 calls the scheduler to perform topological sorting on the pre-built directed acyclic graph DAG operation diagram to determine the task node to be executed from at least one task node included in the DAG operation diagram. If the processing module 71 detects that the task node to be executed is not a termination node, it executes the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface. When the sorting module 70 receives the business sub-process execution result returned by the processing interface, it triggers the sorting module 70 to call the scheduler to perform topological sorting on the pre-built DAG operation diagram to determine the task node to be executed from at least one task node included in the DAG operation diagram. By adopting the embodiment of the present invention, each business sub-process in the target business can be executed based on the topological sorting corresponding to the DAG operation diagram, which can not only improve the utilization rate of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling.

[0093] See also Figure 8 , is a schematic diagram of the structure of a server according to an embodiment of the present invention, wherein the server according to the embodiment of the present invention includes a power supply module and other structures, and includes a processor 801, a storage device 802, and a network interface 803. The processor 801, the storage device 802, and the network interface 803 can exchange data, and the processor 801 implements the corresponding task scheduling function.

[0094] The storage device 802 may include a volatile memory, such as a random-access memory (RAM); the storage device 802 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the storage device 802 may also include a combination of the above-mentioned types of memory.

[0095] The processor 801 may be a central processing unit (CPU) 801. In one embodiment, the processor 801 may also be a graphics processing unit (GPU) 801. The processor 801 may also be a combination of a CPU and a GPU. In the server, multiple CPUs and GPUs may be included as needed to perform corresponding task scheduling. In one embodiment, the storage device 802 is used to store program instructions. The processor 801 may call the program instructions to implement the various methods involved in the embodiments of the present invention.

[0096] In a first possible implementation, the processor 801 of the server calls the program instructions stored in the storage device 802 to call the scheduler to perform topological sorting on the pre-built directed acyclic graph (DAG) operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph, and the DAG operation graph is constructed based on the directed acyclic graph description information; if the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface; when the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to execute the step of topologically sorting the pre-built DAG operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph.

[0097] In one embodiment, in one embodiment, the processor 801 is also used to obtain directed acyclic graph description information through the network interface 803, and the directed acyclic graph description information represents the dependency relationship between the business sub-processes corresponding to the target business; based on the directed acyclic graph description information, a DAG operation diagram corresponding to the target business is constructed, and each task node in the DAG operation diagram corresponds to a business sub-process in the target business.

[0098] In one embodiment, the processor 801 is further configured to output a DAG operation diagram corresponding to the target service through the network interface 803 , where the DAG operation diagram is used to indicate the execution relationship between each service sub-process of the target service.

[0099] In one embodiment, the processor 801 is also used to establish a correspondence between each task node in the at least one task node and each business sub-process in the target business; store the each task node in association with the each business sub-process based on the correspondence; determine the target business sub-process stored in association with the task node to be executed based on the correspondence, and execute the target business sub-process by calling a processing interface.

[0100] In one embodiment, the task nodes to be executed include at least two task nodes, and the processor 801 is specifically used to obtain the business sub-process in the target business corresponding to each of the at least two task nodes; and concurrently execute the business sub-process in the target business corresponding to each of the task nodes by calling the processing interface.

[0101] In one embodiment, the processor 801 is also used to, if the task node to be executed is a termination node, execute the termination process corresponding to the termination node by calling the processing interface; when receiving the termination process execution result returned by the processing interface, output statistical information corresponding to the execution of the target business, the statistical information includes at least one of the following: the execution result of the business sub-process, the time-consuming statistical information corresponding to the execution of the target business, and the number of executions of the business sub-process corresponding to the task node to be executed.

[0102] In one embodiment, the processor 801 is also used to configure a corresponding control strategy for a target task node among the at least one task node, wherein the control strategy includes at least one of a flow control strategy, a timeout control strategy and a retry control strategy, and the target task node is any one of the at least one task node; if the task node to be executed is the target task node, then the control strategy corresponding to the target task node is obtained; based on the control strategy corresponding to the target task node, a processing interface is called to execute a business sub-process in the target business corresponding to the task node to be executed.

[0103] In the embodiment of the present invention, the specific implementation of the processor 801 may refer to the description of the relevant contents in the embodiments corresponding to the aforementioned figures.

[0104] In an embodiment of the present invention, the processor 801 calls the scheduler to perform topological sorting on the pre-built directed acyclic graph DAG operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph. If it is detected that the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface. When the business sub-process execution result returned by the processing interface is received, the scheduler is called to perform topological sorting on the pre-built DAG operation graph to determine the task node to be executed from at least one task node included in the DAG operation graph. By adopting the embodiment of the present invention, each business sub-process in the target business can be executed based on the topological sorting corresponding to the DAG operation graph, which can not only improve the utilization rate of resources, but also intuitively understand the execution relationship between each business sub-process, which is conducive to reducing the maintenance cost of task scheduling.

[0105] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0106] The above disclosure is only part of the embodiments of the present invention, which certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention are still within the scope of the invention.

Claims

1. A task scheduling method, characterized in that: The method comprises: Outputting a directed acyclic graph operation diagram corresponding to the target business, wherein the directed acyclic graph operation diagram includes task nodes corresponding to each business sub-process of the target business, the directed acyclic graph operation diagram is constructed based on the directed acyclic graph description information, and the directed acyclic graph operation diagram is used to indicate the execution relationship of each business sub-process of the target business; According to the directed acyclic graph description information reconfigured by the user or the adjustment operation of the user on the connection relationship between the task nodes in the directed acyclic graph operation graph, the directed acyclic graph operation graph is updated to obtain an updated directed acyclic graph operation graph; Calling a scheduler to perform topological sorting on the updated directed acyclic graph operation graph to determine a task node to be executed from at least one task node included in the updated directed acyclic graph operation graph; If the task node to be executed is not a termination node, the business sub-process in the target business corresponding to the task node to be executed is executed by calling the processing interface based on the control strategy corresponding to the task node to be executed; wherein the control strategy includes at least one of a flow control strategy and a timeout control strategy; the flow control strategy is used to limit the execution of the business sub-process corresponding to the task node to be executed when it is detected that the occupied flow of the business sub-process is greater than or equal to a first preset flow threshold, so as to limit the occupied flow to a second preset flow threshold, wherein the first preset flow threshold is greater than the second preset flow threshold, and the timeout control strategy is used to determine the preset execution result as the execution result of the business sub-process corresponding to the task node to be executed when the execution duration of the business sub-process corresponding to the task node to be executed is greater than or equal to the preset duration threshold; When the business sub-process execution result returned by the processing interface is received, the scheduler is triggered to execute the step of topologically sorting the updated directed acyclic graph operation graph to determine the task node to be executed from at least one task node included in the updated directed acyclic graph operation graph.

2. The method according to claim 1, characterized in that Before the calling scheduler topologically sorts the updated directed acyclic graph directed acyclic graph operation graph, the method further includes: Obtaining directed acyclic graph description information, wherein the directed acyclic graph description information represents the dependency relationship between each business sub-process in the target business; A directed acyclic graph operation graph corresponding to the target business is constructed based on the directed acyclic graph description information.

3. The method according to claim 2, characterized in that After constructing the directed acyclic graph operation graph corresponding to the target business based on the directed acyclic graph description information, and before executing the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface, the method further includes: Establishing a correspondence between each task node in the at least one task node and each business sub-process in the target business; Based on the corresponding relationship, each task node is associated with each business sub-process and stored; The executing of the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface includes: Based on the corresponding relationship, a target business sub-process stored in association with the task node to be executed is determined, and the target business sub-process is executed by calling a processing interface.

4. The method according to claim 1, characterized in that: The task nodes to be executed include at least two task nodes, and the calling of the processing interface to execute the business sub-process in the target business corresponding to the task nodes to be executed includes: Obtaining a business sub-process in a target business corresponding to each task node in the at least two task nodes; The business sub-processes in the target business corresponding to each task node are concurrently executed by calling the processing interface.

5. The method according to claim 1, characterized in that The method further comprises: If the task node to be executed is a termination node, the termination process corresponding to the termination node is executed by calling the processing interface; When the termination process execution result returned by the processing interface is received, statistical information corresponding to the execution of the target business is output, and the statistical information includes at least one of the following: the execution result of the business sub-process, the time-consuming statistical information corresponding to the execution of the target business, and the number of executions of the business sub-process corresponding to the task node to be executed.

6. The method according to claim 1, characterized in that The method further comprises: Configuring a corresponding control strategy for a target task node in the at least one task node, the target task node being any one of the at least one task node; The executing of the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface includes: If the task node to be executed is the target task node, obtaining the control strategy corresponding to the target task node; Based on the control strategy corresponding to the target task node, a processing interface is called to execute a business sub-process in the target business corresponding to the task node to be executed.

7. A task scheduling device, characterized in that: The device comprises: An output module, used to output a directed acyclic graph operation diagram corresponding to the target business, wherein the directed acyclic graph operation diagram includes task nodes corresponding to each business sub-process of the target business, the directed acyclic graph operation diagram is constructed based on the directed acyclic graph description information, and the directed acyclic graph operation diagram is used to indicate the execution relationship of each business sub-process of the target business; A processing module, configured to update the directed acyclic graph operation graph according to the directed acyclic graph description information reconfigured by the user or the adjustment operation of the user on the connection relationship between the task nodes in the directed acyclic graph operation graph, to obtain an updated directed acyclic graph operation graph; A sorting module, used for calling a scheduler to perform topological sorting on the updated directed acyclic graph operation graph, so as to determine a task node to be executed from at least one task node included in the updated directed acyclic graph operation graph; The processing module is also used for, if it is detected that the task node to be executed is not a termination node, executing the business sub-process in the target business corresponding to the task node to be executed by calling the processing interface based on the control strategy corresponding to the task node to be executed; wherein the control strategy includes at least one of a flow control strategy and a timeout control strategy; the flow control strategy is used to limit the execution of the business sub-process corresponding to the task node to be executed when it is detected that the occupied flow of the business sub-process is greater than or equal to a first preset flow threshold, so as to limit the occupied flow to a second preset flow threshold, wherein the first preset flow threshold is greater than the second preset flow threshold, and the timeout control strategy is used to determine the preset execution result as the execution result of the business sub-process corresponding to the task node to be executed when the execution duration of the business sub-process corresponding to the task node to be executed is greater than or equal to the preset duration threshold; The sorting module is also used to trigger the scheduler to execute the step of topologically sorting the updated directed acyclic graph operation graph when receiving the business sub-process execution result returned by the processing interface, so as to determine the task node to be executed from at least one task node included in the updated directed acyclic graph operation graph.

8. A server, characterized in that: The method comprises a processor and a storage device, wherein the processor and the storage device are connected to each other, wherein the storage device is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores program instructions, which, when executed, are used to implement the method according to any one of claims 1 to 6.

Images