Task scheduling system, method and device based on observer mechanism

By introducing an observer mechanism into a distributed cluster and utilizing a scheduling center and database to achieve dynamic resource scheduling and information interaction, the high development cost and low efficiency caused by complex interaction logic in existing technologies are solved, and efficient task processing and task status indication are achieved.

CN118916142APending Publication Date: 2024-11-08ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411104595.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In distributed clusters, existing technologies require writing complex interaction logic to realize information exchange between nodes, resulting in high development costs and low task processing efficiency, and the task is prone to suspension when the main execution body fails.

Method used

An observer mechanism is introduced, which dynamically schedules and processes resources through a scheduling center, writes the execution status to a database, and the observer reads from the database and generates indication information, avoiding direct interaction between the executor and using simple database read and write logic to achieve information exchange.

Benefits of technology

It reduced development costs, improved task processing efficiency, avoided the problem of high task hang time, and achieved decoupled information interaction between executors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916142A_ABST
    Figure CN118916142A_ABST
Patent Text Reader

Abstract

One or more embodiments of the invention provide a task scheduling system, method and device based on an observer mechanism. The system comprises a distributed cluster and a database, a scheduling center is deployed in the distributed cluster, and the scheduling center dynamically schedules processing resources in the distributed cluster according to a predefined cluster resource scheduling strategy so as to allocate the corresponding processing resources to an executor and an observer corresponding to a target business task; the executor executes the target business task by using the allocated processing resources, and writes the execution state of the target business task into the database; and the observer reads the execution state from the database by using the allocated processing resources, generates indication information corresponding to the execution state, and writes the indication information into the database to indicate further operation of the executor and / or other executors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer science and technology, and in particular, to a task scheduling system, method, and device based on an observer mechanism. Background Art

[0002] A distributed cluster is a computer system architecture that connects multiple computing nodes through a network so that these computing nodes can work together.

[0003] In the related art, in order to achieve load balancing of nodes in a distributed cluster, a dynamic scheduling of processing resources is usually adopted to allocate processing resources to various tasks. In order to achieve information interaction between nodes while dynamically scheduling processing resources to complete the task, technicians are usually required to write complex interaction logic, which not only has a high development cost, but also cannot achieve efficient information interaction due to the complex interaction logic, thus reducing the efficiency of task processing. Summary of the invention

[0004] In view of this, one or more embodiments of this specification provide the following technical solutions:

[0005] According to a first aspect of one or more embodiments of this specification, a task scheduling system based on an observer mechanism is proposed, including: a distributed cluster and a database;

[0006] A scheduling center is deployed in the distributed cluster, and the scheduling center dynamically schedules processing resources in the distributed cluster according to a predefined cluster resource scheduling strategy to allocate corresponding processing resources to executors and observers corresponding to target business tasks;

[0007] The executor executes the target business task using the allocated processing resources, and writes the execution status of the target business task into the database;

[0008] The observer uses the allocated processing resources to read the execution status from the database, generates indication information corresponding to the execution status, and writes the indication information into the database to indicate further operations of the executor and / or other executors.

[0009] According to a second aspect of one or more embodiments of the present specification, a task execution method based on an observer mechanism is proposed, which is applied to an executor corresponding to a target business task; the processing resources allocated to the executor in a distributed cluster are dynamically scheduled to the executor by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the observer corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the method includes:

[0010] Executing the target business task using the allocated processing resources, and writing the execution status of the target business task into the database;

[0011] The execution status is read from the database by the observer to generate indication information corresponding to the execution status; the indication information is written into the database by the observer to indicate further operations of the executor and / or other executors.

[0012] According to a third aspect of one or more embodiments of the present specification, a task control method based on an observer mechanism is proposed, which is applied to an observer corresponding to a target business task; the processing resources allocated to the observer in a distributed cluster are dynamically scheduled to the observer by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to an executor corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the method includes:

[0013] Using the allocated processing resources, reading the execution status of the target business task from a database to generate indication information corresponding to the execution status, and writing the indication information into the database;

[0014] The target business task is executed by the executor using the allocated processing resources, and the execution status is written into the database by the executor; the indication information written into the database is used to instruct further operations of the executor and / or other executors.

[0015] According to a fourth aspect of one or more embodiments of the present specification, a task execution device based on an observer mechanism is proposed, which is applied to an executor corresponding to a target business task; the processing resources allocated to the executor in a distributed cluster are dynamically scheduled to the executor by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the observer corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the device includes:

[0016] an execution unit, executing the target business task using the allocated processing resources, and writing the execution status of the target business task into the database;

[0017] The execution status is read from the database by the observer to generate indication information corresponding to the execution status; the indication information is written into the database by the observer to indicate further operations of the executor and / or other executors.

[0018] According to a fifth aspect of one or more embodiments of the present specification, a task control device based on an observer mechanism is proposed, which is applied to an observer corresponding to a target business task; the processing resources allocated to the observer in a distributed cluster are dynamically scheduled to the observer by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to an executor corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the device includes:

[0019] a generating unit, using the allocated processing resources to read the execution status of the target business task from a database to generate indication information corresponding to the execution status, and write the indication information into the database;

[0020] The target business task is executed by the executor using the allocated processing resources, and the execution status is written into the database by the executor; the indication information written into the database is used to instruct further operations of the executor and / or other executors.

[0021] According to the sixth aspect of one or more embodiments of this specification, an electronic device is proposed, comprising: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in the second aspect or the third aspect by running the executable instructions.

[0022] According to a seventh aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the second aspect or the third aspect are implemented.

[0023] According to an eighth aspect of one or more embodiments of the present specification, a computer program product is proposed, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the second aspect or the third aspect.

[0024] As can be seen from the above technical solution, this specification uses the scheduling center deployed in the distributed cluster to dynamically schedule the processing resources in the distributed cluster to allocate the processing resources to the executors and observers of the target business tasks. On this basis, the executor can use the allocated processing resources to execute the target business task and write the execution status of the target business task into the database; and the observer can use the allocated processing resources to read the execution status from the database and generate indication information corresponding to the execution status, and then write the indication information into the database to instruct the executor or other executors to perform further operations.

[0025] It should be understood that the scheduling center in this specification dynamically schedules the processing resources in the distributed cluster to executors and observers, where the executors are responsible for executing the target business tasks, while the observers are used to generate indication information for indicating further operations of the target business tasks based on the execution status of the target business tasks. It can be seen that this specification introduces an observer mechanism, which enables information interaction between different executors, avoiding the direct interaction between the various execution entities working together in the related technology, resulting in the failure of the corresponding task execution when any execution entity fails and cannot interact.

[0026] In addition, the distributed cluster in this specification also includes a database, and the above-mentioned execution status and the indication information corresponding to the execution status are written into the database for reading. It can be seen that the executors and observers in this specification realize information interaction through simple database reading and writing logic rather than direct interaction, thereby avoiding the problem of direct interaction between the various execution entities working together in the distributed cluster of the related technology, and the need to write additional complex interaction logic. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the architecture of a task scheduling system based on an observer mechanism provided by an exemplary embodiment.

[0028] Figure 2 It is a flowchart of a task execution method based on an observer mechanism provided by an exemplary embodiment.

[0029] Figure 3 It is a flowchart of a task control method based on an observer mechanism provided by an exemplary embodiment.

[0030] Figure 4 It is a schematic diagram of a task generation process provided by an exemplary embodiment.

[0031] Figure 5 It is a flowchart of a task execution method provided by an exemplary embodiment.

[0032] Figure 6 It is an interaction diagram of a task execution method provided by an exemplary embodiment.

[0033] Figure 7 It is a structural schematic diagram of a device provided by an exemplary embodiment.

[0034] Figure 8 It is a block diagram of a task execution device based on an observer mechanism provided by an exemplary embodiment.

[0035] Fig. 9 It is a block diagram of a task control device based on an observer mechanism provided by an exemplary embodiment. DETAILED DESCRIPTION

[0036] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0037] In a distributed cluster, in order to achieve load balancing between various computing nodes, dynamic scheduling of processing resources is usually used to allocate processing resources to tasks to be executed. For example, according to the load of each computing node in the distributed cluster, processing resources in nodes with lower loads can be allocated first to avoid high-load operation of nodes with higher loads.

[0038] In related technologies, the same task is usually completed jointly by various computing nodes in a distributed cluster. For example, a corresponding execution module can be generated in each computing node, and the task can be split into multiple sub-tasks and sent to different computing nodes respectively, so that the execution modules in each computing node can collaboratively complete the task. This process is equivalent to allocating the processing resources of each computing node to the task, or scheduling the processing resources of each node to the execution module involved in the task.

[0039] It is worth noting that the processing resource scheduling strategy adopted by the distributed cluster is a dynamic scheduling strategy, which means that while the computing nodes are executing tasks, the scheduling operations of processing resources are also executed synchronously.

[0040] For example, assuming that task A is divided into subtasks a1 and a2, and the two subtasks are executed serially, subtask a1 can be preferentially sent to computing node 1 to be executed by the execution module in computing node 1, and during the execution of subtask a1, the operation of "determining the computing node for executing subtask a2 based on the load status of each computing node in the distributed cluster" is synchronously executed.

[0041] It should be understood that since the collaborative execution of tasks and the dynamic scheduling of processing resources may be performed at the same time, and the information interaction operations during the task execution process are performed autonomously by the execution subject, the relevant technology requires pre-writing of complex interaction logic so that timely and effective information interaction can be achieved between the execution subjects of the task while the distributed cluster dynamically schedules processing resources.

[0042] Continuing with the above example, in the case of serial execution of subtasks, if computing node 1 completes the execution operation of subtask a1, it is necessary to transmit the execution result to the computing node responsible for subtask a2. Due to the use of a dynamic scheduling method, there are multiple situations when completing the execution operation of subtask a. For example, computing node 2 responsible for subtask a2 may have been determined during the execution of subtask a1, and the load condition of computing node 2 has not changed. For another example, although computing node 2 responsible for subtask a2 may have been determined, the load condition of computing node 2 has changed, resulting in the need to re-determine the computing node responsible for subtask a. For another example, when the execution operation of subtask a1 is completed, the computing node responsible for subtask a2 may not have been determined.

[0043] It should be understood that in order to achieve collaborative work of different nodes, after completing the execution operation of subtask a1, computing node 1 needs to prioritize and determine which of the above situations it currently belongs to, and determine the interaction object based on the situation. Of course, this example is only based on "how to determine the execution subject responsible for the next subtask when multiple subtasks are executed serially" as an example to illustrate that in the case of dynamic scheduling of processing resources in distributed clusters, related technologies require pre-writing of complex interaction logic. In actual applications, there may be far more than the few situations in the above examples, and it is not only necessary to consider the interaction object, which means that the interaction logic needs to be written by technicians with professional knowledge, and the development cost is relatively high.

[0044] It is not difficult to see that in the relevant technologies, on the one hand, when collaboratively processing tasks across nodes, the execution entities participating in the collaborative processing interact with each other on their own. Once any execution entity participating in the collaborative processing handles an exception, the task cannot be completed, resulting in a high task suspension rate. On the other hand, due to the strong binding or mutual coupling between the dynamic scheduling operations of processing resources and the collaborative processing operations for tasks, the relevant technologies require technical personnel to pre-write complex interaction logic, which not only has a high development cost, but also cannot achieve efficient information interaction due to the complexity of the interaction logic, thereby reducing the processing efficiency of the task.

[0045] To this end, this specification proposes a task scheduling system based on an observer mechanism. On the one hand, the system introduces an observer mechanism, where the observer controls the operation of the executor according to the execution status of the task, thus avoiding the problem of a high suspension rate caused by the interaction of the executor. On the other hand, it also uses simple database read and write logic to achieve the decoupling of dynamic scheduling operations and collaborative processing operations, thus avoiding the problem of high development cost and low task execution efficiency caused by complex interaction logic.

[0046] The following is an introduction to the multiple execution entities included in the system:

[0047] Figure 1FIG. 1 is a schematic diagram of an architecture of a task scheduling system based on an observer mechanism, shown as an exemplary embodiment of this specification. Figure 1 As shown, the system may include: a distributed cluster 11 and a database 12; wherein a dispatching center 111 is deployed in the distributed cluster 11;

[0048] The scheduling center 111 dynamically schedules the processing resources in the distributed cluster 11 according to the predefined cluster resource scheduling strategy to allocate the corresponding processing resources to the executors 112 and observers 113 corresponding to the target business tasks;

[0049] The executor 112 executes the target business task using the allocated processing resources, and writes the execution status of the target business task into the database 12;

[0050] The observer 113 uses the allocated processing resources to read the execution status from the database 12, generates indication information corresponding to the execution status, and writes the indication information into the database 12 to indicate further operations of the executor 112 and / or other executors.

[0051] In this specification, the dispatch center 111 can be deployed in the distributed cluster 11 in different ways. For example, when the computing nodes in the distributed cluster 11 are divided into master and slave nodes, the dispatch center 111 can be deployed in the master node; for another example, a server independent of the computing nodes can be added to the distributed cluster 11 to deploy the dispatch center 111 in the server. Of course, the examples here are only for illustration, and how to deploy the dispatch center 111 in the distributed cluster 11 can be determined by those skilled in the art according to actual needs, and this specification does not limit this.

[0052] In this specification, the scheduling center 111 predefines a cluster resource scheduling strategy to dynamically schedule the processing resources in the distributed cluster 11. For example, in order to achieve load balancing of the computing nodes in the distributed cluster 11, the scheduling center 111 can monitor the load status of the computing nodes in the distributed cluster 11, and give priority to allocating processing resources to computing nodes with lower loads; for another example, in order to improve the processing efficiency of tasks, the quality of processing resources in each computing node can be determined in advance, and relatively better processing resources can be allocated preferentially. Of course, the examples here are only illustrative. How to predefine the cluster resource scheduling strategy in this specification to dynamically schedule processing resources can be determined by those skilled in the art according to actual needs, and this specification does not limit this.

[0053] In this specification, the execution subject that actually executes the business task can be referred to as the executor, and the execution subject used to generate corresponding indication information according to the execution status of the executor's execution of the business task can be referred to as the observer. In other words, the executors and observers in this specification are both concepts at the logical level, and can be understood as execution modules responsible for performing different operations. Among them, this specification refers to the business task that the executor 112 is responsible for as the target business task, and the observer used to generate indication information corresponding to the execution status of the target business task is referred to as observer 113.

[0054] In this specification, the executor 112 can use the processing resources allocated to itself to execute the target business task and write the execution status of the target business task into the database 12, while the observer 113 can use the processing resources allocated to itself to read the execution status from the database 12 to generate indication information corresponding to the execution status, and write the indication information into the database, thereby instructing the executor 112 and at least one of the other executors to perform further operations.

[0055] It should be understood that the indication information generated by the observer 113 is used to indicate which executor performs which operation, and is related to the execution status of the target business task. For example, when the execution status of the target business task indicates that the executor 112 has failed to execute the target business task, the indication information generated by the observer 113 based on the execution status can be used to instruct the executor 112 to re-execute the target business task; for another example, when the target business task is a subtask of an overall task, and the execution status of the target business task indicates that the executor 112 has successfully executed the target business task, the indication information generated by the observer 113 based on the execution status can be used to trigger the executor corresponding to the next subtask in the overall task to start executing the next business task, wherein the next subtask can correspond to either the executor 112 or another executor. Of course, the examples given here are merely illustrative. For instance, the execution status can also be used to indicate the lack of data required to execute the target business task. In this case, the indication information generated by the observer 113 can be used to instruct the subject maintaining the data to provide the data. The subject can be other executors or other types of information carriers. The execution status and the specific indication information corresponding to the execution status can be determined by technicians in this field according to actual needs, and this specification does not limit this.

[0056] In this specification, a resource scheduling table may be maintained in the scheduling center 111, so as to allocate the processing resources of the corresponding computing node to the executor corresponding to the business task by creating a corresponding relationship between the business task and the computing node in the resource scheduling table. For example, when the scheduling center 111 determines to use the processing resources of a specific computing node to execute the target business task based on the cluster resource scheduling strategy, the corresponding relationship between the target business task and the specific computing node may be maintained in the resource scheduling table to allocate the processing resources on the specific computing node to the executor 112 corresponding to the target business task. On this basis, the specific computing node can pull up the executor 112 locally according to the corresponding relationship to execute the target business task.

[0057] In this specification, the scheduling center 111 can maintain the correspondence between business tasks and corresponding computing nodes by creating relationship table entries corresponding to business tasks in the resource scheduling table, and each computing node writes node information in the relationship table entries to preempt the corresponding business tasks.

[0058] Taking maintaining the correspondence between the target business task and the specific computing node as an example, the scheduling center 111 can preferentially create a relationship table item corresponding to the target business task in the resource scheduling table, and when determining that the specific computing node writes its own node information in the relationship table item corresponding to the target business task to preempt the target business task, determine the status of the specific computing node according to the maintained node status table; if it is confirmed that the specific computing node is online, maintain the relationship table item corresponding to the target business task; and if the target business task is not preempted within the preset time, or the preempted computing node is recorded as offline in the node status table, select the specific computing node through the cluster resource scheduling strategy and write the node information of the specific computing node into the relationship table item corresponding to the target business task.

[0059] In other words, in this example, if a specific computing node has written its own node information in the relationship table item of the target business task to preempt the target business task, and the specific computing node is online, the scheduling center 111 can continue to maintain the content in the table item; if the target business task has not been preempted for a long time, or is preempted by other offline computing nodes, the scheduling center 111 selects the specific computing node based on the cluster resource scheduling strategy, and preempts the target task by writing the node information to maintain the corresponding relationship between the target business task and the specific computing node.

[0060] In this specification, the operation performed by the observer based on the execution status provided by the executor can be called an observation task, wherein the most typical observation task is the operation of generating indication information based on the execution status. On this basis, this specification can be similar to a business task, by creating a correspondence between the observation task and the computing node in the resource scheduling table, so as to allocate the processing resources of the corresponding computing node to the observer corresponding to the observation task. For example, when the scheduling center 111 determines to use the processing resources of any computing node to execute the target observation task based on the cluster resource scheduling strategy, the corresponding relationship between the target observation task and the any computing node can be maintained in the resource scheduling table to allocate the processing resources on the any computing node to the observer 113 corresponding to the target observation task. On this basis, the any computing node can pull up the observer 113 locally according to the corresponding relationship to execute the target observation task.

[0061] Similar to the target business task, the scheduling center 111 can also create a relationship table entry for the target observation task in the resource scheduling table to indicate the pull-up of the observer 113 corresponding to the target observation task. Under this premise, when the scheduling center 111 determines that any computing node writes its own node information in the relationship table entry corresponding to the target observation task to preempt the target observation task, it can also determine the status of any computing node according to the maintained node status table. If any computing node is in an online state, the relationship table entry corresponding to the target observation task can be maintained; and when the target observation task is not preempted within the preset time, or the preempted computing node is recorded as offline in the node status table, any computing node is selected through the cluster resource scheduling strategy and the corresponding node information is written into the relationship table entry corresponding to the target observation task. Among them, when the scheduling center 111 maintains the relationship table entry of the target observation task, any computing node can be used to: pull up the observer 113 corresponding to the target business task locally according to the relationship table entry corresponding to the target observation task. It is worth noting that any computing node used to pull up the observer 113 and the specific computing node used to pull up the executor 112 can be the same node or different nodes.

[0062] In this specification, when multiple executors collaborate to process the same overall task, the multiple executors may correspond to the same observer or to their own observers. When multiple executors correspond to the same observer, the execution status of the multiple executors written into the database 12 based on the business tasks they execute is used by the same observer to generate indication information corresponding to each execution status; and when multiple executors correspond to their own observers, the execution status of each executor written into the database 12 based on the business tasks they execute is used by their own corresponding observers to generate corresponding indication information.

[0063] It should be understood that the correspondence between executors and observers is described here with the same overall task as the granularity. In this specification, the correspondence between executors and observers can also be determined from other dimensions. For example, the entire distributed cluster 11 can be deployed with only one observer to perform observation tasks for all executors. For another example, multiple observers can be deployed in the distributed cluster 11, but each observer corresponds to multiple executors, and the multiple executors corresponding to the same observer are not used to perform the same overall task. Of course, the examples here are only for illustration. How to set the correspondence between observers and executors can be determined by those skilled in the art according to actual needs, and this specification does not limit this.

[0064] In this specification, the correspondence between the executor and the corresponding observer can be maintained in the database 12, so that each observer can read the execution status written by the corresponding executor from the database. For example, after the executor 112 writes the execution status of the target business task it is responsible for into the database 12, the observer 113 can read the execution status written by the corresponding executor 112 according to the correspondence maintained in the database 12, and provide the generated indication information to the database 12 according to the execution status.

[0065] As for the instruction information generated by the observer, if the instruction information is used to indicate the operation of the executor corresponding to the observer, after the observer writes the instruction information into the database 12, the executor can also read the instruction information written by the observer corresponding to the observer from the database according to the corresponding relationship between the executor and the observer. For example, when the instruction information is the instruction information used to instruct the re-execution of the target business task as described above, the executor 112 can read it from the database 12 according to the corresponding relationship.

[0066] In this specification, a task assembler may also be deployed in the distributed cluster 11, and the task assembler refers to the execution subject responsible for generating various overall tasks. In the task assembler, a task assembly table may be maintained, so that after the overall task is generated according to the demand, the composition information of each overall task is maintained in the task assembly table. Among them, the task assembler also synchronizes the maintained task assembly table to the database 12 for the transmission of indication information.

[0067] For example, assume that the overall task to which the target business task that the executor 112 is responsible is assembled by the task assembler, and after the task assembler completes the assembly of the overall task, the assembly information of the overall task can be synchronized to the database 12 so that the database 12 can update the maintained task assembly table.

[0068] It should be understood that when the database 12 maintains a task assembly table and a resource scheduling table, if any executor completes the business task for which he is responsible, the corresponding observer can determine the instruction information generated by itself based on the two tables, and which computing node should be provided to instruct the next executor to perform the business task for which he is responsible.

[0069] For example, still taking the executor 112 and the observer 113 as an example, when the observer 113 reads the execution status written into the database 12 by the executor 112 to characterize that the target business task has been completed, it can determine the next business task responsible for the target business task based on the task assembly table, and based on the resource scheduling table, it can know which computing node the next business task is assigned to. On this basis, it can instruct the corresponding computing node to read the instruction information written by the observer 113 from the database 12, and call its own processing resources to form an executor for processing the next business task.

[0070] It is worth noting that in addition to being responsible for generating the overall task, the assembler can also be responsible for generating the observation tasks corresponding to each business task in the overall task. In other words, the assembler is used not only to generate business tasks, but also to generate observation tasks. In this process, the correspondence between each observation task and its corresponding business task can be determined, and then the correspondence between each executor and observer can be indirectly determined.

[0071] It should also be noted that the computing nodes included in the distributed cluster of this specification can be any type of device. For example, all computing nodes in the cluster can be servers, so the distributed cluster is a server cluster; for another example, the computing nodes in the distributed cluster can also be any type of electronic device, such as PC (Personal Computer), smart phone, tablet computer, etc. Of course, the examples here are only illustrative. In addition to the physical devices in the above examples, the computing nodes in the distributed cluster can also be virtual devices such as virtual machines. It should be understood that any type of subject that can provide processing resources can be used as a computing node in a distributed cluster.

[0072] As can be seen from the above technical solution, this specification uses the scheduling center deployed in the distributed cluster to dynamically schedule the processing resources in the distributed cluster to allocate the processing resources to the executors and observers of the target business tasks. On this basis, the executor can use the allocated processing resources to execute the target business task and write the execution status of the target business task into the database; and the observer can use the allocated processing resources to read the execution status from the database and generate indication information corresponding to the execution status, and then write the indication information into the database to instruct the executor or other executors to perform further operations.

[0073] It should be understood that the scheduling center in this specification dynamically schedules the processing resources in the distributed cluster to executors and observers, where the executors are responsible for executing the target business tasks, while the observers are used to generate indication information for indicating further operations of the target business tasks based on the execution status of the target business tasks. It can be seen that this specification introduces an observer mechanism, which enables information interaction between different executors, avoiding the direct interaction between the various execution entities working together in the related technology, resulting in the failure of the corresponding task execution when any execution entity fails and cannot interact.

[0074] In addition, the distributed cluster in this specification also includes a database, and the above-mentioned execution status and the indication information corresponding to the execution status are written into the database for reading. It can be seen that the executors and observers in this specification realize information interaction through simple database reading and writing logic rather than direct interaction, thereby avoiding the problem of direct interaction between the various execution entities working together in the distributed cluster of the related technology, and the need to write additional complex interaction logic.

[0075] This specification also proposes a task execution method based on the observer mechanism. It is worth noting that this method only takes the executor corresponding to the above-mentioned target business task as the execution subject, and introduces the technical solution of this specification. Most of the operation methods, such as how to realize dynamic resource scheduling and how to realize the interaction between different executors, have been introduced in the system described above. The relevant contents can refer to the above contents and will not be repeated in the following text.

[0076] Figure 2 This is a flowchart of a task execution method based on an observer mechanism, which is shown as an exemplary embodiment of this specification. The method is applied to an executor corresponding to a target business task; the processing resources allocated to the executor in a distributed cluster are dynamically scheduled to the executor by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the observer corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; Figure 2 As shown, the method may include the following steps:

[0077] Step 202, executing the target business task using the allocated processing resources, and writing the execution status of the target business task into the database;

[0078] The execution status is read from the database by the observer to generate indication information corresponding to the execution status; the indication information is written into the database by the observer to indicate further operations of the executor and / or other executors.

[0079] As described above, the dispatch center can be deployed in a distributed cluster in different ways. For example, when the computing nodes in the distributed cluster are divided into master and slave nodes, the dispatch center can be deployed in the master node; for another example, a server independent of the computing nodes can be added to the distributed cluster to deploy the dispatch center in the server.

[0080] As described above, the dispatch center can use different methods to achieve dynamic scheduling of distributed cluster resources. For example, in order to achieve load balancing of computing nodes in a distributed cluster, the dispatch center can monitor the load status of computing nodes in the distributed cluster and give priority to allocating processing resources to computing nodes with lower loads; for another example, in order to improve the processing efficiency of tasks, the quality of processing resources in each computing node can be determined in advance, and relatively better processing resources can be allocated first.

[0081] As described above, the executor can use the processing resources allocated to itself to execute the target business task and write the execution status of the target business task into the database, while the observer can use the processing resources allocated to itself to read the execution status from the database to generate indication information corresponding to the execution status, and write the indication information into the database, thereby instructing the executor and at least one of the other executors to perform further operations.

[0082] As described above, the indication information generated by the observer is used to indicate which execution subject performs which operation, and is related to the execution status of the target business task. For example, when the execution status of the target business task indicates that the executor has failed to execute the target business task, the indication information generated by the observer based on the execution status can be used to instruct the executor to re-execute the target business task; for another example, when the target business task is a subtask of an overall task, and the execution status of the target business task indicates that the executor has successfully executed the target business task, the indication information generated by the observer based on the execution status can be used to trigger the executor corresponding to the next subtask in the overall task to start executing the next business task, wherein the next subtask can correspond to either the executor or another executor.

[0083] As described above, a resource scheduling table can be maintained in the scheduling center, and resource scheduling can be achieved by creating a relational table item corresponding to the business task in the resource scheduling table. For example, the scheduling center can preferentially create a relational table item corresponding to the target business task in the resource scheduling table, and when it is determined that a specific computing node writes its own node information in the relational table item corresponding to the target business task to preempt the target business task, the status of the specific computing node is determined according to the maintained node status table, and if it is confirmed that the specific computing node is in an online state, the relational table item corresponding to the target business task is maintained; and when the target business task is not preempted within a preset time, or the preempted computing node is recorded as offline in the node status table, a specific computing node is selected through the cluster resource scheduling strategy and the node information of the specific computing node is written into the relational table item corresponding to the target business task.

[0084] As described above, the operation performed by the observer based on the execution status provided by the executor can be called an observation task. On this basis, the observation task performed by the observer corresponding to the target business task can be called a target observation task. Similar to the target business task, the scheduling center can also create a relationship table entry for the target observation task in the resource scheduling table to indicate the pull-up of the observer corresponding to the target observation task. Under this premise, when the scheduling center determines that any computing node writes its own node information in the relationship table entry corresponding to the target observation task to preempt the target observation task, it can also determine the status of any computing node according to the maintained node status table. If any computing node is online, the relationship table entry corresponding to the target observation task can be maintained; and when the target observation task is not preempted within the preset time, or the preempted computing node is recorded as offline in the node status table, any computing node is selected through the cluster resource scheduling strategy and the corresponding node information is written into the relationship table entry corresponding to the target observation task.

[0085] As described above, when multiple executors collaborate to handle the same overall business, the multiple executors can correspond to the same observer or have their own observers. When multiple executors correspond to the same observer, the execution status written into the database by the multiple executors based on the business tasks they execute is used by the same observer to generate indication information corresponding to each execution status; and when multiple executors correspond to their own observers, the execution status written into the database by each executor based on the business tasks they execute is used by their own corresponding observers to generate corresponding indication information.

[0086] As can be seen from the above technical solution, the present specification can allocate resources to executors and observers by means of dynamic resource scheduling by the scheduling center in the distributed cluster. In the process of executing business tasks, the executor writes the execution status into the database, so that the observer can read the execution status and generate indication information corresponding to the execution status, thereby indicating the next operation of the executor or other executors.

[0087] It should be understood that this specification introduces an observer mechanism and realizes data interaction between the executor and the observer through simple database reading and writing, avoiding the problem of high development costs caused by the need to pre-write complex interaction logic in related technologies.

[0088] This specification also proposes another task control method based on the observer mechanism. It is worth noting that this method only takes the observer corresponding to the above-mentioned target business task as the execution subject, and introduces the technical solution of this specification. Most of the operation methods, such as how to realize dynamic resource scheduling and how to realize the interaction between different executors, have been introduced in the system described above. The relevant content can refer to the above content and will not be repeated in the following text.

[0089] Figure 3 This is a flowchart of a task control method based on an observer mechanism, which is an exemplary embodiment of this specification. The method is applied to an observer corresponding to a target business task; the processing resources allocated to the observer in a distributed cluster are dynamically scheduled to the observer by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the executor corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; Figure 3 As shown, the method may include the following steps:

[0090] Step 302: using the allocated processing resources to read the execution status of the target business task from a database to generate indication information corresponding to the execution status, and write the indication information into the database;

[0091] The target business task is executed by the executor using the allocated processing resources, and the execution status is written into the database by the executor; the indication information written into the database is used to instruct further operations of the executor and / or other executors.

[0092] As described above, the dispatch center can be deployed in a distributed cluster in different ways. For example, when the computing nodes in the distributed cluster are divided into master and slave nodes, the dispatch center can be deployed in the master node; for another example, a server independent of the computing nodes can be added to the distributed cluster to deploy the dispatch center in the server.

[0093] As described above, the dispatch center can use different methods to achieve dynamic scheduling of distributed cluster resources. For example, in order to achieve load balancing of computing nodes in a distributed cluster, the dispatch center can monitor the load status of computing nodes in the distributed cluster and give priority to allocating processing resources to computing nodes with lower loads; for another example, in order to improve the processing efficiency of tasks, the quality of processing resources in each computing node can be determined in advance, and relatively better processing resources can be allocated first.

[0094] As described above, the scheduling center can create a relationship table entry for the target observation task in the resource scheduling table to indicate the pull-up of the observer corresponding to the target observation task. Under this premise, when determining that any computing node writes its own node information in the relationship table entry corresponding to the target observation task to preempt the target observation task, the scheduling center can also determine the status of any computing node based on the maintained node status table. If any computing node is online, the relationship table entry corresponding to the target observation task can be maintained; and when the target observation task is not preempted within the preset time, or the preempted computing node is recorded as offline in the node status table, any computing node is selected through the cluster resource scheduling strategy and the corresponding node information is written into the relationship table entry corresponding to the target observation task.

[0095] As described above, for the indication information generated by the observer, if the indication information is used to indicate the operation of the executor corresponding to it, after the observer writes the indication information into the database, the executor can also read the indication information written by the observer corresponding to it from the database based on the correspondence between the executor and the observer.

[0096] As described above, when multiple executors collaborate to handle the same overall business, the multiple executors can correspond to the same observer or have their own observers. When multiple executors correspond to the same observer, the execution status written into the database by the multiple executors based on the business tasks they execute is used by the same observer to generate indication information corresponding to each execution status; and when multiple executors correspond to their own observers, the execution status written into the database by each executor based on the business tasks they execute is used by their own corresponding observers to generate corresponding indication information.

[0097] As can be seen from the above technical solution, the present specification can allocate resources to executors and observers by means of dynamic resource scheduling by the scheduling center in the distributed cluster. Among them, the observer can generate corresponding instruction information according to the execution status of the business task written into the database by the executor to indicate the next operation of the executor or other executors.

[0098] It should be understood that this specification introduces an observer mechanism and realizes data interaction between the executor and the observer through simple database reading and writing, avoiding the problem of high development costs caused by the need to pre-write complex interaction logic in related technologies.

[0099] Below, the technical solution of this specification is introduced by taking “multiple executors collaborate to complete an overall task, and each executor has its own observer” as an example.

[0100] Figure 4 FIG. 1 is a schematic diagram of a task generation process shown in an exemplary embodiment of the present specification. In a distributed cluster, a code program for task assembly can be deployed in a specific computing node to form a task in the computing node. Figure 4 The task assembler 414 shown. On this basis, the user can instruct the task assembler 414 to generate an overall task for meeting the corresponding requirements by providing the task assembler 414 with the requirement information.

[0101] Taking the overall task X of "user 1 transfers 1,500 yuan to user 2's bank account based on the balance in the payment platform" as an example, if the task assembler 414 finds that the bank account bound to user 1 in the payment platform is bank M, and user 2's account opening behavior is bank N, then the overall task X that can be constructed by the task assembler 414 can be shown in the following Table 1:

[0102] Subtask x Transfer 1,500 yuan from user 1's platform account to user 1's bank account M Subtask x Transfer 1,500 yuan from user 1's bank account M to user 2's bank account N <![CDATA[Observation task y of subtask x1]]> Generate instruction information based on the transfer status from the platform account to the bank account M <![CDATA[Observation task y of subtask x2]]> Generate instruction information based on the transfer status of bank account M to bank account N

[0103] Table 1

[0104] Based on Table 1, the executor responsible for subtask x1 can be directly called executor x1', the executor responsible for subtask x2 can be called executor x2', the observer responsible for observing task y1 can be called observer y1', and the observer responsible for observing task y2 can be called observer y2'. The corresponding relationship can be referred to in Table 2 below:

[0105]

[0106] Table 2

[0107] It should be understood that the task assembler 414 may also pre-set the indication information generated by the observer in different situations. For example, the setting situation may be as shown in Table 3:

[0108]

[0109]

[0110] Table 3

[0111] After completing the task assembly operation of Tables 1 to 3, the task information of the assembled overall task X can be provided to the task manager 415, so that it can write the task information of the assembled overall task X into the maintained task management table. Further, the task manager 415 can write the task information of the overall task X into the task management table maintained by the database 42.

[0112] After the task information of the overall task X is written into the database 42, the scheduling center 411 can read the task information of the overall task X from the database 42, and create the relationship table items of each subtask included in the overall task X in the maintained resource scheduling table according to the read task information, so that each computing node in the distributed cluster can preempt it. Among them, when the relationship table items of the subtasks are created but not preempted, the records in the resource scheduling table can be shown in the following Table 4:

[0113]

[0114] Table 4

[0115] After completing the creation of task table entries for each subtask, the computing nodes used to deploy executors x1', x2', and observers y1', y2' can be determined based on the dynamic scheduling strategy of the distributed cluster, and then the corresponding computing nodes preempt the corresponding task table entries.

[0116] It is worth noting that each computing node in the distributed cluster can be pre-deployed with codes for implementing the above-mentioned operations such as "transferring" and "generating indication information", and dynamic resource scheduling, or the operation of deploying observers and executors, is only used to determine which nodes' computing resources are used to run these codes. For example, when it is determined to deploy executor x1' to computing node 23 based on the dynamic scheduling strategy, the corresponding execution module can be generated based on the pre-deployed code for executing the "transferring" operation as executor x1' deployed in computing node 23. Correspondingly, computing node 23 can preempt the task list item of subtask x1, and the resource scheduling table after preemption can be shown in Table 5:

[0117]

[0118] Table 5

[0119] Similarly, executor x2', observer y1', and observer y2' can all be deployed in a similar manner, and the corresponding computing nodes can also preempt the task items in the resource scheduling table.

[0120] Of course, in addition to deploying the code in each node in the distributed cluster in advance, you can also determine the computing nodes for deploying executors and observers based on the dynamic scheduling strategy, and then provide the corresponding code to the corresponding computing nodes so that the computing nodes can temporarily generate the corresponding execution modules as executors or observers. This manual does not limit this.

[0121] In this embodiment, in addition to the task assembler 414, a trigger constructor 416 may also be included, which may also be formed in a specific computing node by deploying a code program for "constructing a trigger" in the computing node. The trigger constructor 416 is used to generate a trigger for the overall task, so as to instruct the task manager 415 to trigger and start executing the corresponding overall task when the trigger condition of the overall task is met.

[0122] It is worth noting that the technician can perform a mutual exclusion check on the dispatch center 411 and the task manager 415 to avoid duplicate task creation. Figure 4 The system structure shown is a logical-level system structure. The components included are all logical components and should not be considered as physical components.

[0123] Based on Figure 4 After the overall task X is constructed in the manner shown, the overall task X can be completed by executing each subtask included in the overall task X.

[0124] Figure 5 FIG. 1 is a flowchart of a task execution method shown in an exemplary embodiment of this specification. Figure 5 As shown, X_TASK in the figure is the task information of the overall task X, which is maintained in the database 42. Executor_1 and Executor_2 in the figure are subtasks x1 and x2 included in the overall task X, and Executant_1 and Executant_2 in the figure are executors x1' and x2', respectively. Observer_1 and Observer_2 in the figure are observers y1' and y2', respectively.

[0125] Figure 6 This is an interactive diagram of a task execution method shown in an exemplary embodiment of this specification. Figure 5 and Figure 6 The execution process of the overall task X is introduced. Figure 6 As shown, the method may include the following steps:

[0126] Step 601, executor x1' executes Executor_1.

[0127] In this embodiment, the task manager 415 can determine whether to start executing the overall task X according to the trigger created in advance for the overall task X. For example, the trigger can be a timer, and the overall task X is set to start executing after 1 minute. Then, after the trigger constructor 416 completes the creation of the timer, the timer can start timing, and after one minute, the task manager 415 can write the "instruction information for instructing the start of the execution of the overall task X" into the database 42. Of course, here only the trigger is used as an example for introduction. In actual applications, the trigger can set any type of execution conditions. For example, the execution condition of the trigger can be: execution when receiving a user instruction, execution when the predecessor task of the overall task X is completed, etc. This embodiment does not limit this.

[0128] It is worth noting that, as described above, due to the use of a dynamic scheduling method, the operation of scheduling resources by the scheduling center 411 can be performed synchronously during the assembly process of the overall task X. For example, the scheduling center 411 can determine the computing node 23 for deploying the executor x1' based on a preset scheduling strategy, and instruct the computing node 23 to deploy the executor x1'. After completing the deployment of the executor x1', the computing node 23 can read data from the database 42 in real time. If the "indication information for instructing the start of the execution of the overall task X" written by the task manager 415 is read, the deployed executor x1' is started so that the executor x1' executes Executor_1.

[0129] It should also be noted that, in order to ensure that the execution state State_1 written by executor x1' into database 42 can be read by observer y1' in time and give instructions, the scheduling center 411 in this embodiment can determine the computing node for deploying observer y1' based on the scheduling policy before executor x1' starts to execute Executor_1. Assuming that observer y1' is determined to be deployed on computing node 24, the resource scheduling table maintained in the scheduling center 411 can be updated to the status shown in Table 6:

[0130]

[0131] Table 6

[0132] In order to avoid wasting processing resources, the deployed observer y1' can be in a dormant state, while the executor x1' can be Figure 5As shown, after completing the node check and the task parameter check, the observer y1' is enabled so that the observer y1' starts running. On this basis, the executor x1' can start executing Executor_1.

[0133] Step 602 , executor x1 ′ writes the execution state State_1 of Executor_1 into the database 42 .

[0134] In this embodiment, executor x1' can write the execution state State_1 into the database 42 in real time during the execution of Executor_1, and add the identifier of observer y1' when writing the execution state, indicating that the recipient of the execution state is observer y1', so that observer y1' can determine the data provided to itself from the data maintained by the database 42.

[0135] It is worth noting that due to the use of dynamic scheduling strategy, the scheduling center 411 can synchronously perform the operations of "determining the computing node for deploying executor x2'" and "determining the computing node for deploying observer y2'" during the process of executor x1' executing Executor_1. Continuing with the above example, assuming that it is determined to deploy executor x2' on computing node 28 and observer y2' on computing node 30, the resource scheduling table maintained in the scheduling center 411 can be updated to the status shown in Table 7:

[0136]

[0137] Table 7

[0138] Step 603 , observer y1 ′ reads State_1 from database 42 .

[0139] In this embodiment, the observer y1' can monitor the data in the database 42 to determine whether there is data provided to itself, and when the execution state State_1 written by the executor x1' is read from the database 42, it can determine whether Executor_1 is executed successfully based on the execution state State_1.

[0140] Step 604 , observer y1 ′ determines whether Executor_1 is executed successfully based on State_1 , if so, jump to step 605 , otherwise, jump to step 615 .

[0141] In this embodiment, if it is determined based on the execution state State_1 that Executor_1 is executed successfully, then indication information Indicate_2start for instructing executor x2' to execute Executor_2 is generated, and an identifier for indicating the indication provided to executor x2' is added to the indication information Indicate_2start before being written into the database 42.

[0142] On the contrary, if it is determined based on the execution state State_1 that Executor_1 fails to execute, then an indication message Indicate_1again for instructing executor x1 ′ to re-execute Executor_1 is generated, and an identifier for indicating the indication provided to executor x1 ′ is added to the indication message Indicate_1again and then written into the database 42 .

[0143] Step 605 , the observer y1 ′ generates indication information Indicate_2start for instructing the executor x2 ′ to execute Executor_2 .

[0144] Step 606 , observer y1 ′ writes Indicate_2start into the database 42 .

[0145] Step 607 , the executor x2 ′ reads Indicate_2start from the database 42 .

[0146] In this embodiment, after the observer y1 ′ writes the indication information Indicate_2start into the database 42 , the executor x2 ′ can read Indicate_2start from the database 42 , and then execute Executor_2 according to the indication information.

[0147] It should be emphasized that since the present embodiment adopts a dynamic scheduling method, in the process of the observer y1' generating the indication information Indicate_2start, the scheduling center 411 can determine whether to change the deployment node of the executor x2' according to the processing resource occupancy status of each computing node. For example, when it is determined that the processing resources of the computing node 28 are insufficient, the computing node for deploying the executor x2' can be re-determined. Assuming that the re-determined computing node is node 29, the executor x2' can be deployed in node 29. Accordingly, the preempting node of the table item of the subtask x2 in the resource scheduling table can be changed to node 29. On this basis, the executor x2' in the computing node 29 can read Indicate_2start from the database.

[0148] Similarly, the deployment node of the observer y2' can also be changed by the scheduling center 411 according to the real-time occupancy of the processing resources in the distributed cluster. The processing process is similar to that of the executor x2' and will not be repeated here.

[0149] Step 608 , executor x2 ′ executes Executor_2 .

[0150] Step 609 , executor x2 ′ writes the execution state State_2 of Executor_2 into the database 42 .

[0151] Similar to the execution of Executor_1 by executor x1', executor x2' can also write the execution state State_2 of Executor_2 into database 42 during the execution of Executor_2. The identifier of observer y2' can be added to indicate that observer y2' is the receiver of the execution state.

[0152] In step 610 , observer y2 ′ reads State_2 from database 42 .

[0153] Step 611 , observer y2 ′ determines whether State_2 indicates that Executor_2 is executed successfully. If so, jump to step 612 ; otherwise, jump to step 617 .

[0154] In this embodiment, similar to observer y1', observer y2' also needs to determine whether Executor_2 is executed successfully after reading State_2.

[0155] If the execution is successful, since Executor_2 is the last subtask of the overall task X, the observer y2' can generate notification information X_Finish for notifying that X_TASK has been completed. The observer y2' can add the identification information of the task manager 415 to X_Finish to indicate that the recipient of X_Finish is X_Finish.

[0156] Step 612 , the observer y2 ′ generates notification information X_Finish for notifying that X_TASK has been completed.

[0157] Step 613 , observer y2 ′ writes X_Finish into the database 42 .

[0158] In step 614 , the task manager 415 reads X_Finish and marks the execution status of the entire task X as completed.

[0159] In this embodiment, the task manager 415 may also monitor the database 42 to obtain data provided to itself, wherein when X_Finish is read, the execution status of the entire task X in the task management table is marked as "completed".

[0160] Step 615 , the observer y1 ′ generates indication information Indicate_1again for instructing the executor x1 ′ to re-execute Executor_1 , and writes the information into the database 42 .

[0161] In this embodiment, when the observer y1' determines that Executor_1 is not executed successfully, it can generate the instruction information Indicate_1again "for instructing the executor x1' to re-execute Executor_1", and write it into the database 42 after adding the identifier of the executor x1'. Based on this, the executor x1' can read Indicate_1again from the database 42 and re-execute Executor_1 according to the instruction.

[0162] In step 616 , the executor x1 ′ reads Indicate_1 again from the database 42 and jumps to step 601 .

[0163] Step 617 , the observer y2 ′ generates indication information Indicate_2again for instructing the executor x2 ′ to re-execute Executor_2 , and writes the information into the database 42 .

[0164] In this embodiment, similar to observer y1', observer y2', when determining that Executor_2 is not executed successfully, can generate indication information Indicate_1again "for instructing executor x2' to re-execute Executor_2", and write it into database 42 after adding the identifier of executor x2'. Based on this, executor x2' can read Indicate_2again from database 42 and re-execute Executor_2 according to the indication.

[0165] In step 618 , the executor x2 ′ reads Indicate_2 again from the database 42 and jumps to step 608 .

[0166] It should be emphasized that in order to coordinate the observers of each subtask, Figure 5As shown, a global observer Y corresponding to the overall task X is deployed. The global observer Y can obtain the execution status of the subtasks that the observers of each subtask are responsible for observing, and based on the obtained execution status, send indication information to the corresponding observer to avoid problems such as the observer failing to successfully execute the observed subtask, resulting in the inability to generate indication information in time for the execution status provided by the executor. The relationship between the observers responsible for each observed subtask and the global observer, such as generating different indication information based on different execution status, can refer to the relationship between the executors of each subtask and the observer, and will not be elaborated here.

[0167] As can be seen from the above embodiments, this specification introduces an observer mechanism, and each subtask contained in the overall task can correspond to its own executor and observer. Among them, the information interaction between different executors is completed through the observer, rather than the executors themselves, and when the corresponding subtask fails to execute, the observer instructs the corresponding executor to re-execute the subtask, avoiding the problem in the related technology that due to the interaction between the executors themselves, once a single executor's task fails, the overall task cannot be completed, resulting in a high task suspension rate.

[0168] In addition, the executors and observers in this specification implement information interaction through simple database read and write logic, thereby achieving the decoupling between dynamic scheduling of processing resources and information interaction, avoiding the self-interaction between executors in related technologies, and the synchronous execution of operations for dynamic scheduling of processing resources during the interaction process, which requires the pre-writing of complex interaction logic and causes high development costs.

[0169] Figure 7 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 7 At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710, and may also include hardware required for other functions. One or more embodiments of this specification may be implemented based on software, such as the processor 702 reading the corresponding computer program from the non-volatile memory 710 into the memory 708 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0170] Please refer to Figure 8 , the task execution device based on the observer mechanism can be applied to Figure 7The device shown in the figure is used to implement the technical solution of this specification. The device is applied to the executor corresponding to the target business task; the processing resources allocated to the executor in the distributed cluster are dynamically scheduled to the executor by the scheduling center deployed in the distributed cluster according to the predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the observer corresponding to the target business task by the scheduling center according to the predefined cluster resource scheduling strategy; wherein, the task execution device may include:

[0171] An execution unit 801 executes the target business task using the allocated processing resources, and writes the execution status of the target business task into the database;

[0172] The execution status is read from the database by the observer to generate indication information corresponding to the execution status; the indication information is written into the database by the observer to indicate further operations of the executor and / or other executors.

[0173] Please refer to Fig. 9 , the task control device based on the observer mechanism can be applied to Figure 7 The device shown in the figure is used to implement the technical solution of this specification. The device is applied to the observer corresponding to the target business task; the processing resources allocated to the observer in the distributed cluster are dynamically scheduled to the observer by the scheduling center deployed in the distributed cluster according to the predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the executor corresponding to the target business task by the scheduling center according to the predefined cluster resource scheduling strategy; wherein, the task control device may include:

[0174] The generating unit 901 uses the allocated processing resources to read the execution status of the target business task from the database to generate indication information corresponding to the execution status, and writes the indication information into the database;

[0175] The target business task is executed by the executor using the allocated processing resources, and the execution status is written into the database by the executor; the indication information written into the database is used to instruct further operations of the executor and / or other executors.

[0176] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in any of the above embodiments by running the executable instructions.

[0177] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0178] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

Claims

1. A task scheduling system based on an observer mechanism, comprising: Distributed clusters and databases; A scheduling center is deployed in the distributed cluster, and the scheduling center is used to: dynamically schedule processing resources in the distributed cluster according to a predefined cluster resource scheduling strategy, so as to allocate corresponding processing resources to executors and observers corresponding to target business tasks; The executor executes the target business task using the allocated processing resources, and writes the execution status of the target business task into the database; The observer uses the allocated processing resources to read the execution status from the database, generates indication information corresponding to the execution status, and writes the indication information into the database to indicate further operations of the executor and / or other executors.

2. The system according to claim 1, In the case where the execution status indicates that the executor has failed to execute the target business task, the instruction information is used to instruct the executor to re-execute the target business task; and / or, When the target business task is a subtask of an overall task and the execution status indicates that the executor has successfully executed the target business task, the indication information is used to trigger the executor corresponding to the next subtask in the overall task to start executing the next subtask; wherein, The next subtask corresponds to the executor or other executors.

3. The system according to claim 1, According to the predefined cluster resource scheduling strategy, dynamically schedule the processing resources in the distributed cluster to allocate the corresponding processing resources to the executors corresponding to the target business tasks, including: Maintaining the correspondence between the target business task and the specific computing node in a resource scheduling table, so as to allocate the processing resources on the specific computing node to the executor corresponding to the target business task; The specific computing node locally pulls up the executor according to the corresponding relationship to execute the target business task.

4. The system according to claim 3, The dispatch center maintains the corresponding relationship in the following manner: Creating a relation table entry corresponding to the target business task in the resource scheduling table; In the case where it is determined that the specific computing node writes its own node information in the relationship table entry corresponding to the target business task to preempt the target business task, if it is confirmed according to the maintained node status table that the specific computing node is in an online state, then the relationship table entry corresponding to the target business task is maintained; When the target business task is not preempted within a preset time, or the preempted computing node is recorded as offline in the node status table, the specific computing node is selected through the cluster resource scheduling strategy and the corresponding node information is written into the relationship table item corresponding to the target business task.

5. The system according to claim 4, The dispatch center, Creating a relation table entry corresponding to a target observation task in the resource scheduling table, wherein the target observation task is used to indicate to pull up an observer corresponding to the target business task; In the case where it is determined that any computing node writes its own node information in the relationship table item corresponding to the target observation task to preempt the target observation task, if it is confirmed according to the maintained node status table that any computing node is in an online state, then the relationship table item corresponding to the target observation task is maintained; When the target observation task is not preempted within a preset time, or the preempted computing node is recorded as offline in the node status table, any computing node is selected through the cluster resource scheduling strategy and the corresponding node information is written into the relationship table item corresponding to the target observation task; Any one of the computing nodes locally pulls up an observer corresponding to the target business task according to the relationship table entry corresponding to the target observation task; wherein, any one of the computing nodes and the specific computing node are the same or different nodes.

6. A task execution method based on an observer mechanism, applied to an executor corresponding to a target business task; the processing resources allocated to the executor in a distributed cluster are dynamically scheduled to the executor by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to the observer corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the method comprises: Executing the target business task using the allocated processing resources, and writing the execution status of the target business task into a database; The execution status is read from the database by the observer to generate indication information corresponding to the execution status; the indication information is written into the database by the observer to indicate further operations of the executor and / or other executors.

7. A task control method based on an observer mechanism, applied to an observer corresponding to a target business task; the processing resources allocated to the observer in a distributed cluster are dynamically scheduled to the observer by a scheduling center deployed in the distributed cluster according to a predefined cluster resource scheduling strategy; the processing resources in the distributed cluster are also dynamically scheduled to an executor corresponding to the target business task by the scheduling center according to a predefined cluster resource scheduling strategy; the method comprises: Using the allocated processing resources, reading the execution status of the target business task from a database to generate indication information corresponding to the execution status, and writing the indication information into the database; The target business task is executed by the executor using the allocated processing resources, and the execution status is written into the database by the executor; the indication information written into the database is used to instruct further operations of the executor and / or other executors.

8. An electronic device comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as claimed in any one of claims 6 or 7 by running the executable instructions.

9. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method as claimed in any one of claims 6 or 7.

10. A computer program product comprising a computer program / instructions, which when executed by a processor implement the steps of the method according to any one of claims 6 or 7.