A task distribution method, device, equipment and medium

Abstract

The present invention relates to the field of computer software and digital medical technology, and discloses a task distribution method, device, equipment and medium. The present invention modifies the original task list according to the latest configured task list and the currently running task list to obtain a to-be-configured list with only tasks to be distributed, and then distributes the tasks in the to-be-configured list to normally running temporary nodes and newly added nodes, so that not only the newly added nodes can share the execution of part of the tasks, and the tasks in the to-be-configured list can be reasonably run on the temporary nodes and the newly added nodes, but also the problem of increasing the load of the temporary nodes when the tasks need to be distributed can be avoided, so that the load effect of each temporary node and the newly added node is balanced.

Description

Technical Field

[0001] The present invention relates to the fields of computer application and digital medical technology, and in particular to a task distribution method, device, equipment and medium. Background Art

[0002] Medical cloud refers to the use of cloud computing to create a medical and health service cloud platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, and the Internet of Things, combined with medical technology, to achieve the sharing of medical resources and the expansion of medical scope.

[0003] At present, with the business development of Internet companies, multi-tasking and high-concurrency scenarios have become very common in medical cloud technology. When the task list executed in medical cloud technology is updated, the tasks on the updated task list need to be distributed synchronously. Since the load conditions of machines corresponding to different nodes are different, after the new tasks are distributed, the machines corresponding to individual nodes may be overloaded and crash. Summary of the invention

[0004] The present invention provides a task distribution method, device, computer equipment and medium to solve the problem of abnormal downtime due to excessive load of individual nodes in the existing task distribution process.

[0005] In a first aspect, a task distribution method is provided, comprising:

[0006] Create multiple temporary nodes in the zookeeper cluster, and determine a master node among the multiple temporary nodes, the master node is used to read the original task list and the latest configured task list; the read original task list includes the tasks that have been configured to each of the temporary nodes, and the latest configured task list includes the tasks to be newly configured to each of the temporary nodes;

[0007] Calling the master node to collect task information and node status information corresponding to each of the temporary nodes, and determining a list of currently running tasks, normally running temporary nodes, and abnormally running temporary nodes; the currently running task list includes all tasks currently running on all the temporary nodes;

[0008] Calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to determine whether the original task list has changed;

[0009] If there is a change, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured; the list to be configured includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list;

[0010] Create a new node in k8s, and distribute the tasks in the to-be-configured list to the normally running temporary node and the new node.

[0011] In a second aspect, a task distribution device is provided, comprising:

[0012] A node creation module is used to create multiple temporary nodes in the zookeeper cluster and determine a master node among the multiple temporary nodes, wherein the master node is used to read the original task list and the latest configured task list from the database; the read original task list includes the tasks that have been configured to each of the temporary nodes, and the latest configured task list includes the tasks to be newly configured to each of the temporary nodes;

[0013] A collection module, used to call the master node to collect the task information and node status information corresponding to each of the temporary nodes, and determine the currently running task list, the normally running temporary nodes and the abnormally running temporary nodes; the currently running task list includes all the tasks currently running on all the temporary nodes;

[0014] A judgment module, used for calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to judge whether the original task list has changed;

[0015] A module for acquiring a list to be configured, for modifying the original task list according to the latest configured task list and the currently running task list to obtain a list to be configured if changes occur; the list to be configured includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list;

[0016] The task distribution module is used to create a new node in k8s and distribute the tasks in the to-be-configured list to the normally running temporary nodes and the new nodes.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned task distribution method when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned task distribution method are implemented.

[0019] In the scheme implemented by the above-mentioned task distribution method, device, computer equipment and storage medium, multiple temporary nodes are created in the zookeeper cluster, and a master node is determined among the multiple temporary nodes; the master node is called to collect the task information and node status information corresponding to each temporary node, and the currently running task list, the normally running temporary nodes and the abnormally running temporary nodes are determined; the master node is called to compare the currently running task list and the latest configured task list with the original task list respectively to determine whether the original task list has changed; if it has changed, the original task list is modified according to the latest configured task list and the currently running task list to obtain the list to be configured; a new node is created in k8s, and the tasks in the list to be configured are distributed to the normally running temporary nodes and the new nodes. The present application obtains a to-be-configured list containing only the tasks to be distributed by modifying the original task list according to the latest configured task list and the currently running task list, and then distributes the tasks in the to-be-configured list to the normally operating temporary nodes and newly added nodes. This not only enables the newly added nodes to share the execution of part of the tasks and enables the tasks in the to-be-configured list to run reasonably on the temporary nodes and newly added nodes, but also avoids the problem of increasing the load of temporary nodes when tasks need to be distributed, thereby balancing the load effects of various temporary nodes and newly added nodes, and thus solving the problem of abnormal downtime due to excessive load on individual nodes in the existing task distribution process. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0021] Figure 1 is a schematic diagram of an application environment of a task distribution method in one embodiment of the present invention;

[0022] Figure 2 is a flowchart of a task distribution method in one embodiment of the present invention;

[0023] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S50;

[0024] Figure 4 yes Figure 3 A schematic flow chart of a specific implementation of step S51;

[0025] Figure 5 yes Figure 4 A schematic flow chart of a specific implementation of step S50;

[0026] Figure 6 is a structural diagram of a task distribution device in one embodiment of the present invention;

[0027] Figure 7 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0028] Figure 8 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figure 1 As shown, the task distribution method provided by the embodiment of the present invention is applied to a task distribution network, and the task distribution network may include: a scheduling node 100, a temporary node 200 and a newly added node 300, wherein the scheduling node 100, the temporary node 200 and the newly added node 300 can all be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0031] See also Figure 2 As shown, Figure 2 A flowchart of a task distribution method provided by an embodiment of the present invention includes the following steps:

[0032] S10: creating multiple temporary nodes in the zookeeper cluster, and determining a master node among the multiple temporary nodes, wherein the master node is used to read the original task list and the latest configured task list from the database.

[0033] The read original task list includes tasks that have been configured to each temporary node, and the latest configured task list includes tasks to be newly configured to each temporary node. Here, the original task list is obtained by manually sending it to the database by the server; the latest configured task list can be obtained by manually changing the database by the server, or by sending an updated task list to the database through an update instruction.

[0034] Zookepper is an open source coordination service for distributed applications. It is software that provides consistency functional services for distributed applications. The functions provided include: configuration maintenance, domain name services, distributed synchronization, group services, etc. In this implementation, the zookeeper cluster creates multiple temporary nodes on the server. In actual applications, applications are deployed in cluster mode. At this time, it is necessary to distribute tasks to the application cluster. In order to ensure that there is only one master node to distribute tasks to each temporary node at any time, this application determines a master node from multiple temporary nodes created in the zookeeper cluster, and enables the master node to read the original task list and the latest configured task list from the database to obtain the tasks to be performed by each temporary node. For example, when the latest configured task list is the latest MySQL data after manual change, the original task list is the MySQL data obtained from the database at the beginning. The master node reads MySQL data from the database every minute. If the read MySQL data is changed compared to the previous MySQL data, the changed MySQL data is the latest updated MySQL data, that is, the latest configured task list; and after the latest updated MySQL data is determined, the task list that needs to synchronize the Roma message is determined according to the latest updated MySQL data to obtain all tasks that need to be executed. Here, the task list of the Roma message is used to process the binlog log generated by the addition, deletion and change data of the MySQL database table, and then convert it into a Rocket MQ message queue in JSON format.

[0035] It should be noted that each temporary node updates its corresponding task ID, task name and load to zookepper in real time.

[0036] S20: calling the master node to collect the task information and node status information corresponding to each of the temporary nodes, and determining the list of currently running tasks, normally running temporary nodes and abnormally running temporary nodes.

[0037] Here, the task information corresponding to each temporary node includes the task ID and task name of the temporary node, and the node status information includes load information. The main node can obtain the task ID, task name and load information corresponding to each temporary node according to zookepper, and can determine the currently running task list according to the task name running on each temporary node; at the same time, according to the load of each temporary node, determine that the temporary node with a load less than a preset threshold is a normally operating temporary node, and determine that the temporary node with a load not less than the preset threshold is an abnormally running temporary node.

[0038] The currently running task list includes all the currently running tasks on all the temporary nodes. Since some of the tasks on the original task list will be manually suspended after the tasks on the temporary node are assigned to the temporary node, the currently running task list may be inconsistent with the original task list. Therefore, by determining the currently running task list, the tasks that were stopped in the original task list can be determined.

[0039] S30: calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to determine whether the original task list has changed.

[0040] Here, the change of the original task list includes that there are new tasks to be synchronized in the latest configured task list or there are stopped tasks in the original task list.

[0041] In one embodiment, the calling of the master node to compare the currently running task list and the latest configured task list with the original task list respectively, and judging whether the original task list has changed, may include: judging whether there are new tasks on the latest configured task list and whether there is a first stop task that needs to be stopped on the original task list according to the comparison between the original task list and the latest configured task list; judging whether there is a second stop task that has been stopped on the original task list according to the comparison between the currently running task list and the original task list; if the judgment result is that there is the new task that needs to be configured, and / or there is a first stop task and / or a second stop task that has been stopped on the original task list, then it is determined that the original task list has changed. By comparing the original task list with the latest configured task list, and comparing the currently running task list with the original task list, that is, when at least one of the three situations occurs, namely, there is the new task that needs to be configured, the first stop task on the original task list is stopped, and the second stop task on the original task list is stopped, it can be determined whether the original task list has changed, so as to accurately obtain the list to be configured in step S40.

[0042] S40: If there is a change, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured.

[0043] The to-be-configured list includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list.

[0044] Furthermore, in order to quickly and accurately modify the original list, in some embodiments, the modification of the original task list based on the latest configured task list and the currently running task list may include: adding the new task to the original task list, and / or removing the first stopped task and / or the second stopped task on the original task list from the original task list to complete the modification of the original task list; using the modified original task list as the list to be configured, so as to quickly and accurately complete the modification of the original task list.

[0045] For example, when it is determined that there are new tasks on the latest configured task list based on the comparison between the original task list and the latest configured task list, the new tasks are added to the original task list to generate task list 1. If it is then determined that there are stopped tasks on the original task list, the stopped tasks are removed from task list 1 to generate task list 2. At this time, the generated task list 2 is the list to be configured.

[0046] S50: Create a new node in k8s, and distribute the tasks in the to-be-configured list to the normally running temporary node and the new node.

[0047] It should be noted that k8s is the abbreviation of Kubernetes, an open source platform for automated deployment, expansion, and operation and maintenance of container clusters, and can provide a complete open source solution for container orchestration management. Before creating a new node, traverse all temporary nodes, determine the node with the lowest load, and distribute the new tasks in the list to be configured to the node with the lowest load, and then update the new task distribution results to the zookepper task. At the same time, after listening to the changes in the zookepper task, each temporary node claims its corresponding task list and then starts the synchronization of related tasks.

[0048] Here, each temporary node and newly added node can be an execution device located on the client, so as to realize the synchronous distribution of tasks located on the list to be configured to the execution device on the client. By distributing the tasks in the list to be configured to the normally operating temporary nodes and newly added nodes, not only can the newly added nodes share the execution of part of the tasks, so that the tasks in the list to be configured can be reasonably run on the temporary nodes and newly added nodes, but also the problem of increasing the load of the temporary nodes when there are new tasks to be distributed can be avoided, so that the load effect of each temporary node and the newly added node is balanced.

[0049] See also Figure 3 In order to avoid distributing too many tasks to the newly created nodes, resulting in an excessive load on the newly created nodes, the newly created nodes are created in k8s, and the tasks in the to-be-configured list are distributed to the normally running temporary nodes and the newly created nodes, including:

[0050] S51: Distribute the tasks on the to-be-configured list running on the abnormally running temporary node to each of the newly added nodes.

[0051] For example, the abnormally running temporary nodes are node 1 and node 2. After creating new node 1 and new node 2 in k8s, the tasks on the to-be-configured list running on node 1 are distributed to the new node 1, and the tasks on the to-be-configured list running on node 2 are distributed to the new node 2.

[0052] In some embodiments, see Figure 4 , distributing the tasks on the to-be-configured list running on the abnormally running temporary node to each of the newly added nodes, includes:

[0053] S511: Create a preset number of new nodes in the k8s.

[0054] By creating a preset number of new nodes in k8s, the tasks in the list to be configured can be distributed to the newly created new nodes. For example, if the preset number is 2, after creating new nodes 1 and 2 in k8s, some tasks in the list to be configured can be distributed to new nodes 1 and 2. Here, the preset number may not be less than the number of temporary nodes that are running abnormally.

[0055] S512: Recycling the same number of the abnormally running temporary nodes as the preset number into the zookeeper cluster.

[0056] Since creating a preset number of new nodes in k8s can share the tasks on the abnormally running temporary nodes, in order to make the load of the newly created new nodes too high, the abnormally running temporary nodes equal to the preset number are recycled into the zookeeper cluster, and the tasks on the recycled abnormally running temporary nodes can be distributed to the new nodes created in each step S511 respectively, so as to avoid distributing too many tasks to the created new nodes, resulting in the problem of excessive load on the new nodes.

[0057] S513: Distribute the tasks on the to-be-configured list running on the recovered abnormally running temporary node to each of the newly added nodes.

[0058] Here, when the tasks running on the recovered abnormally running temporary nodes are distributed to a preset number of new nodes, the new nodes corresponding to each of the recovered abnormally running temporary nodes are determined, and each of the new nodes is marked according to the task ID of the corresponding abnormally running temporary node, and then the tasks running on the recovered abnormally running temporary nodes are distributed to each of the new nodes according to the marks of each new node.

[0059] For example, the task IDs of the two abnormally running temporary nodes that are recycled are 001 and 002. According to the task IDs of the two temporary nodes, the newly added node 1 and the newly added node 2 are marked respectively, that is, the newly added node 1 is marked as 001, and the newly added node 2 is marked as 002. The tasks running on the temporary node with task ID 001 are distributed to the newly added node 1 marked as 001, and the tasks running on the temporary node with task ID 002 are distributed to the newly added node 2 marked as 002, so as to complete the task allocation on the preset number of new nodes.

[0060] S514: calling the master node to determine whether there is an abnormally running temporary node according to the node status information corresponding to each of the temporary nodes that have not been recovered.

[0061] Here, the node status information corresponding to each temporary node is sent to zookepper, and the task id, task name and load of each temporary node are obtained, and the temporary node with a load exceeding the preset threshold is regarded as an abnormally running temporary node. In addition, the abnormally running temporary node can also be specified in zookepper. For example, the task name executed by the temporary node with task id 01 is "query task", and the machine corresponding to this temporary node is old and needs to be replaced. Therefore, it can be designated as an abnormally running temporary node in zookepper.

[0062] S515: If yes, repeat the step of creating a preset number of new nodes in the k8s to distribute the tasks on the to-be-configured list running on the recovered abnormally running temporary nodes to each of the new nodes until it is determined that there are no abnormally running temporary nodes.

[0063] After completing step S514, the calling main node can determine whether there are any abnormally running temporary nodes based on the node status information corresponding to each temporary node that has not been recovered. If there are no abnormally running temporary nodes, the creation of a preset number of new nodes in k8s is terminated; if there are abnormally running temporary nodes, steps S511 to S514 are repeated until it is determined that there are no abnormally running temporary nodes, so as to achieve complete replacement of the abnormally running temporary nodes.

[0064] S52: Reclaim the tasks on the temporary nodes that are running normally and are not on the to-be-configured list, and allow the tasks on the to-be-configured list that are running on the temporary nodes that are running normally to continue running on their respective corresponding temporary nodes that are running normally.

[0065] Since the tasks running on each normally operating temporary node are tasks on the to-be-configured list, in order to reduce the steps of distributing the tasks on the to-be-configured list, the tasks on the to-be-configured list running on the normally operating temporary nodes can be directly distributed by allowing the tasks on the to-be-configured list running on the normally operating temporary nodes to continue running on their respective corresponding normally operating temporary nodes.

[0066] S53: Determine whether there are any undistributed tasks on the to-be-configured list. If so, determine the to-be-distributed nodes corresponding to the undistributed tasks according to the load of the normally operating temporary nodes and the newly added nodes, and distribute the undistributed tasks to the to-be-distributed nodes.

[0067] It should be noted that if the list to be configured includes new tasks that are not running on each temporary node, steps S51 and S52 do not distribute the new task. Therefore, the new task is an undistributed task. In order to ensure the load balance of each normally operating temporary node and the newly added node, here, the nodes to be distributed corresponding to the undistributed tasks are determined according to the load of the normally operating temporary nodes and the newly added nodes. For example, undistributed task 1 and undistributed task 2 need to be distributed. The load of temporary node 1 and temporary node 2 is lower than that of other nodes. Therefore, undistributed task 1 and undistributed task 2 are distributed to temporary node 1 and temporary node 2, respectively.

[0068] To quickly replace abnormal nodes with new nodes, see Figure 5, the step of creating a new node in k8s and distributing the tasks in the to-be-configured list to the normally running temporary node and the new node includes:

[0069] S54: Distribute the tasks on the to-be-configured list running on the abnormally running temporary node to the normally running temporary node.

[0070] In order to avoid allocating tasks running on an abnormally running temporary node to a normally running temporary node with a relatively high load, causing the temporary node to run abnormally, here, distributing the tasks on the to-be-configured list running on the abnormally running temporary node to the normally running temporary node can include: sorting the normally running temporary nodes from small to large according to the load corresponding to each of the normally running temporary nodes, and taking the first N normally running temporary nodes in the sorting as target nodes; wherein N is the number of the abnormally running temporary nodes; and distributing the tasks on the to-be-configured list running on the abnormally running temporary node to the target node.

[0071] For example, after sorting the load corresponding to temporary node 01, temporary node 02, temporary node 03 and temporary node 04 from small to large, the order of arrangement is temporary node 01, temporary node 03, temporary node 04 and temporary node 02. If the number of abnormally running temporary nodes is 2, temporary node 01 and temporary node 03 are used as target nodes, and the tasks executed on the two abnormally running temporary nodes are distributed to temporary node 01 and temporary node 03 to ensure that the load of each normally running temporary node is balanced.

[0072] S55: Create in the k8s the same number of new nodes as the temporary nodes that are running abnormally.

[0073] S56: Reclaim the abnormally running temporary node to the zookeeper cluster.

[0074] S57: Distribute all the tasks in the to-be-configured list equally to the normally operating temporary nodes and the newly added nodes.

[0075] By distributing the tasks in the to-be-configured list to the normally running temporary nodes and newly added nodes, not only can the newly added nodes share the execution of the abnormally running temporary nodes, and make the tasks in the to-be-configured list run reasonably on the temporary nodes and the newly added nodes, but also can avoid the problem of increasing the load of the temporary nodes when tasks need to be distributed, so that the load effect of each temporary node and the newly added node is balanced.

[0076] In order to make the load of each normally operating temporary node and the newly added node consistent, the distribution of the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes may include: periodically configuring the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes until all the tasks in the to-be-configured list are redistributed to each normally operating temporary node and the newly added nodes. By periodically configuring the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes, the load of each normally operating temporary node and the newly added nodes can be made consistent while redistributing the tasks in the to-be-configured list.

[0077] For example, the normally operating temporary nodes include node 1, node 2, node 3 and node 4, the newly added nodes include node 5 and node 6, and the tasks in the list to be configured include tasks 1 to 12. First, tasks 1 to 6 are respectively assigned to nodes 1 to 6, so that one task is configured in each node, and then tasks 7 to 12 are respectively assigned to nodes 1 to 6, so that two tasks are configured in each node, so that the load of each node is consistent.

[0078] It can be seen that in the above scheme, multiple temporary nodes are created in the zookeeper cluster, and a master node is determined among the multiple temporary nodes; the master node is called to collect the task information and node status information corresponding to each temporary node, and determine the currently running task list and the normally running temporary node; the master node is called to compare the currently running task list and the latest configured task list with the original task list to determine whether the original task list has changed; if it has changed, the original task list is modified according to the latest configured task list and the currently running task list to obtain the list to be configured; a new node is created in k8s, and the tasks in the list to be configured are distributed to the normally running temporary nodes and the new nodes. The present application obtains a to-be-configured list containing only the tasks to be distributed by modifying the original task list according to the latest configured task list and the currently running task list, and then distributes the tasks in the to-be-configured list to the normally operating temporary nodes and newly added nodes. This not only enables the newly added nodes to share the execution of part of the tasks and enables the tasks in the to-be-configured list to run reasonably on the temporary nodes and newly added nodes, but also avoids the problem of increasing the load on the temporary nodes when tasks need to be distributed, and balances the load effects of the temporary nodes and the newly added nodes, so as to solve the problem of abnormal downtime due to excessive load on individual nodes in the existing task distribution process.

[0079] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0080] In one embodiment, a task distribution device is provided, which corresponds one-to-one to the task distribution method in the above embodiment. Figure 6 As shown, the task distribution device includes a node creation module 101, a collection module 102, a judgment module 103, a to-be-configured list acquisition module 104 and a task distribution module 105. The detailed description of each functional module is as follows:

[0081] The node creation module 101 is used to create multiple temporary nodes in the zookeeper cluster and determine a master node among the multiple temporary nodes, wherein the master node is used to read the original task list and the latest configured task list from the database; the read original task list includes the tasks that have been configured to each of the temporary nodes, and the latest configured task list includes the tasks to be newly configured to each of the temporary nodes;

[0082] The collection module 102 is used to call the master node to collect the task information and node status information corresponding to each of the temporary nodes, and determine the currently running task list, the normally running temporary nodes and the abnormally running temporary nodes; the currently running task list includes all the tasks currently running on all the temporary nodes;

[0083] The judgment module 103 is used to call the master node to compare the currently running task list and the latest configured task list with the original task list respectively to determine whether the original task list has changed;

[0084] The module 104 for acquiring the list to be configured is used to modify the original task list according to the latest configured task list and the currently running task list to obtain the list to be configured if changes occur; the list to be configured includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list;

[0085] The task distribution module 105 is used to create a new node in k8s and distribute the tasks in the to-be-configured list to the normally running temporary nodes and the new nodes.

[0086] In one embodiment, the task distribution module 105 includes:

[0087] A first distribution unit is used to distribute the tasks on the to-be-configured list running on the abnormally running temporary node to each of the newly added nodes;

[0088] A second distribution unit is used to reclaim the tasks on the temporary nodes that are running normally and are not on the to-be-configured list, and to allow the tasks on the to-be-configured list that are running on the temporary nodes that are running normally to continue running on their respective corresponding temporary nodes that are running normally;

[0089] The third distribution unit is used to determine whether there are any undistributed tasks on the to-be-configured list. If so, determine the to-be-distributed nodes corresponding to the undistributed tasks based on the load of the normally operating temporary nodes and the newly added nodes, and distribute the undistributed tasks to the to-be-distributed nodes.

[0090] In one embodiment, the first distribution unit includes:

[0091] A new node creation subunit is used to create a preset number of new nodes in the k8s;

[0092] A recycling subunit, used for recycling the same number of the abnormally running temporary nodes as the preset number into the zookeeper cluster;

[0093] A distribution subunit is used to distribute the tasks on the to-be-configured list running on the recovered abnormally running temporary node to each of the newly added nodes;

[0094] A judgment subunit, used for calling the master node to judge whether there is any abnormally running temporary node according to the node status information corresponding to each of the temporary nodes that have not been recovered;

[0095] The repeating subunit is used to repeat the step of creating a preset number of new nodes in the k8s to distribute the tasks on the to-be-configured list running on the recovered abnormally running temporary nodes to each of the new nodes, if any, until it is determined that there are no abnormally running temporary nodes.

[0096] In one embodiment, the task distribution module 105 includes:

[0097] A first allocation unit is used to distribute the tasks on the to-be-configured list running on the abnormally running temporary node to the normally running temporary node;

[0098] A new node creation unit, used to create in the k8s the same number of new nodes as the temporary nodes that are running abnormally;

[0099] A reclaiming unit, used to reclaim the abnormally running temporary node to the zookeeper cluster;

[0100] The second allocating unit is used to allocate all the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes.

[0101] In one embodiment, the first allocation unit includes:

[0102] The target node determination subunit is used to sort the normally operating temporary nodes from small to large according to the load corresponding to each of the temporary nodes, and use the first N normally operating temporary nodes in the sorting as target nodes; wherein N is the number of the abnormally operating temporary nodes;

[0103] The allocation subunit is used to allocate all tasks on the abnormally running temporary node to the target node.

[0104] In one embodiment, the second allocation unit includes:

[0105] The task distribution subunit is used to periodically configure the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes until all the tasks in the to-be-configured list are redistributed to each of the normally operating temporary nodes and the newly added nodes.

[0106] In one embodiment, the determination module 103 includes:

[0107] A first comparison unit is used to compare the original task list with the latest configured task list to determine whether there is a new task on the latest configured task list and whether there is a first stop task that needs to be stopped on the original task list;

[0108] A second comparison unit is used to compare the currently running task list with the original task list to determine whether there is a stopped second stopped task on the original task list;

[0109] a judging unit, configured to determine that the original task list has changed if a judging result shows that the new task needs to be configured and / or the first stopped task and / or the second stopped task in the original task list has been stopped;

[0110] The to-be-configured list acquisition module 104 includes:

[0111] A modification unit, configured to add the new task to the original task list, and / or remove the first stopped task and / or the second stopped task that are stopped on the original task list from the original task list, so as to complete the modification of the original task list;

[0112] An updating unit is used to use the modified original task list as the to-be-configured list.

[0113] The present invention provides a task distribution device, which first creates multiple temporary nodes in a zookeeper cluster and determines a master node among the multiple temporary nodes; calls the master node to collect task information and node status information corresponding to each temporary node, and determines a currently running task list and a normally running temporary node; calls the master node to compare the currently running task list and the latest configured task list with the original task list respectively to determine whether the original task list has changed; if changed, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured; creates a new node in k8s, and distributes the tasks in the list to be configured to the normally running temporary nodes and the new node. The present application obtains a to-be-configured list containing only the tasks to be distributed by modifying the original task list according to the latest configured task list and the currently running task list, and then distributes the tasks in the to-be-configured list to the normally operating temporary nodes and newly added nodes. This not only enables the newly added nodes to share the execution of part of the tasks and enables the tasks in the to-be-configured list to run reasonably on the temporary nodes and newly added nodes, but also avoids the problem of increasing the load on the temporary nodes when tasks need to be distributed, and balances the load effects of the temporary nodes and the newly added nodes, so as to solve the problem of abnormal downtime due to excessive load on individual nodes in the existing task distribution process.

[0114] For the specific definition of the task distribution device, please refer to the definition of the intelligent question and answer method above, which will not be repeated here. Each module in the above-mentioned task distribution device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0115] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a task distribution method.

[0116] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of the client side of a task distribution method.

[0117] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0118] Create multiple temporary nodes in the zookeeper cluster, and determine a master node among the multiple temporary nodes, where the master node is used to read the original task list and the latest configured task list from the database;

[0119] Calling the master node to collect task information and node status information corresponding to each of the temporary nodes, and determining a list of currently running tasks, normally running temporary nodes, and abnormally running temporary nodes;

[0120] Calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to determine whether the original task list has changed;

[0121] If there is a change, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured;

[0122] Create a new node in k8s, and distribute the tasks in the to-be-configured list to the normally running temporary node and the new node.

[0123] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0124] Create multiple temporary nodes in the zookeeper cluster, and determine a master node among the multiple temporary nodes, where the master node is used to read the original task list and the latest configured task list from the database;

[0125] Calling the master node to collect task information and node status information corresponding to each of the temporary nodes, and determining a list of currently running tasks, normally running temporary nodes, and abnormally running temporary nodes;

[0126] Calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to determine whether the original task list has changed;

[0127] If there is a change, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured;

[0128] Create a new node in k8s, and distribute the tasks in the to-be-configured list to the normally running temporary node and the new node.

[0129] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0131] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0132] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A task distribution method, characterized in that: include: Create multiple temporary nodes in the zookeeper cluster, and determine a master node among the multiple temporary nodes, where the master node is used to read the original task list and the latest configured task list; The read original task list includes tasks that have been configured to each of the temporary nodes, and the latest configured task list includes tasks to be newly configured to each of the temporary nodes; Calling the master node to collect task information and node status information corresponding to each of the temporary nodes, and determining a list of currently running tasks, normally running temporary nodes, and abnormally running temporary nodes; The currently running task list includes all tasks currently running on all the temporary nodes; Calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to determine whether the original task list has changed; If there is a change, the original task list is modified according to the latest configured task list and the currently running task list to obtain a list to be configured; The to-be-configured list includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list; Create a new node in k8s, and distribute the tasks in the to-be-configured list to the normally running temporary node and the new node.

2. The task distribution method according to claim 1, characterized in that: The step of creating a new node in k8s and distributing the tasks in the to-be-configured list to the normally running temporary node and the new node includes: Distribute the tasks on the to-be-configured list running on the abnormally running temporary node to each of the newly added nodes; Reclaim the tasks on the temporary node that is running normally and that are not on the to-be-configured list, and allow the tasks on the to-be-configured list that are running on the temporary node that is running normally to continue running on their respective corresponding temporary nodes that are running normally; Determine whether there are any undistributed tasks on the to-be-configured list. If so, determine the to-be-distributed nodes corresponding to the undistributed tasks based on the load of the normally running temporary nodes and the newly added nodes, and distribute the undistributed tasks to the to-be-distributed nodes.

3. The task distribution method according to claim 2, characterized in that: The distributing the tasks on the to-be-configured list running on the abnormally running temporary node to each of the newly added nodes includes: Create a preset number of new nodes in the k8s; Recycling the same number of the abnormally running temporary nodes as the preset number into the zookeeper cluster; Distributing the tasks on the to-be-configured list running on the recovered abnormally running temporary node to each of the newly added nodes; Calling the master node to determine whether there is any abnormally running temporary node according to the node status information corresponding to each of the temporary nodes that have not been recovered; If so, repeat the step of creating a preset number of new nodes in the k8s to distribute the tasks on the to-be-configured list running on the recovered abnormally running temporary node to each of the new nodes until it is determined that there is no abnormally running temporary node.

4. The task distribution method according to claim 1, characterized in that: The step of creating a new node in k8s and distributing the tasks in the to-be-configured list to the normally running temporary node and the new node includes: Distribute the tasks on the to-be-configured list running on the abnormally running temporary node to the normally running temporary node; Create in the k8s the same number of new nodes as the temporary nodes that are running abnormally; Recycle the temporary node that is running abnormally to the zookeeper cluster; All tasks in the to-be-configured list are evenly distributed to the normally operating temporary nodes and the newly added nodes.

5. The task distribution method according to claim 4, characterized in that: The distributing the tasks on the to-be-configured list running on the abnormally running temporary node to the normally running temporary node includes: According to the load corresponding to each of the temporary nodes operating normally, the nodes are sorted from small to large, and the first N temporary nodes operating normally in the sorting are used as target nodes; wherein N is the number of the temporary nodes operating abnormally; The tasks on the to-be-configured list running on the abnormally running temporary node are distributed to the target node.

6. The task distribution method according to claim 4, characterized in that: The step of evenly distributing all the tasks in the to-be-configured list to the normally operating temporary nodes and the newly added nodes includes: All the tasks in the to-be-configured list are periodically configured to the normally operating temporary nodes and the newly added nodes until the tasks in the to-be-configured list are reallocated to each of the normally operating temporary nodes and the newly added nodes.

7. The task distribution method according to claim 1, characterized in that: The calling of the master node to compare the currently running task list and the latest configured task list with the original task list respectively to determine whether the original task list has changed includes: Comparing the original task list with the latest configured task list, determining whether there is a new task on the latest configured task list, and whether there is a first stop task that needs to be stopped on the original task list; Comparing the currently running task list with the original task list, determining whether there is a stopped second stopped task on the original task list; If the judgment result is that the new task needs to be configured, and / or the first stop task and / or the second stop task in the original task list is stopped, then determining that the original task list has changed; The modifying the original task list according to the latest configured task list and the currently running task list includes: Adding the new task to the original task list, and / or removing the first stopped task and / or the second stopped task on the original task list from the original task list, to complete the modification of the original task list; The modified original task list is used as the to-be-configured list.

8. A task distribution device, characterized in that: include: A node creation module is used to create multiple temporary nodes in the zookeeper cluster and determine a master node among the multiple temporary nodes, wherein the master node is used to read the original task list and the latest configured task list; The read original task list includes tasks that have been configured to each of the temporary nodes, and the latest configured task list includes tasks to be newly configured to each of the temporary nodes; A collection module, used to call the master node to collect the task information and node status information corresponding to each of the temporary nodes, and determine the currently running task list, the normally running temporary nodes and the abnormally running temporary nodes; The currently running task list includes all tasks currently running on all the temporary nodes; A judgment module, used for calling the master node to compare the currently running task list and the latest configured task list with the original task list respectively, to judge whether the original task list has changed; A module for acquiring a list to be configured, for modifying the original task list according to the latest configured task list and the currently running task list to obtain a list to be configured if changes occur; the list to be configured includes tasks on the currently running task list and on the latest configured task list, and tasks newly added to the latest configured task list relative to the original task list; The task distribution module is used to create a new node in k8s and distribute the tasks in the to-be-configured list to the normally running temporary nodes and the new nodes.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the task distribution method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the task distribution method according to any one of claims 1 to 7 are implemented.

Images