A task scheduling method, device, equipment and medium

Abstract

The embodiment of the present invention discloses a task scheduling method, device, equipment and medium, which relates to the field of automatic programming. Among them, the task scheduling method is applied to a task scheduling system with a tree-like distributed architecture, including: obtaining at least one target task in the task center through the root node in the root node layer; according to the attribute information of each target task, using at least one target routing node respectively determined in each routing node layer, forwarding each target task layer by layer to each target execution node in the execution node layer; through each of the target execution nodes, executing the assigned target task. The technical solution of the embodiment of the present invention performs task scheduling through a task scheduling system with a tree-like distributed architecture, improves the dimension of the allocation rules, and reduces the basic setting consumption when executing tasks.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and in particular to a task scheduling method, device, equipment and medium. Background Art

[0002] In order to achieve high-concurrency task processing, the commonly used task scheduling framework is the distributed task scheduling framework, such as xxl-job, elastic-job, etc. These frameworks are based on the original single task scheduling framework, and realize multi-cluster task scheduling through decentralization and sharding-based principles.

[0003] However, in the distributed task scheduling framework, all connected task execution nodes are treated equally, that is, the existing sharding algorithm is random when selecting task execution nodes, and it is difficult to consider the infrastructure consumption value, so it is impossible to control the infrastructure overhead cost. Summary of the invention

[0004] The embodiments of the present invention provide a task scheduling method, apparatus, device and medium, which relate to the field of automatic programming. Task scheduling is performed through a task scheduling system with a tree-like distributed architecture, the dimension of allocation rules is improved, and the basic setting consumption when executing tasks is reduced.

[0005] In a first aspect, an embodiment of the present invention provides a task scheduling method, which is applied to a task scheduling system of a tree-like distributed architecture, wherein the task scheduling system includes a root node layer, at least one routing node layer and an execution node layer, and the method includes:

[0006] Obtain at least one target task in the task center through the root node in the root node layer;

[0007] According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer;

[0008] The assigned target tasks are executed through each target execution node.

[0009] In a second aspect, an embodiment of the present invention further provides a task scheduling device, the device comprising:

[0010] A target task acquisition module, used to acquire at least one target task in the task center through a root node in the root node layer;

[0011] A target task forwarding module is used to forward each target task to each target execution node in the execution node layer layer by layer according to the attribute information of each target task and using at least one target routing node determined in each routing node layer;

[0012] The target task execution module is used to execute the assigned target task through each target execution node.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0014] one or more processors;

[0015] A memory for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the task scheduling method provided by any embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task scheduling method provided by any embodiment of the present invention.

[0018] The technical solution of the embodiment of the present invention obtains at least one target task in the task center through the root node in the root node layer, and then forwards each target task to each target execution node in the execution node layer layer by layer using at least one target routing node determined respectively in each routing node layer according to the attribute information of each target task. Finally, the assigned target task is executed through each target execution node, which solves the problem that the selection of task execution nodes for the node distribution architecture is random and the basic setting consumption cannot be controlled. Task scheduling is performed through the task scheduling system of the tree-like distribution architecture, which improves the dimension of the allocation rules and reduces the basic setting consumption when executing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1a is a flowchart of a task scheduling method in Embodiment 1 of the present invention;

[0020] Figure 1b It is a structural diagram of a task scheduling system with a tree-like distributed architecture in Embodiment 1 of the present invention;

[0021] Figure 2 is a flowchart of a task scheduling method in Embodiment 2 of the present invention;

[0022] Figure 3a is a flowchart of a task scheduling method in Embodiment 3 of the present invention;

[0023] Figure 3b This is a schematic diagram of a task acquisition principle in Embodiment 3 of the present invention;

[0024] Figure 4 It is a structural diagram of a task scheduling device in Embodiment 4 of the present invention;

[0025] Figure 5 It is a structural schematic diagram of a device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] Embodiment 1

[0028] Figure 1a This is a flowchart of a task scheduling method in Example 1 of the present invention. The technical solution of this embodiment is applicable to the situation where task scheduling is performed through a task scheduling system with a tree-like distributed architecture. The method can be executed by a task scheduling device, which can be implemented by software and / or hardware and can be integrated in various general-purpose computer devices.

[0029] To facilitate understanding, we first explain the task scheduling system of the tree-like distributed architecture. Figure 1b As shown, the task scheduling system of the tree-like distributed architecture includes a root node layer, at least one routing node layer and an execution node layer, and each node layer includes at least one node.

[0030] Among them, the root node in the root node layer controls the complete life cycle of the task from the initial state to the start of execution and finally to the end of execution. Specifically, the root node is used to obtain tasks from the task center according to the pre-set task acquisition rules, and distribute them to the routing node layer. When the task is completed, the task execution result will be fed back to the root node for transactional processing, for example, the task execution result will be written back to the database.

[0031] The task scheduling system of the tree-like distributed architecture includes at least one routing node layer. The routing nodes in the routing node layer are used to forward tasks according to pre-set routing rules. For example, tasks are assigned according to the characteristic values ​​of the routing nodes, and tasks with characteristic values ​​of A are assigned to routing nodes with characteristic values ​​of A, and tasks with characteristic values ​​of B are assigned to routing nodes with characteristic values ​​of B. The routing nodes neither change the status of the tasks nor execute the tasks, but are only responsible for sequentially transferring tasks to the lower layers and sequentially transferring the results of task execution to the upper layers. The routing nodes of the routing node layer adjacent to the execution nodes also perform task assignments according to the characteristic values ​​of the execution nodes, for example, tasks with characteristic values ​​of a are assigned to routing nodes with characteristic values ​​of a.

[0032] The execution node layer is located at the bottom layer of the task scheduling system of the tree-like distributed architecture. The execution node is the leaf node in the entire architecture. It is used to receive tasks forwarded by the routing node and execute the tasks, and finally feedback the task execution results to the upper layer.

[0033] The task scheduling method in this embodiment specifically includes the following steps:

[0034] Step 110: Obtain at least one target task in the task center through the root node in the root node layer.

[0035] The root node layer is the top layer of the entire tree-like distribution architecture, which contains at least one root node for acquiring tasks in the task center according to the set task acquisition rules and performing business processing on the received task execution results.

[0036] In this embodiment, first, at least one root node in the root node layer obtains at least one target task in the task center according to the task acquisition rule. Specifically, a distributed task scheduling framework can be used to achieve task acquisition under high concurrency.

[0037] Exemplarily, the root nodes may be sharded in advance according to the scheme of 0-9, and shard values ​​of 1, 2, and 3 are assigned to root node 1, shard values ​​of 4, 5, and 6 are assigned to root node 1, and shard values ​​of 7, 8, 9, and 0 are assigned to root node 3. When each root node acquires a target task at the task center, it may acquire it according to the above shard values, that is, root node 1 acquires tasks whose task numbers end with 1, 2, and 3 as target tasks, root node 2 acquires tasks whose task numbers end with 4, 5, and 6 as target tasks, and root node 3 acquires tasks whose task numbers end with 7, 8, 9, and 0 as target tasks.

[0038] Step 120: According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer.

[0039] Among them, the attribute information of the target task refers to at least one basic attribute information of the target task. For example, the target task is to count the performance consumption of a part of machines in a data center, and the basic attribute information may be the data center where the machine is located, the network area where the machine is located, the machine number, and the IP address (Internet Protocol Address) of the machine, etc.

[0040] In this embodiment, the task scheduling system determines at least one target routing node in each routing node layer in turn according to the attribute information of each target task, and then uses the target routing node in each routing node layer to pass it to the adjacent lower layer in turn along the tree structure, until the routing node in the bottommost routing node layer forwards the target task to the target execution node in the execution node layer, thereby completing the task allocation.

[0041] Exemplarily, the task scheduling system is divided into a root node layer, a routing node layer and an execution node layer. After each root node in the root node layer obtains the corresponding target task (statisticing the performance consumption of a part of machines in a data center), the target task is sent down to the target routing node that matches the data center of the task according to the data center corresponding to each target task; further, each target routing node further sends the target task to the execution node that matches the network area of ​​the task according to the network area corresponding to each target task.

[0042] Step 130: Execute the assigned target tasks through each target execution node.

[0043] In this embodiment, after each target task is sent to each target task execution node, the target task execution node executes the assigned target task. Exemplarily, the target task execution node obtains the performance consumption of the set machine contained in the target task and performs relevant processing on the specific data.

[0044] Optionally, since the task execution time is relatively long, all task forwarding and task execution result feedback operations are performed in an asynchronous calling manner.

[0045] The technical solution of the embodiment of the present invention obtains at least one target task in the task center through the root node in the root node layer, and then forwards each target task to each target execution node in the execution node layer layer by layer using at least one target routing node determined respectively in each routing node layer according to the attribute information of each target task. Finally, the assigned target task is executed through each target execution node, which solves the problem that the selection of task execution nodes for the node distribution architecture is random and the basic setting consumption cannot be controlled. Task scheduling is performed through the task scheduling system of the tree-like distribution architecture, which improves the dimension of the allocation rules and reduces the basic setting consumption when executing tasks.

[0046] Embodiment 2

[0047] Figure 2This is a flowchart of a task scheduling method in Embodiment 2 of the present invention. This embodiment further refines the above embodiment and provides specific steps for forwarding each target task layer by layer to each target execution node in the execution node layer using at least one target routing node determined in each routing node layer according to the attribute information of each target task. Figure 2 A task scheduling method provided in the second embodiment of the present invention is described, which includes the following steps:

[0048] Step 210: Obtain at least one target task in the task center through the root node in the root node layer.

[0049] Optionally, the task center stores task statuses, and each task corresponds to one of an initial state, an execution state, an execution completion state, and a timeout state;

[0050] Obtain at least one target task in the task center through the root node in the root node layer, including:

[0051] The task whose task status is the initial state or the timeout state is obtained in the task center through the root node in the root node layer as the target task.

[0052] Among them, when storing tasks, the task center will store the current status of each task, including the initial state, execution state, completion state and timeout state. Each state is mutually exclusive, and each task currently has only one state. The initial state means that the task has not been assigned or processed, the execution state means that the task is being assigned or being processed, the completion state means that the task has been processed by the execution node at the bottom of the tree-like distribution architecture, and the timeout state means that the task exceeds the set time during the processing process, and it needs to be re-acquired and assigned by the root node for execution.

[0053] In this optional embodiment, when the target task is obtained in the task center through the root node in the root node layer, the task status in the task center will be read. The root node only obtains tasks in the initial state or timeout state as the target task, avoiding multiple root nodes from obtaining the same task at the same time or successively, resulting in repeated allocation and processing of tasks.

[0054] Step 220: Determine at least one target routing node in the routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, and distribute each target task to each target routing node.

[0055] In this embodiment, after the root node obtains the target task, the root node first determines a condition for task allocation based on the acquired attribute information of each target task, for example, one of the attribute information. Further, at least one target routing node that meets the above allocation condition is determined in the routing node layer adjacent to it, and each target task is distributed to each target routing node.

[0056] Exemplarily, the root node determines, based on the target data center (Data Center 1 in City A) corresponding to the target task, a routing node that matches the target data center in the routing node layer adjacent to the root node as the target routing node, and assigns each target task to the target routing point that matches the corresponding target data center.

[0057] Optionally, determining at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node includes:

[0058] Determine the primary task feature value of each target task through the root node according to the acquired attribute information of each target task;

[0059] Determine, by a root node, a target class routing node matching the first-level task feature value according to the first-level task feature value and the node feature value corresponding to at least one type of routing node in the routing node layer adjacent to the root node, and determine at least one target routing node among the target class routing nodes;

[0060] Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes:

[0061] Registering node feature values ​​with the root node layer through each routing node in the routing node layer adjacent to the root node;

[0062] Through the root node layer, each routing node is classified according to its node feature value.

[0063] In this optional embodiment, before the root node obtains the target task, each routing node in the routing node layer adjacent to the root node will register a node characteristic value with the root node layer. The root node layer can classify each routing node according to the node characteristic value registered by each routing node. Routing nodes with the same node characteristic value can be regarded as a cluster, that is, a class of routing nodes, and each node in a class of routing nodes is the same.

[0064] In response to the above-mentioned routing node classification operation, this optional embodiment also provides a method for determining at least one target routing node in a routing node layer adjacent to the root node according to the attribute information of each target task obtained through the root node. Specifically, first, the first-level task characteristic value of each target task is determined according to the attribute information of each target task obtained through the root node, and a class of routing nodes matching the first-level task characteristic value is selected as the target class routing node in the routing node layer adjacent to it, and at least one target routing node is determined in the target class routing node. Since there is no difference between the routing nodes in a class of routing nodes, determining at least one target routing node in the target class routing node can be a random selection or a polling selection, etc., which is not specifically limited here.

[0065] Exemplarily, each routing node in the routing node layer adjacent to the root node will register a node characteristic value with the root node layer. In this routing node layer, the node characteristic value is a data center, that is, each routing node registers its data center with the root node layer. The root node layer can classify each routing node according to the data center registered by each routing node. After obtaining the target task, the root node determines the primary task characteristic value of each target task as the data center to which each target task belongs based on the attribute information of each target task obtained. Further, the root node selects a type of routing node that matches the primary task characteristic value in the routing node layer adjacent to it as the target type routing node, that is, selects a type of routing node whose data center is the same as the data center to which the task belongs as the target type routing node, and then randomly determines at least one target routing node among the target type routing nodes.

[0066] Optionally, the attribute information of the target task includes at least one of the following:

[0067] The data center to which the target task belongs, the network partition to which the target task belongs, the Internet Protocol IP address corresponding to the target task, and the task number.

[0068] Optionally, the first-level task feature value of each target task is determined through the root node according to the acquired attribute information of each target task, including:

[0069] The data center to which each target task belongs is obtained through the root node as the primary task feature value of each target task.

[0070] In this optional embodiment, a method for determining the primary task characteristic value of the target task is provided, specifically, the root node directly uses the acquired data center to which each target task belongs as the primary task characteristic value of each target task. Exemplarily, when the root node reads in the attribute information of the target task that the data center to which the target task belongs is data center 1 in city A, the root node can use the attribute information as the primary task characteristic value of the target task.

[0071] Optionally, after determining at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node and distributing each of the target tasks to each of the target routing nodes, the method further includes:

[0072] The task state of the target task is modified to an execution state through the root node.

[0073] In this optional embodiment, after the root node successfully distributes the target task to the target routing node, the root node will modify the task state of the target task to the execution state to prevent other root nodes from repeatedly acquiring and distributing the target task.

[0074] Step 230, taking the target routing node determined by the root node as the starting point, at least one target routing node is determined in the adjacent lower routing node layer through each target routing node, and the received target task is forwarded to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the execution node layer.

[0075] In this embodiment, after the target routing node in the routing node layer adjacent to the root node layer obtains the target task, a condition for task assignment is further determined based on the task attribute information of the target task, and at least one target routing node is determined in the adjacent lower-layer routing nodes based on the task assignment condition, and the target task is forwarded to the determined target routing node; the routing nodes of each layer repeat the above operation until the target routing node is determined in the routing node layer adjacent to the execution node layer.

[0076] It should be noted that the allocation conditions based on which each layer of routing nodes allocate tasks are different. For example, the allocation condition based on which the first layer of routing nodes allocates tasks is the data center to which the target task belongs, and the allocation condition based on which the second layer of routing nodes allocates tasks is the network area to which the target task belongs.

[0077] Optionally, determining at least one target routing node in an adjacent lower routing node layer by each target routing node includes:

[0078] Determining the acquired intermediate task characteristic values ​​of each target task through each target routing node;

[0079] Determine, by the target routing node, a target class routing node matching the intermediate task feature value according to the intermediate task feature value and the node feature value corresponding to at least one class of routing nodes in the lower routing node layer adjacent to the target routing node, and determine at least one target routing node among the target class routing nodes;

[0080] Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes:

[0081] Registering node feature values ​​in the adjacent upper routing node layer through each routing node in each routing node layer;

[0082] The routing node layer that receives the node characteristic value classifies each routing node according to the node characteristic value of each lower layer routing node.

[0083] In this optional embodiment, before the root node obtains the target task, in addition to the routing nodes in the routing node layer adjacent to the root node, the routing nodes in other layers will also register node feature values ​​with the adjacent upper routing node layer. The upper routing node layer can classify the routing nodes of each lower layer according to the node feature values ​​registered by each lower routing node. The lower routing nodes with the same node feature values ​​are considered as a cluster, that is, a type of routing nodes, and each node in the same type of routing nodes is the same.

[0084] In response to the above-mentioned routing node classification operation, this optional embodiment also provides a method for each target routing node to determine at least one target routing node in an adjacent lower routing node layer. Specifically, each target routing node first determines the intermediate task characteristic value of each target task based on the acquired attribute information of each target task, selects a class of routing nodes matching the intermediate task characteristic value in the adjacent lower routing node layer as the target class routing node, and determines at least one target routing node in the target class routing node. Since there is no difference between the routing nodes in a class of routing nodes, determining at least one target routing node in the target class routing node can be a random selection or a polling selection, etc., which is not specifically limited here.

[0085] Exemplarily, the routing nodes in each routing node layer will register node characteristic values ​​in the adjacent upper routing node layer. In this routing node layer, the node characteristic value is the network area, that is, each routing node registers the network area in which it is located with the adjacent upper routing node layer. The upper routing node layer can classify each routing node according to the network area registered by each lower routing node. After obtaining the target task, the upper routing node determines the intermediate task characteristic value of each target task as the network area to which each target task belongs based on the attribute information obtained. Further, the upper routing node selects a type of routing node that matches the intermediate task characteristic value in the adjacent lower routing node layer as the target routing node, that is, selects a type of routing node whose network area is the same as the network area to which the task belongs as the target routing node, and then randomly determines at least one target routing node in the target routing node. This method allows the task to be assigned to the routing node in the same network area as the one to which it belongs, reducing the consumption of basic settings such as network traffic.

[0086] Optionally, each target routing node determines at least one target routing node in an adjacent lower routing node layer, and forwards the received target task to the determined target routing node, including:

[0087] The target routing node detects whether the routing node layer where it is located is a routing node layer adjacent to the execution node layer. If not, at least one target routing node is determined in the adjacent lower routing node layer, and the received target task is forwarded to the determined target routing node.

[0088] In this optional embodiment, after each target routing node receives the task forwarded by the upper node, it detects through the target routing node whether the routing node layer in which it is located is a routing node layer adjacent to the execution node layer. If not, at least one target routing node is determined in the adjacent lower routing node layer, and the received target task is forwarded to the determined target routing node. If so, the target execution node is directly determined in the execution node layer, and the target task is forwarded to the target execution node.

[0089] Step 240: determine each target execution node in the execution node layer according to the attribute information of each received target task through each target routing node in the routing node layer adjacent to the execution node layer, and send each received target task to each target execution node.

[0090] In this embodiment, after the routing node in the routing node layer adjacent to the execution node layer obtains the target task, the target routing node first determines a condition for task allocation based on the acquired attribute information of each target task, for example, one of the attribute information. Further, at least one target execution node that meets the above allocation condition is determined in the execution node layer adjacent to it, and each target task is distributed to each target execution node.

[0091] Exemplarily, the target routing node determines, according to the task number corresponding to the target task, an execution node matching the task number in the execution node layer as the target execution node, and assigns each target task to a target execution point matching the corresponding task number.

[0092] Optionally, determining each target execution node in the execution node layer according to the received attribute information of each target task through each target routing node in the routing node layer adjacent to the execution node layer includes:

[0093] Determining the acquired secondary task feature value of each target task through the target routing node adjacent to the execution node layer;

[0094] Determine, by the target routing node, a target class execution node matching the secondary task feature value according to the secondary task feature value and the node feature value corresponding to at least one type of execution node in the execution node layer, and determine at least one target execution node among the target class execution nodes;

[0095] Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes:

[0096] Registering node feature values ​​to adjacent routing node layers through each execution node in the execution node layer;

[0097] Each execution node is classified according to the node characteristic value of each execution node through the routing node layer adjacent to the execution node layer.

[0098] In this optional embodiment, before the root node obtains the target task, the execution nodes in the execution node layer will register node feature values ​​with the adjacent routing node layer. The routing node layer can classify each execution node according to the node feature values ​​registered by each execution node. Execution nodes with the same node feature values ​​are considered as a cluster, that is, a class of execution nodes, and each node in a class of execution nodes is the same.

[0099] In response to the above-mentioned execution node classification operation, this optional embodiment also provides a method for determining each target execution node in the execution node layer according to the attribute information of each target task received through each target routing node in the routing node layer adjacent to the execution node layer. Specifically, first, each target routing node determines the secondary task feature value of each target task according to the acquired attribute information of each target task, selects a class of execution nodes matching the secondary task feature value as the target class execution node in the execution node layer adjacent to it, and determines at least one target execution node in the target class execution node. Since there is no difference between the execution nodes in a class of execution nodes, determining at least one target execution node in the target class execution node can be a random selection or a polling selection, etc., which is not specifically limited here.

[0100] Exemplarily, the execution nodes in each execution node layer will register node characteristic values ​​in the adjacent upper routing node layer. The node characteristic value in this execution node layer is the task number type, that is, each execution node registers its corresponding task number type with the adjacent upper routing node layer. The upper routing node layer can classify each execution node according to the task number type registered by each execution node. After obtaining the target task, the upper routing node determines the secondary task characteristic value of each target task as the task number corresponding to each target task based on the attribute information of each target task obtained. Further, the upper routing node selects a type of execution node that matches the secondary task characteristic value in the execution node layer adjacent to it as the target class execution node, that is, selects a type of execution node whose corresponding task number type matches the task number of the task as the target class execution node, and then randomly determines at least one target execution node in the target class execution node.

[0101] In this embodiment, a multi-level task scheduling system is used to classify nodes at each layer according to characteristic values, nodes with the same characteristic values ​​are clustered, and tasks are allocated according to the characteristic values ​​corresponding to the tasks and the characteristic values ​​corresponding to the nodes, so that tasks can always be executed by nodes with the least basic setting consumption.

[0102] Step 250: Execute the assigned target tasks through each target execution node.

[0103] The technical solution of this embodiment is to obtain at least one target task in the task center through the root node in the root node layer, and then determine at least one target routing node in the routing node layer adjacent to the root node according to the attribute information of each target task obtained through the root node, and distribute each target task to each target routing node, take the target routing node determined by the root node as the starting point, determine at least one target routing node in the adjacent lower routing node layer through each target routing node, and forward the received target task to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the execution node layer, further determine each target execution node in the execution node layer according to the attribute information of each received target task through each target routing node in the routing node layer adjacent to the execution node layer, and send each received target task to each target execution node, and finally execute the assigned target task through each target execution node, and perform task scheduling through the task scheduling system of the tree-like distributed architecture, improve the dimension of the allocation rule, and reduce the basic setting consumption when executing tasks.

[0104] Embodiment 3

[0105] Figure 3aThis is a flowchart of a task scheduling method in Embodiment 3 of the present invention. This embodiment further refines the above embodiment and provides specific steps before obtaining at least one target task in the task center through the root node in the root node layer, and specific steps after executing the target task through the target execution node. Figure 3a A task scheduling method provided in Embodiment 3 of the present invention is described, and includes the following steps:

[0106] Step 310: Allocate corresponding task slices to each root node in the root node layer according to a preset slicing rule.

[0107] In this embodiment, in order to achieve task acquisition under high concurrency, a corresponding task slice is allocated to each root node in the root node layer according to the preset sharding rules, so that each root node in the cluster can obtain tasks according to the corresponding task slice, avoiding the situation where each node repeatedly obtains tasks and improving the concurrency of task acquisition.

[0108] Step 320: At least one root node in the root node layer obtains at least one target task corresponding to the task area to which each root node belongs in the task center.

[0109] In this embodiment, on the basis of assigning a corresponding task slice to each root node, at least one root node in the root node layer obtains at least one target task corresponding to the task slice to which each root node belongs in the task center by reading the characteristics of each task in the task center.

[0110] Exemplarily, the root node is pre-sharded according to the 0-9 scheme, such as Figure 3b As shown in the figure, Zookeeper assigns shard values ​​of 1, 2, and 3 to root node 1, 4, 5, and 6 to root node 1, and 7, 8, 9, and 0 to root node 3. When each root node obtains the target task in the task center, it can obtain it according to the above shard values, that is, root node 1 obtains tasks ending with task numbers 1, 2, and 3 as target tasks, root node 2 obtains tasks ending with task numbers 4, 5, and 6 as target tasks, and root node 3 obtains tasks ending with task numbers 7, 8, 9, and 0 as target tasks, thereby improving the concurrency of obtaining tasks.

[0111] Step 330: According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer.

[0112] Optionally, the technical solution of this embodiment further includes: at least one root node in the root node layer and at least one routing node in at least one routing node layer initiate a node status acquisition request to an adjacent lower-layer node according to a set period;

[0113] When the node status fed back by the adjacent lower-layer node is unhealthy, the forwarding task to the lower-layer node is suspended.

[0114] In this optional embodiment, in addition to the execution nodes in the execution node layer, at least one root node in the root node layer and at least one routing node in at least one routing node layer will initiate a node status acquisition request to the adjacent lower-level nodes according to a set period, thereby obtaining the node status fed back by the lower-level nodes. When the node status is healthy, tasks can continue to be assigned to the node. Otherwise, the assignment of tasks to the node is suspended, thereby ensuring the high availability of the task scheduling system.

[0115] Step 340: Execute the assigned target tasks through each target execution node.

[0116] Optionally, after executing the target task through the target execution node, the method further includes:

[0117] The target execution node feeds back the execution result of the target task to the target routing node in the adjacent routing node layer;

[0118] Taking the target routing node determined by the target execution node as the starting point, at least one target routing node is determined in the adjacent upper routing node layer through each target routing node, and the received execution result is forwarded to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the root node layer;

[0119] A target root node is determined in the root node layer through each target routing node in the routing node layer adjacent to the root node layer, and the received execution result is forwarded to the target root node.

[0120] In this optional embodiment, after each target execution node completes the task execution and obtains the task execution result, it will feed back the task execution result to the target routing node in the adjacent routing node layer. Furthermore, taking the target routing node determined by the target execution node as the starting point, at least one target routing node is determined in the adjacent upper routing node layer through each target routing node, and the received execution result is forwarded to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the root node layer. Finally, through each target routing node in the routing node layer adjacent to the root node layer, the target root node is determined in the root node layer, and the received execution result is forwarded to the target root node, so that the root node can perform subsequent transactional operations according to the obtained task execution result, for example, storing the task execution result in a database.

[0121] Optionally, the task scheduling system of the tree-like distributed architecture in this embodiment can not only expand nodes at each level to achieve high-concurrency task distribution and processing, but also can expand vertically to enhance the task allocation dimension and achieve optimal task allocation.

[0122] The technical solution of this embodiment allocates corresponding task slices to each root node in the root node layer according to a preset sharding rule, and then at least one root node in the root node layer obtains at least one target task corresponding to the task slice to which each root node belongs in the task center, and then, according to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer, and finally the allocated target task is executed through each target execution node, and task scheduling is performed through a task scheduling system with a tree-like distributed architecture, thereby improving the dimension of the allocation rules and reducing the basic setting consumption when executing tasks.

[0123] Embodiment 4

[0124] Figure 4 This is a structural diagram of a task scheduling device provided in Embodiment 4 of the present invention. The task scheduling device includes: a target task acquisition module 410, a target task forwarding module 420 and a target task execution module 430.

[0125] A target task acquisition module 410 is used to acquire at least one target task in a task center through a root node in a root node layer;

[0126] The target task forwarding module 420 is used to forward each target task to each target execution node in the execution node layer layer by layer using at least one target routing node determined in each routing node layer according to the attribute information of each target task;

[0127] The target task execution module 430 is used to execute the assigned target task through each target execution node.

[0128] The technical solution of the embodiment of the present invention obtains at least one target task in the task center through the root node in the root node layer, and then forwards each target task to each target execution node in the execution node layer layer by layer using at least one target routing node determined respectively in each routing node layer according to the attribute information of each target task. Finally, the assigned target task is executed through each target execution node, which solves the problem that the selection of task execution nodes for the node distribution architecture is random and the basic setting consumption cannot be controlled. Task scheduling is performed through the task scheduling system of the tree-like distribution architecture, which improves the dimension of the allocation rules and reduces the basic setting consumption when executing tasks.

[0129] Optionally, the task scheduling device further includes:

[0130] A task slice allocation module, used to allocate corresponding task slices to each root node in the root node layer according to a preset slicing rule;

[0131] The target task acquisition module 410 is specifically used for:

[0132] At least one target task corresponding to the task slice to which each root node belongs is obtained in the task center by at least one root node in the root node layer.

[0133] Optionally, the target task forwarding module 420 includes:

[0134] A first target routing node determination unit, configured to determine at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, and distribute each of the target tasks to each of the target routing nodes;

[0135] A second target routing node determination unit is used to determine at least one target routing node in an adjacent lower routing node layer through each target routing node, starting from the target routing node determined by the root node, and forwarding the received target task to the determined target routing node until a target routing node is determined in a routing node layer adjacent to the execution node layer;

[0136] The target execution node determination unit is used to determine each target execution node in the execution node layer through each target routing node in the routing node layer adjacent to the execution node layer according to the attribute information of each received target task, and send each received target task to each target execution node.

[0137] Optionally, the first target routing node determination unit is specifically configured to:

[0138] Determine the primary task feature value of each target task through the root node according to the acquired attribute information of each target task;

[0139] Determine, by a root node, a target class routing node matching the first-level task feature value according to the first-level task feature value and the node feature value corresponding to at least one type of routing node in the routing node layer adjacent to the root node, and determine at least one target routing node among the target class routing nodes;

[0140] The task scheduling device further includes:

[0141] A first node registration module, configured to register a node characteristic value with the root node layer through each routing node in the routing node layer adjacent to the root node;

[0142] The first node classification module is used to classify each routing node according to the node characteristic value of each routing node through the root node layer.

[0143] Optionally, the second target routing node determination unit is specifically configured to:

[0144] Determining the acquired intermediate task characteristic values ​​of each target task through each target routing node;

[0145] Determine, by the target routing node, a target class routing node matching the intermediate task feature value according to the intermediate task feature value and the node feature value corresponding to at least one class of routing nodes in the lower routing node layer adjacent to the target routing node, and determine at least one target routing node among the target class routing nodes;

[0146] The task scheduling device further includes:

[0147] The second node registration module is used to register node characteristic values ​​in the adjacent upper routing node layer through each routing node in each routing node layer;

[0148] The second node classification module is used to classify each routing node according to the node characteristic value of each lower layer routing node through the routing node layer that receives the node characteristic value.

[0149] Optionally, the target execution node determination unit is specifically used to:

[0150] Determining the acquired secondary task feature value of each target task through the target routing node adjacent to the execution node layer;

[0151] Determine, by the target routing node, a target class execution node matching the secondary task feature value according to the secondary task feature value and the node feature value corresponding to at least one type of execution node in the execution node layer, and determine at least one target execution node among the target class execution nodes;

[0152] The task scheduling device further includes:

[0153] The third node registration module is used to register the node characteristic value with the adjacent routing node layer through each execution node in the execution node layer;

[0154] The third node classification module is used to classify each execution node according to the node characteristic value of each execution node through the routing node layer adjacent to the execution node layer.

[0155] Optionally, the second target routing node determination unit is further specifically configured to:

[0156] The target routing node detects whether the routing node layer where it is located is a routing node layer adjacent to the execution node layer. If not, at least one target routing node is determined in the adjacent lower routing node layer, and the received target task is forwarded to the determined target routing node.

[0157] Optionally, the task scheduling device further includes:

[0158] A status acquisition request initiating module, configured to initiate a node status acquisition request to an adjacent lower-layer node according to a set period by at least one root node in the root node layer and at least one routing node in at least one routing node layer;

[0159] The state feedback module is used to suspend forwarding tasks to the lower-layer nodes when the node state fed back by the adjacent lower-layer nodes is an unhealthy state.

[0160] Optionally, the task scheduling device further includes:

[0161] A first result feedback module, configured to feed back the execution result of the target task from the target execution node to a target routing node in an adjacent routing node layer;

[0162] The second task feedback module is used to determine at least one target routing node in an adjacent upper routing node layer through each target routing node, starting from the target routing node determined by the target execution node, and forwarding the received execution result to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the root node layer;

[0163] The third task feedback module is used to determine a target root node in the root node layer through each target routing node in the routing node layer adjacent to the root node layer, and forward the received execution result to the target root node.

[0164] Optionally, the task center stores task statuses, and each task corresponds to one of an initial state, an execution state, an execution completion state, and a timeout state;

[0165] The target task acquisition module 410 is specifically used for:

[0166] Obtain the task whose task status is the initial state or the timeout state in the task center through the root node in the root node layer as the target task;

[0167] The target task forwarding module 420 further includes:

[0168] The state modification unit is used to modify the task state of the target task to an execution state through a root node.

[0169] Optionally, the attribute information of the target task includes at least one of the following:

[0170] The data center to which the target task belongs, the network partition to which the target task belongs, the Internet Protocol IP address corresponding to the target task, and the task number.

[0171] Optionally, the first target routing node determining unit is further specifically configured to:

[0172] The data center to which each target task belongs is obtained through the root node as the primary task feature value of each target task.

[0173] The task scheduling device provided in the embodiment of the present invention can execute the task scheduling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0174] Embodiment 5

[0175] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention is shown in FIG. Figure 5 As shown, the electronic device includes a processor 50 and a memory 51; the number of processors 50 in the device can be one or more. Figure 5 A processor 50 is taken as an example; the processor 50 and the memory 51 in the device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0176] The memory 51, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to a task scheduling method in an embodiment of the present invention (for example, the target task acquisition module 410, the target task forwarding module 420 and the target task execution module 430 in the task scheduling device). The processor 50 executes various functional applications and task scheduling of the device by running the software programs, instructions and modules stored in the memory 51, that is, implements the above-mentioned task scheduling method.

[0177] The method is applied to a task scheduling system of a tree-like distributed architecture, the task scheduling system comprising a root node layer, at least one routing node layer and an execution node layer, including:

[0178] Obtain at least one target task in the task center through the root node in the root node layer;

[0179] According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer;

[0180] The assigned target tasks are executed through each target execution node.

[0181] The memory 51 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include a memory remotely arranged relative to the processor 50, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] Embodiment 6

[0183] Embodiment 6 of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to execute a task scheduling method when executed by a computer processor, the method comprising:

[0184] The method is applied to a task scheduling system of a tree-like distributed architecture, the task scheduling system comprising a root node layer, at least one routing node layer and an execution node layer, including:

[0185] Obtain at least one target task in the task center through the root node in the root node layer;

[0186] According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer;

[0187] The assigned target tasks are executed through each target execution node.

[0188] Of course, the computer executable instructions of the storage medium containing computer executable instructions provided by the embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in the task scheduling method provided by any embodiment of the present invention.

[0189] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, an application server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0190] It is worth noting that in the above-mentioned embodiment of a task scheduling device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0191] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A task scheduling method is applied to a task scheduling system of a tree-like distributed architecture, wherein the task scheduling system comprises a root node layer, at least one routing node layer and an execution node layer, and is characterized in that: include: Obtain at least one target task in the task center through the root node in the root node layer; According to the attribute information of each target task, at least one target routing node determined in each routing node layer is used to forward each target task to each target execution node in the execution node layer layer by layer; Executing the assigned target task through each of the target execution nodes, wherein the attribute information of the target task includes at least one of a data center to which the target task belongs, a network partition to which the target task belongs, an Internet Protocol IP address corresponding to the target task, and a task number; Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes: According to the preset sharding rules, corresponding task slices are allocated to each root node in the root node layer; Obtain at least one target task in the task center through the root node in the root node layer, including: At least one root node in the root node layer obtains at least one target task corresponding to the task slice to which each root node belongs in the task center; The method of forwarding each target task to each target execution node in the execution node layer layer by layer using at least one target routing node determined in each routing node layer according to the attribute information of each target task includes: Determine at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, and distribute each of the target tasks to each of the target routing nodes; Taking the target routing node determined by the root node as the starting point, at least one target routing node is determined in the adjacent lower routing node layer through each target routing node, and the received target task is forwarded to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the execution node layer; Through each target routing node in the routing node layer adjacent to the execution node layer, each target execution node is determined in the execution node layer according to the attribute information of each received target task, and each received target task is sent to each target execution node.

2. The method according to claim 1, characterized in that: Determining at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, including: Determine the primary task feature value of each target task through the root node according to the acquired attribute information of each target task; Determine, by a root node, a target class routing node matching the first-level task feature value according to the first-level task feature value and the node feature value corresponding to at least one type of routing node in the routing node layer adjacent to the root node, and determine at least one target routing node among the target class routing nodes; Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes: Registering node feature values ​​with the root node layer through each routing node in the routing node layer adjacent to the root node; Through the root node layer, each routing node is classified according to its node feature value.

3. The method according to claim 1, characterized in that Determining at least one target routing node in an adjacent lower routing node layer by each target routing node, including: Determining the acquired intermediate task characteristic values ​​of each target task through each target routing node; Determine, by the target routing node, a target class routing node matching the intermediate task feature value according to the intermediate task feature value and the node feature value corresponding to at least one class of routing nodes in the lower routing node layer adjacent to the target routing node, and determine at least one target routing node among the target class routing nodes; Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes: Registering node feature values ​​in the adjacent upper routing node layer through each routing node in each routing node layer; The routing node layer that receives the node characteristic value classifies each routing node according to the node characteristic value of each lower layer routing node.

4. The method according to claim 1, characterized in that: Determining each target execution node in the execution node layer according to the received attribute information of each target task through each target routing node in the routing node layer adjacent to the execution node layer, including: Determining the acquired secondary task feature value of each target task through the target routing node adjacent to the execution node layer; Determine, by the target routing node, a target class execution node matching the secondary task feature value according to the secondary task feature value and the node feature value corresponding to at least one type of execution node in the execution node layer, and determine at least one target execution node among the target class execution nodes; Before acquiring at least one target task in the task center through the root node in the root node layer, the method further includes: Registering node feature values ​​to adjacent routing node layers through each execution node in the execution node layer; Each execution node is classified according to the node characteristic value of each execution node through the routing node layer adjacent to the execution node layer.

5. The method according to claim 1, characterized in that Determining at least one target routing node in an adjacent lower routing node layer by each target routing node, and forwarding the received target task to the determined target routing node, including: The target routing node detects whether the routing node layer where the target routing node is located is a routing node layer adjacent to the execution node layer. If not, at least one target routing node is determined in the adjacent lower routing node layer, and the received target task is forwarded to the determined target routing node.

6. The method according to claim 1, characterized in that Also includes: At least one root node in the root node layer and at least one routing node in at least one routing node layer initiate a node status acquisition request to an adjacent lower-layer node according to a set period; When the node status fed back by the adjacent lower-layer node is an unhealthy state, forwarding tasks to the lower-layer node is suspended.

7. The method according to claim 1, characterized in that After executing the target task through the target execution node, the method further includes: The target execution node feeds back the execution result of the target task to the target routing node in the adjacent routing node layer; Taking the target routing node determined by the target execution node as the starting point, at least one target routing node is determined in the adjacent upper routing node layer through each target routing node, and the received execution result is forwarded to the determined target routing node until the target routing node is determined in the routing node layer adjacent to the root node layer; A target root node is determined in the root node layer through each target routing node in the routing node layer adjacent to the root node layer, and the received execution result is forwarded to the target root node.

8. The method according to claim 1, characterized in that The task center stores task statuses, and each task corresponds to one of the initial status, execution status, completion status, and timeout status; Obtain at least one target task in the task center through the root node in the root node layer, including: Obtain the task whose task status is the initial state or the timeout state in the task center through the root node in the root node layer as the target task; The method further includes: determining at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, and distributing each of the target tasks to each of the target routing nodes; The task state of the target task is modified to an execution state through the root node.

9. The method according to claim 2, characterized in that The attribute information of the target task includes at least one of the following: The data center to which the target task belongs, the network partition to which the target task belongs, the Internet Protocol IP address corresponding to the target task, and the task number.

10. The method according to claim 9, characterized in that The first-level task feature value of each target task is determined through the root node according to the acquired attribute information of each target task, including: The data center to which each target task belongs is obtained through the root node as the primary task feature value of each target task.

11. A task scheduling device, characterized in that: include: A target task acquisition module, used to acquire at least one target task in the task center through a root node in the root node layer; A target task forwarding module, used to forward each target task layer by layer to each target execution node in the execution node layer using at least one target routing node determined in each routing node layer according to the attribute information of each target task, wherein the attribute information of the target task includes at least one of the data center to which the target task belongs, the network partition to which the target task belongs, the Internet Protocol IP address corresponding to the target task, and the task number; A target task execution module, used for executing the assigned target task through each target execution node; The task scheduling device further includes: A task slice allocation module, used for allocating corresponding task slices to each root node in the root node layer according to a preset slicing rule before acquiring at least one target task in the task center through the root node in the root node layer; The target task acquisition module is specifically used for: At least one root node in the root node layer obtains at least one target task corresponding to the task slice to which each root node belongs in the task center; Target task forwarding module, including: A first target routing node determination unit, configured to determine at least one target routing node in a routing node layer adjacent to the root node according to the acquired attribute information of each target task through the root node, and distribute each of the target tasks to each of the target routing nodes; A second target routing node determination unit is used to determine at least one target routing node in an adjacent lower routing node layer through each target routing node, starting from the target routing node determined by the root node, and forwarding the received target task to the determined target routing node until a target routing node is determined in a routing node layer adjacent to the execution node layer; The target execution node determination unit is used to determine each target execution node in the execution node layer through each target routing node in the routing node layer adjacent to the execution node layer according to the attribute information of each received target task, and send each received target task to each target execution node.

12. An electronic device, characterized in that: The device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the task scheduling method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the task scheduling method as described in any one of claims 1 to 10 is implemented.

Images