A decentralized distributed task scheduling method and apparatus

Abstract

This invention relates to the field of task scheduling technology, providing a decentralized distributed task scheduling method and apparatus. When the execution pointer completes a full cycle encompassing all buckets, the completion status of each task on each node is determined based on the tasks' scheduled start and finish times, and the node then sends its own task execution health status to each node in the network. When a new task is dispatched to the network, each node, based on its own task execution health status and the task execution health status of other nodes in the network, initiates a competition request for the new task using a weighted value set by a preset strategy. This invention does not rely on other data consistency components, enabling automatic decentralized synchronization of task configuration and content between services.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, and in particular to a decentralized distributed task scheduling method and apparatus. Background Technology

[0002] In existing timed scheduling frameworks, either most do not support distributed task scheduling, or the specific implementation of distributed task scheduling still relies on third-party data consistency components for task data synchronization and chaotic task execution mechanisms. This results in problems such as low efficiency, complex deployment, chaotic execution, and no effective strategies for handling task execution failures.

[0003] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0004] The technical problem this invention aims to solve is that existing time-based scheduling frameworks either mostly do not support distributed task scheduling, or the specific implementation of distributed task scheduling still relies on third-party data consistency components for task data synchronization and chaotic task execution mechanisms, which suffer from problems such as low efficiency, complex deployment, chaotic execution, and lack of effective strategies for handling task execution failures.

[0005] The present invention adopts the following technical solution: In a first aspect, the present invention provides a decentralized distributed task scheduling method, establishing an NHMS time wheel system with four nested layers: a second-level time wheel, a minute-level time wheel, an hour-level time wheel, and a node-level time wheel, including: The NHMS time wheel is used by each node to place the task into the target bucket on the time wheel that is appropriate for it, based on the task's scheduled start time and specified completion time. Among them, the execution pointer saves the relevant descriptive information of all tasks within one cycle of each time wheel, including the jump between each bucket. When the execution pointer has completed a full cycle encompassing all buckets, the completion status of the corresponding tasks in each node is determined based on the tasks that have scheduled start and completion times, and the task execution health status of the node is then sent to each node in the network. When a new task is assigned to the network, each node initiates a competition request for the new task based on the task execution health status of other nodes in the network and its own task execution health status, using a weight value set by a preset strategy.

[0006] Preferably, one or more bytes are allocated to each time wheel to represent the task execution health status, specifically: In the second-level time wheel, minute-level time wheel, and hour-level time wheel, each task in its respective bucket is attached with the node's identity identifier, the time wheel number parameter value, and the execution result status parameter value indicating whether the task has been completed within the specified completion time. After each node completes one cycle of its execution pointer in each time wheel, an analysis data table is established using the node's identity, the round number parameter value of its respective time wheel, and the execution result status parameter value, along with the task description information in each bucket of each time wheel under the corresponding round number parameter value. The node generates its own task execution health status based on the analysis data table. The task execution health status includes one or more of the following: the total number of tasks that time out at a specified completion time in each time wheel; the number of concurrent tasks executed by each task that timed out at a specified completion time in its own time wheel; the number of concurrent tasks executed by each task that timed out at a specified completion time in other time wheels of the node; and the distribution of each task that timed out at a specified completion time in the buckets of its own time wheel.

[0007] Preferably, each node initiates a competition request for the new task based on the task execution health status of other nodes in the network and its own task execution health status, using a weight value set by a preset strategy. Specifically, this includes: Each node determines its own task execution health status relative to other nodes in the network in the previous minute-level or hour-level time wheel based on the task execution health status of other nodes, and assigns a priority weight value to itself according to the sorting. When each node learns that a new task is to be dispatched, it sends a competition request carrying its own priority to the dispatch server; where the priority is obtained by adjusting the initial priority and its own priority weight value. The dispatch server dispatches the new task to the node with the highest priority based on the priority of the competing requests received from each node.

[0008] Preferably, each node, based on the task execution health status of other nodes, determines its own task execution health status relative to other nodes in the network in the previous minute-level or hour-level time round, and assigns itself a priority weight value according to the ranking, further including: Each node confirms the slot of the time wheel where the task it recently obtained through a competitive request is located, and confirms that there are concurrent tasks in other time wheels that match the slot of the corresponding time wheel, and the dispatch time of the next new task is within the specified completion time of the most recently obtained task, and then adds a second priority weight value on top of the priority weight value.

[0009] Preferably, the task execution health status is reported on a timer cycle that completes one full second rotation of the execution pointer. Each node sends its own task execution health status to other nodes in the network via the network; or, The task execution health status is reported on a timer cycle that completes one full minute rotation of the execution pointer. Each node sends its own task execution health status to all other nodes in the network; or... The task execution health status is reported on a cycle of one complete cycle of the hourly time wheel. Each node sends its own task execution health status to other nodes in the network.

[0010] Preferably, when a network node failure occurs in the network, the method further includes: If a node with reporting authority does not receive the task execution health status of node i during the reporting period, it sends a verification request message to the network. The verification request message carries a query to see if the node that received the verification request message via broadcast also did not receive the task execution health status of node i. If a node with reporting authority receives verification response messages from at least two nodes, all of which indicate that the task execution health status of node i has not been received, then the node with reporting authority returns to the dispatch server the round number of the previous round of task execution health status of node i, as well as the analysis data table of the previous round stored by the node with reporting authority itself. Based on the health status of node i in the previous round of task execution and the analysis data table of nodes with reporting authority, the dispatch server infers the tasks that node i has competed for but has not yet completed, and performs supplementary dispatch in the corresponding network.

[0011] Preferably, when a new node enters the network, the method further includes: When a node with reporting authority receives a node whose task execution health status is at the initial value during the reporting period, it determines that the node is a newly joined node in the network. The node with reporting authority sends the identity identifier of the newly joined node to the dispatch server, so that the dispatch server can pre-set tasks for each time wheel of the newly joined node.

[0012] Preferably, after the dispatch server pre-assigns tasks to each time wheel of the newly joined network node, the method further includes: The newly joined node skips the competition process for task assignment in its current round.

[0013] Secondly, the present invention also provides a decentralized distributed task scheduling apparatus for implementing the decentralized distributed task scheduling method described in the first aspect, the apparatus comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the decentralized distributed task scheduling method described in the first aspect.

[0014] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the decentralized distributed task scheduling method described in the first aspect.

[0015] This invention discloses a decentralized distributed task scheduling method that does not rely on other data consistency components. It enables automatic decentralized synchronization of task configurations and content between services, employs an optimized time wheel algorithm to ensure the normal scheduling of timed tasks, efficiently implements a task timeout handling mechanism, and records the entire task execution status in detail and calculates task scheduling optimization strategies. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0017] Figure 1 This is a schematic diagram of the NHMS time wheel algorithm structure in a decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the node layer time wheel in a decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the minute-level time wheel in a decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of another decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of another decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 7This is a schematic diagram of another decentralized distributed task scheduling method provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a decentralized distributed task scheduling device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] In the description of this invention, the terms "inner", "outer", "longitudinal", "lateral", "upper", "lower", "top", "bottom", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and do not require that this invention must be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0020] In this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0021] In this application, unless otherwise expressly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral part; it can be a direct connection or an indirect connection through an intermediate medium. Furthermore, the term "coupled" can refer to an electrical connection that enables signal transmission.

[0022] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0023] Example 1: Embodiment 1 of the present invention provides a decentralized distributed task scheduling method, such as... Figure 1 As shown, an NHMS time wheel is created by nesting four layers in sequence: a second-level time wheel, a minute-level time wheel, an hour-level time wheel, and a node-level time wheel. Figure 2 As shown, the method includes: In step 201, when each node acquires a task, it places the task into the target bucket on the time wheel that is appropriate to that task, according to the task's scheduled start time and specified completion time.

[0024] Specifically, the execution pointer, during each revolution of the time wheel including the jumps between buckets, stores the relevant descriptive information of all tasks within one jump cycle. For example... Figure 1 As shown, the corresponding time wheels have an iterative jump relationship. The reason is also clear: the node-level time wheel is only used to attach to the nodes contained in the bucket pointed to by its execution pointer to run the hour-level time wheel, minute-level time wheel, and second-level time wheel that follow. The tasks involved in the corresponding time wheels will have mutual inclusion and support relationships. Therefore, it is manifested that the tasks of the upper-level time wheel will be partially transferred to the lower-level execution. However, it is not impossible that, for example, the tasks in the minute-level time wheel do not depend on the second-level time wheel, but can be completed within their own time wheel (but this is a minority).

[0025] The time wheel established in this invention, with the node-layer time wheel as the top layer, can effectively leverage the distributed task scheduling characteristics among various nodes. That is, a task is broken down into multiple subtasks and assigned to different nodes to await execution. Correspondingly, the relationships between these subtasks are carried out by the node-layer time wheel, i.e., the relationships within the node-layer time wheel are... Figure 3 The relationships between the recorded nodes in the front and back buckets (in) Figure 3 In the example shown, the node-level time wheel has 12 buckets, determined by the support relationships between the tasks it assigns. Unlike the hour, minute, and second-level time wheels, whose execution pointers jump based on the computer clock, the node-level time wheel's execution frame jumps based on the completion and timeout status of the tasks recorded in each bucket. This means the node-level time wheel must ensure strict sequential execution dependencies after tasks are laid out within their respective buckets. Specifically, the hour, minute, and second-level time wheels preload tasks from two or three buckets (the actual number can be adjusted based on the total number of tasks) that the node-level time wheel is about to execute. This ensures that the latter three time wheels are not overly bloated while maintaining the continuity and smoothness of task execution. Therefore, in the actual implementation, the execution pointer jump of the node-level time wheel is determined by the completion and timeout status of the tasks in its current bucket. For example, if the tasks recorded in the current bucket are... Figure 3 Once the node task is completed, the execution pointer jumps to the next bucket, and the resulting changes are as described above: the jump of the node layer time wheel will bring about the update of tasks in the following three time wheel buckets.

[0026] In step 202, after the execution pointer has completed a full cycle including all buckets, the completion status of the corresponding task in each node is determined based on the task containing the timed start execution time and timed completion execution time, thereby sending the task execution health status of its own node to each node in the network.

[0027] like Figure 4 As shown, the tasks recorded in buckets 0, 1, and 2 of the minute-level time wheel are illustrated. It can be seen that task 2 in bucket 0 did not complete the required number of jumps, while task 1 in bucket 0 specified a requirement of 4 jumps to complete. This could be because the task is the final stage of a complete task, or it could be that the task is executed on a single node without requiring support between different nodes, thus eliminating the need to limit the number of jumps required for completion.

[0028] In step 203, when a new task is assigned to the network, each node initiates a competition request for the new task based on the task execution health status of other nodes in the network and its own task execution health status, using a weight value set by a preset strategy.

[0029] This invention discloses a decentralized distributed task scheduling method that does not rely on other data consistency components. It enables automatic decentralized synchronization of task configurations and content between services, employs an optimized time wheel algorithm to ensure the normal scheduling of timed tasks, efficiently implements a task timeout handling mechanism, and records the entire task execution status in detail and calculates task scheduling optimization strategies.

[0030] To complete step 203, where each node initiates a competition request for the new task based on the task execution health status of other nodes in the network and its own task execution health status, using a weighted value set by a preset strategy, a feasible support scheme is also provided in conjunction with this embodiment: that is, one or more bytes are allocated for each time round as the task execution health status, specifically: In the second-level, minute-level, and hour-level time wheels, each task within its respective bucket is assigned an identifier of that node, a time wheel number parameter, and an execution result status parameter indicating whether the task was completed within the specified time. (See figure) After each node completes one cycle of its execution pointer in each time wheel, an analysis data table is established using the node's identity, the round number parameter value of its respective time wheel, and the execution result status parameter value, along with the task description information in each bucket of each time wheel under the corresponding round number parameter value.

[0031] The node generates its own task execution health status based on the analysis data table. The task execution health status includes one or more of the following: the total number of tasks that time out at a specified completion time in each time wheel; the number of concurrent tasks executed by each task that timed out at a specified completion time in its own time wheel; the number of concurrent tasks executed by each task that timed out at a specified completion time in other time wheels of the node; and the distribution of each task that timed out at a specified completion time in the buckets of its own time wheel.

[0032] In conjunction with embodiments of the present invention, each node, based on the task execution health status of other nodes in the network and its own task execution health status, initiates a competition request for the new task with a weight value set by a preset strategy, such as... Figure 5 As shown, it specifically includes: In step 301, each node determines its own task execution health status relative to other nodes in the network in the previous minute-level or hour-level time wheel based on the task execution health status of other nodes, and assigns a priority weight value to itself according to the sorting.

[0033] In step 302, when each node learns that a new task is to be dispatched, it sends a competition request carrying its own priority to the dispatch server.

[0034] The priority is obtained by adjusting the initial priority and its own priority weight value.

[0035] In step 303, the dispatch server dispatches the new task to the node with the highest priority based on the priority of the competition requests received from each node.

[0036] In this embodiment of the invention, each node determines its own task execution health status relative to other nodes in the network in the previous minute-level or hour-level time wheel based on the obtained task execution health status of other nodes, and assigns itself a priority weight value according to the ranking, further including: Each node confirms the slot of the time wheel where the task it recently obtained through a competitive request is located, and confirms that there are concurrent tasks in other time wheels that match the slot of the corresponding time wheel, and the dispatch time of the next new task is within the specified completion time of the most recently obtained task, and then adds a second priority weight value on top of the priority weight value.

[0037] In this embodiment of the invention, the task execution health status is reported on a timer cycle that completes one full second rotation of the execution pointer. Each node sends its own task execution health status to other nodes in the network via the network; or... The task execution health status is reported on a timer cycle that completes one full minute rotation of the execution pointer. Each node sends its own task execution health status to all other nodes in the network; or... The task execution health status is reported on a cycle of one complete cycle of the hourly time wheel. Each node sends its own task execution health status to other nodes in the network.

[0038] In this embodiment of the invention, when a network node failure occurs in the network, such as Figure 6 As shown, the method further includes: In step 204, if a node with reporting authority does not receive the task execution health status of node i during the reporting period, it sends a verification request message to the network. The verification request message carries an inquiry into whether the node that received the verification request message via broadcast also did not receive the task execution health status of node i.

[0039] In step 205, if the node with reporting authority receives verification response messages from at least two nodes, all of which indicate that the task execution health status of node i has not been received, then the node with reporting authority returns to the dispatch server the round number of the previous round of task execution health status of node i, as well as the analysis data table of the previous round stored by the node with reporting authority itself.

[0040] In step 206, the dispatch server infers the tasks that node i has competed for but has not yet completed based on the round of task execution health status of node i in the previous round and the analysis data table of nodes with reporting authority, and performs supplementary dispatch in the corresponding network.

[0041] In this embodiment of the invention, when a new node enters the network, such as Figure 7 As shown, the method further includes: In step 204', when a node with reporting authority receives a node whose task execution health status is at the initial value during the reporting period, it determines that the node is a newly joined node in the network. In step 205', the node with reporting authority sends the identity identifier of the newly joined node to the dispatch server, so that the dispatch server can pre-set tasks for each time wheel of the newly joined node.

[0042] In this process, after the dispatch server pre-assigns tasks to each time round of the newly joined network node, the newly joined network node skips the competition process for dispatching tasks in its current round.

[0043] Example 2: like Figure 8The diagram shown is an architectural schematic of a decentralized distributed task scheduling device according to an embodiment of the present invention. The decentralized distributed task scheduling device of this embodiment includes one or more processors 21 and a memory 22. Figure 8 Take a processor 21 as an example.

[0044] Processor 21 and memory 22 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0045] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the decentralized distributed task scheduling method in Embodiment 1. The processor 21 executes the decentralized distributed task scheduling method by running the non-volatile software programs and instructions stored in the memory 22.

[0046] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The program instructions / modules are stored in the memory 22. When executed by one or more processors 21, they perform the decentralized distributed task scheduling method described in Embodiment 1 above, for example, the method described above. Figure 2 , Figures 5-7 The steps shown.

[0048] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0049] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A decentralized distributed task scheduling method, characterized in that, Establish an NHMS time wheel system with four nested layers: second-level time wheel, minute-level time wheel, hour-level time wheel, and node-level time wheel. This includes: The NHMS time wheel is used by each node to place the task into the target bucket on the time wheel that is appropriate for it, based on the task's scheduled start time and specified completion time. Among them, the execution pointer saves the relevant descriptive information of all tasks within one cycle of each time wheel, including the jump between each bucket. When the execution pointer has completed a full cycle encompassing all buckets, the completion status of the corresponding tasks in each node is determined based on the tasks that have scheduled start and completion times, and the task execution health status of the node is then sent to each node in the network. When a new task is assigned to the network, each node determines its own task execution health status relative to other nodes in the previous minute-level or hour-level time wheel based on the task execution health status of other nodes, and assigns a priority weight value to itself according to the ranking. Each node confirms the slot of the time wheel containing the task it recently acquired through a contention request, and if there are concurrent tasks in other time wheels that match the slot of the corresponding time wheel, and the assignment time of the next new task is within the specified completion time of the recently acquired task, a second priority weight value is added to the priority weight value. When each node learns that a new task is to be assigned, it sends a contention request carrying its own priority to the assignment server. The priority is obtained by adjusting the initial priority and its own priority weight value. The assignment server assigns the new task to the node with the highest priority based on the priority in the contention requests received from each node. In this system, tasks within each bucket of the second-level, minute-level, and hour-level time wheels are associated with the node's identity, the round number parameter of their respective time wheels, and the execution result status parameter value indicating whether the task was completed within the specified completion time. After each node completes one cycle of its execution pointer in each time wheel, an analysis data table is established using the node's identity, the round number parameter value of its respective time wheels, and the execution result status parameter value, along with the task description information within each bucket of each time wheel under the corresponding round number parameter value. Based on the analysis data table, the node generates its own task execution health status.

2. The decentralized distributed task scheduling method according to claim 1, characterized in that: The task execution health status includes one or more of the following: the total number of tasks that have timed out at a specified completion time in each time wheel of each node; the number of concurrent tasks executed by each task that timed out at a specified completion time in its own time wheel; the number of concurrent tasks executed by each task that timed out at a specified completion time in other time wheels of the node; and the distribution of each task that timed out at a specified completion time in the buckets of its own time wheel.

3. The decentralized distributed task scheduling method according to claim 1, characterized in that, The task execution health status is reported on a time wheel cycle of one second after the execution pointer completes one revolution. Each node sends its own task execution health status to other nodes in the network. or, The task execution health status is reported on a timer cycle that completes one full minute rotation of the execution pointer. Each node sends its own task execution health status to all other nodes in the network; or... The task execution health status is reported on a cycle of one complete cycle of the hourly time wheel. Each node sends its own task execution health status to other nodes in the network.

4. The decentralized distributed task scheduling method according to claim 3, characterized in that, When a network node failure occurs in the network, the method further includes: If a node with reporting authority does not receive the task execution health status of node i during the reporting period, it sends a verification request message to the network. The verification request message carries a query to see if the node that received the verification request message via broadcast also did not receive the task execution health status of node i. If a node with reporting authority receives verification response messages from at least two nodes, all of which indicate that the task execution health status of node i has not been received, then the node with reporting authority returns to the dispatch server the round number of the previous round of task execution health status of node i, as well as the analysis data table of the previous round stored by the node with reporting authority itself. Based on the health status of node i in the previous round of task execution and the analysis data table of nodes with reporting authority, the dispatch server infers the tasks that node i has competed for but has not yet completed, and performs supplementary dispatch in the corresponding network.

5. The decentralized distributed task scheduling method according to claim 3, characterized in that, When a new node enters the network, the method further includes: When a node with reporting authority receives a node whose task execution health status is at the initial value during the reporting period, it determines that the node is a newly joined node in the network. The node with reporting authority sends the identity identifier of the newly joined node to the dispatch server, so that the dispatch server can pre-set tasks for each time wheel of the newly joined node.

6. The decentralized distributed task scheduling method according to claim 5, characterized in that, After the dispatch server pre-assigns tasks to each time wheel of the newly joined network node, the method further includes: The newly joined node skips the competition process for task assignment in its current round.

7. A decentralized distributed task scheduling device, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the decentralized distributed task scheduling method according to any one of claims 1-6.

8. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the decentralized distributed task scheduling method according to any one of claims 1-6.

Images