Decentralized Timed Task Scheduling Method and System for Cluster Environment

Abstract

The present invention discloses a method and system for decentralized timed task scheduling in a cluster environment, including: when the system starts, reading the list of timed tasks in the database into memory using SQL, creating and registering the timed tasks; monitoring the registered timed tasks, and triggering a task when the Cron expression of a certain timed task is reached; reading the task list in the database to obtain the execution times, the latest execution IP, and the latest random string of the task; determining whether the triggered timed task needs to be executed on the current server; sending a task query request to the server pointed to by the latest execution IP using the TCP protocol; after receiving the query request, checking whether there is a thread with the name of the task in the thread pool and returning the result; when the update is successful, creating a thread with the name of the task in the thread pool and executing the task in the thread; after the timed task is executed, incrementing the execution times of the timed task in the task list by 1.

Description

Technical Field

[0001] The present invention relates to the technical field of timed task scheduling, and in particular, to a method and system for decentralized timed task scheduling in a cluster environment. Background Art

[0002] In large-scale systems, in order to achieve high concurrency and high availability, a cluster deployment method is often adopted, that is, the same set of programs are deployed on multiple servers to form multiple system instances. When the program contains timed tasks, the timed tasks on multiple servers may start simultaneously, resulting in problems such as duplicate data or program exceptions. The commonly used solutions in the industry are to deploy a separate centralized scheduling system to uniformly schedule timed tasks, or to use distributed locks to prevent timed tasks from being executed simultaneously on multiple servers at the same time. However, there are still some drawbacks:

[0003] The method of using a centralized scheduling system: not only requires additional server resources, but also the business system needs to be modified accordingly to adapt to the scheduling system, increasing the system complexity and construction cost; when the centralized scheduling system fails, all timed tasks cannot be executed.

[0004] The method of using distributed locks: when a task needs to be executed for a long time, the distributed lock occupies a long time, which may cause the lock to time out and become invalid, and other servers may obtain the lock and cause the task to be executed simultaneously; when a server fails and stops during the task execution process, the distributed lock fails to be released in time, which may cause all servers to fail to obtain the lock and cause the task not to be executed; when the server clocks are different, one server executes the task first due to a faster clock, and another server may obtain the lock again and execute the task due to a slower clock, resulting in duplicate execution of tasks at the same point in time. Summary of the Invention

[0005] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract, and the title. However, such simplifications or omissions shall not be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: when the system starts, use SQL to read the list of timed tasks in the database into memory, and create and register the timed tasks;

[0008] Monitor the registered timed tasks, and when the Cron expression of a certain timed task is reached, trigger the timed task;

[0009] Read the task list in the database to obtain the execution count, the latest execution IP, and the latest random string of the scheduled task;

[0010] Determine whether the triggered scheduled task needs to be executed on this server. When it needs to be executed on this server, set the task execution count in memory to the task execution count in the database plus 1;

[0011] Send a task query request to the server pointed to by the latest execution IP using the TCP protocol;

[0012] The server pointed to by the latest execution IP receives the query request, checks whether there is a thread with the name of the scheduled task in the thread pool and returns the result. When the return result is no, attempt to update the latest execution IP of the scheduled task in the database task list to the IP of this server;

[0013] When the update is successful, create a thread with the name of the scheduled task in the thread pool and execute the scheduled task in the thread;

[0014] After the scheduled task is executed, increment the execution count of the scheduled task in the database task list by 1.

[0015] As a preferred solution of the scheduled task scheduling method for decentralization in a cluster environment according to the present invention, triggering the scheduled task includes:

[0016] After the task trigger class CronTrigger is registered, the scheduled task registrar ScheduledTaskRegistrar monitors the server clock. When the time of the clock conforms to the Cron expression, the task dispatcher class TaskDispatcher will be started.

[0017] As a preferred solution of the scheduled task scheduling method for decentralization in a cluster environment according to the present invention, when the scheduled task does not need to be executed on this server, set the task execution count in memory to the task execution count in the database, and the task execution is aborted;

[0018] When it needs to be executed on this server, set the task execution count in memory to the task execution count in the database plus 1.

[0019] As a preferred solution of the scheduled task scheduling method for decentralization in a cluster environment according to the present invention, the return result includes two cases, where:

[0020] When the return result is yes, the task execution is aborted;

[0021] When the return result is no, attempt to update the latest execution IP of this task in the database task list to the IP of this server;

[0022] When the update fails, it means that the random string has been updated by other servers, and the task execution is aborted.

[0023] As a preferred solution of the timing task scheduling method for decentralization in a cluster environment according to the present invention, attempting to update the latest execution IP of this task in the database task list to the IP of this server includes:

[0024] Generate a new random string, use the latest random string and the timing task name as the update conditions, and update the newly generated random string and the IP of this server to the table.

[0025] As a preferred solution of the timing task scheduling method for decentralization in a cluster environment according to the present invention, the fields included in the timing task list include the timing task name, Cron expression, execution times, latest execution IP, and latest random string.

[0026] As a preferred solution of the timing task scheduling system for decentralization in a cluster environment according to the present invention, it includes a task registration module, a task scheduling module, a task inspection module, and a database, wherein:

[0027] The task registration module is used to read the timing task list in the database when the system starts, and create and register timing tasks according to the timing task list;

[0028] The task scheduling module is used to check whether the timing task is executed simultaneously or repeatedly when the timing task is triggered, decide whether to execute the task according to the check result, and update the timing task list in the database after executing the timing task;

[0029] The task inspection module is used to respond to the task query request sent by the task scheduling module and query whether there is a thread with the name of the queried task in the thread pool of this server;

[0030] The database is used to save the timing task list.

[0031] As a preferred solution of the timing task scheduling system for decentralization in a cluster environment according to the present invention, it further includes:

[0032] One or more processors;

[0033] A memory that stores operable instructions, and the instructions, when executed by the one or more processors, cause the one or more processors to perform operations, and the operations include the processes of the timing task scheduling method for decentralization in a cluster environment as described above.

[0034] Advantages of the present invention:

[0035] 1. Achieve flexible task management: Using Cron expressions to define the execution time of tasks provides a high degree of flexibility and precision, enabling the system to be customized and optimized according to different business requirements, including but not limited to task creation, registration, triggering, execution, and update;

[0036] 2. Improve task execution efficiency: By generating a random string and using it as an update condition, it effectively prevents multiple nodes from executing the same task simultaneously, ensuring the uniqueness and accuracy of task execution. Dynamically updating the execution IP of the task can ensure that the task is executed on the most suitable server, improving the efficiency of task execution;

[0037] 3. Enhance the fault tolerance of the system: Through the abort mechanism and error handling, the fault tolerance of the system is improved, enabling it to operate stably even in the case of partial component failures. This fault tolerance mechanism ensures the high availability of tasks and reduces task interruptions caused by node failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0039] Figure 1 is a schematic flowchart of the method for decentralized timed task scheduling in a cluster environment shown in the present invention;

[0040] Figure 2 is a schematic diagram of the module structure distribution of the decentralized timed task scheduling system in a cluster environment shown in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them.

[0042] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0043] In the following description, many specific details are set forth to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways different from those described herein. Persons skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0044] According to an embodiment of the present invention, in combination with Figure 1 the flowchart shown, a timing task scheduling method for decentralization in a cluster environment specifically includes the following steps:

[0045] S1. When the system starts, read the timing task list in the database into memory and create and register the timing tasks.

[0046] In an optional implementation, the fields included in the timing task list include the timing task name, Cron expression, execution times, latest execution IP, and latest random string.

[0047] As an example, the table name of the task list in the database is: t_task_list, the timing task name field is: task_id, the random string field is: random_str, the latest execution IP field is: last_ip, and the execution times field is: run_times. Then the SQL statement read is:

[0048] SELECT task_id,cron,run_times,last_ip,random_str FROM t_task_list 。

[0049] In an embodiment of the present invention, use the Cron expression in the timing task list to create a task trigger class CronTrigger and a task scheduling class TaskDispatcher, and register them in the scheduled task registrar ScheduledTaskRegistrar.

[0050] Exemplary code is as follows:

[0051] public void addTask(ScheduledTaskRegistrar taskRegistrar,String cron,TaskDispatcher taskOne) {

[0052] Trigger triggerOne = triggerContext ->{

[0053] Date nextExecTime = null;

[0054] try {

[0055] CronTrigger cronTrigger = new CronTrigger(cron);

[0056] nextExecTime = cronTrigger.nextExecutionTime(triggerContext);

[0057] } catch (Exception e) {

[0058] e.printStackTrace();

[0059] }

[0060] return nextExecTime;

[0061] };

[0062] taskRegistrar.addTriggerTask(taskOne, triggerOne);

[0063] }

[0064] S2. Monitor the registered scheduled tasks. When the Cron expression of a certain scheduled task is reached, trigger the scheduled task.

[0065] As an example, after the task trigger class CronTrigger is registered, the scheduled task registrar ScheduledTaskRegistrar will monitor the server clock. When the time of the clock meets the Cron expression, it will start the task dispatcher class TaskDispatcher.

[0066] S3. Read the task list in the database to obtain the execution times, the latest execution IP, and the latest random string of the task.

[0067] As an example, the SQL statement for reading the scheduled task with the name task1 is:

[0068] SELECT run_times,last_ip,random_str FROM t_task_list

[0069] WHERE task_id='task1'.

[0070] S4. Determine whether the triggered scheduled task needs to be executed on this server. Among them, it should be noted in this step that:

[0071] When the task does not need to be executed on this server, set the number of task executions in memory to the number of task executions in the database, and abort the task execution.

[0072] When the task needs to be executed on this server, set the number of task executions in memory to the number of task executions in the database plus 1.

[0073] Further, before the above steps are implemented, it is necessary to first determine whether the triggered timed task has been executed by other servers, including the following steps:

[0074] (1) Compare whether the number of task executions in the database is greater than the number of task executions in memory for this task;

[0075] (2) When the comparison result is yes, it means that the task has been executed on other servers and does not need to be executed on this server, preventing duplicate execution of the task;

[0076] (3) When the comparison result is no, it means that the task has not been executed on other servers and needs to be executed on this server.

[0077] S5. Send a task query request to the server pointed to by the latest execution IP obtained in step S3 using the TCP protocol.

[0078] S6. The server pointed to by the above IP receives the query request, checks whether there is a thread with the name of this task in the thread pool and returns the result.

[0079] Specifically, in the embodiment of the present invention, all threads are obtained through the enumerate method of the ThreadGroup class, and whether there is a thread with the name of this task is checked one by one.

[0080] Exemplary code is as follows:

[0081] public static boolean findTaskByThreadName(String threadName) {

[0082] ThreadGroup threadGroup = Thread.currentThread().getThreadGroup();

[0083] Thread[] threads = new Thread[threadGroup.activeCount()];

[0084] threadGroup.enumerate(threads);

[0085] for (int i = 0; i < threads.length; i++) {

[0086] if (threadName.equals(threads[i].getName())) {

[0087] return true;

[0088] }

[0089] }

[0090] return false;

[0091] }

[0092] When the return result is yes, the task execution is aborted;

[0093] When the return result is no, attempt to update the latest execution IP of this task in the database task list to the IP of this server.

[0094] In an optional implementation, the method of attempting to update the latest execution IP of this task in the database task list to the IP of this server includes: newly generating a random string, using the latest random string obtained in step S3 and the scheduled task name as the update conditions, and updating the newly generated random string and the IP of this server to the table.

[0095] Exemplarily, assume that a newly generated random string is:

[0096] BD116EB5AB4F1D393E8CE90F77677225, the latest random string obtained in step S3 is: 532B295F85B91CAF7FE3E0C32902C183, the scheduled task name is: task1, and the IP of this server is 192.168.1.10. Then the executed update SQL statement is:

[0097] UPDATE t_task_list

[0098] SET random_str='BD116EB5AB4F1D393E8CE90F77677225',

[0099] last_ip='192.168.1.10'

[0100] WHERE task_id='task1'

[0101] AND random_str='532B295F85B91CAF7FE3E0C32902C183'

[0102] When the update fails, it means that the random string has been updated by other servers, and the task execution is aborted.

[0103] Thus, it can be seen that the embodiment of the present invention controls that the task list can only be updated once at the same time through the random string, preventing multiple servers from updating simultaneously, and can effectively solve the problem of timed tasks being executed simultaneously.

[0104] S7. When the update is successful, create a thread named after the task name in the thread pool, and execute the timed task in this thread.

[0105] S8. After the timed task is executed, increment the execution count of this task in the database task list by 1.

[0106] Exemplarily, the executed update SQL statement is:

[0107] UPDATE t_task_list

[0108] SET run_times = run_times + 1

[0109] WHERE task_id='task1'

[0110] Thus, it can be seen that the task list in the database records the actual execution count of the timed task. By controlling that the execution count of the present invention can only increase sequentially, it can ensure that only one task can be executed at a time point, thus solving the problem of task repeated execution.

[0111] Preferably, some other aspects disclosed in the embodiment of the present invention further propose a timed task scheduling system for decentralization in a cluster environment, including: a task registration module, a task scheduling module, a task inspection module, and a database.

[0112] The task registration module is used to read the timed task list in the database when the system starts, and create and register timed tasks according to the timed task list.

[0113] As an example, the task registration module at least includes a scheduled task registrar ScheduledTaskRegistrar and one or more task trigger classes CronTrigger. For each task in the timed task list, a task trigger class CronTrigger is created correspondingly.

[0114] The task scheduling module is used to check whether a scheduled task is executed simultaneously or repeatedly when the scheduled task is triggered, determine whether the task needs to be executed according to the check result, and update the scheduled task list in the database after the task is executed.

[0115] As an example, the task scheduling module includes at least one or more task scheduling classes TaskDispatcher, and a task scheduling class TaskDispatcher is created for each task in the scheduled task list.

[0116] The task checking module is used to respond to the task query request sent by the task scheduling module and query whether there is a thread with the name of the task to be queried in the thread pool of the server.

[0117] The database is used to store the scheduled task list.

[0118] It should be noted that the system further includes one or more processors and a memory.

[0119] The memory is used to store executable instructions, and when these instructions are executed by one or more processors, the one or more processors perform operations, and these operations include the processes of the method for decentralized scheduled task scheduling in a cluster environment in the foregoing embodiments, especially Figure 1 the processes of the shown method.

[0120] In an alternative embodiment, the system further includes a computer-readable medium storing software, and the software includes instructions executable by one or more computers, and these instructions, when executed in this way, cause the one or more computers to perform operations, and these operations include the processes of the method for decentralized scheduled task scheduling in a cluster environment in the foregoing embodiments, especially Figure 1 the processes of the shown method.

[0121] It should also be noted in the embodiments of the present invention that through the decentralized design, the dependence on a single central scheduler is reduced, making the entire system more robust and better able to cope with node failures or network problems. At the same time, decentralized scheduling can better adapt to heterogeneous cluster environments composed of nodes of different types and performances, making the system more general and flexible.

[0122] Preferably, as the cluster scale grows, the decentralized scheduling method can be more easily extended without large-scale modification or upgrade of the central scheduler. In the embodiments of the present invention, the central scheduler is removed, simplifying the system architecture, thereby reducing the costs and complexities of maintenance and upgrade.

[0123] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory.

[0124] The method can be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner.

[0125] Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program can be implemented in assembly or machine language.

[0126] In any case, the language can be a compiled or interpreted language.

[0127] In addition, for this purpose the program is capable of running on a programmed application-specific integrated circuit.

[0128] The processes described herein (or variations and / or combinations thereof) can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executed jointly on one or more processors, by hardware, or by a combination thereof. The computer program includes a plurality of instructions executable by one or more processors.

[0129] Further, the method can be implemented in any type of computing platform operably connected, including but not limited to personal computers, minicomputers, mainframes, workstations, network or distributed computing environments, separate or integrated computer platforms, or in communication with charged particle tools or other imaging devices.

[0130] Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer and, when read by the computer, can be used to configure and operate the computer to execute the processes described herein.

[0131] In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network.

[0132] When such media includes instructions or programs that implement the above-described steps in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media.

[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for scheduling timed tasks in a decentralized cluster environment, characterized in that: include: When the system starts, use SQL to read the scheduled task list in the database into memory, create and register the scheduled task; Monitor registered scheduled tasks, and trigger the scheduled task when the Cron expression of a scheduled task is reached; Read the task list in the database to obtain the execution times, the latest execution IP and the latest random string of the scheduled task; Determining whether the triggered scheduled task needs to be executed on the server and triggering the scheduled task includes: After the task trigger class CronTrigger is registered, the scheduled task register ScheduledTaskRegistrar will monitor the server clock. When the clock time meets the Cron expression, the task scheduling class TaskDispatcher will be started; when the scheduled task does not need to be executed on this server, the task execution count in the memory is set to the task execution count in the database, and the task execution is terminated; when it needs to be executed on this server, the task execution count in the memory is set to the task execution count in the database plus 1; Use TCP protocol to send a task query request to the server indicated by the latest execution IP; The server indicated by the latest execution IP receives the query request, checks whether there is a thread named as the scheduled task in the thread pool and returns the result. When the returned result is no, try to update the latest execution IP of the scheduled task in the database task list to the IP of this server; the returned result includes two situations, among which: When the return result is yes, the task execution is terminated; If the result is negative, try to update the latest execution IP of the task in the database task list to the IP of this server; generate a new random string, use the latest random string and the scheduled task name as the update condition, and update the newly generated random string and the IP of this server to the table; When the update fails, it means that the random string has been updated by other servers and the task execution is aborted; When the update is successful, a thread with the name of the scheduled task is created in the thread pool, and the scheduled task is executed in the thread; After the scheduled task is executed, the execution count of the scheduled task in the database task list is increased by 1.

2. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 1, characterized in that: The fields in the scheduled task list include the scheduled task name, Cron expression, execution times, latest execution IP, and latest random string.

3. A decentralized scheduled task scheduling system for a cluster environment, characterized in that: It includes task registration module, task scheduling module, task checking module and database, among which: The task registration module is used to read the scheduled task list in the database when the system starts, and create and register the scheduled task according to the scheduled task list; The task scheduling module is used to check whether the scheduled task is executed simultaneously or repeatedly when the scheduled task is triggered, determine whether the task needs to be executed according to the check result, and update the scheduled task list in the database after executing the scheduled task; The task checking module is used to respond to the task query request sent by the task scheduling module and query whether there is a thread with the name of the task being queried in the thread pool of the server; The database is used to store the scheduled task list; The implementation of the registration module, the task scheduling module, the task checking module and the database specifically includes: When the system starts, use SQL to read the scheduled task list in the database into memory, create and register the scheduled task; Monitor registered scheduled tasks, and trigger the scheduled task when the Cron expression of a scheduled task is reached; Read the task list in the database to obtain the execution times, the latest execution IP and the latest random string of the scheduled task; Determining whether the triggered scheduled task needs to be executed on the server and triggering the scheduled task includes: After the task trigger class CronTrigger is registered, the scheduled task register ScheduledTaskRegistrar will monitor the server clock. When the clock time meets the Cron expression, the task scheduling class TaskDispatcher will be started; when the scheduled task does not need to be executed on this server, the task execution count in the memory is set to the task execution count in the database, and the task execution is terminated; when it needs to be executed on this server, the task execution count in the memory is set to the task execution count in the database plus 1; Use TCP protocol to send a task query request to the server indicated by the latest execution IP; The server indicated by the latest execution IP receives the query request, checks whether there is a thread named as the scheduled task in the thread pool and returns the result. When the returned result is no, try to update the latest execution IP of the scheduled task in the database task list to the IP of this server; the returned result includes two situations, among which: When the return result is yes, the task execution is terminated; If the result is negative, try to update the latest execution IP of the task in the database task list to the IP of this server; generate a new random string, use the latest random string and the scheduled task name as the update condition, and update the newly generated random string and the IP of this server to the table; When the update fails, it means that the random string has been updated by other servers and the task execution is aborted; When the update is successful, a thread with the name of the scheduled task is created in the thread pool, and the scheduled task is executed in the thread; After the scheduled task is executed, the execution count of the scheduled task in the database task list is increased by 1.

4. The decentralized scheduled task scheduling system for cluster environment according to claim 3 is characterized in that: Also includes: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of the method for decentralized scheduled task scheduling in a cluster environment as described in any one of claims 1 to 2.

Images