Timed task scheduling method and system for cluster environment decentralization

By reading and updating the scheduled task list in the database in the cluster environment, ensuring that the tasks are executed on the appropriate server, solving the problem of repeated execution of timing tasks, and achieving efficient and flexible task management and system fault tolerance.

CN119003136AActive Publication Date: 2024-11-22ZHONGBO INFORMATION TECH RES INST CO LTD

Patent Information

Application Number
CN202411476080.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-22
Publication Date
2024-11-22
Estimated Expiration
2044-10-22

AI Technical Summary

Technical Problem

In a cluster environment, timing tasks may be started at the same time, resulting in duplicate data or program exceptions. The existing centralized scheduling system and distributed locking methods have problems with repeated tasks caused by high resource consumption, increased complexity, lock timeout and inconsistent clocks.

Method used

By reading the timed task list in the database to memory when the system starts, creating and registering the timed task, monitoring the Cron expression triggering task, determining whether the task is executed on this server, using the TCP protocol to send query requests to the latest execution IP, and updating the thread Pool and database task list ensures that tasks are executed on the appropriate server and prevents repeated execution.

Benefits of technology

It realizes flexible task management, improves task execution efficiency and system fault tolerance, ensures task uniqueness and high availability, and reduces task interruptions caused by node failure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003136A_ABST
    Figure CN119003136A_ABST
Patent Text Reader

Abstract

The invention discloses a timed task scheduling method and system for cluster environment decentralization, and the method comprises the steps: reading a timed task list in a database to a memory by using an SQL when a system is started, and creating and registering a timed task; the registered timed tasks are monitored, and when the Cron expression of a certain timed task is reached, the task is triggered; reading the task list in the database to obtain the execution times, the latest execution IP and the latest random character string of the task; judging whether the triggered timed task needs to be executed in the server or not; sending a task query request to a server pointed by the latest execution IP by using a TCP protocol; after receiving a query request, checking whether a thread pool has a thread of which the name is the task name or not, and returning a result; when updating succeeds, a thread with the name being the task name is created in a thread pool, and the task is executed in the thread; and after the timed tasks are executed, adding 1 to the execution times of the timed tasks in the task list.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of scheduled task scheduling, and in particular to a method and system for scheduling scheduled tasks in a decentralized cluster environment. Background Art

[0002] In large-scale systems, in order to achieve high concurrency and high availability, cluster deployment is often used, that is, the same set of programs is deployed on multiple servers to form multiple system instances. When the program contains scheduled tasks, scheduled tasks on multiple servers may be started at the same time, causing problems such as duplicate data or program exceptions. The common solution in the industry is to deploy a centralized scheduling system to uniformly schedule scheduled tasks, or use distributed locks to prevent scheduled tasks from being executed on multiple servers at the same time. However, there are still some disadvantages: The method of using a centralized scheduling system: not only does it require additional server resources, but the business system also needs to undergo certain supporting modifications to adapt to the scheduling system, which increases system complexity and construction costs; when a centralized scheduling system fails, all scheduled tasks will fail to execute.

[0003] Method of using distributed locks: When a task takes a long time to execute, the distributed lock will be occupied for a long time, which will cause the lock to time out and become invalid. Other servers may obtain the lock and cause the tasks to be executed at the same time. When a server fails during the execution of a task and stops, the distributed lock will not be released in time, causing all servers to fail to obtain the lock and cause the task to not be executed. When the server clocks are different, one server will execute the task first because of its fast clock, and the other server will obtain the lock again to execute the task because of its slow clock, causing the task to be executed repeatedly at the same time. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract of the specification and the title of the invention of this application to avoid blurring the purpose of this section, the abstract of the specification and the title of the invention, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: when the system starts, SQL is used to read the scheduled task list in the database into the memory, and the scheduled task is created and registered; Monitor registered scheduled tasks, and trigger the scheduled task when the Cron expression of a scheduled task is reached; Read the task list in the database to obtain the execution times, the latest execution IP and the latest random string of the scheduled task; Determine whether the triggered scheduled task needs to be executed on the current server. If it needs to be executed on the current server, set the task execution count in the memory to the task execution count in the database plus 1. Use TCP protocol to send a task query request to the server indicated by the latest execution IP; The server indicated by the latest execution IP receives the query request, checks whether there is a thread with the name of the scheduled task in the thread pool and returns the result. When the returned result is no, try to update the latest execution IP of the scheduled task in the database task list to the IP of this server; When the update is successful, a thread with the name of the scheduled task is created in the thread pool, and the scheduled task is executed in the thread; After the scheduled task is executed, the execution count of the scheduled task in the database task list is increased by 1.

[0007] As a preferred solution of the method for scheduling a decentralized scheduled task in a cluster environment according to the present invention, triggering the scheduled task includes: After the task trigger class CronTrigger is registered, the scheduled task register ScheduledTaskRegistrar will monitor the server clock. When the clock time matches the Cron expression, the task scheduling class TaskDispatcher will be started.

[0008] As a preferred solution of the method for scheduling decentralized scheduled tasks in a cluster environment described in the present invention, when the scheduled task does not need to be executed on the server, the number of task executions in the memory is set to the number of task executions in the database, and the task execution is terminated; When it is necessary to execute on this server, the task execution count in the memory is set to the task execution count in the database plus 1.

[0009] As a preferred solution of the method for decentralized scheduled task scheduling in a cluster environment described in the present invention, the returned result includes two cases, wherein: When the return result is yes, the task execution is terminated; If the result is negative, try to update the latest execution IP of the task in the database task list to the IP of this server; When the update fails, it means that the random string has been updated by another server and the task execution is aborted.

[0010] As a preferred solution of the method for scheduling timed tasks in a decentralized cluster environment described in the present invention, an attempt is made to update the latest execution IP of the task in the database task list to the IP of the server, including: Generate a new random string, use the latest random string and scheduled task name as update conditions, and update the newly generated random string and the IP of this server to the table.

[0011] As a preferred solution of the method for decentralized scheduled task scheduling in a cluster environment described in the present invention, the fields contained in the scheduled task list include scheduled task name, Cron expression, execution times, latest execution IP, and latest random string.

[0012] As a preferred solution of the decentralized scheduled task scheduling system for cluster environment described in the present invention, it includes a task registration module, a task scheduling module, a task checking module and a database, wherein: The task registration module is used to read the scheduled task list in the database when the system starts, and create and register the scheduled task according to the scheduled task list; The task scheduling module is used to check whether the scheduled task is executed simultaneously or repeatedly when the scheduled task is triggered, determine whether the task needs to be executed according to the check result, and update the scheduled task list in the database after executing the scheduled task; The task checking module is used to respond to the task query request sent by the task scheduling module and query whether there is a thread with the name of the task being queried in the thread pool of the server; The database is used to store the scheduled task list.

[0013] As a preferred solution of the decentralized scheduled task scheduling system for cluster environment described in the present invention, it also includes: one or more processors; A memory stores operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of the aforementioned decentralized scheduled task scheduling method for a cluster environment.

[0014] Beneficial effects of the present invention: 1. Flexible task management: Using Cron expressions to define the execution time of tasks provides a high degree of flexibility and precision, allowing the system to be customized and optimized according to different business needs, including but not limited to task creation, registration, triggering, execution and update; 2. Improve task execution efficiency: By generating a random string and using it as the update condition, it effectively prevents multiple nodes from executing the same task at the same time, ensuring the uniqueness and accuracy of task execution. Dynamically updating the task execution IP can ensure that the task is executed on the most appropriate server, improving the efficiency of task execution; 3. Enhance system fault tolerance: The system's fault tolerance is improved through the abort mechanism and error handling, and it can maintain stable operation even when some components fail. This fault tolerance mechanism ensures high availability of tasks and reduces task interruptions caused by node failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 It is a flow chart of a method for decentralized scheduled task scheduling in a cluster environment shown in the present invention; Figure 2 This is a schematic diagram of the module structure distribution of the decentralized scheduled task scheduling system for a cluster environment shown in the present invention. DETAILED DESCRIPTION

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0017] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making any creative work should fall within the scope of protection of the present invention.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] According to an embodiment of the present invention, Figure 1 The flowchart shown is a method for scheduling decentralized scheduled tasks in a cluster environment, which specifically includes the following steps: S1. When the system starts, it reads the scheduled task list in the database into the memory, creates and registers the scheduled task.

[0020] In an optional implementation, the fields included in the scheduled task list include the scheduled task name, Cron expression, execution times, latest execution IP, and latest random string.

[0021] As an example, the table name of the task list in the database is: t_task_list, the scheduled task name field is: task_id, the random string field is: random_str, the latest execution IP field is: last_ip, and the execution times field is: run_times. The SQL statement to be read is: SELECT task_id,cron,run_times,last_ip,random_str FROM t_task_list.

[0022] In the embodiment of the present invention, the Cron expression in the scheduled task list is used to create a task trigger class CronTrigger and a task scheduling class TaskDispatcher, and register them in the scheduled task register ScheduledTaskRegistrar.

[0023] The example code is as follows: public void addTask(ScheduledTaskRegistrar taskRegistrar,String cron,TaskDispatcher taskOne) { Trigger triggerOne = triggerContext ->{ Date nextExecTime = null; try { CronTrigger cronTrigger = new CronTrigger(cron); nextExecTime = cronTrigger.nextExecutionTime(triggerContext); } catch (Exception e) { e.printStackTrace(); } return nextExecTime; }; taskRegistrar.addTriggerTask(taskOne, triggerOne);} S2. Monitor the registered scheduled tasks and trigger the scheduled task when the Cron expression of a scheduled task is reached.

[0024] As an example, after the task trigger class CronTrigger is registered, the scheduled task register ScheduledTaskRegistrar will monitor the server clock. When the clock time matches the Cron expression, the task scheduling class TaskDispatcher will be started.

[0025] S3. Read the task list in the database to obtain the execution count, latest execution IP, and latest random string of the task.

[0026] As an example, the SQL statement to read the scheduled task named task1 is: SELECT run_times,last_ip,random_str FROM t_task_list WHERE task_id='task1'.

[0027] S4: Determine whether the triggered scheduled task needs to be executed on this server. The following points should be noted in this step: When the task does not need to be executed on this server, the task execution times in the memory are set to the task execution times in the database, and the task execution is terminated; When the task needs to be executed on this server, the task execution count in the memory is set to the task execution count in the database plus 1.

[0028] Furthermore, before the above steps are implemented, it is necessary to determine whether the triggered scheduled task has been executed by other servers, including the following steps: (1) Compare whether the number of executions of the task in the database is greater than the number of executions of the task in the memory; (2) When the comparison result is yes, it means that the task has been executed on other servers and does not need to be executed on this server, preventing the task from being executed repeatedly; (3) When the comparison result is no, it means that the task has not been executed on other servers and needs to be executed on this server.

[0029] S5. Use the TCP protocol to send a task query request to the server indicated by the latest execution IP obtained in step S3.

[0030] S6. The server indicated by the above IP receives the query request, checks whether there is a thread with the name of the task in the thread pool and returns the result.

[0031] Specifically, in the embodiment of the present invention, all threads are obtained through the enumerate method of the ThreadGroup class, and it is checked one by one whether there is a thread with the task name.

[0032] The example code is as follows: public static boolean findTaskByThreadName(String threadName) { ThreadGroup threadGroup = Thread.currentThread().getThreadGroup(); Thread[] threads = new Thread[threadGroup.activeCount()]; threadGroup.enumerate(threads); for (int i = 0; i <threads.length; i++) { if(threadName.equals(threads[i].getName())) { return true; } } return false; } When the return result is yes, the task execution is terminated; If the returned result is negative, try to update the latest execution IP of the task in the database task list to the IP of this server.

[0033] In an optional implementation, the method for attempting to update the latest execution IP of the task in the database task list to the IP of this server includes: generating a new random string, using the latest random string obtained in step S3 and the scheduled task name as update conditions, and updating the newly generated random string and the IP of this server to the table.

[0034] For example, assume that a new random string is generated as: BD116EB5AB4F1D393E8CE90F77677225, the latest random string obtained in step S3 is: 532B295F85B91CAF7FE3E0C32902C183, the scheduled task name is: task1, the IP address of this server is 192.168.1.10, and the update SQL statement executed is: UPDATE t_task_list SET random_str='BD116EB5AB4F1D393E8CE90F77677225', last_ip='192.168.1.10' WHERE task_id='task1' AND random_str='532B295F85B91CAF7FE3E0C32902C183' When the update fails, it means that the random string has been updated by another server and the task execution is aborted.

[0035] It can be seen that the embodiment of the present invention controls the task list to be updated only once at the same time through a random string, thereby preventing multiple servers from updating at the same time, and can effectively solve the problem of simultaneous execution of scheduled tasks.

[0036] S7. When the update is successful, a thread with the name of the task is created in the thread pool, and the scheduled task is executed in the thread.

[0037] S8. After the scheduled task is executed, the number of executions of the task in the database task list is increased by 1.

[0038] For example, the update SQL statement executed is: UPDATE t_task_list SET run_times = run_times + 1 WHERE task_id='task1' It can be seen that the task list in the database records the actual execution times of the scheduled tasks. The present invention controls the execution times to increase sequentially, which can ensure that a task can only be executed once at a time point, thus solving the problem of repeated execution of tasks.

[0039] Preferably, other aspects disclosed in the embodiments of the present invention further propose a decentralized scheduled task scheduling system for a cluster environment, comprising: a task registration module, a task scheduling module, a task checking module, and a database.

[0040] The task registration module is used to read the scheduled task list in the database when the system starts, and create and register scheduled tasks according to the scheduled task list.

[0041] As an example, the task registration module includes at least one scheduled task register ScheduledTaskRegistrar, one or more task trigger classes CronTrigger, and a task trigger class CronTrigger is created for each task in the scheduled task list.

[0042] The task scheduling module is used to check whether the scheduled task is executed simultaneously or repeatedly when the scheduled task is triggered, decide whether the task needs to be executed based on the check result, and update the scheduled task list in the database after the task is executed.

[0043] As an example, the task scheduling module includes at least one or more task scheduling classes TaskDispatcher, and a task scheduling class TaskDispatcher is created for each task in the scheduled task list.

[0044] The task checking module is used to respond to the task query request sent by the task scheduling module, and query whether the thread pool of this server has a thread with the name of the queried task.

[0045] The database is used to save the scheduled task list.

[0046] It should be noted that the system also includes one or more processors and memories.

[0047] The memory is used to store operable instructions, which, when executed by one or more processors, cause one or more processors to perform operations, including the process of the decentralized scheduled task scheduling method for a cluster environment of the aforementioned embodiment, especially Figure 1 The process of the method shown.

[0048] In an optional embodiment, the system further includes a computer-readable medium storing software, the software including instructions executable by one or more computers, the instructions causing one or more computers to perform operations by such execution, the operations including the process of the method for decentralized scheduled task scheduling in a cluster environment of the aforementioned embodiment, in particular Figure 1 The process of the method shown.

[0049] It should also be noted that the embodiment of the present invention reduces the dependence on a single central scheduler through a decentralized design, making the entire system more robust and better able to cope with node failures or network problems. At the same time, decentralized scheduling can better adapt to a heterogeneous cluster environment composed of nodes of different types and performances, making the system more versatile and flexible.

[0050] Preferably, as the cluster size grows, the decentralized scheduling method can be more easily expanded without large-scale modification or upgrade of the central scheduler. In the embodiment of the present invention, the central scheduler is removed, the system architecture is simplified, and the cost and complexity of maintenance and upgrades are reduced.

[0051] It should be appreciated that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory.

[0052] The method may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes a computer to operate in a specific and predefined manner.

[0053] Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system, however, the program can be implemented in assembly or machine language if desired.

[0054] In any case, the language may be a compiled or interpreted language.

[0055] Furthermore, the program can be run on an application specific integrated circuit programmed for this purpose.

[0056] The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.

[0057] Further, the method may be implemented in any type of computing platform operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device.

[0058] Various aspects of the present invention may be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage media, RAM, ROM, etc., so that it can be read by a programmable computer, which, when the storage medium or device is read by the computer, can be used to configure and operate the computer to perform the processes described herein.

[0059] Additionally, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network.

[0060] The invention described herein includes these and other various types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor.

[0061] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for scheduling timed tasks in a decentralized cluster environment, characterized in that: include: When the system starts, use SQL to read the scheduled task list in the database into memory, create and register the scheduled task; Monitor registered scheduled tasks, and trigger the scheduled task when the Cron expression of a scheduled task is reached; Read the task list in the database to obtain the execution times, the latest execution IP and the latest random string of the scheduled task; Determine whether the triggered scheduled task needs to be executed on the current server. If it needs to be executed on the current server, set the task execution count in the memory to the task execution count in the database plus 1. Use TCP protocol to send a task query request to the server indicated by the latest execution IP; The server indicated by the latest execution IP receives the query request, checks whether there is a thread with the name of the scheduled task in the thread pool and returns the result. When the returned result is no, try to update the latest execution IP of the scheduled task in the database task list to the IP of this server; When the update is successful, a thread with the name of the scheduled task is created in the thread pool, and the scheduled task is executed in the thread; After the scheduled task is executed, the execution count of the scheduled task in the database task list is increased by 1.

2. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 1, characterized in that: Triggering the scheduled task includes: After the task trigger class CronTrigger is registered, the scheduled task register ScheduledTaskRegistrar will monitor the server clock. When the clock time matches the Cron expression, the task scheduling class TaskDispatcher will be started.

3. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 1, characterized in that: When the scheduled task does not need to be executed on this server, the task execution times in the memory are set to the task execution times in the database, and the task execution is terminated; When it is necessary to execute on this server, the task execution count in the memory is set to the task execution count in the database plus 1.

4. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 1, characterized in that: The returned result includes two cases: When the return result is yes, the task execution is terminated; If the result is negative, try to update the latest execution IP of the task in the database task list to the IP of this server; When the update fails, it means that the random string has been updated by another server and the task execution is aborted.

5. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 4, characterized in that: Try to update the latest execution IP of the task in the database task list to the IP of this server, including: Generate a new random string, use the latest random string and scheduled task name as update conditions, and update the newly generated random string and the IP of this server to the table.

6. The method for scheduling decentralized scheduled tasks in a cluster environment according to claim 1, characterized in that: The fields in the scheduled task list include the scheduled task name, Cron expression, execution times, latest execution IP, and latest random string.

7. A decentralized scheduled task scheduling system for a cluster environment, characterized in that: It includes task registration module, task scheduling module, task checking module and database, among which: The task registration module is used to read the scheduled task list in the database when the system starts, and create and register the scheduled task according to the scheduled task list; The task scheduling module is used to check whether the scheduled task is executed simultaneously or repeatedly when the scheduled task is triggered, determine whether the task needs to be executed according to the check result, and update the scheduled task list in the database after executing the scheduled task; The task checking module is used to respond to the task query request sent by the task scheduling module and query whether there is a thread with the name of the task being queried in the thread pool of the server; The database is used to store the scheduled task list.

8. The decentralized scheduled task scheduling system for cluster environment according to claim 7, characterized in that: Also includes: one or more processors; A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of the method for decentralized scheduled task scheduling in a cluster environment as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Distributed calling system for timed tasks

    CN112416581A

  • Remote task regulation and control method and device, equipment and storage medium

    CN113590347A

  • Distributed task scheduling method and device, electronic equipment and storage medium

    CN114090198A

  • Distributed multi-subtask scheduling method and system

    CN116166393A

  • K-DB database multi-node cluster scheduling method and system, device, and medium

    US20240311391A1

Cited By

  • Task scheduling method, device and equipment for server cluster

    CN119883635A