Method for removing distributed timing task by consistent hash and adding distributed lock

Abstract

The application provides a method for adding a distributed lock by a consistent Hash to remove a distributed timing task, comprising the following steps: S1, each service instance is started, and a timing task trigger time point and a rule are set according to a cron expression; S2, each service instance node registers and updates own heartbeat information in a database according to a set time; S3, each service instance checks heartbeat information of all instance nodes from the database according to the set time, and eliminates a node whose heartbeat information is not updated in time; S4, at the set timing task trigger time point, each node service instance triggers a task; and S5, each node service instance obtains existing alive instance nodes from the database. The application has the beneficial effects that the distributed timing task no longer adds a database distributed lock, and no longer has a dependency on system time of each service instance node, so that the timing task will not be repeatedly executed even if there is a time difference between nodes.

Description

Technical Field

[0001] This invention belongs to the field of microservices and cloud computing technology, and in particular relates to a method for removing distributed timed tasks and adding distributed locks through consistent hashing. Background Technology

[0002] The distributed scheduled task implemented in this way has the following two problems. First, by adding a distributed lock to the database, it relies on the system time of each node running the service to be consistent. If the time is inconsistent, the distributed lock will lose its effect, and the scheduled task will still be executed repeatedly. Second, if a service instance crashes after acquiring the lock, the acquired lock will not be released in time, which will prevent other service instances from acquiring the lock, and the scheduled task will never be executed. Summary of the Invention

[0003] In view of this, the present invention aims to propose a method for removing the need for distributed timed tasks and adding distributed locks through consistent hashing, in order to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0005] The method of removing distributed locks from distributed scheduled tasks using consistent hashing includes the following steps:

[0006] S1. Each service instance starts up, and the timing and pattern of scheduled task triggering are set according to the cron expression;

[0007] S2. Each service instance node registers and updates its own heartbeat information in the database according to the set time.

[0008] S3. Each service instance checks the heartbeat information of all instance nodes from the database at a set time and removes nodes that have not updated their heartbeat information after the timeout.

[0009] S4. At the set time point for the scheduled task, each node service instance triggers the task.

[0010] S5. Each node service instance retrieves currently active instance nodes from the database;

[0011] S6. Add virtual nodes to each node using consistent hashing;

[0012] S7. Use the scheduled task name and consistent hashing to elect the instance node to execute the task;

[0013] S8. The elected node service instance executes a scheduled task, while the unelected service instance nodes continue to wait for the next election time.

[0014] Furthermore, in step S2, each service instance node registers and updates its own heartbeat information in the database every 2 seconds.

[0015] Furthermore, in step S3, each service instance checks the heartbeat information of all instance nodes from the database every 4 seconds and removes nodes that have not updated their heartbeat information after the timeout.

[0016] Furthermore, in step S6, 100 virtual nodes are added to each node through consistent hashing.

[0017] Furthermore, by utilizing consistent hashing, an instance node is elected among multiple service instances to execute tasks, thereby ensuring that numerous scheduled tasks are executed evenly across various instance nodes and preventing the same scheduled task from being executed repeatedly across different instances.

[0018] Furthermore, this solution discloses an electronic device, including a processor and a memory communicatively connected to the processor and used to store processor-executable instructions, wherein the processor is used to execute a method for removing distributed timed tasks and adding distributed locks through consistent hashing.

[0019] Furthermore, this solution discloses a server, including at least one processor and a memory communicatively connected to the processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the processor to cause the at least one processor to perform a method of adding a distributed lock by removing a distributed timed task through consistent hashing.

[0020] Furthermore, this solution discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a method for removing distributed timed tasks and adding distributed locks through consistent hashing.

[0021] Compared with existing technologies, the method of removing distributed timed tasks and adding distributed locks through consistent hashing described in this invention has the following advantages:

[0022] (1) The method of removing the distributed lock added by the distributed timed task through consistent hashing described in this invention means that the distributed timed task no longer adds a database distributed lock and no longer depends on the system time of each service instance node. Even if there is a time difference between the nodes, the timed task will not be executed repeatedly.

[0023] (2) The method of adding distributed locks to distributed scheduled tasks by removing distributed scheduled tasks through consistent hashing described in this invention will remove the instance node from the scheduled task scheduling cluster in a timely manner when one of the service instance nodes fails. The scheduled tasks previously executed on that node will re-elect the task execution node, and there will be no phenomenon that some scheduled tasks cannot be executed because one service fails. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0025] Figure 1 This is a schematic diagram of a traditional timed task scheduling method as described in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram illustrating the method for removing distributed timed tasks and adding distributed locks through consistent hashing, as described in an embodiment of the present invention. Detailed Implementation

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0028] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] As per the instruction manual Figure 1 As shown, currently, scheduled tasks for distributed clusters generally prevent duplicate execution across clusters by adding distributed locks to the database. The specific implementation follows this process:

[0030] 1. Set the trigger time for scheduled tasks for each service instance;

[0031] 2. After the distributed cluster starts, at a predetermined time, each running instance of the service tries to acquire a lock from the database, but only one running instance of the service can acquire the lock at any given time.

[0032] 3. Service instances that successfully acquire the lock will execute a scheduled task, while service instances that fail to acquire the lock will continue to wait for the next scheduled task time to acquire the lock.

[0033] 4. After the service instance that has acquired the lock finishes executing the scheduled task, it releases the lock and continues to wait for the next scheduled task time to acquire the lock;

[0034] The distributed scheduled task implemented in this way has the following two problems. First, by adding a distributed lock to the database, it relies on the system time of each node running the service to be consistent. If the time is inconsistent, the distributed lock will lose its effect, and the scheduled task will still be executed repeatedly. Second, if a service instance crashes after acquiring the lock, the acquired lock will not be released in time, which will prevent other service instances from acquiring the lock, and the scheduled task will never be executed.

[0035] The purpose of this invention is to provide a scheme for distributed timed scheduling based on a consistent hashing algorithm, so as to solve the problems mentioned in the background.

[0036] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0037] This invention provides a scheme for implementing distributed timed task scheduling based on a consistent hashing algorithm, comprising the following steps:

[0038] S01. Each service instance starts up and the scheduled task triggering time and pattern are set according to the cron expression;

[0039] S02. Each service instance node registers and updates its own heartbeat information in the database every 2 seconds.

[0040] S03. Every 4 seconds, each service instance checks the heartbeat information of all instance nodes from the database and removes nodes that have not updated their heartbeat information after the timeout.

[0041] S04. At the set time point for the scheduled task triggering, each node service instance triggers the task.

[0042] S05. Each node service instance retrieves currently active instance nodes from the database;

[0043] S06. Add 100 virtual nodes to each node using consistent hashing;

[0044] S07. Using the scheduled task name, elect the instance node to execute the task using consistent hashing;

[0045] S08. The elected node service instance executes a scheduled task, while the unelected service instance nodes continue to wait for the next time point for election.

[0046] Furthermore, by utilizing consistent hashing, an instance node is elected among multiple service instances to execute tasks, thereby ensuring that numerous scheduled tasks are executed evenly across various instance nodes and preventing the same scheduled task from being executed repeatedly across different instances.

[0047] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0048] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the division of units described above is merely a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The aforementioned units may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for removing distributed scheduled tasks and adding distributed locks through consistent hashing, characterized in that... Includes the following steps: S1. Each service instance starts up, and the timing and pattern of scheduled task triggering are set according to the cron expression; S2. Each service instance node registers and updates its own heartbeat information in the database according to the set time. S3. Each service instance checks the heartbeat information of all instance nodes from the database at a set time and removes nodes that have not updated their heartbeat information after the timeout. S4. At the set time point for the scheduled task, each node service instance triggers the task. S5. Each node service instance retrieves currently active instance nodes from the database; S6. Add virtual nodes to each node using consistent hashing; S7. Use the scheduled task name and consistent hashing to elect the instance node to execute the task; S8. The elected instance nodes execute scheduled tasks, while the unelected instance nodes continue to wait for the next election time. In step S2, each service instance node registers and updates its heartbeat information in the database every 2 seconds; In step S3, each service instance checks the heartbeat information of all instance nodes from the database every 4 seconds and removes nodes that have not updated their heartbeat information after the timeout. In step S6, 100 virtual nodes are added to each node using consistent hashing; By using consistent hashing, an instance node is elected from among multiple service instances to execute tasks, thus ensuring that numerous scheduled tasks are executed evenly across various instance nodes and preventing the same scheduled task from being executed repeatedly across different instances.

2. An electronic device, comprising a processor and a memory communicatively connected to the processor and used for storing processor-executable instructions, characterized in that: The processor is used to execute the method described in claim 1 for removing distributed timed tasks and adding distributed locks through consistent hashing.

3. A server, characterized in that: The device includes at least one processor and a memory communicatively connected to the processor. The memory stores instructions executable by the at least one processor, which are executed by the processor to cause the at least one processor to perform the method of adding a distributed lock by removing a distributed timed task through consistent hashing as described in claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the method of adding a distributed lock by removing a distributed timed task through consistent hashing as described in claim 1.

Images