Redis-based asynchronous task security scheduling method and system

Abstract

The invention relates to a Redis-based asynchronous task security scheduling method and system, and the method comprises the steps: creating an asynchronous task, and storing the asynchronous task to a task queue for scheduling; scheduling the asynchronous tasks through atomic operation executed by the script, preferentially acquiring the overtime asynchronous tasks from the task execution set, or popping up the asynchronous tasks to be executed from the task queue, and recording the acquired asynchronous tasks to the task execution set to track a scheduling state; prolonging the maximum execution time of the asynchronous task by updating the expiration time of the asynchronous task in the task execution set; and after the asynchronous task is completed or failed, deleting the asynchronous task from the task execution set. Compared with the prior art, the scheme has the advantages that the task scheduling stability in a high-concurrency scene is remarkably improved, the task loss risk is reduced, non-blocking access and dynamic scheduling are supported, the scheme is suitable for high-reliability systems such as financial systems and industrial control systems, the system performance is optimized, and the implementation complexity is simplified.

Description

Technical Field

[0001] The present invention relates to the technical field of asynchronous task scheduling, and in particular to a Redis-based asynchronous task security scheduling method and system. Background Art

[0002] With the widespread adoption of distributed systems and cloud computing, asynchronous task scheduling technology has played a significant role in decoupling task publishing and execution, improving system performance, and supporting concurrent processing. Among existing asynchronous task scheduling implementations, the key-value store-based task queue mechanism is widely adopted. A typical implementation involves the following: the task publisher stores pending tasks in a Redis task queue. The task executor then pops a task from the queue and performs operations based on the task information. This approach leverages Redis's high performance and data structure support to achieve fast task access, resolving the mismatch between task publishing and execution speeds. It also supports concurrent operations and distributed deployment for both task publishers and executors.

[0003] However, the existing technology has significant limitations. After the task executor pops a task from the task queue, if an abnormal situation occurs during the task execution, the task will not be able to continue to execute, and the task will be lost because it has been removed from the task queue. This task loss problem is unacceptable in systems with high reliability requirements. Although some existing technologies attempt to alleviate this problem through logging or task retry mechanisms, these methods usually increase system complexity and cannot fundamentally ensure the reliability of the task. Therefore, there is an urgent need for a technical solution that can effectively prevent task loss and ensure the safe scheduling of asynchronous tasks to improve the reliability and stability of the system. Summary of the Invention

[0004] The purpose of the present invention is to provide a Redis-based asynchronous task security scheduling method and system to solve the problem of task loss in the current system using asynchronous task scheduling.

[0005] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides a method for securely scheduling asynchronous tasks based on Redis, the method comprising:

[0006] Create asynchronous tasks and store them in the task queue for scheduling;

[0007] Schedule asynchronous tasks through atomic operations executed by scripts, prioritize obtaining timed-out asynchronous tasks from the task execution set, or pop out pending asynchronous tasks from the task queue, and record the obtained asynchronous tasks in the task execution set to track the scheduling status;

[0008] By updating the expiration time of asynchronous tasks in the task execution collection, the maximum execution time of asynchronous tasks is extended;

[0009] After the asynchronous task is completed or fails, the asynchronous task is removed from the task execution collection.

[0010] As a further improvement of an embodiment of the present invention, the method further includes that the step of creating an asynchronous task includes:

[0011] The asynchronous task is stored in a combination format according to the maximum execution time and task content; the combination format is connected by a predetermined delimiter; the task content is a JSON string containing information required for asynchronous task execution;

[0012] The asynchronous task is inserted into the left side of the task queue using a list operation of a key-value store to support non-blocking task scheduling submission.

[0013] As a further improvement of an embodiment of the present invention, the method further includes that the step of performing the atomic operation by the script includes:

[0014] Retrieving, from the task execution set, asynchronous tasks with scores less than the current system time through a script execution mechanism of key-value storage, wherein the scores represent expiration times of the asynchronous tasks, and the retrieved asynchronous tasks are timed-out tasks;

[0015] If a timed-out asynchronous task is retrieved, its maximum execution time is parsed, the score of the timed-out asynchronous task in the task execution set is updated to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, and the timed-out asynchronous task is rescheduled and returned;

[0016] If no timed-out asynchronous task is retrieved, popping an asynchronous task from the right side of the task queue to continue scheduling;

[0017] If the asynchronous task is successfully popped up, its maximum execution time is parsed, the asynchronous task is recorded in the task execution set, and its score is set to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, so as to be included in the scheduling management;

[0018] If no asynchronous task pops up, an empty result is returned and the scheduling operation is suspended.

[0019] As a further improvement of an embodiment of the present invention, the method further includes: the scheduling implementation of the atomic operation also includes:

[0020] The key value is stored in a Redis database, the task queue is a Redis list for storing asynchronous tasks to be scheduled, the task execution set is a Redis ordered set for managing the status of scheduled tasks, and the score is the score in the Redis ordered set, indicating the expiration time of the task;

[0021] The script is a Redis Lua script that implements task scheduling and allocation through atomic execution. The list operation includes LPUSH and RPOP commands to support non-blocking access and scheduling of asynchronous tasks.

[0022] As a further improvement of an embodiment of the present invention, the method further includes that the atomic operation and the extension and deletion operations include:

[0023] Use Redis's ZRANGEBYSCORE command to retrieve asynchronous tasks whose scores are less than the current system time, so as to prioritize timed-out tasks.

[0024] Use Redis's ZADD command to update the score of asynchronous tasks in the task execution set to maintain the scheduling status;

[0025] Use Redis's ZADD INCR command to extend the expiration time of asynchronous tasks to dynamically adjust the scheduling time limit;

[0026] Use the Redis ZREM command to delete the asynchronous task to end its scheduling lifecycle.

[0027] As a further improvement of an embodiment of the present invention, the method further includes that extending the asynchronous task includes:

[0028] Obtaining the asynchronous task data returned by the atomic operation and the time increment specified by the task executor for extending the expiration time;

[0029] The score of the asynchronous task in the task execution set is updated using an incremental update operation of the key-value store, where the updated score is the sum of the original score and the time increment, to support dynamic scheduling management.

[0030] As a further improvement of an embodiment of the present invention, the method further includes that deleting the asynchronous task includes:

[0031] Using the delete operation of the key-value store, remove the completed or failed asynchronous task from the task execution set and end its scheduling state;

[0032] If the asynchronous task fails to execute, the asynchronous task is re-recorded into the task queue to be re-included in the scheduling process for retry.

[0033] To achieve one of the above-mentioned objectives of the invention, an embodiment of the present invention further provides a Redis-based asynchronous task security scheduling system, the system comprising a task creation module, a task acquisition module, a task extension module and a task termination module;

[0034] The task creation module is used to create asynchronous tasks and store the asynchronous tasks in the task queue for scheduling;

[0035] The task acquisition module is used to schedule asynchronous tasks through atomic operations executed by scripts, preferentially acquire timed-out asynchronous tasks from the task execution set, or pop out asynchronous tasks to be executed from the task queue, and record the acquired asynchronous tasks into the task execution set to track the scheduling status;

[0036] The task extension module is used to extend the maximum execution time of the asynchronous task by updating the expiration time of the asynchronous task in the task execution set;

[0037] The task ending module is used to delete the asynchronous task from the task execution set after the asynchronous task is completed or fails.

[0038] In order to achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention also provides an electronic device, including a memory and a processor, characterized in that the memory stores a computer program that can be run on the processor, and when the program is executed on the processor, the steps in the above-mentioned Redis-based asynchronous task security scheduling method are implemented.

[0039] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores a computer program, and is characterized in that when the computer program is executed by a processor, the steps in the above-mentioned Redis-based asynchronous task security scheduling method are implemented.

[0040] Compared to existing technologies, this invention provides a Redis-based method and system for secure asynchronous task scheduling. This system implements atomic task acquisition and recording via Redis Lua scripts, effectively preventing task loss in concurrent environments. Redis lists and ordered sets, combined with a dynamic extension mechanism, ensure task reliability and flexibility. Compared to existing technologies, this solution significantly improves task scheduling stability in high-concurrency scenarios, reduces the risk of task loss, supports non-blocking access and dynamic scheduling, and is suitable for high-reliability systems such as finance and industrial control, optimizing system performance and simplifying implementation complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the overall flow chart of the Redis-based asynchronous task security scheduling method described in the present invention.

[0042] Figure 2 This is a schematic diagram of the processing flow of the Lua script for obtaining tasks in the Redis-based asynchronous task security scheduling method described in the present invention.

[0043] Figure 3 This is a schematic diagram of the architecture of the Redis-based asynchronous task security scheduling system described in the present invention. DETAILED DESCRIPTION

[0044] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0045] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0046] In the first embodiment of the present invention, the present invention provides a method for securely scheduling asynchronous tasks based on Redis, such as Figure 1 As shown, the method includes,

[0047] Create asynchronous tasks and store them in the task queue for scheduling;

[0048] Schedule asynchronous tasks through atomic operations executed by scripts, prioritize obtaining timed-out asynchronous tasks from the task execution set, or pop out pending asynchronous tasks from the task queue, and record the obtained asynchronous tasks in the task execution set to track the scheduling status;

[0049] By updating the expiration time of asynchronous tasks in the task execution collection, the maximum execution time of asynchronous tasks is extended;

[0050] After the asynchronous task is completed or fails, the asynchronous task is removed from the task execution collection.

[0051] In a specific embodiment of the present invention, the step of creating an asynchronous task is specifically as follows:

[0052] The asynchronous task is stored in a combination format according to the maximum execution time and task content; the combination format is connected by a predetermined delimiter; the task content is a JSON string containing information required for asynchronous task execution;

[0053] The asynchronous task is inserted into the left side of the task queue using a list operation of a key-value store to support non-blocking task scheduling submission.

[0054] It should be noted that the maximum execution time, measured in milliseconds, represents the maximum allowable duration of a task's execution and is used for subsequent timeout management and dynamic scheduling. Task content is formatted as a JSON string and contains all the information required for asynchronous task execution, such as the task type, input parameters, execution logic, or target address. The maximum execution time and task content are concatenated using a predefined delimiter (such as an underscore "") to form a unified task data format, such as "maxExeTime_taskContent." This combined format ensures the integrity and parsability of task data, facilitating parsing and processing by the task executor in subsequent steps.

[0055] Furthermore, the asynchronous task is inserted into the left side of the task queue using Redis's list operation. Specifically, the formatted task data is pushed into the Redis list structure through Redis's LPUSH command. Redis lists support efficient header insertion operations, ensuring that tasks are stored in a first-in-first-out order, adapting to the non-blocking submission requirements of asynchronous tasks. The task queue serves as a storage container for tasks to be executed, supporting the decoupling of task publishers and executors, so that the task publisher can continue other operations without waiting for task execution, thereby achieving task scheduling in a high-concurrency and distributed environment. For example, in a distributed Web service, the task publisher can quickly insert multiple asynchronous tasks into the task queue through the LPUSH command, and the task executors can independently obtain tasks from the queue without interfering with each other.

[0056] In a specific embodiment of the present invention, the steps of performing atomic operations through script execution are specifically as follows:

[0057] Retrieving, from the task execution set, asynchronous tasks with scores less than the current system time through a script execution mechanism of key-value storage, wherein the scores represent expiration times of the asynchronous tasks, and the retrieved asynchronous tasks are timed-out tasks;

[0058] If a timed-out asynchronous task is retrieved, its maximum execution time is parsed, the score of the timed-out asynchronous task in the task execution set is updated to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, and the timed-out asynchronous task is rescheduled and returned;

[0059] If no timed-out asynchronous task is retrieved, popping an asynchronous task from the right side of the task queue to continue scheduling;

[0060] If the asynchronous task is successfully popped up, its maximum execution time is parsed, the asynchronous task is recorded in the task execution set, and its score is set to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, so as to be included in the scheduling management;

[0061] If no asynchronous task pops up, an empty result is returned and the scheduling operation is suspended.

[0062] It should be noted that Redis's script execution mechanism is used to retrieve asynchronous tasks with scores less than the current system time from the task execution set. The task execution set uses a Redis ordered set data structure, in which each asynchronous task is associated with a score that represents the expiration time of the asynchronous task, typically the sum of the current time at the time of task assignment and the maximum execution time. Redis's ZRANGEBYSCORE command is used to retrieve tasks with scores less than the current system time. These tasks are considered timed-out tasks, indicating that their execution may not have been completed due to an exception (such as a program crash or server failure). Calling ZRANGEBYSCORE through a Lua script ensures that the retrieval operation and other concurrent operations are executed in a single atomic environment, avoiding race conditions.

[0063] Furthermore, if a timed-out asynchronous task is retrieved, the task data is parsed to extract its maximum execution time. The task data format is "maxExeTime_taskContent", and the maximum execution time and task content are separated by a string splitting operation. Subsequently, the score of the asynchronous task in the task execution set is updated and set to the sum of the current system time and the maximum execution time, for example, by updating the score value through the ZADD command of Redis. The updated score represents the new expiration time of the task, allowing the task executor to retry executing the task. After the update is completed, the script returns the data of the asynchronous task for use by the task executor.

[0064] Furthermore, if no timed-out asynchronous task is retrieved, indicating that there is currently no timed-out task to be retried, the script pops up an asynchronous task from the right side of the task queue through the Redis RPOP command. The task queue uses the Redis list data structure to store asynchronous tasks to be executed, and implements first-in-first-out task allocation through the RPOP command. The RPOP operation is executed in the Lua script to ensure the atomicity of the popped-out task and prevent multiple executors from obtaining the same task at the same time. For example, in a distributed system, multiple executors can concurrently obtain tasks through RPOP, and the script ensures the mutual exclusivity of task allocation.

[0065] Furthermore, if an asynchronous task is successfully popped up, the script parses the maximum execution time for that task and, similarly, extracts maxExeTime from "maxExeTime_taskContent" through string segmentation. The asynchronous task is then recorded in the "running tasks" collection, and the task data is added using the Redis ZADD command. Its score is set to the sum of the current system time and the maximum execution time. This score serves as the task's expiration time and is used for subsequent timeout detection and status management. This recording operation ensures that the task is moved from the queue to the "running" state, preventing task loss.

[0066] Furthermore, if an asynchronous task fails to pop up (i.e., the task queue is empty), the script returns an empty result, indicating that there are no available tasks to execute. Returning an empty result allows the task executor to enter a waiting state or execute other logic, avoiding invalid operations. The atomicity of Lua scripts ensures that returning an empty result is consistent with other operations, maintaining the integrity of the system state.

[0067] In a specific embodiment of the present invention, the scheduling implementation of the atomic operation further includes:

[0068] The key value is stored in a Redis database, the task queue is a Redis list for storing asynchronous tasks to be scheduled, the task execution set is a Redis ordered set for managing the status of scheduled tasks, and the score is the score in the Redis ordered set, indicating the expiration time of the task;

[0069] The script is a Redis Lua script that implements task scheduling and allocation through atomic execution. The list operation includes LPUSH and RPOP commands to support non-blocking access and scheduling of asynchronous tasks.

[0070] It should be noted that the key-value store is the Redis database, a high-performance in-memory key-value storage system that supports a variety of data structures and atomic operations and is widely used in task scheduling in distributed systems. The task queue uses the Redis list data structure to store asynchronous tasks to be executed. The Redis list supports efficient head and tail operations, which is suitable for implementing a first-in-first-out task queue, ensuring that the task publisher can submit tasks quickly and the task executor can obtain tasks in order. The task execution set uses the Redis ordered set data structure to track the status of the asynchronous tasks being executed. Each asynchronous task is associated with a score in the ordered set, and the score represents the expiration time of the task, which is defined as the sum of the current system time at the time of task assignment and the maximum execution time. The Redis ordered set supports fast retrieval of timed-out tasks through score sorting and range query functions.

[0071] Furthermore, the script is a Redis Lua script, leveraging Redis's single-threaded execution mechanism to achieve atomic operations. Redis Lua scripts allow multiple operations (such as task retrieval, popping, and recording) to be encapsulated within a single execution environment, ensuring that these operations are not interrupted by other concurrent operations, thereby preventing task loss or duplicate assignment. The Lua script is invoked via the Redis EVAL command, with the current system time as an input parameter, which is used to compare task scores and execute task assignment logic. Within the script, Redis commands (such as ZRANGEBYSCORE, RPOP, and ZADD) can be called to retrieve timed-out tasks from the executing task set or pop new tasks from the task queue, logging them into the executing set. For example, the script first calls ZRANGEBYSCORE 'DoingZSet' 0 currentTime to retrieve tasks with scores less than the current time. If no results are found, RPOP 'TaskQueue' is called to retrieve new tasks and record the task status in the DoingZSet. This atomic operation ensures that multiple executors do not simultaneously retrieve the same task in highly concurrent scenarios, ensuring the reliability of asynchronous task scheduling.

[0072] In a specific implementation scenario of the present invention, Figure 2 The figure shows the processing flow of the Lua script for obtaining a task. The Lua script for obtaining a task used in the present invention is:

[0073] / / ARGV[1] -- nowMS current system time, single transmission: ms

[0074] local tiList = redis.call('zrangebyscore', 'DoingZSet', 0, ARGV[1], 'limit', 0, 1);

[0075] if #tiList > 0 then

[0076] / / There is a task being executed, but it has not been completed due to exceeding the maximum execution time

[0077] local task = tiList[1];

[0078] local s, e = string.find(task, '_');

[0079] local maxExeTime = string.sub(task, 1, s - 1);

[0080] local score = maxExeTime + ARGV[1];

[0081] / / Update task score = current system time (ms) + maxExeTime (maximum task execution time)

[0082] redis.call('zadd', 'DoingZSet', score, task);

[0083] return task;

[0084] end;

[0085] / / Get the task from the task queue, write the executing set and return the task content

[0086] local task = redis.call('rpop', 'TaskQueue');

[0087] if not task then

[0088] local s, e = string.find(task, '_');

[0089] local maxExeTime = string.sub(task, 1, s - 1);

[0090] local score = maxExeTime + ARGV[1];

[0091] redis.call('zadd', 'DoingZSet', score, task);

[0092] return task;

[0093] end;

[0094] / / No tasks to execute

[0095] return nil;

[0096] Furthermore, list operations include Redis's LPUSH and RPOP commands, which are used to implement asynchronous task access operations. The LPUSH command inserts asynchronous tasks into the left side of the task queue, supporting non-blocking submission by the task publisher. The task publisher quickly pushes task data into the queue through LPUSH without waiting for task execution, thereby decoupling task release and execution. The RPOP command pops a task from the right side of the task queue, supporting the task acquisition operation of the task executor. The RPOP operation is executed in a Lua script to ensure the atomicity of the popped task and the recorded status. For example, in a distributed microservice architecture, the task publisher inserts the order processing task through LPUSH, and the task executor obtains the task through RPOP. The script ensures the mutual exclusivity of task allocation. The high efficiency of LPUSH and RPOP supports the fast access of asynchronous tasks and adapts to the needs of high concurrency and distributed environments.

[0097] In a specific embodiment of the present invention, the atomic operation and the extension and deletion operations are specifically,

[0098] Use Redis's ZRANGEBYSCORE command to retrieve asynchronous tasks whose scores are less than the current system time, so as to prioritize timed-out tasks.

[0099] Use Redis's ZADD command to update the score of asynchronous tasks in the task execution set to maintain the scheduling status;

[0100] Use Redis's ZADD INCR command to extend the expiration time of asynchronous tasks to dynamically adjust the scheduling time limit;

[0101] Use the Redis ZREM command to delete the asynchronous task to end its scheduling lifecycle.

[0102] It should be noted that the set in task execution is a Redis ordered set, in which each asynchronous task is associated with a score representing the task's expiration time, which is usually the sum of the current system time at the time of task assignment and the maximum execution time. The ZRANGEBYSCORE command is used to query tasks with scores within a specified range. The format is ZRANGEBYSCORE DoingZSet-inf currentTime, where currentTime is the current system time. This command returns tasks with scores less than the current time, that is, asynchronous tasks that have timed out, indicating that these tasks may not have been completed due to an exception on the executor. In atomic operations, ZRANGEBYSCORE is called through a Lua script to ensure that retrieval and other operations are completed in a single execution environment to avoid concurrency conflicts.

[0103] Furthermore, the ZADD command is used to add or update the score value of an element in an ordered set. The format is ZADDDoingZSet score taskData, where taskData is the asynchronous task data and score is the new expiration time, calculated as the sum of the current system time and the task's maximum execution time. When a timed-out task is retrieved or a new task is popped from the task queue, the script parses the maximum execution time in the task data and updates or records the task's score via ZADD. This operation ensures the accuracy of task status and supports subsequent timeout detection and task reassignment.

[0104] Furthermore, the ZADD INCR command increments the score of a specified element in an ordered set by an increment value. The format is ZADD DoingZSet INCR delay taskData, where delay is the delay time specified by the task executor, and taskData is the asynchronous task data to be delayed. If the task executor determines that the task cannot be completed within the original maximum execution time, ZADD INCR can be used to dynamically extend the task's expiration time. This operation allows the task executor to flexibly adjust the execution time limit, preventing tasks from being reassigned due to timeouts, and supports the dynamic scheduling of asynchronous tasks.

[0105] Furthermore, the ZREM command is used to remove a specified element from an ordered set. Its format is ZREM DoingZSettaskData, where taskData represents the data of a completed or failed asynchronous task. When an asynchronous task completes or fails due to an unrecoverable error, the task executor uses ZREM to remove the task from the set of executing tasks, marking the end of its lifecycle. If a task fails, it can be reinserted into the task queue via LPUSH to support retries, while ZREM ensures that the failed task no longer occupies DoingZSet resources. This operation cleans up task status and optimizes system resource utilization.

[0106] In a specific embodiment of the present invention, the asynchronous task is extended, specifically,

[0107] Obtaining the asynchronous task data returned by the atomic operation and the time increment specified by the task executor for extending the expiration time;

[0108] The score of the asynchronous task in the task execution set is updated using an incremental update operation of the key-value store, where the updated score is the sum of the original score and the time increment, to support dynamic scheduling management.

[0109] It should be noted that asynchronous task data is returned by atomic operation steps (executed via Redis Lua scripts) in the format of "maxExeTime_taskContent," which contains the maximum execution time and task content. The task executor determines whether to extend the task's expiration time based on the task's actual execution progress and specifies the extension time. The executor calculates the extension time based on task complexity or external factors. The extension time is transmitted to the scheduling system through the task executor's call interface (such as an API request), ensuring that the scheduling system can accurately obtain task data and extension time.

[0110] Furthermore, the task execution set is a Redis ordered set, where each asynchronous task is associated with a score representing the task's expiration time (typically the sum of the current time at the time of assignment and the maximum execution time). The Redis ZADD INCR command is used to perform incremental updates. The format is ZADD DoingZSet INCR delay taskData, where delay is the delay time and taskData is the asynchronous task data. This command increments the original score of the specified task by the delay value, generating a new expiration time. This operation is executed using Redis's atomicity, ensuring the reliability and consistency of score updates and avoiding data conflicts in concurrent environments.

[0111] In a specific embodiment of the present invention, deleting an asynchronous task is specifically as follows:

[0112] Using the delete operation of the key-value store, remove the completed or failed asynchronous task from the task execution set and end its scheduling state;

[0113] If the asynchronous task fails to execute, the asynchronous task is re-recorded into the task queue to be re-included in the scheduling process for retry.

[0114] It should be noted that the task execution set uses the Redis ordered set data structure to store currently executing asynchronous tasks. Each task is associated with a score representing its expiration time. The Redis ZREM command is used to remove a specified task. The format is ZREM DoingZSet taskData, where taskData is the asynchronous task data, including the maximum execution time and task content. When the task executor confirms that the task has completed successfully (e.g., the execution logic has completed and a result has been returned) or has failed due to an unrecoverable error (e.g., data format error or unavailable external dependencies), the corresponding task data is removed from the DoingZSet using the ZREM command. This operation clears the task status, frees up memory resources, and prevents completed or failed tasks from occupying system resources.

[0115] Furthermore, the task queue uses a Redis list data structure to store asynchronous tasks to be executed. Failed tasks are reinserted into the left side of the task queue via Redis's LPUSH command in the format of LPUSH TaskQueuetaskData, where taskData is the original data of the failed task (such as "maxExeTime_taskContent"). The reinserted task will wait for the next allocation in a first-in-first-out (FIFO) order, allowing other executors to re-acquire and attempt to execute it. To avoid infinite retries, the system can set an upper limit on the number of retries, record the number of retries through metadata fields in the task data (such as "retryCount"), and update it during insertion.

[0116] In the second embodiment of the present invention, the present invention provides a Redis-based asynchronous task security scheduling system, such as Figure 3 As shown, the system includes a task creation module 1, a task acquisition module 2, a task extension module 3 and a task termination module 4;

[0117] The task creation module 1 is used to create asynchronous tasks and store the asynchronous tasks in a task queue for scheduling;

[0118] The task acquisition module 2 is used to schedule asynchronous tasks through atomic operations executed by scripts, preferentially acquire timed-out asynchronous tasks from the task execution set, or pop out asynchronous tasks to be executed from the task queue, and record the acquired asynchronous tasks into the task execution set to track the scheduling status;

[0119] The task extension module 3 is used to extend the maximum execution time of the asynchronous task by updating the expiration time of the asynchronous task in the task execution set;

[0120] The task ending module 4 is used to delete the asynchronous task from the task execution set after the asynchronous task is completed or fails.

[0121] In a third embodiment of the present invention, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the program is executed on the processor, the steps of the above-mentioned Redis-based asynchronous task security scheduling method are implemented.

[0122] In a fourth embodiment of the present invention, the present invention provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned Redis-based asynchronous task security scheduling method.

[0123] In summary, the present invention provides a Redis-based asynchronous task security scheduling method and system, which implements atomic operations for task acquisition and recording through Redis Lua scripts, effectively preventing task loss in concurrent environments. Redis lists and ordered sets are utilized, combined with a dynamic extension mechanism, to ensure task reliability and flexibility. Compared to existing technologies, this solution significantly improves the stability of task scheduling in high-concurrency scenarios, reduces the risk of task loss, supports non-blocking access and dynamic scheduling, and is suitable for high-reliability systems such as finance and industrial control, optimizing system performance and simplifying implementation complexity.

[0124] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the modules described above can refer to the corresponding process in the aforementioned method implementation, and will not be repeated here.

[0125] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional modules in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0127] The above-mentioned integrated modules implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above-mentioned software functional modules are stored in a storage medium and include a number of instructions for causing a computer system (which may be a personal computer, server, or network system, etc.) or a processor to execute some of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A Redis-based asynchronous task security scheduling method, characterized by: include, Create asynchronous tasks and store them in the task queue for scheduling; Schedule asynchronous tasks through atomic operations executed by scripts, prioritize obtaining timed-out asynchronous tasks from the task execution set, or pop out pending asynchronous tasks from the task queue, and record the obtained asynchronous tasks in the task execution set to track the scheduling status; By updating the expiration time of asynchronous tasks in the task execution collection, the maximum execution time of asynchronous tasks is extended; After the asynchronous task is completed or fails, the asynchronous task is removed from the task execution collection.

2. The Redis-based asynchronous task security scheduling method according to claim 1, characterized in that: The steps of creating an asynchronous task include: The asynchronous task is stored in a combination format according to the maximum execution time and task content; the combination format is connected by a predetermined delimiter; the task content is a JSON string containing information required for asynchronous task execution; The asynchronous task is inserted into the left side of the task queue using a list operation of a key-value store to support non-blocking task scheduling submission.

3. The Redis-based asynchronous task security scheduling method according to claim 2, characterized in that: The steps of the atomic operation performed by the script include: Retrieving, from the task execution set, asynchronous tasks with scores less than the current system time through a script execution mechanism of key-value storage, wherein the scores represent expiration times of the asynchronous tasks, and the retrieved asynchronous tasks are timed-out tasks; If a timed-out asynchronous task is retrieved, its maximum execution time is parsed, the score of the timed-out asynchronous task in the task execution set is updated to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, and the timed-out asynchronous task is rescheduled and returned; If no timed-out asynchronous task is retrieved, popping an asynchronous task from the right side of the task queue to continue scheduling; If the asynchronous task is successfully popped up, its maximum execution time is parsed, the asynchronous task is recorded in the task execution set, and its score is set to the sum of the current system time and the maximum execution time of the timed-out asynchronous task, so as to be included in the scheduling management; If no asynchronous task pops up, an empty result is returned and the scheduling operation is suspended.

4. The Redis-based asynchronous task security scheduling method according to claim 3, characterized in that: The scheduling implementation of the atomic operation also includes: The key value is stored in a Redis database, the task queue is a Redis list for storing asynchronous tasks to be scheduled, the task execution set is a Redis ordered set for managing the status of scheduled tasks, and the score is the score in the Redis ordered set, indicating the expiration time of the task; The script is a Redis Lua script that implements task scheduling and allocation through atomic execution. The list operation includes LPUSH and RPOP commands to support non-blocking access and scheduling of asynchronous tasks.

5. The Redis-based asynchronous task security scheduling method according to claim 4, characterized in that: The atomic operations and the extension and deletion operations include: Use Redis's ZRANGEBYSCORE command to retrieve asynchronous tasks whose scores are less than the current system time, so as to prioritize timed-out tasks. Use Redis's ZADD command to update the score of asynchronous tasks in the task execution set to maintain the scheduling status; Use Redis's ZADD INCR command to extend the expiration time of asynchronous tasks to dynamically adjust the scheduling time limit; Use the Redis ZREM command to delete the asynchronous task to end its scheduling lifecycle.

6. The Redis-based asynchronous task security scheduling method according to claim 3, characterized in that: The extended asynchronous task includes: Obtaining the asynchronous task data returned by the atomic operation and the time increment specified by the task executor for extending the expiration time; The score of the asynchronous task in the task execution set is updated using an incremental update operation of the key-value store, where the updated score is the sum of the original score and the time increment, to support dynamic scheduling management.

7. The Redis-based asynchronous task security scheduling method according to claim 6, characterized in that: The deletion of the asynchronous task includes: Using the delete operation of the key-value store, remove the completed or failed asynchronous task from the task execution set and end its scheduling state; If the asynchronous task fails to execute, the asynchronous task is re-recorded into the task queue to be re-included in the scheduling process for retry.

8. A Redis-based asynchronous task security scheduling system, characterized by: It includes task creation module, task acquisition module, task extension module and task completion module; The task creation module is used to create asynchronous tasks and store the asynchronous tasks in the task queue for scheduling; The task acquisition module is used to schedule asynchronous tasks through atomic operations executed by scripts, preferentially acquire timed-out asynchronous tasks from the task execution set, or pop out asynchronous tasks to be executed from the task queue, and record the acquired asynchronous tasks into the task execution set to track the scheduling status; The task extension module is used to extend the maximum execution time of the asynchronous task by updating the expiration time of the asynchronous task in the task execution set; The task ending module is used to delete the asynchronous task from the task execution set after the asynchronous task is completed or fails.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program that can be run on the processor, and when the program is executed on the processor, the steps in the Redis-based asynchronous task security scheduling method as described in any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the Redis-based asynchronous task security scheduling method are implemented.

Images