A method, apparatus and device for implementing a distributed delay task

Abstract

The application provides a method, device and equipment for implementing a distributed delay task, the method comprising: obtaining a task to be processed, submitting the task to a database Redis after obtaining a distributed lock with a task ID as a Key; periodically traversing a task executor queue, each time the current task executor is traversed, accessing the database Redis with a task type corresponding to the current task executor as a Key, querying a target task in the queue with the most advanced expiration time of the task to be processed of the corresponding task type, confirming whether the target task is expired, otherwise traversing the next current task executor, if the target task is expired, obtaining a distributed lock with an ID of the target task, executing the target task through a working thread of the current task executor, and traversing the next current task executor, if the distributed lock is not obtained, traversing the next current task executor. The application solves the consistency problem and the idempotency problem with lower complexity, and effectively manages different types of tasks.

Description

Technical Field

[0001] This invention relates to the field of message queue execution technology, and in particular to a method, apparatus and device for implementing distributed delayed tasks. Background Technology

[0002] Currently, distributed delayed tasks are primarily implemented using the open-source message queue system RabbitMQ and the platform database Redis, which is based on open-source rapid development software. Using RabbitMQ for distributed delayed tasks relies on RabbitMQ's operation and maintenance, thus increasing complexity and cost. In contrast, implementing distributed delayed tasks using the Redis database leverages the characteristics of the Redis ZSet data structure. It automatically sorts tasks according to their execution time, then periodically polls the ZSet collection, retrieves and executes tasks that meet the criteria, and finally removes the task from the ZSet collection after execution.

[0003] Current technologies that utilize the Redis ZSet data structure to implement distributed delayed tasks based on Redis do not consider the scenario where the task producers are deployed in a cluster, leading to the same task being generated and written to Redis by multiple producers. Without proper control, this can result in duplicate task generation, a problem known as idempotency. Furthermore, when the task consumers are deployed in a cluster, existing technologies introduce intermediate components to ensure that a task is consumed by only one consumer, addressing the consistency issue of multiple consumers repeatedly executing tasks. However, introducing intermediate components not only increases complexity and development / maintenance costs but also fails to effectively manage and coordinate different types of tasks. Summary of the Invention

[0004] This application provides a method, apparatus, and device for implementing distributed delayed tasks. It utilizes Redis distributed locks to solve the idempotency problem of multiple task producers submitting tasks and the consistency problem of multiple task consumers consuming tasks. At the same time, it periodically traverses the task executors corresponding to different types of tasks on the consumer side to effectively manage and mobilize different types of tasks.

[0005] Firstly, this application provides a method for implementing distributed delayed tasks, serving as a production end, including:

[0006] Obtain the task to be processed, and use the task ID to obtain a distributed lock from the Redis database;

[0007] If the distributed lock is not acquired, the task to be submitted will be automatically abandoned.

[0008] If the distributed lock is acquired, a task structure corresponding to the task to be processed is generated based on the task information of the task to be processed and the predefined task structure.

[0009] The generated task structure is stored in a ZSet data structure and submitted to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

[0010] In one or more embodiments, the task structure includes at least one of the following information items:

[0011] Task ID, Task Type, Task Execution Expiration Time, Extended Data, Data Required for Task Execution;

[0012] The extended data includes the task's request context information.

[0013] Secondly, this application provides a method for implementing distributed delayed tasks, serving as a consumer end, including:

[0014] The task executor queue is periodically traversed. Each time the current task executor is reached, the database Redis is accessed using the task type corresponding to the current task executor as the key.

[0015] Query the score of at least one pending task corresponding to the task type in the task queue, and determine the target task with the earliest execution due time among the at least one pending task.

[0016] If it is determined that the target task has not expired, the next current task executor is traversed; otherwise, the distributed lock of the database Redis is obtained using the ID of the target task.

[0017] If the distributed lock is not acquired, the execution of the target task is automatically abandoned, and the next current task executor is traversed.

[0018] If the distributed lock is acquired, the value of the target task is retrieved from the Redis database, the target task is executed through the worker thread of the current task executor, and the process iterates through the next current task executor.

[0019] In one or more embodiments, the task executor is generated in the following manner:

[0020] The task types corresponding to different task executors are marked with annotations;

[0021] For task executors of different task types, define the processing logic for processing the data in the task structure;

[0022] The processing logic and corresponding annotations defined for each task type are structured into a task executor for that task type.

[0023] In one or more embodiments, the processing logic defined for each task type and the corresponding annotations are structured into a task executor for that task type, including:

[0024] The processing logic and corresponding annotations defined for each task type are structured into a task executor that includes the task type and reflection call parameters.

[0025] The reflection invocation parameters are used to execute the target task based on the processing logic that processes the data in the task structure, using the value of the target task obtained from the Redis database.

[0026] In one or more embodiments, the task executor further includes:

[0027] A task confirmation interface for reading user instructions;

[0028] Specifically, the task executor reads user instructions from the task confirmation interface through the current task executor's worker thread, and deletes the target task after it is completed, or resubmits the target task and resets its task execution expiration time, based on the user instructions.

[0029] In one or more embodiments, the method for confirming whether a target task has expired includes:

[0030] If the score of the target task is later than the current system clock, then the target task has expired; otherwise, the target task has not expired.

[0031] In one or more embodiments, retrieving the value of the target task from the Redis database and executing the target task through the worker thread of the current task executor includes:

[0032] Retrieve the value of the target task from the Redis database, and use the extended data in the value of the target task to restore the request context of the target task in the worker thread of the current task executor.

[0033] After restoring the request context, the target task is executed using the data required for task execution from the value of the target task obtained from the Redis database.

[0034] In one or more embodiments, it further includes:

[0035] After the target task is completed by the worker thread of the current task executor, the Redis database automatically deletes the target task.

[0036] Thirdly, this application provides an apparatus for implementing distributed delayed tasks, serving as a production end, comprising:

[0037] The distributed lock acquisition module is used to acquire tasks to be processed and obtain a distributed lock from the Redis database using the task's ID.

[0038] The task abandonment module is used to automatically abandon the submission of the pending task if the distributed lock is not acquired.

[0039] The task structure definition module is used to generate a task structure corresponding to the task to be processed based on the task information of the task to be processed and a predefined task structure if the distributed lock is acquired.

[0040] The task submission module is used to store the generated task structure in a ZSet data structure and submit it to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

[0041] Fourthly, this application provides an apparatus for implementing distributed delayed tasks, serving as a consumer end, comprising:

[0042] The task executor traversal module is used to periodically traverse the task executor queue. Each time the current task executor is traversed, the database Redis is accessed using the task type corresponding to the current task executor as the key.

[0043] The module for finding due tasks is used to query the score of at least one pending task corresponding to the task type in the task queue, and determine the target task with the earliest execution due time among the at least one pending task.

[0044] The distributed lock acquisition module is used to determine that if the target task has not expired, it traverses the next current task executor; otherwise, it acquires the distributed lock of the database Redis using the ID of the target task.

[0045] The task abandonment module is used to automatically abandon the execution of the target task and traverse the next current task executor if the distributed lock is not acquired.

[0046] The task execution module is used to, if the distributed lock is acquired, retrieve the value of the target task from the Redis database, execute the target task through the worker thread corresponding to the current task executor, and traverse the next current task executor.

[0047] Fifthly, this application provides an apparatus for implementing a distributed delayed task, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the aforementioned method for implementing a distributed delayed task.

[0048] Sixthly, embodiments of this application provide a computer storage medium storing a computer program for causing a computer to execute the above-described method for implementing a distributed delay task.

[0049] The method, apparatus, and device for implementing distributed delayed tasks provided in this application have the following beneficial effects:

[0050] By leveraging the distributed locks of the Redis database, the idempotency problem in scenarios with multiple task producers and the consistency problem in scenarios with multiple task consumers can be solved with less cost and lower computational complexity. At the same time, a method of identifying consumer tasks using custom annotations is adopted, and the methods corresponding to different types of tasks are structured into task executors. The task executor queue is periodically traversed and the task executors are executed asynchronously, so as to effectively manage and execute different types of tasks.

[0051] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram illustrating the implementation of distributed delayed tasks using RabbitMQ message queues, provided in an embodiment of this application.

[0054] Figure 2 This is a schematic diagram of a method for implementing distributed delayed tasks in a production environment, provided in an embodiment of this application.

[0055] Figure 3 A schematic diagram illustrating multiple production ends competing to submit the tasks to be processed, provided for an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of a method for implementing distributed delayed tasks on the consumer side, provided in an embodiment of this application.

[0057] Figure 5 A schematic diagram of an apparatus for implementing distributed delayed tasks in a production environment, provided in an embodiment of this application;

[0058] Figure 6 A schematic diagram of a device for implementing distributed delayed tasks on a consumer end, provided in an embodiment of this application;

[0059] Figure 7 This is a device diagram for implementing distributed delayed tasks, provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] During development, we often encounter requirements regarding delayed tasks. For example:

[0062] Orders not paid within 30 minutes of being generated will be automatically cancelled;

[0063] • Send a text message to the user 60 seconds after the order is generated.

[0064] In related technologies, the open-source message queue system RabbitMQ and the platform database Redis, which is based on open-source rapid development software, are used to implement distributed delayed tasks.

[0065] Using RabbitMQ for delayed queues is a common approach. However, RabbitMQ itself doesn't directly support delayed queue functionality. Instead, it indirectly implements distributed delayed tasks through the TTL and DXL attributes of RabbitMQ message queues. Figure 1 As shown.

[0066] TTL refers to the message's lifespan. RabbitMQ allows you to set message expiration times. Once a message in the buffer queue exceeds its TTL, it becomes a Dead Letter. DLX refers to a Dead Letter Exchange. The queue bound to a Dead Letter Exchange is a Dead Letter queue. Once a Dead Letter appears in the queue, the message is rerouted to another Exchange so it can be consumed again.

[0067] However, implementing distributed delayed tasks using RabbitMQ depends on RabbitMQ's operation and maintenance, which increases complexity and cost.

[0068] Implementing distributed delayed tasks using the Redis database leverages the inherent sorting capabilities of Redis's sorted sets (ZSets). Each member of a sorted set is associated with a score attribute, which is used to sort the members in ascending order. Tasks are treated as members of the ZSet, with each member's score representing its execution due date. The ZSet is periodically polled; if a task meets the criteria, it is retrieved, executed, and then removed from the ZSet.

[0069] Current technologies that utilize the ZSet data structure in Redis database to implement distributed delayed tasks do not consider the situation where the task production end is deployed in a cluster, resulting in the same task being generated and written to the Redis database on multiple production ends. If this is not controlled, it will lead to the duplicate generation of tasks, i.e., the idempotency problem.

[0070] Meanwhile, when the task consumers are deployed in a cluster, multiple consumers may retrieve the same task from the Redis database after the execution time has arrived but before it has been completed. This will result in the task being executed repeatedly, causing consistency issues.

[0071] Existing technologies ensure that a task is consumed by only one consumer by introducing a single-process deployed intermediate component. However, the introduction of the intermediate component not only increases complexity and development and maintenance costs, but also fails to effectively manage and mobilize different types of tasks.

[0072] In view of the above problems, this application provides a method, apparatus and device for implementing distributed delayed tasks. It utilizes the distributed lock of the database Redis to solve the idempotency problem in multiple task production scenarios and the consistency problem of multiple task consumers in consuming tasks with less cost and lower computational complexity. At the same time, it adopts a method of identifying consumer tasks with custom annotations, and structures the methods corresponding to different types of tasks into task executors. It periodically traverses the task executor queue and executes the task executors asynchronously, so as to effectively manage and execute different types of tasks.

[0073] This application provides a method for implementing distributed delayed tasks, applicable to production environments, such as... Figure 2 As shown, it includes:

[0074] Step S201: Obtain the task to be processed, and use the task ID to obtain a distributed lock from the Redis database;

[0075] Once a task is generated, it possesses a unique ID across multiple production endpoints. This ID may have certain business attributes; for example, for an order-related task, the ID could be set to the order number. The ID of a task remains consistent across multiple production endpoints. After obtaining the ID of the task, each production endpoint uses that ID as the key to retrieve a unique distributed lock corresponding to that task's ID from the Redis database.

[0076] Step S202: If the distributed lock is not acquired, the task to be submitted is automatically abandoned.

[0077] If a producer fails to acquire the distributed lock corresponding to the ID of the task to be processed, it means that the distributed lock for the ID of the task to be processed has been preemptively acquired by another producer. In this case, the producer automatically abandons the submission of the task to be processed. A diagram illustrating multiple producers competing to submit the task to be processed is shown below. Figure 3 As shown.

[0078] Step S203: If the distributed lock is acquired, then generate the task structure corresponding to the task to be processed based on the task information of the task to be processed and the predefined task structure.

[0079] In one or more embodiments, the task structure includes at least one of the following information items:

[0080] Task ID, Task Type, Task Execution Expiration Time, Extended Data, Data Required for Task Execution;

[0081] The task type indicates which type of task the task to be processed belongs to. A task of a certain type means that the method for processing this type of task is consistent. The task execution due date indicates when the task to be processed will be executed. The extended data contains the task's request context information. The data required for task execution includes the key data required for the specific execution of the task.

[0082] Step S204: Store the generated task structure as a ZSet data structure and submit it to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

[0083] Redis is a database that stores data using key-value pairs. The value is retrieved based on the key, and each key corresponds to a unique value. The sorted set (ZSet) data structure of Redis used in this embodiment includes the following attributes:

[0084] Key: Task type;

[0085] Value: The task structure;

[0086] The score attribute is the task's expiration time.

[0087] The production side stores the task structure generated by the tasks to be processed into an ordered set ZSet in the Redis database as a ZSet data structure, forming a task queue.

[0088] This application defines a unique task ID on the task producer side and uses this task ID as the key of the distributed lock. Only the producer side that acquires the distributed lock can submit the task, thus ensuring idempotency.

[0089] This application provides a method for implementing distributed delayed tasks, applied to the consumer end, such as... Figure 4 As shown, it includes:

[0090] Step S401: Periodically traverse the task executor queue. Each time the current task executor is reached, access the database Redis using the task type corresponding to the current task executor as the key.

[0091] In one or more embodiments, the task executor is generated in the following manner:

[0092] The task types corresponding to different task executors are marked with annotations;

[0093] For task executors of different task types, define the processing logic for processing the data in the task structure;

[0094] The processing logic and corresponding annotations defined for each task type are structured into a task executor for that task type.

[0095] The processing logic that processes the data in the task structure is also the method of consuming tasks. By setting annotations corresponding to different task types, the method of consuming tasks is marked. Therefore, each task executor corresponds to a task type, and task executors that process different task types constitute a task executor queue.

[0096] By using custom annotations to identify consumption tasks, the methods corresponding to different types of tasks are structured into task executors, and a task processor is defined for each type of task, thus achieving effective management of different types of tasks.

[0097] In one or more embodiments, the processing logic defined for each task type and the corresponding annotations are structured into a task executor for that task type, including:

[0098] The processing logic and corresponding annotations defined for each task type are structured into a task executor that includes the task type and reflection call parameters.

[0099] The reflection invocation parameters are used to execute the target task based on the processing logic that processes the data in the task structure, using the target task ID as the key and the value of the target task obtained from the Redis database.

[0100] In one or more embodiments, the task executor further includes:

[0101] A task confirmation interface for reading user instructions;

[0102] Specifically, the task executor reads user instructions from the task confirmation interface through the current task executor's worker thread, and deletes the target task after it is completed, or resubmits the target task and resets its task execution expiration time, based on the user instructions.

[0103] Whether or not the task confirmation interface is set in the task executor depends on whether the present invention needs to provide the user with the task confirmation interface when setting the task executor, so as to delete the task or resubmit the task and reset the task execution expiration time of the target task according to the user's instructions.

[0104] This application completes the consumer registration process during the initial deployment. After the application scans the service containing the consumption method corresponding to a certain type of task and starts up, it scans and finds all methods marked with the custom annotation, structures this method information into task executors, and stores them in the task executor queue, thus completing the registration process.

[0105] This application uses custom annotations to identify task consumption methods. After the application scans for this method, it structures it into a task executor based on the relevant information and stores this executor in a task executor queue. A periodic scheduled task iterates through this queue and executes the task executors asynchronously. This effectively manages and executes different types of tasks.

[0106] Step S402: Query the score of at least one pending task corresponding to the task type in the task queue, and determine the target task with the earliest execution due time among the at least one pending task.

[0107] Using the task type corresponding to the current task executor as the key, query the Redis database for the task with the lowest score in the task queue corresponding to the task type, that is, the task with the earliest execution expiration time and closest to the current system clock, as the target task.

[0108] Step S403: If the target task has not expired, iterate through the next current task executor; otherwise, use the ID of the target task to obtain the distributed lock of the Redis database.

[0109] In one or more embodiments, the method for confirming whether a target task has expired includes:

[0110] If the score of the target task is later than the current system clock, then the target task has expired; otherwise, the target task has not expired.

[0111] The term "target task due date" refers to the target task's execution time being later than the current system clock.

[0112] Step S404: If the distributed lock is not acquired, the execution of the target task is automatically abandoned, and the next current task executor is traversed.

[0113] Upon the expiration of the target task, the consumer retrieves a distributed lock from the Redis database using the target task's ID as the key. Acquiring the distributed lock indicates that the current task consumer has obtained the resources to execute the target task and can proceed to the next task execution step. If the distributed lock is not acquired, the task execution is automatically abandoned, and the consumer iterates through the next current task executor, entering the next worker thread.

[0114] On the task consumer side, the task's unique ID is used as the key of the distributed lock. Only consumers that have acquired the distributed lock can consume and execute the task, thus ensuring consistency.

[0115] Step S405: If the distributed lock is acquired, retrieve the value of the target task from the Redis database, execute the target task through the worker thread of the current task executor, and iterate through the next current task executor.

[0116] In one or more embodiments, retrieving the value of the target task from the Redis database and executing the target task through the worker thread of the current task executor includes:

[0117] Retrieve the value of the target task from the Redis database, and use the extended data in the value of the target task to restore the request context of the target task in the worker thread of the current task executor.

[0118] After restoring the request context, the target task is executed using the data required for task execution from the value of the target task obtained from the Redis database.

[0119] Asynchronous task consumption always involves the issue of request context propagation. During the process of retrieving a task from the thread pool and executing it on the current worker thread, user data loss is inevitable. Therefore, when the target task is executed, the request context information of the task at the time of submission is first obtained from the extended data of the corresponding task data in the target task's value. This information is then set into the request context of the current worker thread. This ensures that the request context of the task submission is restored within this worker thread, recovering user data and allowing users to seamlessly pass on the request context.

[0120] This application includes the request context in the extended data of the task data when the task is submitted. When the task starts executing on the consumer side, the request context is retrieved and set into the request context of the worker thread executing the task, thus restoring the request context information.

[0121] After restoring the context, the task's consumption method is invoked using the principle of Java reflection. The data required for task execution is then used in the target task's value to perform the actual task processing.

[0122] In one or more embodiments, it further includes:

[0123] After the target task is completed by the worker thread of the current task executor, the Redis database automatically deletes the target task.

[0124] After task processing is complete, if the task is automatically deleted, it is removed from the Redis task queue. If it is not automatically deleted, the task is deleted based on the task confirmation interface. The task confirmation interface has two methods: one is to delete the target task, and the other is to add the target task back to the task queue and reset its execution expiration time.

[0125] This application provides the task confirmation interface Acker to solve the problem of how to re-consume tasks after they fail. Through this interface, task data can be manually deleted and, when a task fails, the task can be added back to the task queue and the execution time can be reset.

[0126] All task consumers periodically traverse the task executor queue. Once the task with the lowest score among the tasks corresponding to the current task executor has expired, one consumer uses the task ID to obtain a unique distributed lock from the Redis database and executes the task. Other consumers then automatically traverse the next task executor. All consumers execute task executors asynchronously, thus achieving effective management and execution of different types of tasks.

[0127] Based on the same inventive concept, embodiments of this application also provide an apparatus for implementing distributed delayed tasks, serving as a production end, such as... Figure 5 As shown, it includes:

[0128] The distributed lock acquisition module 501 is used to acquire tasks to be processed and to acquire distributed locks from the Redis database using the task ID;

[0129] The task abandonment module 502 is used to automatically abandon the submission of the pending task if the distributed lock is not acquired.

[0130] The task structure definition module 503 is used to generate a task structure corresponding to the task to be processed based on the task information of the task to be processed and a predefined task structure if the distributed lock is acquired.

[0131] The task submission module 504 is used to store the generated task structure in a ZSet data structure and submit it to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

[0132] Based on the same inventive concept, embodiments of this application also provide a device for implementing distributed delayed tasks, as a consumer end, such as... Figure 6 As shown, it includes:

[0133] The task executor traversal module 601 is used to periodically traverse the task executor queue. Each time the current task executor is traversed, the database Redis is accessed using the task type corresponding to the current task executor as the key.

[0134] The module 602 for finding due tasks is used to query the score of at least one pending task in the task queue that corresponds to the task type, and to determine the target task with the earliest execution due time among the at least one pending task.

[0135] The distributed lock acquisition module 603 is used to determine that if the target task has not expired, it traverses the next current task executor; otherwise, it acquires the distributed lock of the database Redis using the ID of the target task.

[0136] The task abandonment module 604 is used to automatically abandon the execution of the target task and traverse the next current task executor if the distributed lock is not acquired.

[0137] The task execution module 605 is used to, if the distributed lock is acquired, retrieve the value of the target task from the Redis database, execute the target task through the worker thread corresponding to the current task executor, and traverse the next current task executor.

[0138] Based on the same inventive concept, this application provides a device 700 for implementing distributed delayed tasks, such as... Figure 7 As shown, it includes at least one processor 702; and a memory 701 communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described method for implementing a distributed delay task.

[0139] The memory 701 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions. The memory 701 may be volatile memory, such as random-access memory (RAM); it may also be non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may be any one or a combination of the above-mentioned volatile and non-volatile memory types.

[0140] The processor 702 can be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. It can also be a hardware chip. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Based on the same inventive concept, embodiments of this application provide a computer storage medium storing a computer program for causing a computer to execute the aforementioned method for implementing a distributed delay task.

[0141] Based on the same inventive concept, embodiments of this application provide a computer program medium, wherein the computer storage medium stores a computer program, and the computer program is used to enable a computer to execute the above-described method for implementing a distributed delay task.

[0142] The aforementioned storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0143] This disclosure also provides a computer program product that, when run on an electronic device, causes the electronic device to execute any of the methods described above for implementing a distributed delayed task, or any possible method involved in implementing a distributed delayed task.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0145] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0147] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0148] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0149] The technical solutions provided in this application have been described in detail above. Specific examples have been used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0150] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood by those skilled in the art from the computer program instructions that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of one or more computer-usable storage media containing computer-usable program code (implementing each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams). These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for implementing distributed delayed tasks, applied in a production environment, characterized in that, include: Obtain the task to be processed, and use the task ID to obtain a distributed lock from the Redis database; If the distributed lock is not acquired, the task to be submitted will be automatically abandoned. If the distributed lock is acquired, a task structure corresponding to the task to be processed is generated based on the task information of the task to be processed and the predefined task structure. The task structure includes at least one of the following information items: task ID, task type, task execution expiration time, extended data, and data required for task execution. The extended data includes the request context information of the task, which is used to restore the request context of the task when the task executor on the consumer side executes the task. The generated task structure is stored in a ZSet data structure and submitted to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

2. A method for implementing distributed delayed tasks, applied to the consumer end, characterized in that, include: The task executor queue is periodically traversed. Each time the current task executor is reached, the database Redis is accessed using the task type corresponding to the current task executor as the key. Query the score of at least one pending task corresponding to the task type in the task queue, and determine the target task with the earliest execution due time among the at least one pending task. If it is determined that the target task has not expired, the next current task executor is traversed; otherwise, the distributed lock of the database Redis is obtained using the ID of the target task. If the distributed lock is not acquired, the execution of the target task is automatically abandoned, and the next current task executor is traversed. If the distributed lock is acquired, the value of the target task is retrieved from the Redis database. Using the extended data in the value of the target task, the request context of the target task is restored in the worker thread of the current task executor. After restoring the request context, the target task is executed using the data required for task execution from the value of the target task obtained from the Redis database, and the next current task executor is traversed.

3. The method as described in claim 2, characterized in that, The task executor is generated in the following manner: The task types corresponding to different task executors are marked with annotations; For task executors of different task types, define the processing logic for processing the data in the task structure; The processing logic and corresponding annotations defined for each task type are structured into a task executor for that task type.

4. The method as described in claim 3, characterized in that, The processing logic defined for each task type and its corresponding annotations are structured into task executors for that task type, including: The processing logic and corresponding annotations defined for each task type are structured into a task executor that includes the task type and reflection call parameters. The reflection invocation parameters are used to execute the target task based on the processing logic that processes the data in the task structure, using the value of the target task obtained from the Redis database.

5. The method as described in claim 4, characterized in that, The task executor also includes: A task confirmation interface for reading user instructions; Specifically, the user instruction is read from the task confirmation interface through the worker thread of the current task executor. According to user instructions, after the target task is completed, delete the target task, or resubmit the target task and reset the task execution expiration time.

6. The method as described in claim 2, characterized in that, The following methods are used to confirm whether a target task is due: If the score of the target task is later than the current system clock, then the target task has expired; otherwise, the target task has not expired.

7. The method as described in claim 2, characterized in that, Also includes: After the target task is completed by the worker thread of the current task executor, the Redis database automatically deletes the target task.

8. An apparatus for implementing distributed delayed tasks, serving as a production end, characterized in that, include: The distributed lock acquisition module is used to acquire tasks to be processed and obtain a distributed lock from the Redis database using the task's ID. The task abandonment module is used to automatically abandon the submission of the pending task if the distributed lock is not acquired. The task structure definition module is used to generate a task structure corresponding to the task to be processed based on the task information of the task to be processed and a predefined task structure if the distributed lock is acquired. The task structure includes at least one of the following information items: task ID, task type, task execution expiration time, extended data, and data required for task execution. The extended data includes the request context information of the task, which is used to restore the request context of the task when the task executor on the consumer side executes the task. The task submission module is used to store the generated task structure in a ZSet data structure and submit it to the task queue of the Redis database. The ZSet data structure includes the task type as the key, the task structure as the value, and the task execution expiration time as the score.

9. A device for implementing distributed delayed tasks, serving as a consumer, characterized in that: include: The task executor traversal module is used to periodically traverse the task executor queue. Each time the current task executor is traversed, the database Redis is accessed using the task type corresponding to the current task executor as the key. The module for finding due tasks is used to query the score of at least one pending task corresponding to the task type in the task queue, and determine the target task with the earliest execution due time among the at least one pending task. The distributed lock acquisition module is used to determine that if the target task has not expired, it traverses the next current task executor; otherwise, it acquires the distributed lock of the database Redis using the ID of the target task. The task abandonment module is used to automatically abandon the execution of the target task and traverse the next current task executor if the distributed lock is not acquired. The task execution module is used to, if the distributed lock is acquired, retrieve the value of the target task from the Redis database, and restore the request context of the target task in the current task executor's worker thread using the extended data in the value of the target task. After restoring the request context, the target task is executed using the data required for task execution from the value of the target task obtained from the Redis database, and the next current task executor is traversed.

10. An electronic device for implementing distributed delayed tasks, characterized in that, The method includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

11. A computer program medium, characterized in that, The computer storage medium stores a computer program that enables the computer to perform the method as described in any one of claims 1-7.

Images