Delay task processing method and device, computer device and storage medium

Abstract

The embodiment of the application belongs to the field of cloud technology, and relates to a delay task processing method, which comprises the following steps: analyzing a received delay task request to obtain a push time; determining a first delay task and a second delay task, storing the first delay task into a time wheel, and storing the second delay task into a relational database to obtain a task data table; synchronizing the task data table to big data; screening task messages in the task data table to obtain a third delay task, and storing the third delay task into the time wheel; when the time wheel is driven to a current time slot, matching expired delay tasks corresponding to the current time slot to form a delay task list; and pushing the delay task list to a message middleware. The application further provides a delay task processing device, a computer device and a storage medium. In addition, the application also relates to blockchain technology, and the delay task list can be stored in the blockchain. The application can realize dynamic setting of a delay time, support high-precision triggering, and is not limited by a time span.

Description

Technical Field

[0001] This application relates to the field of message processing technology, and in particular to a delayed task processing method, apparatus, computer device, and storage medium. Background Technology

[0002] In daily life, there are often scenarios that require delayed message delivery; such messages can be called delayed tasks. Here, "delayed delivery" means that after the producer generates a message, the consumer cannot receive the message immediately, but must wait for a specified period of time (which can be on the order of seconds, hours, or days) before the consumer can receive the message and process it accordingly.

[0003] Existing products supporting delayed tasks, such as RocketMQ, only support delayed tasks within 2 hours, and the precision is limited, requiring fixed delays of 1 second, 5 seconds, 10 seconds, etc., as specified in the template. Some open-source distributed scheduled task frameworks, such as Elastic-Job, only support triggering at fixed times according to cron expression specifications (the trigger time needs to be specified in advance). However, some business scenarios require calculating the trigger time according to certain business rules, and the trigger time cannot be determined in advance; it needs to be triggered at a specific second, possibly days, months, or years later. Current technologies struggle to meet these requirements. Summary of the Invention

[0004] The purpose of this application is to provide a delayed task processing method, apparatus, computer device, and storage medium to solve the technical problem in the related art that delayed task triggering is difficult to apply to business scenarios with uncertain triggering times, high precision, and long time spans.

[0005] To address the aforementioned technical problems, this application provides a delayed task processing method, employing the following technical solution:

[0006] Receive delayed task requests from message middleware and parse the delayed task requests to obtain the push time;

[0007] Based on the push time, a first delayed task and a second delayed task are determined. The first delayed task is stored in the time wheel, and the second delayed task is stored in the relational database to obtain a task data table.

[0008] The task data table is synchronized to the big data platform to obtain synchronized data.

[0009] The delayed tasks in the synchronized data are filtered by a partitioning method to obtain the third delayed task, and the third delayed task is stored in the time wheel;

[0010] When the time wheel drives to the current time slot, it matches the currently triggered delayed tasks in the time wheel that correspond to the current time slot, and forms a delayed task list.

[0011] The list of delayed tasks is pushed to the message middleware so that the delayed tasks in the list are consumed.

[0012] Furthermore, the step of storing the first delayed task to the time wheel includes:

[0013] The target time slot corresponding to the first delayed task is determined based on the push time of the first delayed task;

[0014] The first delayed task is stored in the target time slot corresponding to the time wheel.

[0015] Furthermore, prior to the step of storing the first delayed task to the time wheel, the method further includes:

[0016] Create a preset number of time slots in Redis based on the cycle period, and create pointers in key-value format;

[0017] The time wheel is obtained based on the time slot and the pointer.

[0018] Furthermore, the step of storing the second delayed task in a relational database to obtain a task data table includes:

[0019] Extract key information from the second delayed task;

[0020] The key information is written into the corresponding key field of the data table in the relational database to obtain the task data table.

[0021] Furthermore, the step of synchronizing the task data table to the big data platform includes:

[0022] The big data computing framework is used to obtain task data tables from the relational database;

[0023] The task data table is stored in a large data storage file for synchronization.

[0024] Furthermore, the step of filtering the delayed tasks in the task data table using a partitioning method to obtain the third delayed task includes:

[0025] Based on the push time, the task data table in the big data storage file is filtered using the partitioning method;

[0026] Select the delayed tasks that meet the criteria as the third delayed task.

[0027] Furthermore, the step of matching the currently triggered delayed task in the time wheel corresponding to the current time slot includes:

[0028] The time interval is calculated based on the push time of the first delayed task and the current time;

[0029] Based on the cycle period of the time wheel and the time interval, the currently triggered delayed task is determined.

[0030] To address the aforementioned technical problems, this application also provides a delayed task processing device, which employs the following technical solution:

[0031] The receiving module is used to receive delayed task requests from the message middleware and parse the delayed task requests to obtain the push time.

[0032] The storage module is used to determine a first delayed task and a second delayed task based on the push time, store the first delayed task in a time wheel, and store the second delayed task in a relational database to obtain a task data table;

[0033] The synchronization module is used to synchronize the task data table to the big data database to obtain synchronized data;

[0034] The filtering module is used to filter the delayed tasks in the synchronized data through a partitioning method to obtain the third delayed task, and store the third delayed task in the time wheel;

[0035] The matching module is used to match the currently triggered delayed tasks in the time wheel corresponding to the current time slot when the time wheel drives to the current time slot, and form a delayed task list.

[0036] The push module is used to push the list of delayed tasks to the message middleware so that the delayed tasks in the list of delayed tasks can be consumed.

[0037] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0038] The computer device includes a memory and a processor, the memory storing computer-readable instructions, and the processor executing the computer-readable instructions to implement the steps of the delayed task processing method described above.

[0039] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0040] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the delayed task processing method described above.

[0041] Compared with the prior art, the embodiments of this application have the following main advantages:

[0042] This application receives delayed task requests from message middleware, parses the delayed task requests to obtain the push time, determines the first and second delayed tasks based on the push time, stores the first delayed task in a time wheel, and stores the second delayed task in a relational database to obtain a task data table, synchronizes the task data table to a large database to obtain synchronized data, filters the task messages in the synchronized data using a partitioning method to obtain the third delayed task, and stores the third delayed task in the time wheel, and when the time wheel drives to the current time slot, it matches the delayed tasks that have expired in the time wheel corresponding to the current time slot to form a delayed task list, and pushes the delayed task list to the message middleware so that the delayed tasks in the delayed task list can be consumed. This application stores delayed tasks through a time wheel and a relational database respectively, with a large storage capacity, adds delayed tasks according to the push time of the delayed tasks, can dynamically set the delay time without specifying the delay time in advance, pushes through the time wheel with accuracy down to the second, supports high-precision triggering, and is not limited by the time span. Attached Figure Description

[0043] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0045] Figure 2 This is a flowchart of one embodiment of the delayed task processing method according to this application;

[0046] Figure 3 This is a schematic diagram of a structure of an embodiment of the delayed task processing device according to this application;

[0047] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0049] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0051] This application provides a delayed task processing method that can be applied to, for example... Figure 1 In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 is used as a medium to provide a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0052] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0053] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0054] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0055] It should be noted that the delayed task processing method provided in the embodiments of this application is generally composed of... server Execution, accordingly, delayed task processing devices are generally located in server middle.

[0056] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0057] Continue to refer to Figure 2 The flowchart illustrates an embodiment of the delayed task processing method according to this application, including the following steps:

[0058] Step S201: Receive the delayed task request from the message middleware and parse the delayed task request to obtain the push time.

[0059] In this context, the delayed tasks in the message middleware are produced by the message producer on the business side. The business side only needs to send the delayed tasks to the message middleware and no longer needs to listen to the delayed tasks by scanning the database or other means. The delayed tasks are then listened to by the delayed server.

[0060] In this embodiment, the message middleware is Kafka. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data from consumers on a website. It is widely used in the big data field. Kafka can be used to temporarily store data, enabling near real-time data synchronization between two or more databases. Data stored in Kafka is stored record by record, using offsets as indexes. Each offset uniquely corresponds to a record. Depending on the operation type of the incremental data, it can be categorized into one of three operation types: add, delete, or modify. A typical application scenario is that a business database writes incremental data to Kafka, and the database reads data from Kafka, ensuring near real-time consistency between the two databases. This allows the database to meet ad-hoc query and analysis requirements. In simpler terms, Kafka is a log cluster where various servers send their logs to the cluster for unified aggregation and storage. Other machines then pull messages from the cluster for analysis and processing, such as ELT and data mining.

[0061] In this embodiment, the business side interacts directly with Kafka, so there are no bottlenecks in terms of performance, availability, or reliability.

[0062] In this embodiment, the message middleware receives the delayed task that the business side needs to push, then generates a delayed task request and sends it to the delayed server. The delayed task includes, but is not limited to, a business side identifier, a delayed task ID, a push time, and task content. The delayed task ID only needs to be unique within the business side.

[0063] Based on the received delayed task, a corresponding delayed task request is generated. The delayed task request carries the push time of the delayed task. After receiving the delayed task request, the message middleware parses it to obtain the push time of the delayed task.

[0064] Step S202: Determine the first delayed task and the second delayed task based on the push time, store the first delayed task in the time wheel, and store the second delayed task in the relational database to obtain the task data table.

[0065] In this embodiment, the first delayed task is the delayed task that needs to be pushed now, and the second delayed task is the delayed task that needs to be pushed tomorrow or after tomorrow. The first delayed task is stored in the time wheel, and the second delayed task is stored in the relational database in the form of a table.

[0066] Specifically, the first delayed task is stored in the time wheel, and the second delayed task is stored in a relational database in the form of a table.

[0067] The time wheel is a circular data structure (implemented using a regular array), similar to a clock divided into 60 slots, each representing one second. Each slot stores tasks to be executed at a specific time (there may be multiple tasks, stored in a linked list). A pointer rotates once per second (from slot 0 to slot 59), retrieving the task from the current slot. Some tasks may be executed in the current round, while others may wait for several rounds before execution. For example, if the current time is 13:54:03 on April 24, 2022, and the pointer is pointing to slot 3, adding three delayed tasks at this moment: Task 1 is pushed at 13:54:10 on April 24, 2022; Task 2 is pushed at 13:55:10 on April 24, 2022; and Task 3 is pushed at 13:54:10 on April 25, 2022. Three delayed tasks are assigned based on the push time. Task 1, Task 2, and Task 3 are all assigned to slot 10. When the pointer moves from slot 3 to slot 10 after 7 seconds, the three tasks are retrieved. It is found that only Task 1 needs to be executed now, Task 2 needs to wait for the next cycle, and Task 3 needs to wait for 24*60 cycles before it can be executed.

[0068] The relational database used is TiDB, an open-source distributed HTAP (Hybrid Transactional and Analytical Processing) database that is compatible with MySQL, supports unlimited horizontal scaling, and features strong consistency and high availability. In most cases, it can easily migrate SQL data from MySQL to TiDB without modifying the code. It also has the advantage of being able to migrate sharded MySQL clusters in real time using TiDB tools.

[0069] In some optional implementations of this embodiment, a time wheel is implemented using Redis. Specifically, a preset number of time slots are created in Redis according to the cycle period, and pointers are created in key-value form. Based on the time slots and pointers created above, a time wheel is obtained.

[0070] The cycle period is the time it takes for the time wheel to complete one revolution. If the cycle period is 60 seconds, 60 Lists (slot0 to slot59) can be created, each representing a time slot. The time wheel rotates in 1 second per slot. Pointers are then created using key-value pairs, where the value points to the specific time slot name based on the current time in seconds. For example, if the current time is 14:15:01, the pointer will point to slot1.

[0071] By storing delayed tasks using a time wheel created in Redis, delayed tasks can be triggered more efficiently and with higher precision.

[0072] In this embodiment, the step of storing the first delayed task to the time wheel includes:

[0073] The target time slot corresponding to the first delayed task is determined based on the push time of the first delayed task;

[0074] Store the first delayed task in the target time slot corresponding to the time wheel.

[0075] When adding a delayed task, the target time slot (which time slot to place) of the first delayed task is calculated based on the time difference between the push time of the first delayed task and the current time, and the first delayed task is stored in the corresponding target time slot.

[0076] Assuming the time wheel is divided into 60 time slots, each representing 1 second, and the current time is 14:15:01, with the pointer pointing to slot1, and the push time for the first delayed task being 14:16:02, then the target time slot is calculated to be slot2, and the first delayed task is stored in slot2. Alternatively, if the time wheel is divided into 60 time slots, each representing 1 second, a simpler approach is to directly determine the target time slot based on the number of seconds since the first delayed task was pushed. For example, if the push time for the first delayed task is 14:16:02, and the number of seconds since that push time is 2, then the target time slot is determined to be slot2.

[0077] In this embodiment, storing the first delayed task using a time wheel enables rapid processing of delayed tasks, reducing the business processing pressure under large data volume conditions.

[0078] Step S203: Synchronize the task data table to the big data database to obtain synchronized data.

[0079] In this embodiment, the task data table in the relational database is synchronized to the big data database, and the big data is used for analysis, which can improve the processing efficiency of delayed tasks.

[0080] Specifically, a big data computing framework is used to retrieve task data tables from a relational database, and the task data tables are stored in a big data storage file for synchronization.

[0081] The big data computing framework used is Spark, an in-memory computing framework that reliably, efficiently, and scalably processes large amounts of data in a distributed manner. Using Spark for big data processing can improve processing speed. The big data storage files are HDFS files, a commonly used storage system in the big data field. HDFS is a distributed file system where data exists in the form of files.

[0082] In this embodiment, the task data table in the relational database is synchronized to HDFS, which means that the task data table is synchronized to the big data database.

[0083] It should be noted that the big data synchronization operation is performed at a predetermined time. In this embodiment, the task data table is synchronized to the big data at 0:00 every day.

[0084] Step S204: Filter the delayed tasks in the synchronized data using a partitioning method to obtain the third delayed task, and store the third delayed task in the time wheel.

[0085] In this embodiment, the DPP (Dynamic Partition Pruning) feature of the big data computing framework Spark is used to trigger a batch data analysis at a predetermined time (0:00 every day) and select the delayed tasks triggered on that day as the third delayed task.

[0086] In some alternative implementations, the steps described above for filtering delayed tasks in the synchronized data using a partitioning method to obtain the third delayed task include:

[0087] Based on the push time, the task data table in the big data storage file is filtered using a partitioning method;

[0088] Select the delayed tasks that meet the criteria as the third delayed task.

[0089] The partitioning method of the big data computing framework is Spark's DPP feature, which is used to filter the synchronized data.

[0090] It should be understood that each delayed task in the task data table contains a pushTime field, meaning that each delayed task in the synchronized data also contains a pushTime field. When the scheduled time (0:00 every day) arrives, the filtering operation is pushed down to the data source by DPP according to the filter condition pushTime, skipping those partition directories that do not meet the filter conditions, and directly filtering out all delayed tasks that meet the condition pushTime = the day.

[0091] The third delayed task is stored in the time wheel using the same method as the first delayed task, which will not be described again here.

[0092] In this embodiment, the big data tools DPP and HDFS are used to handle massive latency tasks without performance bottlenecks.

[0093] Step S205: When the time wheel drives to the current time slot, the currently triggered delayed tasks in the time wheel corresponding to the current time slot are matched to form a delayed task list.

[0094] In this embodiment, the time wheel is driven at a preset rate. When the time wheel pointer is driven to a certain time slot, all delayed tasks under that time slot are retrieved. The time interval is calculated based on the push time of the first delayed task and the current time. Based on the cycle period of the time wheel and the time interval, the currently triggered delayed task is determined.

[0095] Specifically, based on the time interval and the cycle period of the time wheel, the number of execution rounds (round) of all delayed tasks under this time slot is calculated. Delayed tasks with round=0 are taken out as the currently triggered delayed tasks, while the rest remain in this time slot.

[0096] For example, slot 10 has three delayed tasks. Task 1 was pushed at 13:54:10 on April 24, 2022, Task 2 was pushed at 13:55:10 on April 24, 2022, and Task 3 was pushed at 13:54:10 on April 25, 2022. Now it is 13:54:10. The pointer jumps to slot 10, retrieves these three tasks, and calculates the number of rounds for each delayed task. The results are: Task 1: round = 0, Task 2: round = 1, Task 3: round = 24 * 60. Therefore, Task 1 is selected as the currently triggered delayed task.

[0097] This embodiment uses a time wheel to obtain currently triggered delayed tasks, thereby improving triggering accuracy and task acquisition accuracy.

[0098] It should be emphasized that, to further ensure the privacy and security of the delayed task list, the aforementioned delayed task list can also be stored in a node of a blockchain.

[0099] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0100] Step S206: Push the list of delayed tasks to the message middleware so that the delayed tasks in the list can be consumed.

[0101] In this embodiment, a list of delayed tasks consisting of all currently triggered delayed tasks is obtained from the Redis time wheel and pushed to the message middleware Kafka. The message middleware Kafka pushes the list to the corresponding business end, and the business end receives the task triggering request and completes the business triggering.

[0102] In this embodiment, the business side only needs to produce one Kafka message (send a delayed task request) and consume one Kafka message (trigger the business at the push time), which is simple to use and has a simple process.

[0103] This application stores delayed tasks using a time wheel and a relational database, providing large storage capacity. Delayed tasks are added based on their push time, allowing for dynamic setting of delay times without prior specification. Pushing via the time wheel can be accurate to the second, supporting high-precision triggering. Furthermore, it is not limited by time span.

[0104] In some optional implementations of this embodiment, the step of storing the second delayed task in a relational database to obtain a task data table includes:

[0105] Extract key information from the second delayed task;

[0106] The key information is written into the corresponding key fields of the data table in the relational database to obtain the task data table.

[0107] Key information includes, but is not limited to, business-side identifier, delayed task ID, push time, and task content. The relational database tables contain corresponding key fields. Key information is joined with these key fields using a join operation. After the join, the corresponding key information is written into the data table, thus obtaining the task data table.

[0108] In this embodiment, the relational database is TiDB. The second delayed task is stored in TiDB, which makes the storage capacity larger and the dynamic expansion and contraction of storage is imperceptible to the business.

[0109] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0111] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0112] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a delayed task processing device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0113] like Figure 3 As shown, the delayed task processing device 300 described in this embodiment includes: a receiving module 301, a storage module 302, a synchronization module 303, a filtering module 304, a matching module 305, and a push module 306.

[0114] in:

[0115] The receiving module 301 is used to receive the delayed task request from the message middleware and parse the delayed task request to obtain the push time.

[0116] The storage module 302 is used to determine a first delayed task and a second delayed task based on the push time, store the first delayed task in the time wheel, and store the second delayed task in the relational database to obtain a task data table;

[0117] Synchronization module 303 is used to synchronize the task data table to the big data database to obtain synchronized data;

[0118] The filtering module 304 is used to filter the delayed tasks in the synchronization data by a partitioning method to obtain the third delayed task, and store the third delayed task in the time wheel;

[0119] The matching module 305 is used to match the currently triggered delayed tasks in the time wheel corresponding to the current time slot when the time wheel drives to the current time slot, and form a delayed task list;

[0120] The push module 306 is used to push the list of delayed tasks to the message middleware so that the delayed tasks in the list of delayed tasks can be consumed.

[0121] It should be emphasized that, to further ensure the privacy and security of the delayed task list, the aforementioned delayed task list can also be stored in a node of a blockchain.

[0122] Based on the aforementioned delayed task processing device, delayed tasks are stored using a time wheel and a relational database, respectively. The storage capacity is large, and delayed tasks are added according to the push time of the delayed tasks. The delay time can be dynamically set without the need to specify the delay time in advance. The time wheel push can be accurate to the second, supporting high-precision triggering. In addition, it is not limited by the time span.

[0123] In this embodiment, the storage module 302 includes a calculation submodule and a storage submodule. The calculation submodule is used to determine the target time slot corresponding to the first delayed task based on the push time of the first delayed task. The storage submodule is used to store the first delayed task into the target time slot corresponding to the time wheel.

[0124] By storing the first delayed task using a time wheel, the delayed task can be processed quickly, reducing the business processing pressure under the condition of large data volume.

[0125] In this embodiment, the aforementioned delayed task processing device 300 further includes a creation module, which is used to create a preset number of time slots in Redis according to the cycle period, and create pointers in key-value form; and obtain a time wheel based on the time slots and the pointers.

[0126] By storing delayed tasks using a time wheel created in Redis, delayed tasks can be triggered more efficiently and with higher precision.

[0127] In some optional implementations of this embodiment, the storage module 302 further includes an extraction submodule and a writing submodule. The extraction submodule is used to extract key information of the second delayed task; the writing submodule is used to write the key information into the key field corresponding to the data table of the relational database to obtain the task data table.

[0128] In this embodiment, the second delayed task is stored in a relational database, which results in a larger storage capacity and the dynamic expansion and contraction of storage is imperceptible to the business.

[0129] In this embodiment, the synchronization module 303 is further used for:

[0130] The big data computing framework is used to obtain task data tables from the relational database;

[0131] The task data table is stored in a large data storage file for synchronization.

[0132] By synchronizing task data tables from a relational database to a large dataset, the processing efficiency of delayed tasks can be improved.

[0133] In this embodiment, the screening module 304 is further used for:

[0134] Based on the push time, the task data table in the big data storage file is filtered using the partitioning method;

[0135] Select the delayed tasks that meet the criteria as the third delayed task.

[0136] In this embodiment, using a partitioning method to filter the task data table can improve filtering efficiency.

[0137] In some optional implementations, the matching module 305 includes a calculation submodule and a matching submodule. The calculation submodule is used to calculate the time interval based on the push time of the first delayed task and the current time. The matching submodule is used to determine the currently triggered delayed task based on the cycle period of the time wheel and the time interval.

[0138] By using a time wheel to obtain currently triggered delayed tasks, the triggering accuracy and task acquisition accuracy can be improved.

[0139] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0140] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0141] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0142] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for delayed task processing methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0143] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the delayed task processing method.

[0144] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0145] This embodiment implements delayed task processing as described in the above embodiment by having the processor execute computer-readable instructions stored in memory. The method involves storing delayed tasks using a time wheel and a relational database, providing a large storage capacity. Delayed tasks are added based on their push time, allowing for dynamic setting of delay times without prior specification. Pushing via the time wheel provides accuracy down to the second, supporting high-precision triggering. Furthermore, it is not limited by time span.

[0146] This application also provides another implementation, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the delayed task processing method described above. The delayed tasks are stored using a time wheel and a relational database, resulting in a large storage capacity. Delayed tasks are added based on their push time, enabling dynamic setting of delay times without prior specification. Pushing via the time wheel can be accurate to the second, supporting high-precision triggering. Furthermore, it is not limited by time span.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0148] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for handling delayed tasks, characterized in that, Includes the following steps: Receive delayed task requests from message middleware and parse the delayed task requests to obtain the push time; Based on the push time, a first delayed task and a second delayed task are determined. The first delayed task is stored in a time wheel, and the second delayed task is stored in a relational database to obtain a task data table. The first delayed task is the delayed task that needs to be pushed at present, and the second delayed task is the delayed task that needs to be pushed tomorrow or after tomorrow. The task data table is synchronized to the big data database to obtain synchronized data; wherein, the task data table is obtained from the relational database using the big data computing framework Spark, and the task data table is stored in the big data storage file HDFS for synchronization to obtain synchronized data. The delayed tasks in the synchronized data are filtered by a partitioning method to obtain a third delayed task, and the third delayed task is stored in the time wheel; wherein, based on the push time, the dynamic partitioning and pruning feature of the big data computing framework Spark is used to filter the delayed tasks in the synchronized data, and the delayed tasks with the push time of the current day are selected as the third delayed task. When the time wheel drives to the current time slot, it matches the currently triggered delayed tasks in the time wheel that correspond to the current time slot, and forms a delayed task list. The list of delayed tasks is pushed to the message middleware so that the delayed tasks in the list are consumed.

2. The delayed task processing method according to claim 1, characterized in that, The step of storing the first delayed task to the time wheel includes: The target time slot corresponding to the first delayed task is determined based on the push time of the first delayed task; The first delayed task is stored in the target time slot corresponding to the time wheel.

3. The delayed task processing method according to claim 1, characterized in that, The method further includes the following steps before the step of storing the first delayed task to the time wheel: Create a preset number of time slots in Redis based on the cycle period, and create pointers in key-value format; The time wheel is obtained based on the time slot and the pointer.

4. The delayed task processing method according to claim 1, characterized in that, The step of storing the second delayed task into a relational database to obtain a task data table includes: Extract key information from the second delayed task; The key information is written into the corresponding key field of the data table in the relational database to obtain the task data table.

5. The delayed task processing method according to claim 1, characterized in that, The steps for synchronizing the task data table to the big data platform include: The big data computing framework is used to obtain task data tables from the relational database; The task data table is stored in a large data storage file for synchronization.

6. The delayed task processing method according to claim 5, characterized in that, The step of filtering the delayed tasks in the task data table using a partitioning method to obtain the third delayed task includes: Based on the push time, the task data table in the big data storage file is filtered using the partitioning method; Select the delayed tasks that meet the criteria as the third delayed task.

7. The delayed task processing method according to claim 1, characterized in that, The step of matching the currently triggered delayed task in the time wheel corresponding to the current time slot includes: The time interval is calculated based on the push time of the first delayed task and the current time; Based on the cycle period of the time wheel and the time interval, the currently triggered delayed task is determined.

8. A delayed task processing device, characterized in that, include: The receiving module is used to receive delayed task requests from the message middleware and parse the delayed task requests to obtain the push time. The storage module is used to determine a first delayed task and a second delayed task based on the push time, store the first delayed task in a time wheel, and store the second delayed task in a relational database to obtain a task data table; wherein, the first delayed task is the delayed task that needs to be pushed at present, and the second delayed task is the delayed task that needs to be pushed tomorrow or after tomorrow; The synchronization module is used to synchronize the task data table to the big data database to obtain synchronized data. Specifically, the task data table is obtained from the relational database using the big data computing framework Spark, and the task data table is stored in the big data storage file HDFS for synchronization to obtain synchronized data. The filtering module is used to filter delayed tasks in the synchronized data through a partitioning method to obtain a third delayed task, and store the third delayed task in the time wheel; wherein, by utilizing the dynamic partitioning and pruning characteristics of the big data computing framework Spark, a batch data analysis is triggered at a predetermined time to filter out the delayed tasks triggered on the same day as the third delayed task. The matching module is used to match the currently triggered delayed tasks in the time wheel corresponding to the current time slot when the time wheel drives to the current time slot, and form a delayed task list. The push module is used to push the list of delayed tasks to the message middleware so that the delayed tasks in the list of delayed tasks can be consumed.

9. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the delayed task processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the delayed task processing method as described in any one of claims 1 to 7.

Images