Service delay pushing method
By deploying MySQL and Redis queues in the cluster, using the delay queue function of the Redisson client, the problem of inaccurate delay push point-in-time control during data interaction is solved, precise control and high availability are achieved, and service quality is improved.
Patent Information
- Application Number
- CN202510311047.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-06-13
AI Technical Summary
In the process of data interaction, due to the huge amount of data, the delayed push time control is inaccurate, resulting in a large number of error logs, affecting service quality, and may cause data consistency problems during cluster deployment.
By deploying MySQL and Redis queues in the cluster, delayed queue functions provided by the Redisson client are used to achieve delayed push. The Redisson client relies on Redis's ordered collection and list data structure, combining Lua scripts and time wheel mechanisms to provide safe and reliable delay task processing.
It realizes precise control of delayed push time points, avoids multiple retry and data consistency problems, improves service quality, and ensures high availability and disaster recovery.
Smart Images
Figure CN120144679A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of software development, and particularly relates to a method for delaying business push. Background Art
[0002] With the development of the information industry, the construction of one - thing services for various types of service platforms is in full swing. When constructing one - thing services, since one - thing services are composed of single - item combinations among multiple departments, it will inevitably involve data interaction among multiple different departments, and may also involve data interaction with different application systems. During the data interaction process, due to the large amount of data in many application systems, it takes a certain amount of time for data synchronization. Therefore, after the completion of the current item, the processing information of the next item cannot be immediately pushed. It is necessary to wait for a certain period of time until the data synchronization of the application system is completed before the data can be pushed. Currently, the common practice is to set a push at regular time intervals until the interface successfully returns.
[0003] The above - described solution for delayed push is based on database polling and interface retry mechanisms, and its disadvantages are obvious:
[0004] Firstly, the control of time points is inaccurate. When polling database records, only a fixed time can be set, and it is impossible to accurately control the time points for each item to be executed.
[0005] Secondly, due to multiple retries of the interface, a large number of error logs will be generated, affecting the service quality.
[0006] Finally, when performing cluster deployment, the timing task needs to consider the data consistency problem. Improper use of distributed locks is likely to cause deadlocks or other exceptions.
[0007] Based on the above problems, the present invention proposes a method for delaying business push. Summary of the Invention
[0008] In order to make up for the deficiencies of the prior art, the present invention provides a simple and efficient method for delaying business push.
[0009] The present invention is implemented through the following technical solutions:
[0010] A method for delaying business push, characterized by comprising the following steps:
[0011] Step S1, deploy a relational database MySQL and a Redis queue in the cluster. The Redis queue is responsible for providing the underlying technical support for delay processing, and the relational database MySQL is responsible for providing ultimate reliability guarantee and disaster recovery function when the Redis queue fails;
[0012] Step S2: Introduce the relevant dependencies of the open-source framework Redisson in the project. As a client for distributed coordination of Redis queues, Redisson is responsible for providing a safe and reliable shortcut method for the delay function;
[0013] Step S3: Customize and configure the delay push time for each item according to the actual business requirements;
[0014] Step S4: When the project starts, register a JavaBean for the delay queue in the container and customize the execution methods corresponding to each delay type;
[0015] Step S5: When receiving the completion message pushed by the business department, if there is a business that needs to be pushed to the next level, determine whether delay push is required. If delay push is not required, do nothing and still push the lower-level business according to the old logic. If delay push is required, first store the corresponding business data in the relational database MySQL, set the business status to pending push, and then put the business into the delay queue, using the custom type and business number as the key, and set the delay time according to the configuration;
[0016] When the delay time arrives, the program matches the corresponding execution method according to the type contained in the key, passes the business number as a parameter, calls the method to execute, and changes the business status in the relational database MySQL to pushed after the execution is completed.
[0017] The delay queue (RDelayedQueue) of the Redisson client depends on the sorted set and list data structures of Redis, as well as the Lua script and time wheel mechanism for implementation;
[0018] In addition, the Redisson client uses the publish / subscribe mode of the Redis queue and the delay queue of the JDK to achieve more efficient delay task processing.
[0019] The delay queue of the Redisson client consists of three parts: a delay queue of zset type and two blocking queues of list type;
[0020] Among them, the delay queue of zset type is used to store tasks with execution times, and each task has an associated score representing its scheduled execution time; when the task reaches the scheduled execution time, the corresponding task will be removed from the delay queue of zset type and pushed into the processing queue.
[0021] When the Redisson client starts, it first subscribes to a key, and at the same time listens to a blocking queue using the BLPOP key 0 command until data is returned;
[0022] When a task is placed in the delay queue, the Redisson client first puts the data into a zset collection, and at the same time publishes the subscribed key above, with the content being the expiration time; at the same time, the Redisson client starts a delay task, and the delay time is the publishing time;
[0023] When the delay task of the Redisson client reaches the execution time, it takes out the data from the zset-type delay queue and puts it into the blocking queue. At this time, the Redisson client returns the corresponding data, calls back the take() method, and executes the specified business method.
[0024] In the Java language, during initialization, use the Redisson client to construct a delay queue and expose 2 methods externally, which are used to add the mapping between types and execution methods and add tasks respectively;
[0025] When the business JavaBean is initialized, call the addConsumerFuction method to add the mapping between types and execution methods;
[0026] If you need to add tasks during the business execution process, call the addTask method.
[0027] Create a separate thread to consume the delay task. When there is no task in the queue, the take() method will block until a task returns; at this time, take out the corresponding execution method according to the type from the consumer function mapping consumerFunctionMap for invocation, and record and operate on the records of the relational database MySQL according to business requirements.
[0028] A business delay push device, characterized in that: it includes a memory and a processor; the memory is used to store computer programs, and the processor is used to implement the above method steps when executing the computer programs.
[0029] A readable storage medium, characterized in that: a computer program is stored on the readable storage medium, and the computer program implements the above method steps when executed by a processor.
[0030] The beneficial effect of the present invention is that: the business delay push method provides a general and reliable solution for delaying the push of a large amount of business data, and can freely adjust the time for delaying the push to the lower level according to the time required for data synchronization. While ensuring the accuracy and customization of the business push time, it can also avoid sending a large number of useless requests, improve the service quality, and ensure high availability, reliability and disaster recovery. Description of the Drawings
[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.
[0032] Appendix Figure 1 It is a schematic diagram of the business delay push method of the present invention. Specific implementation manners
[0033] In order to enable those skilled in the art of this technology to better understand the technical solutions in the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in combination with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0034] The business delay push method includes the following steps:
[0035] Step S1: Deploy a relational database MySQL and a Redis queue in the cluster. The Redis queue is responsible for providing the underlying technical support for delay processing, and the relational database MySql is responsible for providing the final reliability guarantee and disaster recovery function when the Redis queue fails;
[0036] When the Redis cluster fails, the data in Redis may be lost. At this time, disaster recovery can be performed through the records in the MySQL database.
[0037] Step S2: Introduce the relevant dependencies of the open-source framework Redisson in the project. Redisson, as a client for distributed coordination of the Redis queue, is responsible for providing a safe, reliable, and fast method for the delay function;
[0038] Step S3: Customize and configure the delay push time for each matter according to the actual business requirements;
[0039] Step S4: When the project starts, register a JavaBean of a delay queue in the container, and customize and set the execution methods corresponding to each delay type;
[0040] Step S5: When receiving the completion message pushed by the business department, if there is a business that needs to be pushed to the next level, it is judged whether a delayed push is required. If a delayed push is not required, no processing is done, and the old logic is still used to push the subordinate business. If a delayed push is required, the corresponding business data is first stored in the relational database MySQL, and the business status is set to pending push. Then, the business is placed in the delay queue, with the custom type and business number as the key, and the delay time is set according to the configuration.
[0041] When the delay time arrives, the program matches the corresponding execution method according to the type included in the key, passes the business number as a parameter, calls the method to execute, and changes the business status in the relational database MySQL to pushed after the execution is completed.
[0042] The delay queue (RDelayedQueue) of the Redisson client depends on the sorted set and list data structures of Redis, as well as the Lua script and time wheel mechanism for implementation.
[0043] In addition, the Redisson client uses the publish / subscribe mode of the Redis queue and the delay queue of the JDK to achieve more efficient delay task processing.
[0044] The delay queue of the Redisson client consists of three parts: a delay queue of zset type and two blocking queues of list type.
[0045] Among them, the delay queue of zset type is used to store tasks with execution times. Each task has an associated score, representing its scheduled execution time. When the task reaches the scheduled execution time, the corresponding task will be removed from the delay queue of zset type and pushed into the processing queue.
[0046] When the Redisson client starts, it subscribes to a key first, and at the same time listens to a blocking queue with the BLPOP key 0 command until data is returned.
[0047] When a task is placed in the delay queue, the Redisson client first puts the data into a zset collection, and at the same time publishes the key subscribed above, with the content being the expiration time. At the same time, the Redisson client starts a delay task, and the delay time is the publish time.
[0048] When the delay task of the Redisson client reaches the time to execute, it retrieves the data from the delay queue of zset type and puts it into the blocking queue. At this time, the Redisson client returns the corresponding data, calls back the take() method, and executes the specified business method.
[0049] In the Java language, during initialization, a Redisson client is used to construct a delay queue, and two methods are exposed externally, which are used to add the mapping of types and execution methods and add tasks respectively;
[0050] When the business JavaBean is initialized, the addConsumerFuction method is called to add the mapping of types and execution methods;
[0051] If tasks need to be added during business execution, the addTask method is called.
[0052] A separate thread is created to consume delay tasks. When there are no tasks in the queue, the take() method will block until a task is returned; at this time, the corresponding execution method is retrieved from the consumer function mapping consumerFunctionMap according to the type and called, and the relational database MySQL records are recorded and operated according to business requirements.
[0053] The business delay push device includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method steps when executing the computer program.
[0054] A computer program is stored on the readable storage medium, and the computer program implements the above method steps when executed by a processor.
[0055] The business delay push method can ensure the precise control of the delay push time point, and the execution time point of each case of each matter is accurate to the second level; at the same time, it can also ensure that there will be no multiple retries. Professional personnel can estimate the data synchronization completion time, and the interface can be successful once without retry; finally, it can also avoid the consistency problems that may occur in a distributed environment, greatly improving the service quality.
[0056] The above-described embodiments are only one of the specific implementation manners of the present invention, and the common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.
Claims
1. A delayed service push method, characterized in that: The following steps are involved: Step S1: Deploy relational database MySQL and Redis queue in the cluster. Redis queue is responsible for providing underlying technical support for delay processing, and relational database MySql is responsible for providing disaster recovery function when Redis queue fails. Step S2: Introduce the open source framework Redisson related dependencies into the project. Redisson, as the client of the distributed coordination Redis queue, is responsible for providing a shortcut method for the delay function; Step S3: Customize the time for delayed push of each item according to actual business needs; Step S4: When the project starts, a JavaBean of a delay queue is registered in the container, and the execution method corresponding to each delay type is customized; Step S5: When receiving the completion message pushed by the business department, if there is a business at the next level that needs to be pushed, determine whether delayed push is required. If delayed push is required, first store the corresponding business data in the relational database MySQL, and set the business status to be pushed, then put the business into the delayed queue, use the custom type and business number as the key, and set the delay time according to the configuration; When the delay time is reached, the program matches the corresponding execution method according to the type contained in the key, passes the business number as a parameter, calls the method for execution, and changes the business status of the relational database MySQL to pushed after the execution is completed.
2. The delayed service push method according to claim 1, characterized in that: The delay queue of the Redisson client relies on the ordered set and list data structures of Redis, as well as the Lua script and time wheel mechanism implementation; In addition, the Redisson client uses the publish / subscribe mode of the Redis queue and the delay queue of the JDK to implement delayed task processing.
3. The delayed service push method according to claim 2, characterized in that: The delay queue of the Redisson client consists of three parts: a zset-type delay queue and two list-type blocking queues; Among them, the zset type delay queue is used to store tasks with execution time. Each task has an associated score, which represents its scheduled execution time. When a task reaches the scheduled execution time, the corresponding task will be removed from the zset type delay queue and pushed to the processing queue.
4. The delayed service push method according to claim 3, characterized in that: When the Redisson client starts, it first subscribes to a key and uses the BLPOP key 0 command to listen to a blocking queue until data is returned. When a task is placed in the delay queue, the Redisson client first puts the data into a zset collection and publishes the subscribed key with the expiration time. At the same time, the Redisson client starts a delay task with the delay time being the publishing time. When the delayed task of the Redisson client is executed on time, the data is taken out from the delayed queue of the zset type and placed in the blocking queue. At this time, the Redisson client returns the corresponding data, calls back the take() method, and executes the specified business method.
5. The delayed service push method according to claim 4, characterized in that: In Java language, during initialization, a delay queue is constructed using the Redisson client, and two methods are exposed to the outside world, one for adding a mapping between types and execution methods, and the other for adding tasks. When the business JavaBean is initialized, the addConsumerFuction method is called to add the mapping between the type and the execution method; If you need to add a task during business execution, call the addTask method.
6. The delayed service push method according to claim 5, characterized in that: Create a separate thread to consume delayed tasks. When there are no tasks in the queue, the take() method will be blocked until a task returns. At this time, the corresponding execution method is taken out from the consumer function map consumerFunctionMap according to the type and called, and the relational database MySQL records are recorded and operated according to business needs.
7. A delayed service push device, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the method steps as claimed in any one of claims 1 to 6 when executing the computer program.
8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 6 are implemented.