Method, system and equipment for realizing service timing delay processing based on Rocket MQ, and medium

Abstract

The invention discloses a method, a system, equipment and a medium for realizing service timing delay processing based on Rocket MQ, belongs to the technical field of information processing, and solves the defects of poor flexibility, resource waste, insufficient scheduling flexibility and incapability of meeting any time precision requirement when a traditional method is used for processing a timing delay message. The defects limit the application of the message queue in some specific service scenes and influence the overall performance and reliability of the system. The method comprises the steps of collecting service data set by a user, extracting delay information in the service data, creating a message producer and generating message content according to the delay information, and creating a delay consumption theme and a corresponding message theme consumer in Rocket MQ message middleware; and according to the delay information in the service data, sending the service data to a corresponding queue of the delayed consumption theme, and carrying out sequential consumption by a message theme consumer according to the delay information. The method is suitable for application scenes such as order system management.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a technology for implementing service timing delay based on RocketMQ. Background Art

[0002] In the face of high-concurrency business scenarios, timed delayed processing is a common business processing method. After the user submits the business data, the program does not process it immediately, but caches it through middleware such as message queues, and consumes the message body to process the business after the delay time is triggered, achieving the effect of timed push.

[0003] With the widespread application of distributed systems and microservice architectures, message queues, as an efficient data exchange middleware, play an important role in decoupling system components and improving system scalability and reliability. RocketMQ, as a high-performance, high-throughput distributed message queue, is widely used in various business scenarios.

[0004] However, in actual applications, message queues such as RocketMQ have limitations in processing timed delayed messages. Specifically, the delayed message function natively supported by RocketMQ usually only provides a limited number of fixed delay levels, such as 1s, 5s, 10s, 30s, 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m, 1h, 2h, etc. This fixed delay level design is difficult to meet the needs of complex business scenarios for delayed messages with arbitrary time precision.

[0005] For example, in an e-commerce system, it may be necessary to automatically close unpaid orders after a certain period of time based on the order payment status. If the order needs to be automatically closed after 35 minutes, and RocketMQ only provides a fixed delay level of 30 minutes, then additional timing processing is required on the consumer side, which not only increases the complexity of the system, but may also introduce additional delays and uncertainties.

[0006] Therefore, the message queue in the prior art has the problem of poor flexibility and inability to meet any time accuracy requirements when processing timed delayed messages. These problems limit the application of message queues in certain specific business scenarios and affect the overall performance and reliability of the system. Summary of the invention

[0007] The present invention proposes a method, system, device and medium for implementing business timed delay processing based on RocketMQ, the purpose of which is to solve the problems that traditional methods usually rely on timed tasks or manual scheduling, have poor flexibility, waste of resources, insufficient scheduling flexibility, and cannot meet any time accuracy requirements when processing timed delay messages. These defects limit the application of message queues in certain specific business scenarios and affect the overall performance and reliability of the system.

[0008] A method for implementing service timing delay processing based on RocketMQ, the method comprising:

[0009] S1: Collect the business data set by the user, extract the delay information in the business data, create a message producer according to the delay information, generate the message content, and create a delay consumption topic in the RocketMQ message middleware, and create a corresponding message topic consumer;

[0010] S2: According to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, and the message topic consumer consumes in sequence according to the delay information to complete the method.

[0011] Furthermore, a preferred solution is provided: the delay information includes: delay processing time, delay message duration and delay level.

[0012] Furthermore, a preferred solution is provided: the message producer is created according to the delayed processing time; according to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, that is, the business data is sent to the corresponding queue of the delayed consumption topic according to the delay level in the business data; the message topic consumer consumes sequentially according to the delayed message duration and the delay level.

[0013] Furthermore, a preferred solution is provided: the delay processing time is determined according to the time when the message is sent and the delay duration set by the user in the business data.

[0014] Furthermore, a preferred solution is provided: the step of generating the message content includes:

[0015] The message producer generates a delay level according to the delayed processing time and the time to receive the business data. The generation method is: the delayed processing time minus the time to receive the business data. The generated delay level is divided into 18 levels. The delay level ranges from 1 second to 2 hours. If the delay level exceeds 2 hours, it is set to 2 hours.

[0016] The message producer sets the service data and the delay level into the delay message body to generate a delay message, and sends the delay message to the corresponding queue of the message middleware according to the delay level.

[0017] Furthermore, a preferred solution is provided: the sequential consumption includes:

[0018] The message topic consumer pulls all message data from the delayed consumption topic and starts multiple consumer threads to process the messages;

[0019] The multiple consumer threads determine, for the message data, whether the current time reaches the consumption time of the message according to the delayed processing time set for the message. If so, the message is consumed; otherwise, the message is resent to the delay queue.

[0020] Furthermore, a preferred solution is provided: when the multiple consumer threads consume messages, if the consumption time is not reached, a new delay level is obtained by subtracting the message delay level and the message receiving time from the delay processing time, and the new delay level is sent to the queue.

[0021] The present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for implementing service timing delay processing based on RocketMQ according to any combination of the above-mentioned solutions.

[0022] The present invention also provides a computer-readable storage medium, which is used to store a computer program, and the computer program executes the steps of a method for implementing service timing delay processing based on RocketMQ according to any one of claims 1 to 7.

[0023] The present invention also provides a system for implementing service timing delay processing based on RocketMQ, the system comprising:

[0024] Delay message production module: used to collect business data set by users, extract delay information from business data, create message producers according to the delay information, generate message content, create delay consumption topics in RocketMQ message middleware, and create corresponding message topic consumers;

[0025] Delayed message consumption module: used to send the business data to the corresponding queue of the delayed consumption topic according to the delay information in the business data, and the message topic consumer consumes in sequence according to the delay information.

[0026] Compared with the prior art, the advantages of the present invention are:

[0027] The present invention describes a method of dynamically generating a RocketMQ delay level in the RocketMQ message middleware according to the delay processing time and message generation time selected by the user, which is stored by different delay queues to achieve a processing effect of a custom delay level, and can meet the business demand for delayed messages of arbitrary time precision, overcome the limitations of the fixed delay level in the prior art, and improve the flexibility of delay processing. The present invention implements refined delay processing on the basis of RocketMQ, reduces the need for additional timing processing on the consumer side, thereby reducing the complexity of the system and improving overall performance and reliability. The present invention can more accurately execute delayed tasks, such as order closing, coupon issuance, etc., according to the time set by the user, thereby improving user experience and satisfaction.

[0028] In summary, the method of implementing business timed delay processing based on RocketMQ shows significant advantages in flexibility, system performance, user experience, etc.

[0029] The present invention is applicable to application scenarios such as order system management. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 It is a flowchart of a method for implementing service timing delay processing based on RocketMQ according to a specific implementation mode of the present invention;

[0032] Figure 2 It is a schematic diagram of a framework of a method for implementing service timing delay processing based on RocketMQ according to a specific implementation mode 1 of the present invention;

[0033] Figure 3 This is a structural diagram of a system for implementing service timing delay processing based on RocketMQ according to a second specific implementation mode of the present invention;

[0034] Figure 4 This is a schematic diagram of the delay levels described in the first specific implementation mode of the present invention. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0036] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0037] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0038] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific implementation methods disclosed below.

[0040] Implementation method 1:

[0041] Reference Figure 1 , Figure 2 , Figure 4 This embodiment will be described.

[0042] A method for implementing service timing delay processing based on RocketMQ, comprising:

[0043] S1: Collect the business data set by the user and extract the delay information in the business data, the delay information includes: delay processing time, delay message duration and delay level; create a message producer according to the delay information, generate message content, and create a delay consumption topic in RocketMQ message middleware, and create a corresponding message topic consumer;

[0044] Furthermore, the delayed consumption topic is a RocketMQ message topic set by the program;

[0045] Furthermore, the message topic consumer is a created RocketMQ message consumer program;

[0046] Furthermore, in this step, the message producer is created according to the delayed processing time;

[0047] Furthermore, the message content includes business data, business timing processing time, submission time, delay level

[0048] S2: According to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, and the message topic consumer consumes in sequence according to the delay information to complete the method.

[0049] Furthermore, according to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, that is, according to the delay level in the business data, the business data is sent to the corresponding queue of the delayed consumption topic; the message topic consumer consumes sequentially according to the delayed message duration and the delay level.

[0050] Furthermore, the delay processing time is determined according to the time when the message is sent and the delay time length set by the user in the business data.

[0051] Furthermore, the step of generating the message content includes:

[0052] The message producer generates a delay level according to the delayed processing time and the time of receiving business data. The generation method is: the delayed processing time is subtracted from the time of receiving business data. The generated delay level is divided into 18 levels. The delay processing time corresponding to the delay level is between 1 second and 2 hours. If the delay time exceeds 2 hours, it is set to the delay level corresponding to 2 hours; Figure 4 As shown, it can be divided into 18 levels of delay queues, corresponding to delay times such as 1s, 5s, 10s, 30s, 1min, 2min, 3min, 4min, 5min, 6min, 7min, 8min, 9min, 10min, 20min, 30min, 1h, 2h, etc.

[0053] The generation method is: delay processing time minus message generation time, and determine whether the calculated time difference is greater than 2 hours. If it is greater than 2 hours, the delay level is set to 18; if it is greater than 1 hour and less than 2 hours, it is set to 17; if it is less than 1 hour and greater than 30 minutes, it is set to 16; if it is less than 30 minutes and greater than 20 minutes, it is set to 15; and so on. By comparing the above calculated time difference with the delay time, the initial delay level corresponding to the message is set.

[0054] The message producer sets the business data and delay level into the delayed message body to generate a delayed message. The message content includes the message generation time, the message timing delay processing time, the message initial delay level and the user business data. The message producer sends the delayed message to the corresponding queue of the message middleware according to the delay level. Specifically: the message middleware sets the message processing program Broker configuration to support the delay level according to the business processing scenario. The Broker will send the delayed message to a new message topic for storing delayed messages, and generate corresponding delay queues in the message topic.

[0055] Furthermore, the sequential consumption includes:

[0056] The message topic consumer pulls all message data from the delayed consumption topic and starts multiple consumer threads to process the messages;

[0057] The multiple consumer threads determine, for the message data, whether the current time reaches the consumption time of the message according to the delayed processing time set for the message. If so, the message is consumed; otherwise, the message is resent to the delay queue.

[0058] Furthermore, when the multiple consumer threads consume messages, if the consumption time is not reached, a new delay level is obtained by subtracting the message delay level and the message receiving time from the delay processing time, and the new delay level is sent to the queue.

[0059] The method for implementing service timed delay processing based on RocketMQ described in this embodiment divides different delay queues and creates threads about delayed messages in RocketMQ message middleware according to the predetermined delay consumption duration to monitor whether there are delayed messages. It can ensure that the messages in each delay queue have the same delay duration, thereby avoiding the problem that consumers may be forced to wait for the longest delayed message before consuming the subsequent messages, causing unnecessary delays, and improving message consumption efficiency.

[0060] To make this implementation easier to understand, please refer to Figure 2 , Figure 2A flow chart of a method for implementing service timing delay processing based on RocketMQ provided in this embodiment may specifically include:

[0061] The message producer generates the message delay level according to the delay processing time set by the business data submitted by the user and the time to receive the business data. The generation rule is the delay processing time minus the time to receive the business data. The generated delay level is divided into 18 levels, ranging from 1 second to 2 hours. The producer generates the timed delay message with the business data and delay level and sends it to RocketMQ.

[0062] The core thread pulls the message data in all delay queues from the delay consumption topic in batches, and starts multiple consumer threads to process the messages concurrently;

[0063] The consumer determines whether the current time reaches the consumption time of the message based on the timer delay processing time set for the message data. If so, the message is consumed. Otherwise, the delay level is recalculated based on the current time minus the message timing time, a delayed message is generated, and resent to the delay queue.

[0064] Implementation method 2:

[0065] Reference Figure 3 This embodiment will be described.

[0066] This embodiment provides a system for implementing service timing delay processing based on RocketMQ, the system comprising:

[0067] Delay message production module: used to collect business data set by users, extract delay information from business data, create message producers according to the delay information, generate message content, create delay consumption topics in RocketMQ message middleware, and create corresponding message topic consumers;

[0068] Delayed message consumption module: used to send the business data to the corresponding queue of the delayed consumption topic according to the delay information in the business data, and the message topic consumer consumes in sequence according to the delay information.

[0069] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0070] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0071] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0072] It will be appreciated by those skilled in the art that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may be provided in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Furthermore, the present disclosure may be provided in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present disclosure rather than to limit its protection scope. Although the present disclosure has been described in detail with reference to the above implementation modes, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation modes of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the disclosed claims to be approved.

Claims

1. A method for implementing service timing delay processing based on RocketMQ, characterized in that: The method comprises: S1: Collect the business data set by the user, extract the delay information in the business data, create a message producer according to the delay information, generate the message content, and create a delay consumption topic in the RocketMQ message middleware, and create a corresponding message topic consumer; S2: According to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, and the message topic consumer consumes in sequence according to the delay information to complete the method.

2. According to a method for implementing service timing delay processing based on RocketMQ according to claim 1, it is characterized in that: The delay information includes: delay processing time, delay message duration and delay level.

3. According to a method for implementing service timing delay processing based on RocketMQ according to claim 2, it is characterized in that: The message producer is created according to the delayed processing time; according to the delay information in the business data, the business data is sent to the corresponding queue of the delayed consumption topic, that is, the business data is sent to the corresponding queue of the delayed consumption topic according to the delay level in the business data; the message topic consumer consumes sequentially according to the delayed message duration and delay level.

4. According to a method for implementing service timing delay processing based on RocketMQ according to claim 2, it is characterized in that: The delay processing time is determined according to the time when the message is sent and the delay time length set by the user in the service data.

5. According to a method for implementing service timing delay processing based on RocketMQ according to claim 2, it is characterized in that: The step of generating message content comprises: The message producer generates a delay level according to the delayed processing time and the time to receive the business data. The generation method is: the delayed processing time minus the time to receive the business data. The generated delay level is divided into 18 levels. The delay level ranges from 1 second to 2 hours. If the delay level exceeds 2 hours, it is set to 2 hours. The message producer sets the service data and the delay level into the delay message body to generate a delay message, and sends the delay message to the corresponding queue of the message middleware according to the delay level.

6. A method for implementing service timing delay processing based on RocketMQ according to claim 5, characterized in that: The sequential consumption includes: The message topic consumer pulls all message data from the delayed consumption topic and starts multiple consumer threads to process the messages; The multiple consumer threads determine, for the message data, whether the current time reaches the consumption time of the message according to the delayed processing time set for the message. If so, the message is consumed; otherwise, the message is resent to the delay queue.

7. A method for implementing service timing delay processing based on RocketMQ according to claim 6, characterized in that: When the multiple consumer threads consume messages, if the consumption time has not been reached, a new delay level is obtained by subtracting the message delay level and the message receiving time from the delay processing time, and the new delay level is sent to the queue.

8. Computer device, characterized in that It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a method for implementing service timing delay processing based on RocketMQ according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program executes the steps of a method for implementing service timing delay processing based on RocketMQ according to any one of claims 1 to 7.

10. A system for implementing service timing delay processing based on RocketMQ, characterized in that: The system comprises: Delay message production module: used to collect business data set by users, extract delay information from business data, create message producers according to the delay information, generate message content, create delay consumption topics in RocketMQ message middleware, and create corresponding message topic consumers; Delayed message consumption module: used to send the business data to the corresponding queue of the delayed consumption topic according to the delay information in the business data, and the message topic consumer consumes in sequence according to the delay information.

Images