Message processing method, device, platform and storage medium

Abstract

The present invention discloses a message processing method, device, platform and storage medium. The method comprises: obtaining a message to be sent and a delay duration of the corresponding delayed sending; dividing the delay duration into at least two sub-durations; based on the at least two sub-durations, placing the message to be sent into a delay queue corresponding to each sub-duration to store the corresponding sub-duration; after the delay queue corresponding to the corresponding sub-duration stores the corresponding sub-duration, taking out the message to be sent and placing it into a delay queue corresponding to another sub-duration for storage; after the total duration stored in the delay queue reaches the delay duration, sending the message to be sent.

Description

Technical Field

[0001] The present invention relates to the technical field of information management, and in particular to a message processing method, device, platform and storage medium. Background Art

[0002] In daily life, there are often scenarios involving delayed message sending, which can be called delayed messages. Here, the so-called "delayed sending" means that after the producer generates a message, the consumer cannot receive the message immediately, but waits for a specified period of time before the consumer can receive the message and process it accordingly.

[0003] In the related art, in order to achieve delayed sending of messages, some delay queues that can delay for a fixed period of time can be set, and the messages that need to be delayed are delivered to the delay queues, and the messages are delayed.

[0004] However, in the related art, the method of sending messages with delay still needs to be optimized. Summary of the invention

[0005] To solve related technical problems, embodiments of the present invention provide a message processing method, device, platform and storage medium.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] An embodiment of the present invention provides a message processing method, including:

[0008] Get the message to be sent and the corresponding delay time for delayed sending;

[0009] Splitting the delay duration into at least two sub-durations;

[0010] Based on the at least two sub-periods, the message to be sent is placed in a delay queue corresponding to each sub-period to store the corresponding sub-period; wherein, after the delay queue corresponding to the corresponding sub-period stores the corresponding sub-period, the message to be sent is taken out and placed in a delay queue corresponding to another sub-period for storage;

[0011] After the total duration stored in the delay queue reaches the delay duration, the message to be sent is sent.

[0012] In the above solution, splitting the delay duration into at least two sub-durations includes:

[0013] The delay duration is divided into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; the first parameter represents the duration for which the message must be stored in the corresponding delay queue.

[0014] In the above solution, the delay duration is divided into at least two sub-durations according to the first parameter of at least one delay queue, including:

[0015] According to the first parameter of each delay queue in at least one delay queue and in combination with the splitting strategy, the delay duration is split into at least two sub-durations.

[0016] In the above scheme, based on the at least two sub-time lengths, delivering the message to be sent to the delay queue corresponding to each sub-time length to store the corresponding sub-time length includes:

[0017] Sorting the at least two sub-durations to obtain a sorting result;

[0018] According to the sorting result, the messages to be sent are sequentially placed in the delay queue corresponding to the corresponding sub-duration and the corresponding sub-duration is stored.

[0019] In the above solution, the obtaining of the message to be sent includes:

[0020] Receive messages sent by producers with a delay time less than or equal to the preset time.

[0021] or,

[0022] Obtain from the database the messages to be sent whose delay duration is less than or equal to the preset duration.

[0023] In the above scheme, the method further comprises:

[0024] The messages to be sent by the producer with a delay time longer than the preset time are stored in the database.

[0025] In the above scheme, the method further comprises:

[0026] Obtaining scene information of the message to be sent;

[0027] The sending of the message to be sent includes:

[0028] Determine at least one consumer corresponding to the message to be sent based on the scenario information;

[0029] The message to be sent is sent to the at least one consumer.

[0030] The embodiment of the present invention further provides a message processing device, including:

[0031] The acquisition module is used to obtain the message to be sent and the delay time of the corresponding delayed sending;

[0032] A splitting module, used for splitting the delay duration into at least two sub-durations;

[0033] A delivery module, configured to place the message to be sent into a delay queue corresponding to each sub-time length based on the at least two sub-time lengths, and store the corresponding sub-time length; wherein, after the delay queue corresponding to the corresponding sub-time length stores the corresponding sub-time length, the message to be sent is taken out and placed into a delay queue corresponding to another sub-time length for storage;

[0034] The sending module is used to send the message to be sent after the total time length stored in the delay queue reaches the delay time length.

[0035] The embodiment of the present invention further provides a message processing platform, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein:

[0036] The processor is used to execute the steps of any of the above methods when running the computer program.

[0037] An embodiment of the present invention further provides a storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0038] The message processing method, device, platform and storage medium provided by the embodiment of the present invention obtain the message to be sent and the delay duration of the corresponding delayed sending; split the delay duration into at least two sub-durations; based on the at least two sub-durations, place the message to be sent in the delay queue corresponding to each sub-duration to store the corresponding sub-duration; wherein, after the delay queue corresponding to the corresponding sub-duration stores the corresponding sub-duration, take out the message to be sent and place it in the delay queue corresponding to another sub-duration for storage; after the total duration stored in the delay queue reaches the delay duration, send the message to be sent. The scheme of the embodiment of the present invention, by splitting the delay duration into several sub-durations according to the delay duration of the message to be sent, so that the message to be sent is delivered to the queues corresponding to several sub-durations for delay, thus realizing a delay queue based on a preset delay preset duration, and being able to delay the delayed message for any duration. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a message processing method according to an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the system composition of an application embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the architecture layer for implementing the delay platform function in an application embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of the delayed flow process of an application embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of a message processing device according to an embodiment of the present invention;

[0044] Figure 6 The figure is a schematic diagram of the hardware structure of the message processing platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0046] Before describing the embodiments of the present invention, the following terms are explained:

[0047] Generally, a delayed message (also called a delayed message) is a message that cannot be consumed immediately by a consumer (an application that receives and processes the received message) after the producer (which can be understood as the application that generates the message to be sent) generates the message (i.e., it is received and processed immediately), but needs to wait until the specified sending time before it can be consumed. In actual applications, a delay queue is usually used to implement message delay. Among them, a delay queue (also called a delayed message queue) has the following three properties:

[0048] First, the delay queue is a queue and has queue functions;

[0049] Second, the delay queue has a delay function;

[0050] Third, the delay queue stores messages.

[0051] In the process of delaying the message to be sent, after the message is placed in the delay queue, the message can only be obtained from the delay queue after the delay time corresponding to the delay queue is reached, otherwise it cannot be obtained. There are many application scenarios for delayed messages. For example, after a user places an order, he needs to pay within a specified time (for example, 30 minutes). Before the specified time arrives, a message can be sent to remind the user to pay; after the order is generated, a text message is sent 1 minute later (the message is generated by the producer) to remind the user to pay, and after the text message is sent, another text message is sent 1 minute later to remind the user to pay.

[0052] At present, the task scheduling middleware (also known as message middleware) in the open source environment can only set a limited number of delay queues, and each delay queue only supports a fixed duration corresponding to the delay. Therefore, the task scheduling middleware in the existing open source environment cannot support the specified delay of any duration, and only supports the delay tasks of a fixed period of time, and cannot support the specified message delay of any time. Take RocketMQ (a message middleware product name) message middleware as an example. RocketMQ will set up ordinary queues and delay queues, among which the number of delay queues is 18, and the 18 delay queues correspond to 18 fixed-duration delays. After RocketMQ obtains the delayed message, it will put the delayed message into the corresponding delay queue for a fixed delay according to the delay level corresponding to the delayed message. After the delay is completed, the message will be put into the ordinary queue and sent to the consumer. Since RocketMQ only has 18 delay queues, RocketMQ only supports 18 fixed-duration delays, and cannot support the specified delay of any time.

[0053] Based on this, in various embodiments of the present invention, a message to be sent and a corresponding delay duration for delayed sending are obtained; the delay duration is divided into at least two sub-durations; based on the at least two sub-durations, the message to be sent is placed in a delay queue corresponding to each sub-duration and stored in the corresponding sub-duration; wherein, after the delay queue corresponding to the corresponding sub-duration stores the corresponding sub-duration, the message to be sent is taken out and placed in a delay queue corresponding to another sub-duration for storage; after the total duration stored in the delay queue reaches the delay duration, the message to be sent is sent.

[0054] The solution of the embodiment of the present invention is to split the delay duration into several sub-durations according to the delay duration of the message to be sent, so that the message to be sent is delivered to the queues corresponding to the several sub-durations for delay. In this way, a delay queue based on preset delay durations is realized, which can delay the delayed message for any duration.

[0055] The embodiment of the present invention provides a message processing method, which is applied to a message processing platform, such as Figure 1 As shown, the method includes:

[0056] Step 101: Obtain the message to be sent and the corresponding delayed sending delay duration;

[0057] Step 102: Split the delay duration into at least two sub-durations;

[0058] wherein each sub-duration in at least one sub-duration is the same or different;

[0059] Step 103: Based on the at least two sub-durations, the message to be sent is placed in a delay queue corresponding to each sub-duration and the corresponding sub-duration is stored;

[0060] After the corresponding sub-duration is stored in the delay queue corresponding to the corresponding sub-duration, the message to be sent is taken out and placed in the delay queue corresponding to another sub-duration for storage;

[0061] Step 104: After the total duration stored in the delay queue reaches the delay duration, the message to be sent is sent.

[0062] In actual application, after receiving the message to be sent, the message processing platform obtains the corresponding delay time, where the delay time can be any time.

[0063] In actual application, in one embodiment, splitting the delay duration into at least two sub-durations includes:

[0064] The delay duration is divided into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; the first parameter represents the duration for which the message must be stored in the corresponding delay queue.

[0065] In actual application, the message processing platform may pre-set at least one delay queue, each delay queue corresponds to a first parameter, and the first parameter of each delay queue is different.

[0066] In actual application, the delay duration can be split based on the splitting strategy.

[0067] Based on this, in one embodiment, the delay duration is split into at least two sub-durations according to the first parameter of at least one delay queue, including:

[0068] According to the first parameter of each delay queue in at least one delay queue and in combination with the splitting strategy, the delay duration is split into at least two sub-durations.

[0069] In actual application, multiple splitting strategies can be set according to the delay requirements.

[0070] Exemplarily, in order to reduce the number of times a message is delivered in a queue, a sub-duration with the longest duration can be split from the delay duration each time according to the delay duration and the first parameter corresponding to at least one delay queue (for example, the first parameters of the 18 delay queues are set to be 1 second, 5 seconds, 10 seconds, 30 seconds, 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 20 minutes, 30 minutes, 1 hour, and 2 hours, respectively; when the delay duration is 16 seconds, according to the set first parameter, the longest duration that can be split is 10 seconds when the first split is performed, the longest duration that can be split is 5 seconds when the second split is performed, and the longest duration that can be split is 1 second when the third split is performed), so that after at least two splits, at least two sub-durations are obtained; in order to reduce delay decomposition The amount of calculation brought about can also be randomly split from the delay time according to the first parameter corresponding to the delay time and at least one delay queue, so that at least two sub-times are obtained after at least two splits; in order to avoid the message delay time being too long each time, a sub-time with the shortest time can be split from the delay time according to the first parameter corresponding to the delay time and at least one delay queue, so that at least two sub-times are obtained after at least two splits; in order to reduce the queue capacity of the delay queue (referring to the maximum value of the amount of messages that can be stored in the delay queue) and optimize the queue processing speed, a sub-time can be split from the delay time according to the queue capacity of each delay queue, and the queue capacity corresponding to the sub-time is the smallest, so that at least two sub-times are obtained after at least two splits. The embodiment of the present invention does not limit the splitting strategy.

[0071] Accordingly, based on the set splitting strategy, after obtaining at least two sub-durations, the messages to be sent can be delivered to the delay queues corresponding to the sub-durations in a certain order.

[0072] In one embodiment, delivering the message to be sent to a delay queue corresponding to each sub-duration based on the at least two sub-durations to store the corresponding sub-duration includes:

[0073] Sorting the at least two sub-durations to obtain a sorting result;

[0074] According to the sorting result, the messages to be sent are sequentially placed in the delay queue corresponding to the corresponding sub-duration and the corresponding sub-duration is stored.

[0075] Here, in actual application, the sub-time lengths can be sorted according to their duration to obtain a sorting result. For example, the at least two sub-time lengths can be sorted in order from long to short to obtain a sorting result; the at least two sub-time lengths can also be sorted in order from short to long to obtain a sorting result; the at least two sub-time lengths can also be sorted randomly to obtain a sorting result.

[0076] In actual application, the queue capacity corresponding to each sub-time length may be sorted to obtain a sorting result. For example, the at least two sub-time lengths may be sorted in order from small to large in terms of queue capacity to obtain a sorting result; or the at least two sub-time lengths may be randomly sorted according to the queue capacity to obtain a sorting result.

[0077] In actual application, in order to improve the delay processing efficiency, the messages to be sent can be divided according to the delay time of the messages to be sent. When processing messages, only the messages to be sent within the preset time are delayed, and the messages to be sent outside the preset time are stored.

[0078] Based on this, in one embodiment, obtaining the message to be sent includes:

[0079] Receive messages sent by producers with a delay time less than or equal to the preset time.

[0080] or,

[0081] Obtain from the database the messages to be sent whose delay duration is less than or equal to the preset duration.

[0082] Here, the preset duration can be set according to the delay requirement. For example, the messages to be sent on the day are set to be less than or equal to the preset duration; for another example, the messages to be sent within two hours before the message delay time is reached are set to be less than or equal to the preset duration.

[0083] In actual application, when the delay time of the message sent by the producer is greater than the preset time, the message to be sent can be stored in the database first. When the delay time corresponding to the message to be sent is within the preset time, the message to be sent is obtained from the database.

[0084] In actual application, scenario information may be set for the message to be sent, so as to send the message to be sent to at least one consumer based on the scenario information.

[0085] Based on this, in one embodiment, scenario information of the message to be sent is obtained; at least one consumer corresponding to the message to be sent is determined based on the scenario information; and the message to be sent is sent to the at least one consumer.

[0086] Here, the consumer refers to the application that receives the message. Scenario information is an information tag used to identify the message, so that the message can be distinguished based on the identified scenario information. Here, the scenario information of the message may include the business information of the message, or the name of the message producer, etc. For example, when a customer generates an order, the order system will generate a corresponding message, and the scenario information of the message can be set to "generate order" (i.e. business information) or "order system" (i.e. the name of the message producer).

[0087] In actual application, the scene information subscription relationship between producers and consumers can be set in advance. Based on the scene information subscription relationship, the consumer corresponding to the message to be sent with scene information is determined. For example, every time the order system (for the producer) generates an order, it will generate a message with scene information A. The SMS system (for the consumer) and the payment system (for the consumer) are set to subscribe to all messages with scene information A in advance. Then, every time the order system generates a message, based on the subscription relationship, the SMS system and the payment system will receive the message.

[0088] The message processing method provided by the embodiment of the present invention obtains the message to be sent and the delay duration of the corresponding delayed sending; splits the delay duration into at least two sub-durations; based on the at least two sub-durations, places the message to be sent into the delay queue corresponding to each sub-duration to store the corresponding sub-duration; wherein, after the delay queue corresponding to the corresponding sub-duration stores the corresponding sub-duration, takes out the message to be sent and places it into the delay queue corresponding to another sub-duration for storage; after the total duration stored in the delay queue reaches the delay duration, sends the message to be sent. The scheme of the embodiment of the present invention, by splitting the delay duration into several sub-durations according to the delay duration of the message to be sent, delivers the message to be sent to the queues corresponding to the several sub-durations for delay, thus realizing a delay queue based on a preset delay preset duration, and being able to delay the delayed message for any duration.

[0089] The present invention is further described in detail below in conjunction with application examples.

[0090] This application embodiment provides an implementation scheme for task scheduling (also understood as task delay) in a computer software system. The scheduling time of a message (also understood as the delay duration) can be positioned to any duration. By using multiple queues (also understood as multiple buckets), message persistence is optimized to provide high-availability, high-performance, and high-concurrency processing capabilities, and based on the high performance of RocketMq, consumers can achieve millisecond-level scheduling responses (RocketMq's message processing capability is at the millisecond level, therefore, this application embodiment performs message processing based on RocketMq, and consumers can also achieve millisecond-level responses and message reception).

[0091] This application embodiment can be applied to multiple message flow scenarios. For example, after a user places an order, he needs to pay within a specified time (for example, 30 minutes). A message is sent 15 minutes before the specified time to remind the user to pay.

[0092] Taking the above scenario as an example, combined with Figure 2 , introduce the components and their corresponding functions in this solution:

[0093] (1) Producer (for example, the order system in the above scenario): After the user completes the order, the user sends a message that requires delay (equivalent to the message to be sent in the above embodiment) to the delay platform, providing a message delay of 15 minutes and a message content of "You have an unpaid order, please pay it in time."

[0094] (2) Delay platform (Broker): used to store messages after receiving them; split the message delay duration, and store the messages in different delay queues according to the split durations, to implement delay service. When the message expires (the specified delay time for the arrival of the message), the target message queue in the delay platform can recognize the message expiration at the millisecond level and push the message to the consumer;

[0095] (3) Consumer (e.g. the SMS service system in the above scenario): used to accurately receive messages from the delay platform at the millisecond level and complete the tasks in the message. Here, the number of consumers can be one or more. When there are multiple consumers, multiple applications can subscribe to producer A's messages through the delay platform. After producer A generates a message, the message can be sent to multiple applications that have subscribed to producer A's message, that is, producer A's message can be sent to multiple consumers.

[0096] It should be noted that producers, delay platforms and consumers can be deployed on multiple devices, and the devices communicate with each other wirelessly or wired. In actual applications, the hardware resources of each device are independently deployed based on demand.

[0097] From the above description, we can see that the delay platform mainly realizes the delay of messages. Figure 3 , introduces the architecture layers that implement the delay platform function:

[0098] Graphical management interface (Dashboard in English) layer: It is the front-end operation interface of the delay platform, used to manage the creation and allocation of permissions for each producer and consumer, view the message running status, and monitor the platform operation status in real time;

[0099] Software Development Kit (SDK) layer: mainly set up on the access side (referring to producers or consumers) (it can also be understood that the SDK layer is installed on the device where the access is located), used to help R&D personnel connect producers and consumers to the delay platform, so that when producers and consumers send delayed messages and consumers receive delayed messages, the delay platform functions are used uniformly in the code program; specifically, the SDK layer is mainly responsible for overall planning, mainly including encapsulating the intermediate steps of producer or consumer access and message delay processing, so that producers only need to provide messages and other information necessary for sending messages, and consumers only need to determine which producer's message to receive, and the specific execution of the message sending process is performed by the SDK layer. The SDK layer can specifically process the messages generated by the producer in a unified format, determine the unique identifier of the message, the name of the stored queue, the account password, the IP address and port number of the consumer corresponding to the message, etc. Here, in actual application, the functions of the SDK layer can be implemented through the SDK, and developers do not need to spend a lot of time developing source code.

[0100] Interface (API) layer: It is the functional interface exposed by the delay platform to the outside world (for example, the function of communicating with projects connected to the SDK layer (referring to producers or consumers)), and provides interfaces such as receiving delay messages, message feedback, user access API authentication, configuration rule API with the delay platform, and query message interface (which can be expressed as Trace API in English); among them, the delay message receiving interface and the message feedback interface are used to receive messages sent by producers through the SDK, or to send messages to consumers through the SDK; the user access API authentication interface, the configuration rule API with the delay platform, and the Trace API are used to provide interfaces for functions such as message query and rule configuration.

[0101] API-Service layer: It is the core service layer that implements the delay function for the delay platform. It is used for message encapsulation, adaptation, core delay logic, delay flow and other functions. It splits the delay duration based on the preset splitting strategy, determines the order of message sending (that is, the order of sending the delay queue), etc.

[0102] Persistence layer: It can be an external storage system connected to the delay platform, used to provide storage functions for the delay platform, and persistently store the messages that need to be delayed in the delay platform. It can be stored through mongoDB (the name of a database based on distributed file storage), Mysql (the name of a relational database management system) and RocketMq (the name of a message middleware). Among them, mongoDB is used to store process information such as message flow logs; Mysql is used to store messages with expiration time exceeding the current day as cold data; RocketMq is set up with ordinary queues and 18 delay queues to realize storage during message delay processing.

[0103] In this application embodiment, the scheduling of delayed messages (which can also be understood as delaying delayed messages) is mainly implemented in the "API-Service layer" and "persistence layer". The scheduling time (which can also be understood as the delay duration) can be located to any duration, and the message expiration can be identified in milliseconds.

[0104] The following will describe in detail the core delay logic and the specific implementation of the delay flow function of the delay platform of this application embodiment. The optimization of the persistence layer can ensure that the scheduling time is not limited by the delay queue delay time of the whole minute and second. When the delay queue delay time is in milliseconds, the scheduling time can also be located in milliseconds.

[0105] Combination Figure 4 , the delayed flow process includes the following steps:

[0106] Step 1: The producer sends a message to the delay platform and indicates the message expiration time (i.e. the specified time to send the message to the consumer);

[0107] Step 2: The delay platform (specifically API-Service) calculates the expiration duration (here it can be understood as the delay duration) according to the expiration time, splits the duration to obtain at least two sub-durations, and delivers the message to the corresponding delay queue according to the obtained at least two sub-durations to implement the delay service. After the message expires, the message is delivered to the target message queue;

[0108] Step 3: The consumer connects to the consumer queue of the delay platform (i.e., the target message queue) and can obtain the messages pushed from the delay platform in real time.

[0109] The following is a detailed description of how to split the duration.

[0110] (1) The delay platform defines 18 queues with fixed durations;

[0111] Delayed messages cannot be delivered to the target message queue immediately, but must be delivered to the delay queue for delay first. Here, since RocketMq is provided with 18 delay queues, in this application embodiment, the delay platform based on RocketMq is also pre-set with 18 invisible delay queues (referring to the queues only for internal use of the delay platform). Delayed messages must first be placed in these delay queues for storage for a certain length of time. When the delay time of this message is reached, it is delivered to the normal queue (here refers to the target message queue, used to send messages to consumers). The storage duration of messages for the 18 delay queues is: 1 second, 5 seconds, 10 seconds, 30 seconds, 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 20 minutes, 30 minutes, 1 hour, and 2 hours.

[0112] Each of the 18 delay queues corresponds to a delay level. Delay level 1 corresponds to sending messages after a delay of 1 second; delay level 2 corresponds to sending messages after a delay of 5 seconds; delay level 3 corresponds to sending messages after a delay of 10 seconds, and so on. These 18 queues will store messages strictly according to the corresponding duration.

[0113] (2) Split the expiration time and repeatedly deliver it to some of the 18 fixed-time delay queues;

[0114] 18 fixed durations only support 18 durations of delay, and cannot achieve any delay duration, so the expiration duration needs to be split. The purpose is to break the whole into parts and convert between fixed and non-fixed durations.

[0115] The following uses the expiration time of 6 seconds as an example to explain how to split the time:

[0116] Step 1: First, deliver the message to the delay queue with a fixed length of "5 seconds";

[0117] Step 2: After 5 seconds expire, the delay platform receives the message and determines that 6 seconds - 5 seconds = 1 second; and continues to deliver the message to the delay queue with a fixed length of "1 second";

[0118] Step 3: After 1 second expires, the delay platform receives the message, determines that the delay duration has been completed, and directly delivers the message to the consumer queue (which can also be understood as a normal queue) so that it can be sent to the consumer and consumed by the consumer.

[0119] Here, in order to improve the delay efficiency and optimize the message processing speed, the delay platform can use a cross-day persistent storage method for messages with particularly long expiration times. That is, the delay platform defines that the messages of the day are repeatedly delivered in the memory queue; and the cross-day messages (that is, the messages after the second day) will be stored in the database of the persistence layer. The purpose is: when memory is scarce, it is necessary to control the total amount of messages circulating in the memory. Therefore, only the messages of the day are repeatedly delivered according to a fixed length of time to achieve efficient memory use.

[0120] Specifically, take the expiration time of 48 hours as an example:

[0121] Step 1: If the expiration time of the message exceeds 0:00 on the same day, the message is first stored in the database;

[0122] Step 2: At 0:00 every day, the messages that expire on that day are retrieved from the database and the expiration time is split;

[0123] Step 3: Based on the 18 delay queues, first deliver the message to the delay queue with a fixed length of "2 hours";

[0124] Step 4: After 2 hours expires, the delay platform receives the expiration message; determines that 24 hours - 2 hours = 20 hours; and continues to deliver to the delay queue with a fixed length of "2 hours";

[0125] Step 5: Repeat step 4;

[0126] Step 6: Until it is delivered to the last delay queue; after the delay is completed, the message is directly delivered to the consumer queue and consumed by the consumer.

[0127] Here, after the SDK layer receives the message generated by the producer, it can unify the format of the message and send the message to the delay platform for delay processing after the unified format processing. The message after the unified format processing may include the following: scene key value (which can be expressed as sceneKey in English), platform retention time (holderTime), time unit (TimeUnit timeUnit), message body (body). Among them, the scene key value is used to verify whether the message generated by the producer has the authority to perform delay processing; the platform retention time is used to store the delay duration of the message; the message body stores message-related information, such as delay platform information and business function information, among which the platform information and business function information are stored separately.

[0128] In this application embodiment, the function of the delay platform is independent of the program where the accessor (producer, consumer) is located, and can be used across platforms, which can reduce the pressure on the accessor's system operation performance. At the same time, in actual application, it can be deployed across computer rooms and regions, so it can provide distributed scheduling functions.

[0129] The solution provided by this application embodiment has the following advantages:

[0130] (1) The limitation of 18 queues that can only delay for a fixed time has been broken. The solution provided by this embodiment can achieve a delay of any length.

[0131] In the related art, the storage durations corresponding to the 18 queues are: 1 second, 5 seconds, 10 seconds, 30 seconds, 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, 10 minutes, 20 minutes, 30 minutes, 1 hour, and 2 hours, so other delay durations (such as 7 seconds) are not supported. In this application embodiment, a delay of any duration can be achieved, for example, 7 seconds can be split into 5 seconds + 1 second + 1 second, and delivered to 18 delay queues for delay according to the split duration, so as to achieve a delay of any duration.

[0132] (2) Combine queues and databases to store massive amounts of information, while ensuring that delayed messages can be effectively pushed on the same day and delivered to consumers in milliseconds before expiration;

[0133] In the related art, all messages are stored in a database or in a queue. When massive information flows, there will be a large delay and poor performance. In severe cases, expired messages may not be pushed to consumers in time. In the present application embodiment, a distinction is made between hot and cold storage (for example, messages with an expiration date of the same day are hot data, and messages with an expiration date exceeding the same day are cold data). Messages that expire on the same day are treated as hot data and stored in a memory queue (i.e., a delay queue) to enhance effectiveness and ensure that they are pushed to consumers in time when they expire. Messages with an expiration date exceeding the same day are treated as cold data and stored in a database (i.e., Mysql). Massive storage memory can be used to store massive messages without affecting the processing of the hot data of the same day.

[0134] (3) The SDK layer encapsulates other functions besides the time segmentation and repeated delivery of delayed messages, so that applications (i.e., producers and consumers) accessing the delay platform can flexibly and conveniently use the delay scheduling function;

[0135] In the related art, when sending and consuming messages, a large amount of code is written into the access program to drive the function of the delay queue by the program; for example, when the access party implements time segmentation and repeated delivery, a large amount of development code is written each time the access is made; in the present application embodiment, other common functions after extracting functions such as time segmentation and repeated delivery are encapsulated into an SDK; so that R&D personnel can simplify the access, producers only need to provide the expiration time and message content, and consumers only need to determine which queue to consume the message. When there is an expired message, the solution of the present application embodiment will actively push it to the consumer, reducing the consumer's constant polling to see if there is an expired message, greatly simplifying consumer access.

[0136] (4) The queue maintains a producer to send a delayed message. When the message expires, multiple consumers can obtain the same message at the same time; reducing the consumer's ability to reconfigure the timing;

[0137] In the related art, a producer can only have one consumer; if multiple consumers consume the same message, multiple producers are required to send multiple messages to meet the scenario corresponding to multiple consumers; in the embodiment of the present invention, a message sent by a producer can be routed to multiple consumers by this solution.

[0138] (5) The function of the above-mentioned delay platform is independent of the program where the accessor (producer, consumer) is located and can be used across platforms. Providing distributed scheduling function can reduce the operating performance pressure of the accessor system;

[0139] In the related technology, the producer program and the delay platform are the same program, and the consumer program also runs in the same program as the delay platform, coupling and influencing each other; in the present application embodiment, it is independent of the program where the accessor (producer, consumer) is located, can be used across platforms, and can reduce the pressure on the accessor's system operating performance; at the same time, this solution can be deployed across computer rooms and regions, so it can provide distributed scheduling functions.

[0140] (6) The delay platform only focuses on forwarding messages and does not need to perform other functions. It can be uniquely configured on hardware devices to achieve targeted resource investment;

[0141] In the related art, the producer and consumer programs may be a high-concurrency program, while the delay platform is a high-storage and high-concurrency system; therefore, if the two are deployed together in one system, it will require a high investment in hardware resources; in this application embodiment, the delay platform is deployed independently. When high storage is required, the central processing unit (CPU) investment can be reduced and mass storage devices can be added instead; when the concurrent access volume is large, the storage devices can be reduced and the memory and CPU investment can be increased. Therefore, it is more flexible and cost-effective than the traditional solution.

[0142] (7) At the SDK layer, the message format is unified, and the message format can be used uniformly for functions with delayed scheduling;

[0143] In the related art, there are too many types of message formats, the formats are not unified, and the message content is coupled with a large amount of business information; in this application embodiment, the message format is unified, and the message content stores the delayed platform information and the business function information separately without affecting each other. After unification, it is convenient for delayed platform upgrades and business upgrades.

[0144] In summary, the performance of traditional delay methods is affected by the consumer's ability to process messages. Once the consumer delays in processing messages, the subsequent expired messages in the delay queue cannot be pushed to the consumer in time; the consumer consumes the delayed messages. The present embodiment solves this problem by using a multi-bucket method to isolate messages with different delay times, and pushes them to consumers in a queue with multiple consumer threads, so that consumers will not consume the delayed messages. At the same time, traditional delay methods mostly use database storage. When repeatedly querying expired consumption, it is affected by the amount of messages. Once the database is paralyzed, the entire delay function will be completely paralyzed. The present embodiment solves this problem by using RocketMQ message middleware to store messages. It has high availability, high performance, and high concurrency processing capabilities, and supports any delay time in the business.

[0145] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides a message processing device, which is arranged on a message processing platform, such as Figure 5 As shown, the message processing device 500 includes: an acquisition module 501, a splitting module 502, a delivery module 503 and a sending module 504; wherein,

[0146] The acquisition module 501 is used to acquire the message to be sent and the delay duration of the corresponding delayed sending;

[0147] The splitting module 502 is used to split the delay duration into at least two sub-durations;

[0148] The delivery module 503 is used to place the message to be sent into the delay queue corresponding to each sub-time length based on the at least two sub-time lengths to store the corresponding sub-time length; wherein, after the delay queue corresponding to the corresponding sub-time length stores the corresponding sub-time length, the message to be sent is taken out and placed into the delay queue corresponding to another sub-time length for storage;

[0149] The sending module 504 is used to send the message to be sent after the total time length stored in the delay queue reaches the delay time length.

[0150] In one embodiment, the splitting module 502 is further used to:

[0151] The delay duration is divided into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; the first parameter represents the duration for which the message must be stored in the corresponding delay queue.

[0152] In one embodiment, the splitting module 502 is further used to:

[0153] According to the first parameter of each delay queue in at least one delay queue and in combination with the splitting strategy, the delay duration is split into at least two sub-durations.

[0154] In one embodiment, the delivery module 503 is further configured to:

[0155] Sorting the at least two sub-durations to obtain a sorting result;

[0156] According to the sorting result, the messages to be sent are sequentially placed in the delay queue corresponding to the corresponding sub-duration and the corresponding sub-duration is stored.

[0157] In one embodiment, the acquisition module 501 is further used for:

[0158] Receive messages sent by producers with a delay time less than or equal to the preset time.

[0159] or,

[0160] Obtain from the database the messages to be sent whose delay duration is less than or equal to the preset duration.

[0161] In one embodiment, the acquisition module 501 is further used for:

[0162] The messages to be sent by the producer with a delay time longer than the preset time are stored in the database.

[0163] In one embodiment, the acquisition module 501 is further used for:

[0164] Obtaining scene information of the message to be sent;

[0165] The sending module 504 is further used for:

[0166] Determine at least one consumer corresponding to the message to be sent based on the scenario information;

[0167] The message to be sent is sent to the at least one consumer.

[0168] Here, the function of the message processing device 500 is equivalent to the function of the delay platform in the above application embodiment.

[0169] In actual application, the acquisition module 501, the splitting module 502, the delivery module 503 and the sending module 504 can be implemented by a processor in the message processing device.

[0170] It should be noted that: when the message processing device provided in the above embodiment processes messages, only the division of the above program modules is used as an example. In actual application, the above processing can be assigned to different program modules as needed, that is, the internal structure of the terminal is divided into different program modules to complete all or part of the processing described above. In addition, the message processing device provided in the above embodiment and the message processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0171] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a message processing platform, such as Figure 6 As shown, the message processing platform 600 (equivalent to the delay platform in the above application embodiment) includes:

[0172] Communication interface 601, capable of exchanging information with other devices (such as network devices, terminals, etc.);

[0173] A processor 602, connected to the communication interface 601 to implement information interaction with other devices, and used to execute the method provided by one or more of the above technical solutions when running a computer program;

[0174] The memory 603 is used to store a computer program that can be executed on the processor 602 .

[0175] Specifically, the processor 602 is configured to perform the following operations:

[0176] Obtaining the message to be sent and the corresponding delay duration of the delayed sending through the communication interface 601;

[0177] Splitting the delay duration into at least two sub-durations;

[0178] Based on the at least two sub-periods, the message to be sent is placed in a delay queue corresponding to each sub-period to store the corresponding sub-period; wherein, after the delay queue corresponding to the corresponding sub-period stores the corresponding sub-period, the message to be sent is taken out and placed in a delay queue corresponding to another sub-period for storage;

[0179] After the total duration stored in the delay queue reaches the delay duration, the message to be sent is sent.

[0180] In one embodiment, the processor 602 is further configured to perform the following operations:

[0181] The delay duration is divided into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; the first parameter represents the duration for which the message must be stored in the corresponding delay queue.

[0182] In one embodiment, the processor 602 is further configured to perform the following operations:

[0183] According to the first parameter of each delay queue in at least one delay queue and in combination with the splitting strategy, the delay duration is split into at least two sub-durations.

[0184] In one embodiment, the processor 602 is further configured to perform the following operations:

[0185] Sorting the at least two sub-durations to obtain a sorting result;

[0186] According to the sorting result, the messages to be sent are sequentially placed in the delay queue corresponding to the corresponding sub-duration and the corresponding sub-duration is stored.

[0187] In one embodiment, the processor 602 is further configured to perform the following operations:

[0188] Receive messages sent by producers with a delay time less than or equal to the preset time.

[0189] or,

[0190] Obtain from the database the messages to be sent whose delay duration is less than or equal to the preset duration.

[0191] In one embodiment, the processor 602 is further configured to perform the following operations:

[0192] The messages to be sent by the producer with a delay time longer than the preset time are stored in the database.

[0193] In one embodiment, the processor 602 is further configured to perform the following operations:

[0194] Obtaining scene information of the message to be sent;

[0195] The sending of the message to be sent includes:

[0196] Determine at least one consumer corresponding to the message to be sent based on the scenario information;

[0197] The message to be sent is sent to the at least one consumer.

[0198] It should be noted that: the specific process of the processor 602 performing the above operations is detailed in the method embodiment, which will not be repeated here.

[0199] Of course, in actual application, the various components in the message processing platform 600 are coupled together through the bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604 .

[0200] The memory 603 in the embodiment of the present invention is used to store various types of data to support the operation of the message processing platform 600. Examples of such data include: any computer program used to operate on the message processing platform 600.

[0201] The method disclosed in the above embodiment of the present invention can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 602 or the instruction in the form of software. The above processor 602 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 603. The processor 602 reads the information in the memory 603 and completes the steps of the above method in combination with its hardware.

[0202] In an exemplary embodiment, the message processing platform 600 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0203] It can be understood that the memory 603 of the embodiment of the present invention can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0204] In an exemplary embodiment, the embodiment of the present invention further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 603 storing a computer program, and the computer program can be executed by a processor 602 of a message processing platform 600 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0205] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0206] In addition, the technical solutions described in the embodiments of the present invention can be arbitrarily combined without conflict.

[0207] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A message processing method, characterized in that: include: Obtain the message to be sent and the delay duration of the corresponding delayed sending, where the delay duration is any length; Splitting the delay duration into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; The first parameter represents the length of time the message must be stored in the corresponding delay queue; Based on the at least two sub-periods, the message to be sent is placed in a delay queue corresponding to each sub-period to store the corresponding sub-period; wherein, after the delay queue corresponding to the corresponding sub-period stores the corresponding sub-period, the message to be sent is taken out and placed in a delay queue corresponding to another sub-period for storage; After the total duration stored in the delay queue reaches the delay duration, the message to be sent is sent.

2. The method according to claim 1, characterized in that The step of splitting the delay duration into at least two sub-durations according to a first parameter of at least one delay queue includes: According to the first parameter of each delay queue in at least one delay queue and in combination with the splitting strategy, the delay duration is split into at least two sub-durations.

3. The method according to claim 1, characterized in that The delivering the message to be sent to the delay queue corresponding to each sub-duration based on the at least two sub-durations to store the corresponding sub-duration includes: Sorting the at least two sub-durations to obtain a sorting result; According to the sorting result, the messages to be sent are sequentially placed in the delay queue corresponding to the corresponding sub-duration and the corresponding sub-duration is stored.

4. The method according to claim 1, characterized in that: The obtaining of the message to be sent includes: Receive messages sent by producers with a delay time less than or equal to the preset time. or, Obtain from the database the messages to be sent whose delay duration is less than or equal to the preset duration.

5. The method according to claim 4, characterized in that The method further comprises: The messages to be sent by the producer with a delay time longer than the preset time are stored in the database.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining scene information of the message to be sent; The sending of the message to be sent includes: Determine at least one consumer corresponding to the message to be sent based on the scenario information; The message to be sent is sent to the at least one consumer.

7. A message processing device, characterized in that: include: An acquisition module is used to acquire a message to be sent and a delay duration of the corresponding delayed sending, wherein the delay duration is any duration; A splitting module, used for splitting the delay duration into at least two sub-durations according to a first parameter of each delay queue in at least one delay queue; The first parameter represents the length of time the message must be stored in the corresponding delay queue; A delivery module, configured to place the message to be sent into a delay queue corresponding to each sub-time length based on the at least two sub-time lengths, and store the corresponding sub-time length; wherein, after the delay queue corresponding to the corresponding sub-time length stores the corresponding sub-time length, the message to be sent is taken out and placed into a delay queue corresponding to another sub-time length for storage; The sending module is used to send the message to be sent after the total time length stored in the delay queue reaches the delay time length.

8. A message processing platform, characterized in that: include: A processor and a memory for storing a computer program capable of running on the processor; wherein, When the processor is used to run the computer program, the steps of the method according to any one of claims 1 to 6 are performed.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images