Message processing method and device based on message queue system, equipment and storage medium

Abstract

The embodiment of the invention relates to a message processing method and device based on a message queue system, equipment and a storage medium, and the method comprises the steps: receiving a message withdrawing request sent by a producer of a message queue, and determining a withdrawn delay message indicated by the message withdrawing request as a to-be-withdrawn message, the to-be-withdrawn message is recorded in a withdraw message database, the withdraw message database is used for recording delay messages belonging to the to-be-withdrawn message in a delay message database, and the delay messages with delay time are stored in the delay message database; and in response to the expiration of the delay time of the first delay message in the delay message database, determining whether the retracement message database contains the first delay message, and if the retracement message database contains the first delay message, indicating that the first delay message is retraced, and at the moment, discarding the first delay message. According to the embodiment of the invention, the withdrawing function of the delay message is realized, so that the waste of system resources caused by the processing of the delay message is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a message processing method, apparatus, device, and storage medium based on a message queue system. Background Art

[0002] A message is a communication mechanism used to transfer information between different systems, components, or services. A message queue system is a container that provides message storage and transmission functions. Each message queue in the message queue system has a corresponding producer and consumer. The messages produced by the producer can be of a type that requires immediate consumption by the consumer or of a type that requires consumption by the consumer after a specified delay time. Such messages that require consumption by the consumer after a specified delay time can be regarded as delayed messages.

[0003] Normally, when the delay time specified for a delayed message expires, an event corresponding to the delayed message needs to be triggered for execution. However, in some cases, even when the delayed message expires, it is not necessary to process the delayed message. For example, after the delayed message expires, the event corresponding to the delayed message does not need to be executed in the business, and at this time, processing the delayed message is instead a waste of system resources. Summary of the Invention

[0004] To solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a message processing method, apparatus, device, and storage medium based on a message queue system.

[0005] In a first aspect, an embodiment of the present disclosure provides a message processing method based on a message queue system, the method including:

[0006] Receiving a message recall request sent by a producer of a message queue in the message queue system, determining the delayed message indicated to be recalled by the message recall request as a message to be recalled, and recording the message to be recalled in a recall message database; the recall message database is used to record the delayed messages in the delayed message database that belong to the messages to be recalled, and the delayed message database stores delayed messages with a delay time;

[0007] In response to the expiration of the delay time of a first delayed message in the delayed message database, determining whether the recall message database contains the first delayed message;

[0008] If the recall message database contains the first delayed message, discarding the first delayed message.

[0009] In an optional implementation manner, the method further includes:

[0010] If the first delayed message is not included in the recall message database, send the first delayed message to the target message queue so that the consumer of the target message queue reads and processes the first delayed message from the target message queue.

[0011] In an alternative embodiment, the message recall request carries target service parameters for indicating the recall of delayed messages matching the target service parameters. Determining the delayed message indicated by the message recall request to be recalled as the message to be recalled includes:

[0012] Match the target service parameters from the delayed message database, and determine the delayed messages that match and are hit as the messages to be recalled.

[0013] In an alternative embodiment, the message recall request further carries an expiration time condition. The expiration time condition is used to determine the delayed message indicated by the message recall request to be recalled according to the expiration time of the delayed message. Determining the delayed message indicated by the message recall request to be recalled as the message to be recalled includes:

[0014] Match the target service parameters from the delayed message database, and determine the delayed messages that match and are hit and whose expiration time meets the expiration time condition as the messages to be recalled.

[0015] In an alternative embodiment, the method further includes:

[0016] Receive a second delayed message, and determine the physical node corresponding to the second delayed message from the distributed cluster based on a preset routing policy as the first target physical node; a recall message database and a delayed message database are respectively deployed on each physical node in the distributed cluster;

[0017] Store the second delayed message in the delayed message database on the first target physical node.

[0018] In an alternative embodiment, after receiving the message recall request sent by the producer of the message queue in the message queue system, it further includes:

[0019] Determine the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy as the second target physical node;

[0020] Correspondingly, recording the message to be recalled in the recall message database includes:

[0021] Record the message to be recalled in the recall message database on the second target physical node.

[0022] In an alternative embodiment, determining the physical node corresponding to the delayed message from the distributed cluster based on the preset routing policy as the first target physical node includes:

[0023] Calculating a hash value according to the target service parameter in the second delayed message, and determining the hash node corresponding to the hash value on the preset hash ring as the first target hash node; each hash node in the preset hash ring has a corresponding hash value range and physical node;

[0024] Determining the physical node corresponding to the first target hash node as the first target physical node.

[0025] In an alternative embodiment, determining the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy as the second target physical node includes:

[0026] Calculating a hash value according to the target service parameter in the message recall request, and determining the hash node corresponding to the hash value on the preset hash ring as the second target hash node;

[0027] Determining the physical node corresponding to the second target hash node according to the historical physical node list corresponding to the second target hash node as the second target physical node; the historical physical node list includes the change records of the physical nodes corresponding to the second target hash node.

[0028] In an alternative embodiment, the method further includes:

[0029] Receiving a third delayed message, and determining the physical node corresponding to the third delayed message from the distributed cluster based on a random policy as the third target physical node; a recall message database and a delayed message database are respectively deployed on each physical node in the distributed cluster;

[0030] Storing the third delayed message in the delayed message database on the third target physical node.

[0031] In an alternative embodiment, after receiving the message recall request, it further includes:

[0032] Broadcasting and distributing the message recall request to each physical node in the distributed cluster, and querying the delayed message database of each physical node to determine the physical node corresponding to the message recall request as the fourth target physical node;

[0033] Correspondingly, recording the message to be recalled in the recall message database includes:

[0034] Record the to-be-withdrawn message in the withdrawal message database on the fourth target physical node.

[0035] In an optional implementation, the method further includes:

[0036] In response to the physical node corresponding to the third target hash node on the preset hash ring being changed from the fifth target physical node to the sixth target physical node, migrate the data in the withdrawal message database and the delayed message database on the fifth target physical node to the sixth target physical node.

[0037] In a second aspect, the present disclosure provides a message processing apparatus based on a message queue system. The apparatus includes:

[0038] A first receiving module, configured to receive a message withdrawal request sent by a producer of a message queue in the message queue system, determine the delayed message indicated to be withdrawn by the message withdrawal request as the to-be-withdrawn message, and record the to-be-withdrawn message in the withdrawal message database; the withdrawal message database is used to record the delayed messages belonging to the to-be-withdrawn messages in the delayed message database, and the delayed message database stores the delayed messages with a delay time;

[0039] A first determination module, configured to determine whether the first delayed message in the delayed message database is included in the withdrawal message database in response to the expiration of the delay time of the first delayed message in the delayed message database;

[0040] A discarding module, configured to discard the first delayed message if the first delayed message is included in the withdrawal message database.

[0041] In a third aspect, an embodiment of the present disclosure further provides an electronic device. The electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the message processing method based on the message queue system provided by the embodiment of the present disclosure.

[0042] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium. The storage medium stores a computer program, and the computer program is used to execute the message processing method based on the message queue system provided by the embodiment of the present disclosure.

[0043] In a fifth aspect, the present disclosure provides a computer program product. The computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the above method is implemented.

[0044] The technical solution provided by the embodiment of the present disclosure has the following advantages compared with the prior art:

[0045] In the message processing method based on a message queue system provided by an embodiment of the present disclosure, first, a message recall request sent by a producer of a message queue in the message queue system is received, and the delayed message indicated to be recalled by the message recall request is determined as the message to be recalled, and the message to be recalled is recorded in a recall message database, which is used to record the delayed messages in the delayed message database that belong to the message to be recalled, and the delayed message database stores delayed messages with a delay time; in response to the expiration of the delay time of the first delayed message in the delayed message database, it is determined whether the first delayed message is included in the recall message database. If the first delayed message is included in the recall message database, it means that the first delayed message has been recalled, and at this time, the first delayed message is discarded.

[0046] By recording the delayed message indicated to be recalled by the message recall request in the recall message database, the embodiment of the present disclosure realizes the recall function of the delayed message by querying the recall message database to determine whether the delayed message has been recalled when the delayed message expires, and discarding the delayed message when it is determined that the delayed message has been recalled, thereby reducing the waste of system resources caused by the processing of the delayed message. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale.

[0048] Figure 1 It is a schematic diagram of a message processing method based on a message queue system provided by an embodiment of the present disclosure;

[0049] Figure 2 It is a schematic diagram of the relationship between a message queue system and a delayed message system provided by an embodiment of the present disclosure;

[0050] Figure 3 It is a schematic diagram of a delayed message interaction protocol provided by an embodiment of the present disclosure;

[0051] Figure 4 It is a schematic flowchart of another message processing method based on a message queue system provided by an embodiment of the present disclosure;

[0052] Figure 5 It is a schematic diagram of a preset hash ring and physical nodes in a distributed cluster provided by an embodiment of the present disclosure;

[0053] Figure 6 It is a schematic flowchart of another message processing method based on a message queue system provided by an embodiment of the present disclosure;

[0054] Figure 7 Schematic diagram of a processing flow for delayed messages and withdrawal message requests provided by an embodiment of the present disclosure;

[0055] Figure 8 Schematic diagram of expanding a distributed cluster provided by an embodiment of the present disclosure;

[0056] Figure 9 Schematic diagram of the structure of a message processing device based on a message queue system provided by an embodiment of the present disclosure;

[0057] Figure 10 Schematic diagram of the structure of a message processing device based on a message queue system provided by an embodiment of the present disclosure. Detailed implementation manners

[0058] Embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0059] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0060] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0061] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0062] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0063] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of these messages or information.

[0064] A message is a communication mechanism used to transfer information between different systems, components, or services. A message queue system is a container that provides message storage and transmission functions. Each message queue in the message queue system has a corresponding producer and consumer. The messages produced by the producer can be of a type that requires immediate consumption by the consumer or a type that requires consumption by the consumer after a specified delay time. Such messages that require consumption by the consumer after a specified delay time can be regarded as delayed messages.

[0065] In the related art, the processing process of delayed messages can include steps such as sending delayed messages and consuming expired delayed messages. For example, in the scenario of detecting order timeout, a delayed message can be used to cancel the timeout order after a specified time. Specifically, when a user places an order, a delayed message of "detect whether the order is timeout" triggered after 10 minutes is sent to the message queue system, and the message queue system is used to process the delayed message of "detect whether the order is timeout" when the delayed message expires.

[0066] However, if the event corresponding to the delayed message has been completed before the delayed message expires, for example, the order has been completed, at this time, the detection of whether the order is timeout will still be executed. In fact, it is meaningless to detect whether the order is timeout when the event corresponding to the delayed message has been completed, and it will instead waste system resources.

[0067] The embodiments of the present disclosure provide a message processing method based on a message queue system. First, a message recall request is received, and the delayed message indicated to be recalled by the message recall request is determined as the message to be recalled, and the message to be recalled is recorded in a recall message database. The recall message database is used to record the delayed messages in the delayed message database that belong to the message to be recalled. The delayed message database stores delayed messages with a delay time. In response to the expiration of the delay time of the first delayed message in the delayed message database, it is determined whether the recall message database contains the first delayed message. If the recall message database contains the first delayed message, it means that the first delayed message has been recalled, and at this time, the first delayed message is discarded.

[0068] In the embodiments of the present disclosure, the delayed message indicated to be withdrawn by the message withdrawal request is recorded in the withdrawal message database, so that when the delayed message expires and is executed, it is determined whether the delayed message has been withdrawn by querying the withdrawal message database, and when it is determined that the delayed message has been withdrawn, the delayed message is discarded, realizing the function of withdrawing the delayed message, thereby reducing the waste of system resources caused by the processing of the delayed message.

[0069] Based on this, the embodiments of the present disclosure provide a message processing method based on a message queue system, as Figure 1 shown in the flowchart of a message processing method based on a message queue system provided by the embodiments of the present disclosure. This method can be executed by a message processing device based on the message queue system, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 1 shown, the method includes:

[0070] S101: Receive a message withdrawal request sent by a producer of a message queue in the message queue system, determine the delayed message indicated to be withdrawn by the message withdrawal request as the message to be withdrawn, and record the message to be withdrawn in the withdrawal message database.

[0071] Wherein, the withdrawal message database is used to record the delayed messages belonging to the messages to be withdrawn in the delayed message database, and the delayed message database stores the delayed messages with a delay time.

[0072] The message processing method based on the message queue system provided by the embodiments of the present disclosure can be applied to any physical node, which can be any physical node in a distributed cluster or a physically deployed device alone, etc.

[0073] The message processing method based on the message queue system provided by the embodiments of the present disclosure can be executed by the message queue system. Specifically, a delayed message system is deployed in the message queue system, and the message processing method based on the message queue system provided by the embodiments of the present disclosure is executed by the delayed message system deployed in the message queue system.

[0074] In the embodiments of the present disclosure, the message queue system refers to a system that provides message storage and transmission functions. After a message is sent to the message queue system, the message queue system needs to provide message routing for the message and ensure the delivery of the message.

[0075] In a message queue system, the messages produced by the producer of the message queue can be of the type that requires immediate consumption by the consumer, or of the type that requires consumption by the consumer after a specified delay time. Such delayed messages that are consumed by the consumer after a specified delay time are usually processed by a delayed message system deployed in the message queue system. The delayed message system can be implemented as a sub-module within the message queue system. The delayed message system exists as an independent and extended subsystem of the message queue system and can implement the processing of delayed messages.

[0076] In addition, the message processing method based on the message queue system provided in the embodiments of the present disclosure can also be implemented in the form of an external plugin, and the embodiments of the present disclosure do not limit its specific implementation form.

[0077] In the embodiments of the present disclosure, the delayed message database is used to store delayed messages with a delay time, and the delayed messages allow the messages to be consumed only after the specified delay time.

[0078] Each message queue in the message queue system has a corresponding producer and consumer. The producer of the message queue is responsible for producing messages and writing the messages into the message queue. For example, when a user places an order, the producer can produce a delayed message of "detecting whether the order times out" that is triggered after 10 minutes and send it to the message queue; the consumer is used to read and process the delayed message from the message queue. In practical applications, when the producer of the message queue in the message queue system delivers a delayed message, information such as the delay time and target service parameters is usually carried in the delayed message. After receiving the delayed message carrying the above message sent by the producer, the delayed message is first stored in the delayed message database.

[0079] Delayed messages usually need to be consumed only after reaching the specified delay time. However, in some cases, it may be necessary to cancel or withdraw these delayed messages. For example, in the scenario of detecting order timeout, if the user completes the order payment or cancels the order within the delay time (for example, at the 2nd minute), then the originally set delayed message of "detecting whether the order times out" after 10 minutes needs to be withdrawn. At this time, the producer of the message queue in the message queue system can send a message withdrawal request to the delayed message system to request the delayed message system to perform a withdrawal process on the already set delayed message.

[0080] In the embodiments of the present disclosure, when receiving the message withdrawal request sent by the producer of the message queue in the message queue system, the delayed message indicated to be withdrawn by the message withdrawal request is first determined as the message to be withdrawn, and the message to be withdrawn is recorded in the withdrawal message database.

[0081] In an alternative embodiment, the message recall request may carry target service parameters. The target service parameters are key attributes of the delayed message and can be used to indicate the recall of the delay matching the target service parameters. The target service parameters may include, for example, parameter information such as product identifiers and user identifiers. Using the target service parameters, a delayed message or a type of delayed message can be determined. Therefore, after receiving the message recall request, the target service parameters can be matched in the delayed message database, and the delayed message that matches the target service parameters is determined as the message to be recalled and recorded in the recalled message database.

[0082] Among them, the recalled message database is used to record the delayed messages in the delayed message database that belong to the messages to be recalled. That is to say, the messages recorded in the recalled message database refer to the delayed messages that have been recalled. So that when the delayed messages in the delayed message database expire for consumption later, it can be determined whether the delayed message has been recalled by querying whether the recalled message database records the delayed message.

[0083] It can be seen that the embodiments of the present disclosure carry target service parameters in the message recall request, so that later, after receiving the message recall request, the delayed messages in the delayed message database that match the target service parameters obtained by parsing can be determined as the messages to be recalled.

[0084] In practical applications, when determining the delayed messages to be recalled that match the target service parameters from the delayed message database based on the target service parameters, there may be multiple delayed messages with different delay times for the same target service parameters. If the recall time range of the delayed messages is not specified, all the delayed messages that match the target service parameters will be recalled.

[0085] Therefore, the embodiments of the present disclosure can also carry an expiration time condition in the message recall request to recall the delayed messages within a specified time range using the expiration time condition.

[0086] In the embodiments of the present disclosure, the expiration time condition is used to determine the delayed messages indicated to be recalled by the message recall request according to the expiration time of the delayed messages. Exemplarily, assuming that the expiration time condition is to recall the delayed messages with an expiration time less than 10 minutes, after multiple delayed messages with the same target service parameters are matched, the delayed messages with an expiration time less than 10 minutes among the multiple matched delayed messages can be determined as the messages to be recalled.

[0087] In an embodiment of the present disclosure, when a message recall request is received, the message recall request is first parsed to obtain the expiration time condition and the target service parameter carried in the message recall request. Then, the target service parameter is matched in the delayed message database, and the delayed messages in the delayed message database whose expiration time meets the expiration time condition among the matched delayed messages are determined as the messages to be recalled.

[0088] Exemplarily, assume that the target service parameter carried in the message recall request is user A, and the expiration time condition is to expire within 10 minutes. After receiving the message recall request, the delayed messages in the delayed message database that match user A and whose expiration time corresponding to the delay time is less than 10 minutes can be determined as the messages to be recalled. For example, message 1, whose corresponding target service parameter is user A and the expiration time corresponding to the delay time is 5 minutes, and message 2, whose corresponding target service parameter is user A and the expiration time corresponding to the delay time is 8 minutes.

[0089] S102: In response to the expiration of the delay time of the first delayed message in the delayed message database, determine whether the recall message database contains the first delayed message.

[0090] In an embodiment of the present disclosure, since the delayed messages stored in the delayed message database have delay time information, it is possible to determine whether a delayed message has expired based on the delay time. Specifically, when it is detected that the delay time of the first delayed message in the delayed message database has expired, it is determined whether the recall message database contains the first delayed message.

[0091] S103: If the recall message database contains the first delayed message, discard the first delayed message.

[0092] In an embodiment of the present disclosure, since the recall message database is used to record the delayed messages in the delayed message database that belong to the messages to be recalled, when it is determined that the recall message database contains the first delayed message, it means that the first delayed message has been recalled. At this time, there is no need to process the first delayed message. Specifically, the first delayed message can be discarded to avoid processing the first delayed message, thereby saving the resources for processing the recalled delayed messages and improving the system performance.

[0093] In an alternative embodiment, if it is determined that the recall message database does not contain the first delayed message, the first delayed message is sent to the target message queue so that the consumer of the target message queue reads and processes the first delayed message from the target message queue system.

[0094] In practical applications, in the publish / subscribe mode of a message queue system, expired delayed messages can be sent to a target topic in a target message queue, and the target topic is used to process the delayed messages. Among them, the target topic allows multiple publishers to send delayed messages to the target topic, and at the same time, multiple subscribers or consumers can receive the delayed messages by subscribing to the target topic. Since the message publishing and consumption in the target topic are asynchronous, that is, the publisher of the message queue does not need to wait for the response of the subscriber or consumer after sending the message. Therefore, the embodiments of the present disclosure can improve the processing efficiency of the delayed messages by processing the expired delayed messages through the target topic.

[0095] In the message processing method based on a message queue system provided by the embodiments of the present disclosure, first, a message recall request sent by a producer of a message queue in the message queue system is received, and the delayed message indicated to be recalled by the message recall request is determined as the message to be recalled, and the message to be recalled is recorded in a recall message database, and the recall message database is used to record the delayed messages belonging to the message to be recalled in the delayed message database, and the delayed message database stores the delayed messages with delay times; in response to the expiration of the delay time of the first delayed message in the delayed message database, it is determined whether the recall message database contains the first delayed message. If the recall message database contains the first delayed message, it means that the first delayed message has been recalled, and at this time, the first delayed message is discarded.

[0096] The embodiments of the present disclosure record the delayed messages indicated to be recalled by the message recall request in the recall message database, so as to determine whether the delayed message has been recalled by querying the recall message database when the delayed message expires, and discard the delayed message when it is determined that the delayed message has been recalled, realizing the recall function of the delayed message, thereby reducing the waste of system resources caused by the processing of the delayed message.

[0097] As Figure 2 shown, it is a schematic diagram of the processing of a delayed message provided by the embodiments of the present disclosure.

[0098] When the delayed message system receives the delayed message A sent by the delayed message writing application, the delayed message A is written into the delayed message database, where the delayed message database is used to store the delayed messages with delay times; when the delay time of the delayed message A in the delayed message database expires, if the recall message database does not contain the delayed message A, the delayed message A is sent to the target message queue, such as the topic 1 in cluster 1 corresponding to message queue A. The delayed message reading application processes the delayed message A through the topic 1 in cluster 1 corresponding to message queue A.

[0099] When the delayed message system receives the delayed message B sent by the delayed message writing application, it writes the delayed message B into the delayed message database. When the delay time of the delayed message B expires, if it is determined that the rollback message database contains the delayed message B, the delayed message B is discarded, that is, the delayed message B is not processed.

[0100] In an optional implementation, the delayed message may carry delay time, target service parameters, processing object location information, etc. Among them, the processing object location information may include information such as message queue type, message queue cluster, and target topic name. The processing object location information is used to determine the target message queue or target topic in the delayed message database after the delayed message expires.

[0101] As Figure 2 shown, there are multiple delayed messages stored in the delayed message database. When it is detected that the delay time of the first delayed message in the delayed message database expires, if it is determined that the rollback message database does not contain the first delayed message, first determine the target message queue or target topic for processing the delayed message based on the processing object location information carried in the delayed message. Exemplarily, assuming that the delayed message carries information about message queue A, then message queue A is determined as the target message queue, and the first delayed message is sent to the target message queue so that the consumer of the target message queue reads and processes the first delayed message from the target message queue; for another example, assuming that the delayed message carries information such as message queue A, cluster 1, and topic 1, then the topic 1 in cluster 1 corresponding to the message queue A type can be determined as the target topic, and the first delayed message is sent to the target topic 1 to process the first delayed message on the target topic 1.

[0102] As Figure 3 shown, it is a schematic diagram of a delayed message interaction protocol provided by an embodiment of the present disclosure. The interaction protocol may include a delayed message sending protocol, a delayed message rollback protocol, and a delayed message query protocol. The delayed message sending protocol is used to rule the message attributes carried in the delayed message to implement the sending of the delayed message. The delayed message rollback protocol is used to stipulate the information carried in the message rollback request. The delayed message query protocol is used to rule the information carried in the query request for querying the delayed message.

[0103] As Figure 3 shown, the delayed message rollback protocol stipulates that the message rollback request may carry information such as message queue type, message queue cluster, topic name, target service parameters, etc. Optionally, the message rollback request may also carry an expiration time condition. Based on the expiration time condition and the target service parameters, delayed messages with the same target service parameters and the expiration time corresponding to the delay time meeting the expiration time condition can be determined.

[0104] In an alternative embodiment, there may be multiple delayed messages under the same target service parameter, and each delayed message has a different delay time. The user needs to know the specific expiration time to determine which delayed message to withdraw.

[0105] Therefore, the embodiments of the present disclosure also provide an interface for querying different message statuses of the target service parameter, which is used to query the expiration time of the delayed message. As Figure 3 shown, the delayed message query protocol stipulates that the query request can carry information such as the message queue type, message queue cluster, topic name, target service parameter, etc.

[0106] In practical applications, a distributed cluster can be used to implement the message processing method based on the message queue system provided by the embodiments of the present disclosure. Each physical node in the distributed cluster can be called a Group shard, and each Group shard runs independently. The data written by the user will be distributed in different Group shards. By splitting the data written by the user and storing it on multiple different physical nodes, the data load balancing of the distributed cluster can be improved.

[0107] Based on this, the embodiments of the present disclosure also provide a message processing method based on the message queue system. As Figure 4 shown, it is a schematic flowchart of another message processing method based on the message queue system provided by the embodiments of the present disclosure. The method includes:

[0108] S401: Receive a second delayed message, and determine the physical node corresponding to the second delayed message from the distributed cluster based on a preset routing policy as the first target physical node.

[0109] In the embodiments of the present disclosure, a delayed message database and a withdrawn message database are respectively deployed on the physical nodes of the distributed cluster. Among them, the delayed message database is used to store delayed messages with delay times, and the withdrawn message database is used to record the delayed messages that have been withdrawn in the delayed message database.

[0110] In the embodiments of the present disclosure, when receiving a second delayed message, first determine the physical node corresponding to the second delayed message from the distributed cluster based on a preset routing policy as the first target physical node.

[0111] Among them, the preset routing policy may include a consistent hashing distribution policy, which can be used to ensure that delay messages with the same target service parameters can fall on the same physical node. When receiving the second delay message, the first target physical node corresponding to the second delay message can be determined from the distributed cluster based on the preset routing policy. The embodiments of the present disclosure do not specifically limit the preset routing policy. The preset routing policy can be any policy that can implement the above-mentioned determination of the physical node corresponding to the second delay message from the distributed cluster. For example, it may include a range allocation policy, a random distribution policy, a geographical location awareness distribution policy, a weight allocation policy, a module allocation policy, a time partition policy, etc.

[0112] In practical applications, a hash ring can also be used to achieve load balancing of delay messages, so that data can be more evenly distributed to each physical node in the distributed cluster.

[0113] In an optional implementation manner, after receiving the second delay message, a hash value can also be calculated according to the target service parameters in the second delay message, and the hash node corresponding to the hash value on the preset hash ring can be determined as the first target hash node.

[0114] Among them, a hash ring is a data structure used for data distribution and load balancing in a distributed system, and is suitable for scenarios where a large amount of data is evenly distributed to multiple nodes. For example, in the scenario of a distributed cluster, each hash node in the preset hash ring has a corresponding hash value range and physical node.

[0115] As Figure 5 shown, it is a schematic diagram of a preset hash ring and physical nodes in a distributed cluster provided by an embodiment of the present disclosure. The distributed cluster includes physical node 1, physical node 2, and physical node 3. Each physical node corresponds to one or more hash nodes in the preset hash ring. Specifically, physical node 1 includes hash node 1 and hash node 2, physical node 2 includes hash node 3 and hash node 4, physical node 3 includes hash node 5 and hash node 6, and the hash value range of hash node 1 is 0x000000 to 0x222222.

[0116] It can be seen that in the embodiments of the present disclosure, by calculating the hash value of the target service parameters in the second delay message and based on the hash value, the physical node corresponding to the first target hash node on the preset hash ring is determined as the first physical node corresponding to the second delay message, so that delay messages with the same target service parameters can be routed to the same physical node.

[0117] In addition, after determining the first target hash node corresponding to the second delayed message and the first target physical node corresponding to the first target hash node, the correspondence between the target service parameter of the second delayed message and the first target hash node, as well as the correspondence between the first target hash node and the first target physical node, can be stored in the metadata engine of the delayed message system.

[0118] It can be seen that in the embodiments of the present disclosure, by storing the correspondence between the target service parameter and the hash node, as well as the correspondence between the hash node and the physical node in the distributed cluster, it is possible to quickly locate the hash node where the delayed message is located subsequently, thereby improving the data access efficiency. In addition, by storing the above correspondences, a large number of traversal and search operations during data storage and retrieval can be avoided, making data access faster and more accurate.

[0119] S402: Store the second delayed message in the delayed message database on the first target physical node.

[0120] In the embodiments of the present disclosure, after determining the first target physical node corresponding to the second delayed message, the second delayed message is stored in the delayed message database on the first target physical node, thereby realizing writing the delayed message into the delayed message database.

[0121] It can be seen that when receiving the second delayed message in the embodiments of the present disclosure, by storing the delayed message in the delayed message database on the first target physical node, it is possible to query the delayed message database subsequently when receiving a message recall request to determine the delayed message indicated to be recalled by the message recall request.

[0122] Based on the above embodiments, the embodiments of the present disclosure further provide a message processing method based on a message queue system, as Figure 6 shown in the flowchart of another message processing method based on a message queue system provided by the embodiments of the present disclosure. The method includes:

[0123] S601: Receive a message recall request, and determine the delayed message indicated to be recalled by the message recall request as the message to be recalled.

[0124] In the embodiments of the present disclosure, the message recall request may carry target service parameters. The target service parameters are the key attributes of the delayed message and can be used to indicate the recall of the delayed message that matches the target service parameters, such as parameter values of product identifiers, user identifiers, etc. Therefore, after receiving the message recall request, the target service identifier can be matched from the delayed message database, and the delayed message that is matched and hit is determined as the message to be recalled, and the message to be recalled is recorded in the recall message database.

[0125] It can be seen that in the embodiments of the present disclosure, by carrying the target service parameter in the message recall request, it is possible to subsequently, after receiving the message recall request, based on the parsed target service parameter, determine the delayed message that matches and hits the target service parameter in the delayed message database as the message to be recalled.

[0126] S602: Determine the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy as the second target physical node.

[0127] In the embodiments of the present disclosure, after determining the first target physical node corresponding to the second delayed message based on the preset routing policy and storing the second delayed message in the delayed message database on the first target physical node, when receiving a message recall request, it is also possible to determine the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy as the second target physical node; then record the message to be recalled in the recall message database on the second target physical node.

[0128] It can be seen that in the embodiments of the present disclosure, by using the same preset routing policy, it is possible to ensure the consistency of the delayed message and the recall message during the processing process, thereby reducing the problem of message routing errors caused by inconsistent routing policies and improving the reliability of delayed message processing.

[0129] S603: Record the message to be recalled in the recall message database on the second target physical node.

[0130] In the embodiments of the present disclosure, after determining the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy as the second target physical node, record the message to be recalled in the recall message database on the second target physical node.

[0131] In another alternative implementation, when receiving a message recall request, it is also possible to first determine the delayed message recalled by the message recall request as the message to be recalled; then calculate a hash value based on the target service parameter in the message recall request and determine the hash node corresponding to the hash value of the target service parameter on the preset hash ring as the second target physical node.

[0132] S604: In response to the expiration of the delay time of the first delayed message in the delayed message database, determine whether the recall message database contains the first delayed message.

[0133] In the embodiments of the present disclosure, since the delayed messages stored in the delayed message database have delay time information, it is possible to determine whether a delayed message has expired based on the delay time. Specifically, when it is detected that the delay time of the first delayed message in the delayed message database has expired, it is determined whether the first delayed message is included in the withdrawal message database.

[0134] S605: If the first delayed message is included in the withdrawal message database, discard the first delayed message.

[0135] In the embodiments of the present disclosure, since the withdrawal message database is used to record the delayed messages in the delayed message database that belong to the messages to be withdrawn, when it is determined that the first delayed message is included in the withdrawal message database, it indicates that the first delayed message has been withdrawn. At this time, there is no need to process the first delayed message. Specifically, the first delayed message can be discarded to avoid processing the first delayed message, thereby saving the resources for processing the withdrawn delayed messages and improving the system performance.

[0136] In practical applications, in the process of implementing delayed message processing using a distributed architecture, a random policy can also be used to determine a corresponding physical node for a delayed message from a distributed cluster.

[0137] In the embodiments of the present disclosure, first, a third delayed message is received, and a physical node corresponding to the third delayed message is determined from a distributed cluster based on a random policy as the third target physical node; wherein, a withdrawal message database and a delayed message database are respectively deployed on each physical node in the distributed cluster.

[0138] In the embodiments of the present disclosure, after determining the third target physical node corresponding to the third delayed message, the third delayed message is stored in the delayed message database on the third target physical node.

[0139] In practical applications, after receiving a message withdrawal request, it is also necessary to determine the physical node corresponding to the message withdrawal request from a distributed cluster, so that the delayed message indicated to be withdrawn by the message withdrawal request can be recorded in the withdrawal message database subsequently. Since in the embodiments of the present disclosure, a physical node may be randomly selected to store a delayed message, when withdrawing the delayed message subsequently, it may be impossible to determine on which physical node the delayed message specifically exists.

[0140] Therefore, after receiving a message recall request, the embodiments of the present disclosure can also broadcast and distribute the message recall request to each physical node in the distributed cluster, and query its own delayed message database through each physical node to determine the physical node corresponding to the message recall request as the fourth target physical node; after determining the fourth target physical node corresponding to the message recall request, record the message to be recalled corresponding to the message recall request in the recall message database on the fourth target physical node.

[0141] It can be seen that in the scenario of using a distributed architecture to implement delayed message processing, the embodiments of the present disclosure can use a random policy to determine a corresponding physical node from the distributed cluster for a received delayed message for storing the delayed message. In addition, when a recall message request is received, the recall message request can be broadcast and distributed to each physical node in the distributed cluster, so that each physical node queries its own delayed message database to determine the physical node corresponding to the recall message request, and then record the message to be recalled corresponding to the message recall request in the recall message database of the physical node.

[0142] In practical applications, a delayed message engine is an engine in a delayed message system for processing delayed messages and recall message requests. The delayed message engine can adopt a distributed architecture. Among them, the distributed cluster can include multiple physical nodes, and a delayed message database, a recall message database, an index database, and a memory space manager can be respectively deployed on each physical node. As Figure 7 shown, it is a schematic diagram of a processing flow of delayed messages and recall message requests provided by the embodiments of the present disclosure. Among them, the memory space manager is used to write the delayed messages with expired delay time into the target message processing module for processing the expired delayed messages on the target message processing module.

[0143] ① When receiving the delayed message A sent by the producer of the message queue in the message queue system, first determine the physical node corresponding to the delayed message A from the distributed cluster based on the preset routing policy recorded in the metadata, such as physical node 1; ② Write the delayed message A into the delayed message database on physical node 1; ③ Then construct corresponding index information for the delayed message A and write the index information corresponding to the delayed message A into the index database.

[0144] ④ When the memory space manager on physical node 1 detects that the delay time of the delayed message A in the delayed message database has expired, first determine whether the recall message database contains the delayed message A; ⑤ If it is determined that the recall message database does not contain the delayed message A, send the delayed message A to the target message queue so that the consumer of the target message queue can read and process the first delayed message from the target message queue.

[0145] ⑥When receiving a message recall request, first determine the physical node corresponding to the message recall request from the distributed cluster based on the preset routing policy recorded in the metadata, such as physical node 2; ⑦Then, based on the target service parameters carried in the message recall request, query the delayed messages in the index database with the same target service parameters, and determine the delayed messages with the same target service parameters as the messages to be recalled; ⑧Record the messages to be recalled in the recall message database.

[0146] ⑨When the memory space manager on physical node 2 detects that the delay time of delayed message B in the delayed message database has expired, ⑩determine whether the recall message database contains delayed message B. If the recall message database contains delayed message B, then discard delayed message B.

[0147] In a preferred implementation, in order to further improve the read / write performance and access performance of the database, key-value storage capabilities can also be provided for the delayed message data, recall message database, and index database deployed in the physical node. For example, components such as redis, rocksdb, leveldb, and hbase can be used to implement the key-value storage capabilities of each of the above databases.

[0148] In an alternative implementation, the metadata of the delayed message system also records hash node information, physical node information, and the mapping association records between physical nodes and hash nodes. Among them, the hash node information can include the number information of hash nodes on the preset hash ring, and the hash value range corresponding to each hash node. By recording the hash value range corresponding to the hash node, it is convenient to determine the physical node corresponding to the delayed message based on the hash value of the target service parameter in the future; the physical node information refers to the information of each physical node in the distributed cluster, such as IP address, available disk information, etc.

[0149] The mapping association record between the physical node and the hash node is used to record the hash nodes allocated to each physical node. Each physical node can be associated with one or more hash nodes. Since there may be expansion or contraction operations on the physical node, the mapping association record will record the hash node allocation situation for each physical node every time.

[0150] In the embodiments of the present disclosure, the database in each physical node in the distributed cluster is a key-value database. Since the key-value database stores data in the form of key-value pairs, the access speed of the data is relatively fast, thereby improving the processing performance of the delayed message.

[0151] In an alternative embodiment, the data structure of the delayed message can be composed of key-value data. Among them, the key data of the delayed message can include fields such as hash node number, expiration time, delay time, message queue type, message queue cluster, topic name, target service parameter, etc., and the value data of the delayed message can include the data written by the service.

[0152] Since the key database of the delayed message includes expiration time information, after receiving the delayed message, it can be sorted in the delayed message database according to the hash node number and expiration time in the key data of the delayed message through the dictionary of the key data, so that the latest stored delayed message in the delayed message database is the message that will expire soon, so that the delayed message system can determine whether the delayed message has expired by reading the expiration time.

[0153] In practical applications, after writing the delayed message into the delayed message database, an index corresponding to the delayed message can also be generated and written into the index database. Among them, the delayed message index structure stored in the index database can be composed of key-value data. Among them, the key data in the delayed message index can include fields such as hash node number, target service parameter, message queue type, message queue cluster, topic name, expiration time, etc., and the value data in the delayed message index can be the key data of the delayed message. Subsequently, when receiving a message recall request, the delayed message indicated by the message recall request can be determined by querying the index database and the delayed message database as the message to be recalled.

[0154] The recall message database is used to record the delayed messages in the delayed message database that belong to the messages to be recalled. The data structure in the recall message database can be composed of key-value data. Among them, the key data can include fields such as hash node number, target service parameter, message queue type, message queue cluster, topic name, expiration time, etc., and the value data can be the key data of the delayed message.

[0155] When the delayed message system reads the delayed message database, it will also read the data in the recall message database so that when sending the delayed message to the target message queue, the correct target service parameter can be read.

[0156] It can be seen that the embodiments of the present disclosure record the delayed messages recalled by the message recall request in the recall message database, so as to determine whether the delayed message has been recalled by querying the recall message database when the delayed message expires and is executed, and discard the delayed message when it is determined that the delayed message has been recalled, realizing the recall function of the delayed message, thereby reducing the waste of system resources caused by the processing of the delayed message.

[0157] In practical applications, in order to optimize the resource allocation of distributed clusters and improve resource utilization, the physical nodes in the distributed clusters are often expanded or reduced in size. The expansion operation refers to increasing the number of physical nodes in the distributed cluster to improve the processing capacity of the distributed cluster, and the reduction operation refers to reducing the number of physical nodes in the distributed cluster to reduce unnecessary resource usage and thus reduce costs.

[0158] like Figure 8 As shown, it is a schematic diagram of expanding the capacity of a distributed cluster provided by an embodiment of the present disclosure. Among them, the original distributed cluster includes physical node 1, physical node 2 and physical node 3, physical node 1 corresponds to hash node 1 and hash node 2, physical node 2 corresponds to hash node 3 and hash node 4, and physical node 3 corresponds to hash node 5 and hash node 6. After adding physical node 4, the mapping association between hash node 6 and physical node 3 is released, and hash node 6 is reallocated to the newly added physical node 4 to realize the expansion operation of the distributed cluster.

[0159] When expanding or shrinking the physical nodes in a distributed cluster, data inconsistency may occur. For example, when a delayed message is received, assuming that the hash value is calculated based on the target service parameters in the delayed message, the physical node corresponding to hash node 6, that is, physical node 3, is determined as the physical node corresponding to the delayed message. At this time, if Figure 8 The expansion operation shown, then, when a message withdrawal request corresponding to the delayed message is subsequently received, a hash value is calculated based on the target business parameters in the message withdrawal request, and the physical node corresponding to the determined hash node 6 is the newly added physical node 4.

[0160] Therefore, in order to maintain the consistency of data in the distributed cluster, the embodiment of the present disclosure may also utilize a node expansion method based on data migration to manage the data in each physical node on the distributed cluster.

[0161] In an embodiment of the present disclosure, in response to the physical node having a corresponding relationship with the third target hash node on the preset hash ring being changed from the fifth target physical node to the sixth target physical node, the data in the pullback message database and the delayed message database on the fifth target physical node are migrated to the sixth target physical node.

[0162] For example, in Figure 8When the physical node corresponding to the hash node 6 on the preset hash ring shown changes from physical node 3 to physical node 4, it is necessary to migrate the data in the rollback message database and the delayed message database on physical node 3 to physical node 4, so that when a message rollback request is received subsequently, the data can be read from the changed physical node 4, thereby maintaining the consistency of the data in the physical node.

[0163] In practical applications, after the mapping association relationship between the physical node and the hash node changes, it is first necessary to suspend receiving the rollback message request. After migrating the data on the original physical node to the changed physical node, then resume receiving the rollback message request and process the rollback message request.

[0164] In an optional implementation manner, after the expansion operation or the contraction operation is completed, the physical node corresponding to the hash node on the preset hash ring may change, and some historical data may still be stored in the original physical node. At this time, the data cannot be read from the new physical node. Therefore, the embodiments of the present disclosure can also determine the physical node corresponding to the hash node according to the list of historical physical nodes corresponding to the hash node, thereby maintaining the consistency of the data.

[0165] In the embodiments of the present disclosure, when a message rollback request is received, first calculate the hash value according to the target service parameter in the message rollback request, and determine the hash node corresponding to the hash value on the preset hash ring, which is determined as the second target hash node; then determine the physical node corresponding to the second target hash node according to the list of historical physical nodes corresponding to the second target hash node, as the second target physical node.

[0166] Among them, the list of historical physical nodes includes the change records of the physical nodes corresponding to the second target hash node. For example, the physical node corresponding to the hash node 6 was physical node 3 before a certain time point, and the physical node corresponding to it was physical node 4 after this time point.

[0167] In another optional implementation manner, it is also possible to only record the latest allocated physical node corresponding to the second target hash node. When the delayed message corresponding to the rollback message request cannot be queried in the database of the latest allocated physical node, broadcast and distribute the rollback message request to each physical node in the distributed cluster, and each physical node queries its own delayed message database to determine the physical node corresponding to the rollback message request, so that the delayed message indicated to be rolled back by the rollback message request can be recorded in the rollback message database on this physical node subsequently.

[0168] It can be seen that the embodiments of the present disclosure can manage the data in each physical node on the distributed cluster by using the node expansion method of data migration, thereby maintaining the consistency of the data in the distributed database.

[0169] To implement the above embodiments, the present disclosure also proposes a message processing apparatus based on a message queue system. Figure 9 As shown in the structural schematic diagram of a message processing apparatus based on a message queue system provided by an embodiment of the present disclosure, the apparatus can be implemented by software and / or hardware and is generally integrated in an electronic device. Figure 9 As shown, the apparatus includes:

[0170] A first receiving module 901, configured to receive a message recall request sent by a producer of a message queue in the message queue system, determine the delayed message indicated to be recalled by the message recall request as the message to be recalled, and record the message to be recalled in a recall message database; the recall message database is used to record the delayed messages belonging to the message to be recalled in the delayed message database, and the delayed message database stores delayed messages with a delay time;

[0171] A first determining module 902, configured to determine whether the recall message database contains the first delayed message in response to the expiration of the delay time of the first delayed message in the delayed message database;

[0172] A discarding module 903, configured to discard the first delayed message if the recall message database contains the first delayed message.

[0173] In an alternative embodiment, the apparatus further includes:

[0174] A sending module, configured to send the first delayed message to a target message queue if the recall message database does not contain the first delayed message, so that a consumer of the target message queue reads and processes the first delayed message from the target message queue.

[0175] In an alternative embodiment, the message recall request carries target service parameters for indicating to recall the delayed messages matching the target service parameters. The first determining module includes:

[0176] A first determining sub-module, configured to match the target service parameters from the delayed message database and determine the delayed messages that are matched and hit as the messages to be recalled.

[0177] In an alternative embodiment, the message recall request further carries an expiration time condition for determining the delayed messages indicated to be recalled by the message recall request according to the expiration time of the delayed messages. The first determining module includes:

[0178] A second determination sub-module, configured to match the target service parameter from the delay message database, and determine, as the messages to be withdrawn, the delay messages in the delay messages that match and whose expiration time meets the expiration time condition.

[0179] In an optional implementation manner, the apparatus further includes:

[0180] A second determination module, configured to receive a second delay message, and determine, as a first target physical node, the physical node corresponding to the second delay message from a distributed cluster based on a preset routing policy; a withdrawal message database and a delay message database are respectively deployed on each physical node in the distributed cluster;

[0181] A first storage module, configured to store the second delay message in the delay message database on the first target physical node.

[0182] In an optional implementation manner, the apparatus further includes:

[0183] A third determination module, configured to determine, as a second target physical node, the physical node corresponding to the message withdrawal request from the distributed cluster based on the preset routing policy;

[0184] Correspondingly, the first receiving module includes:

[0185] A first recording sub-module, configured to record the messages to be withdrawn in the withdrawal message database on the second target physical node.

[0186] In an optional implementation manner, the third determination module includes:

[0187] A third determination sub-module, configured to calculate a hash value according to the target service parameter in the second delay message, and determine, as a first target hash node, the hash node corresponding to the hash value on a preset hash ring; each hash node in the preset hash ring has a corresponding hash value range and a physical node;

[0188] A fourth determination sub-module, configured to determine the physical node corresponding to the first target hash node as the first target physical node.

[0189] In an optional implementation manner, the third determination module includes:

[0190] A fifth determination sub-module, configured to calculate a hash value according to the target service parameter in the message withdrawal request, and determine the hash node corresponding to the hash value on the preset hash ring as a second target hash node;

[0191] A sixth determination sub-module, configured to determine, according to the historical physical node list corresponding to the second target hash node, the physical node corresponding to the second target hash node as the second target physical node; the historical physical node list includes the change records of the physical nodes corresponding to the second target hash node.

[0192] In an optional implementation manner, the apparatus further includes:

[0193] A second receiving module, configured to receive a third delay message, and determine, based on a random policy, the physical node corresponding to the third delay message in the distributed cluster as the third target physical node; a rollback message database and a delay message database are respectively deployed on each physical node in the distributed cluster;

[0194] A second storage module, configured to store the third delay message in the delay message database on the third target physical node.

[0195] A first receiving module, the apparatus further includes:

[0196] A broadcast distribution module, configured to broadcast and distribute the message rollback request to each physical node in the distributed cluster, and determine, by each physical node querying its own delay message database, the physical node corresponding to the message rollback request as the fourth target physical node;

[0197] Correspondingly, the first receiving module includes:

[0198] A second recording sub-module, configured to record the message to be rolled back in the rollback message database on the fourth target physical node.

[0199] In an optional implementation manner, the apparatus further includes:

[0200] A migration module, configured to, in response to the physical node corresponding to the third target hash node on the preset hash ring changing from the fifth target physical node to the sixth target physical node, migrate the data in the rollback message database and the delay message database on the fifth target physical node to the sixth target physical node.

[0201] In the message processing device based on the message queue system provided by the embodiments of the present disclosure, first, a message recall request sent by a producer of a message queue in the message queue system is received, the delayed message indicated to be recalled by the message recall request is determined as the message to be recalled, and the message to be recalled is recorded in a recall message database, where the recall message database is used to record the delayed messages belonging to the message to be recalled in a delayed message database, and the delayed message database stores delayed messages with a delay time; in response to the expiration of the delay time of a first delayed message in the delayed message database, it is determined whether the first delayed message is included in the recall message database. If the first delayed message is included in the recall message database, it indicates that the first delayed message has been recalled, and at this time, the first delayed message is discarded.

[0202] In the embodiments of the present disclosure, after receiving the message recall request, by determining the delayed message indicated to be recalled by the message recall request as the message to be recalled and recording it in the recall message database, the recall function for the delayed message is implemented. Since the embodiments of the present disclosure support the recall of delayed messages, after the delayed message expires, it is possible to determine whether the delayed message has been recalled by querying the recall message database, and when it is determined that the delayed message has been recalled, the delayed message is discarded. Compared with a message system that does not support the delayed message recall function, the message processing method based on the message queue system provided by the embodiments of the present disclosure can reduce resource occupancy.

[0203] The message processing device based on the message queue system provided by the embodiments of the present disclosure can execute the message processing method based on the message queue system provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0204] In addition to the above methods and devices, the embodiments of the present disclosure also provide a computer-readable storage medium, in which instructions are stored. When the instructions run on a terminal device, the terminal device implements the message processing method based on the message queue system described in the embodiments of the present disclosure.

[0205] The embodiments of the present disclosure also provide a computer program product, where the computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the message processing method based on the message queue system described in the embodiments of the present disclosure is implemented.

[0206] In addition, the embodiments of the present disclosure also provide a message processing device based on the message queue system. Refer to Figure 10 as shown, it may include:

[0207] A processor 1001, a memory 1002, an input device 1003, and an output device 1004. The number of processors 1001 in the message processing device based on the message queue system can be one or more. Figure 10 Here, one processor is taken as an example. In some embodiments of the present disclosure, the processor 1001, the memory 1002, the input device 1003, and the output device 1004 can be connected through a bus or other means. Among them, Figure 10 Here, connection through a bus is taken as an example.

[0208] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing of the message processing device based on the message queue system by running the software programs and modules stored in the memory 1002. The memory 1002 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc. In addition, the memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. The input device 1003 can be used to receive input digital or character information, and generate signal inputs related to user settings and function controls of the message processing device based on the message queue system.

[0209] Specifically, in this embodiment, the processor 1001 loads the executable files corresponding to the processes of one or more application programs into the memory 1002 according to the following instructions, and the processor 1001 runs the application programs stored in the memory 1002, so as to implement various functions of the above-mentioned message processing device based on the message queue system.

[0210] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0211] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A message processing method based on a message queue system, characterized in that: The method comprises: Receive a message withdrawal request sent by a producer of a message queue in a message queue system, determine the delayed message indicated to be withdrawn by the message withdrawal request as a message to be withdrawn, and record the message to be withdrawn in a withdrawal message database; the withdrawal message database is used to record delayed messages belonging to the messages to be withdrawn in the delayed message database, and the delayed message database stores delayed messages with a delay time; In response to expiration of a delay time of a first delayed message in the delayed message database, determining whether the first delayed message is included in the withdrawn message database; If the recall message database includes the first delayed message, the first delayed message is discarded.

2. The method according to claim 1, characterized in that The method further comprises: If the first delayed message is not included in the withdrawn message database, the first delayed message is sent to the target message queue, so that a consumer of the target message queue reads and processes the first delayed message from the target message queue.

3. The method according to claim 1, characterized in that The message withdrawal request carries a target service parameter, which is used to indicate the withdrawal of a delayed message matching the target service parameter, and the determining the delayed message indicated to be withdrawn by the message withdrawal request as a message to be withdrawn includes: The target service parameters are matched from the delayed message database, and the matched delayed messages are determined as messages to be withdrawn.

4. The method according to claim 3, characterized in that The message withdrawal request also carries an expiration time condition, where the expiration time condition is used to determine the delayed message indicated by the message withdrawal request to be withdrawn according to the expiration time of the delayed message, and determining the delayed message indicated by the message withdrawal request to be withdrawn as the message to be withdrawn includes: The target service parameters are matched from the delayed message database, and delayed messages whose expiration time meets the expiration time condition among the matched delayed messages are determined as messages to be withdrawn.

5. The method according to claim 1, characterized in that The method further comprises: Receive a second delayed message, and determine a physical node corresponding to the second delayed message from a distributed cluster based on a preset routing strategy as a first target physical node; a fallback message database and a delayed message database are respectively deployed on each physical node in the distributed cluster; The second delay message is stored in a delay message database on the first target physical node.

6. The method according to claim 5, characterized in that After receiving the message withdrawal request sent by the producer of the message queue in the message queue system, the method further includes: Determine, based on the preset routing strategy, a physical node corresponding to the message withdrawal request from the distributed cluster as a second target physical node; Correspondingly, recording the to-be-retracted message in a retracted message database includes: The message to be withdrawn is recorded in a withdrawal message database on the second target physical node.

7. The method according to claim 5 or 6, characterized in that: The determining, based on a preset routing strategy, from a distributed cluster a physical node corresponding to the delayed message as a first target physical node includes: Calculate a hash value according to the target service parameter in the second delay message, and determine a hash node corresponding to the hash value on a preset hash ring as a first target hash node; each hash node in the preset hash ring has a corresponding hash value range and a physical node; The physical node corresponding to the first target hash node is determined as the first target physical node.

8. The method according to claim 7, characterized in that The determining, based on the preset routing strategy, from the distributed cluster a physical node corresponding to the message withdrawal request as a second target physical node includes: Calculate a hash value according to the target service parameter in the message withdrawal request, and determine a hash node corresponding to the hash value on the preset hash ring as a second target hash node; According to the historical physical node list corresponding to the second target hash node, the physical node corresponding to the second target hash node is determined as the second target physical node; the historical physical node list includes the change record of the physical node corresponding to the second target hash node.

9. The method according to claim 1, characterized in that: The method further comprises: Receive a third delayed message, and determine a physical node corresponding to the third delayed message from a distributed cluster based on a random strategy as a third target physical node; a fallback message database and a delayed message database are respectively deployed on each physical node in the distributed cluster; The third delay message is stored in a delay message database on the third target physical node.

10. The method according to claim 9, characterized in that After receiving the message withdrawal request, the method further includes: Broadcasting and distributing the message withdrawal request to each physical node in the distributed cluster, and determining the physical node corresponding to the message withdrawal request as the fourth target physical node by querying the delayed message database of each physical node; Correspondingly, recording the to-be-retracted message in a retracted message database includes: The message to be withdrawn is recorded in a withdrawal message database on the fourth target physical node.

11. The method according to claim 1, characterized in that: The method further comprises: In response to the physical node corresponding to the third target hash node on the preset hash ring being changed from the fifth target physical node to the sixth target physical node, the data in the recall message database and the delayed message database on the fifth target physical node are migrated to the sixth target physical node.

12. A message processing device based on a message queue system, characterized in that: The device comprises: A first receiving module is used to receive a message withdrawal request sent by a producer of a message queue in a message queue system, and determine the delayed message indicated to be withdrawn by the message withdrawal request as a message to be withdrawn, and record the message to be withdrawn in a withdrawal message database; the withdrawal message database is used to record delayed messages belonging to the messages to be withdrawn in the delayed message database, and the delayed message database stores delayed messages with a delay time; A first determining module, configured to determine whether the withdrawal message database contains the first delayed message in response to expiration of a delay time of the first delayed message in the delayed message database; A discarding module is used to discard the first delayed message if the first delayed message is included in the withdrawal message database.

13. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the message processing method based on the message queue system as described in any one of claims 1-11 above.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the message processing method based on the message queue system as described in any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images