Message processing method and device
By deploying local message processing components and local message queues on the message production and consumer side, the problem of complex logic and poor performance of transactional message processing in the prior art is solved, efficient message processing and transaction consistency is achieved, and system resource occupation is alleviated.
Patent Information
- Application Number
- CN202411648701.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-03-11
AI Technical Summary
In the prior art, the processing logic of transactional message sending and receiving is complex, and the processing delay increases in high concurrency scenarios, and the processing performance is poor. At the same time, log file management and cleaning occupy system resources, affecting disk IO performance.
The local message processing component and local message queue are deployed on the message production side and the consumer side respectively. The messages are persisted to the local database before local transaction submission, and the messages are added to the message queue after the transaction submission is successful to achieve asynchronous processing.
The processing logic of transactional message sending and receiving is simplified, processing performance is improved, data inconsistency is avoided, business processing and transaction consistency between message production and consumer side and message sending is alleviated, and log file management is used to occupancy on system resources.
Smart Images
Figure CN119597502A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a message processing method and device. Background Art
[0002] A transactional message queue based on a database is a technology that ensures that message delivery and database operations are completed atomically in the same transaction. In related technologies, the processing logic of transactional message sending and receiving is complex, and transactional message queues need to coordinate transactions between multiple services or databases to ensure data consistency. Moreover, in high-concurrency scenarios, when the number of message processing reaches 100,000, the processing delay will increase significantly, and the processing performance will be poor. Summary of the invention
[0003] The present disclosure provides a message processing method and device to solve one of the technical problems in the related art at least to a certain extent. The technical solution of the present disclosure is as follows:
[0004] According to a first aspect of an embodiment of the present disclosure, a message processing method is provided, which is applied to a message production end, and includes: before a local transaction is committed, uploading a message generated during the execution of the local transaction to a local message processing component; determining a local target message queue corresponding to the message through the local message processing component; calling a message persister of the local target message queue to save the message to a local database and register a transaction listener of the local transaction; and when the transaction listener detects that the local transaction is successfully committed, adding the message to the local target message queue, so as to send the message to a message consumer end through the local target message queue.
[0005] According to the second aspect of an embodiment of the present disclosure, a message processing method is provided, which is applied to a message consumer end, including: receiving a message sent by a message producer end, and uploading the message to a local message processing component; determining a local target message queue corresponding to the message through the local message processing component; calling a message persister of the local target message queue to save the message in a local database; and adding the message to the local target message queue to send the message to a local message processing module through the local target message queue.
[0006] According to a third aspect of an embodiment of the present disclosure, a message processing device is provided, which is applied to a message production end, and includes: an upload module, which is used to upload the message generated during the execution of the local transaction to a local message processing component before the local transaction is committed; a first determination module, which is used to determine the local target message queue corresponding to the message through the local message processing component; a first calling module, which is used to call the message persister of the local target message queue to save the message to a local database and register a transaction listener of the local transaction; a first processing module, which is used to add the message to the local target message queue when the transaction listener detects that the local transaction is successfully committed, so as to send the message to a message consumer end through the local target message queue.
[0007] According to the fourth aspect of an embodiment of the present disclosure, a message processing device is provided, which is applied to a message consumption end, and includes: a third processing module, used to receive a message sent by a message production end, and upload the message to a local message processing component; a second determination module, used to determine a local target message queue corresponding to the message through the local message processing component; a second calling module, used to call a message persister of the local target message queue to save the message in a local database; and a fourth processing module, used to add the message to the local target message queue, so as to send the message to the local message processing module through the local target message queue.
[0008] According to the fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the message processing method as described in the embodiment of the first aspect of the present disclosure, or to implement the message processing method as described in the embodiment of the second aspect of the present disclosure.
[0009] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the message processing method as described in the embodiment of the first aspect of the present disclosure, or to execute the message processing method as described in the embodiment of the second aspect of the present disclosure.
[0010] According to the seventh aspect of the embodiments of the present disclosure, a computer program product is provided, including: a computer program, which, when executed by a processor, implements the message processing method as described in the embodiments of the first aspect of the present disclosure, or implements the message processing method as described in the embodiments of the second aspect of the present disclosure.
[0011] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:
[0012] 1. By using the local components and local message queues on the message production side to complete the message sending (before the local transaction is committed, the message is first persisted in the local database, and after the local transaction is successfully committed, the message is queued and sent), the processing logic of transactional message sending is simplified. At the same time, this process decouples message processing and business logic processing, allowing asynchronous message processing and improving processing performance;
[0013] Similarly, by using the local components and local message queues of the message consumer end (when processing messages asynchronously) to send messages to the local message processing module for processing (after receiving the message, first persist the message, then queue the message and send it to the local message processing module for processing), the processing logic of transactional message reception is simplified and the processing performance is improved;
[0014] 2. For the message production end, the message is added to the local target message queue only after the transaction listener monitors that the local transaction is successfully submitted, which can ensure the atomicity of message sending and local transactions. That is, if the local transaction fails, the message will not be sent to the message consumer end, thereby avoiding the problem of data inconsistency and ensuring the transaction consistency of business processing and message sending at the message production end. For the message consumer end, asynchronous message processing (after receiving the message, the local component and local message queue of the message consumer end are used to send the message to the local message processing module for processing) can also ensure the transaction consistency of message reception and business processing at the message consumer end;
[0015] 3. In the present disclosure, local databases are used to persist messages at both the message production and consumption ends, which can alleviate the problems of system resource occupation and high disk IO performance requirements caused by log file management and cleanup to a certain extent, and can prevent messages from being lost due to temporary network failures or unavailability of message queue services. After failure recovery, these messages can be reloaded from the local database and sent or processed.
[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0018] Figure 1 It is a schematic diagram of a message processing principle in the related technology;
[0019] Figure 2 is a flowchart of a message processing method shown in the first embodiment of the present disclosure;
[0020] Figure 3 is a flowchart of a message processing method shown in the second embodiment of the present disclosure;
[0021] Figure 4 is a flowchart of a message processing method shown in the third embodiment of the present disclosure;
[0022] Figure 5 is a schematic diagram of the principle of the message processing method shown in the fourth embodiment of the present disclosure;
[0023] Figure 6 is a flowchart of a message processing method provided by the fifth embodiment of the present disclosure;
[0024] Figure 7 is a flowchart of a message processing method provided by the sixth embodiment of the present disclosure;
[0025] Figure 8 is a flowchart of a message processing method provided by the seventh embodiment of the present disclosure;
[0026] Fig. 9 is a schematic diagram of the principle of a message processing method shown in the eighth embodiment of the present disclosure;
[0027] Fig.10 is a structural diagram of a message processing device shown in the ninth embodiment of the present disclosure;
[0028] Fig.11 is a structural diagram of a message processing device shown in the tenth embodiment of the present disclosure;
[0029] Fig.12 It is a schematic structural diagram of an electronic device 1200 shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0030] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0032] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0033] Database-based transactional message queues are a technology that ensures that message delivery and database operations are completed atomically in the same transaction. Currently, mainstream transactional message systems have implemented this approach, which can basically meet the data consistency requirements of distributed applications, especially when dealing with complex business scenarios that require operations across multiple services or databases, such as banking and financial services. Mainstream implementations include RocketMQ, SofaMQ, etc.
[0034] Its principle is as follows Figure 1 As shown, through Figure 1 You can clearly know the message sending and consumption mechanism, and its workflow is as follows:
[0035] Broker is mainly responsible for the storage, transmission, query, high availability, etc. of messages. When it starts, it will register its own information with each NameServer (NameServer acts as a registration center and is responsible for the management of broker and routing information). This information includes its own IP and port number, and the topics of this Broker.
[0036] After startup, the Producer will establish a connection with the NameServer and regularly obtain Broker information from the NameServer. When sending a message, it will look for the corresponding Broker address based on the topic to which the message needs to be sent. If there is a Broker address, a request will be sent to this Broker; if not found, whether to send the message will be determined based on whether automatic topic creation is allowed.
[0037] After receiving the message from the Producer, the Broker will store the message and make it persistent. If there is a slave node, it will also actively synchronize it to the slave node to achieve data backup.
[0038] After the Consumer is started, it will also establish a connection with the NameServer, and regularly obtain information about the Broker and the corresponding topic from the NameServer. Then, it will find the address of the corresponding Broker according to the topic information it needs to subscribe to, and then establish a connection with the Broker to obtain messages and consume.
[0039] However, the above solution has the following disadvantages:
[0040] 1. The processing logic of sending and receiving transactional messages is complex. Transactional message queues need to coordinate transactions between multiple services or databases to ensure data consistency.
[0041] 2. In high-concurrency scenarios, when the message processing of current mainstream products reaches 100,000 levels, the processing delay will increase significantly and the processing performance will be poor.
[0042] 3. Log files are generally used to persist messages. The management and cleaning of log files requires system resources and has high requirements for disk IO performance.
[0043] In response to the above problems, the present disclosure proposes a message processing method and device to simplify the processing logic of transactional message sending and receiving, improve processing performance, and alleviate the problems of system resource occupation and high disk IO performance requirements caused by log file management and cleanup.
[0044] The message processing method and device of the embodiments of the present disclosure are described below with reference to the accompanying drawings.
[0045] Figure 2 It is a flowchart of the message processing method shown in the first embodiment of the present disclosure.
[0046] It should be noted that the message processing method of the embodiment of the present disclosure is applied to the message production end.
[0047] like Figure 2 As shown, the message processing method includes the following steps:
[0048] Step 201: before the local transaction is committed, the message generated during the execution of the local transaction is uploaded to the local message processing component.
[0049] The number of messages generated during the execution of a local transaction may be one or more.
[0050] In order to simplify the processing logic of message sending, the present disclosure deploys a local message processing component at the message production end, such as a message component core (MessageCore), so as to process and send messages generated during the execution of local transactions at the message production end through the component.
[0051] As a possible implementation method, the message production end starts a local transaction to process the business logic, and uploads the messages generated during the execution of the local transaction to the local message processing component before the business code processing is completed and the local transaction is committed. Optionally, the message production end can call the sending method of the message component core (MessageCore) to upload the message to the local message processing component.
[0052] Step 202: Determine the local target message queue corresponding to the message through the local message processing component.
[0053] It should be noted that multiple local message queues are deployed in the message production end. After the message production end uploads one or more messages generated during the execution of the local transaction to the local message processing component, the local message processing component can determine the local target message queue corresponding to each message from the multiple local message queues.
[0054] As a possible implementation method, each message carries target message queue information, such as target, so that the message producer can determine the local target message queue corresponding to the message from multiple local message queues through a local message processing component according to the target message queue information carried in the message.
[0055] Step 203: Call the message persister of the local target message queue to save the message to the local database and register the transaction listener of the local transaction.
[0056] In order to ensure business continuity and data integrity, in the embodiment of the present disclosure, after the message production end determines the local target message queue corresponding to the message, it can call the message persister of the local target message queue to save the message to the local database. Thus, by persisting the message to the local database, even if there is a temporary network failure or the message queue service is unavailable, the message will not be lost, and after the failure is recovered, the message can be reloaded from the local database to continue processing the unfinished business logic, thereby reducing the risk of business interruption.
[0057] In the present disclosure, using a local database to persist messages can alleviate the problems of system resource occupation and high disk IO performance requirements caused by log file management and cleaning to a certain extent.
[0058] It should be noted that in the embodiment of the present disclosure, in addition to calling the message persister of the local target message queue to persist the message, the message producer also calls the message persister of the local target message queue to persist the message and register the transaction listener of the local transaction to monitor whether the local transaction is provided successfully.
[0059] As an example, the save method of the message persister (MessageRoller) of the local target message queue can be called to persist the message, and the message persister (MessageRoller) of the local target message queue will automatically register a transaction listener with the current local transaction after the save method writes the table to listen to the transaction submission success event.
[0060] As a possible implementation manner, the message production end may obtain the configuration information of the local target message queue through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
[0061] As an example, a local message processing component, such as a message component core (MessageCore), can match the local target message queue corresponding to the message according to the target of the message, and read the configuration information of the local target message queue. Then, according to the persistence configuration information of the target message queue, the message persister of the local target message queue is called.
[0062] Step 204: When the transaction listener detects that the local transaction is successfully committed, the message is added to the local target message queue, so as to send the message to the message consumer through the local target message queue.
[0063] For the atomicity of message sending and local transactions, in the embodiment of the present disclosure, the message producer adds the message to the local target message queue only after the transaction listener detects that the local transaction is successfully submitted, so as to send the message to the message consumer through the local target message queue. Therefore, if the local transaction fails, the message will not be sent to the message consumer, thus avoiding the problem of data inconsistency and ensuring the transaction consistency between the business processing of the message producer and the message sending.
[0064] In summary, by using the local components and local message queues on the message production side to complete message sending (before the local transaction is committed, the message is first persisted in the local database, and after the local transaction is successfully committed, the message is queued and sent), the processing logic of transactional message sending is simplified. At the same time, this process decouples message processing and business logic processing, allows asynchronous message processing, and improves processing performance.
[0065] The message processing method of the embodiment of the present disclosure uploads the message generated during the execution of the local transaction to the local message processing component before the local transaction is committed; determines the local target message queue corresponding to the message through the local message processing component; calls the message persister of the local target message queue to save the message to the local database and registers the transaction listener of the local transaction; when the transaction listener detects that the local transaction is successfully committed, adds the message to the local target message queue to send the message to the message consumer end through the local target message queue. Therefore, by uploading the message to the local message processing component before the local transaction is committed and taking a series of measures to complete the sending of the message, the processing logic of transactional message sending is simplified, the processing performance is improved, and the transaction consistency between the business processing of the message production end and the message sending is ensured.
[0066] In order to clearly illustrate how to send messages to the message consumer through the local target message queue in the above embodiment, the present disclosure proposes another message processing method.
[0067] Figure 3 It is a flowchart of the message processing method shown in the second embodiment of the present disclosure.
[0068] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message production end.
[0069] like Figure 3 As shown, the message processing method includes the following steps:
[0070] Step 301: before the local transaction is committed, the message generated during the execution of the local transaction is uploaded to the local message processing component.
[0071] Step 302: Determine the local target message queue corresponding to the message through the local message processing component.
[0072] Step 303: Call the message persister of the local target message queue to save the message to the local database and register the transaction listener of the local transaction.
[0073] Step 304: When the transaction listener detects that the local transaction is successfully committed, the message is added to the local target message queue.
[0074] Step 305: sort the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue.
[0075] The indication information of the message is used to indicate the basis for sorting the message, for example, it may be the message type, label, content, etc.
[0076] In the disclosed embodiment, the message production end can sort the message to the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue. For example, the message with message tag flag=A is sent to the local recipient message queue 1 for processing, and the message with message tag flag=B is sent to the local recipient message queue 2 for processing.
[0077] Step 306: Send the message to the message consumer through the local target recipient message queue, or call a third-party message adapter through the local target recipient message queue to send the message to the message consumer.
[0078] In the present disclosure, the local target recipient message queue can directly send messages to the message consumer, or call the message middleware adapter interface to send messages to the message consumer through the message middleware adapter (i.e. call a third-party message adapter to send messages to the message consumer).
[0079] It should be noted that the execution process of steps 301 to 304 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0080] The message processing method of the disclosed embodiment sorts the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue; sorts the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue, thereby different messages can be sorted into different queues according to their content or type, thereby realizing classified processing. Moreover, the messages sorted into specific recipient queues can be processed in parallel, reducing the waiting time for message processing.
[0081] It should be noted that in this disclosure, if a message fails to be sent or times out, it can be re-added to the local target message queue. Figure 4 This process is described below.
[0082] Figure 4 It is a flowchart of a message processing method shown in the third embodiment of the present disclosure.
[0083] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message production end.
[0084] like Figure 4 As shown, the message processing method includes the following steps:
[0085] Step 401: before the local transaction is committed, the message generated during the execution of the local transaction is uploaded to the local message processing component.
[0086] Step 402: Determine the local target message queue corresponding to the message through the local message processing component.
[0087] Step 403: Call the message persister of the local target message queue to save the message to the local database and register the transaction listener of the local transaction.
[0088] Step 404: When the transaction listener detects that the local transaction is successfully committed, the message is added to the local target message queue, so as to send the message to the message consumer through the local target message queue.
[0089] Step 405: When the message fails to be sent or times out, the local message restorer is called to query the message from the local database and add the message to the local target message queue again.
[0090] In the present disclosure, messages that fail to be sent or are not sent due to timeout will be retrieved from the local database by a message recoverer (MessageRecover) and re-added to the local target message queue, thereby reducing instability and data inconsistency caused by message loss or processing failure.
[0091] The message processing method of the embodiment of the present invention, when a message fails to be sent or times out, calls a local message recoverer to query the message from the local database and re-adds the message to the local target message queue. Thus, through the local message recoverer, when a message fails to be sent or times out, the message can be recovered from the local database and put back into the queue, thereby ensuring that the message will not be lost.
[0092] In order to explain the above embodiment clearly, an example is given for illustration.
[0093] Figure 5 It is a schematic diagram of the principle of the message processing method shown in the fourth embodiment of the present disclosure.
[0094] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message production end.
[0095] like Figure 5 As shown, the message processing method includes the following steps:
[0096] Step 501, call the put method to send a message.
[0097] The business application starts a local transaction to process business logic, and calls the sending method of the message component core (MessageCore) after the business code processing is completed and before the local transaction is committed.
[0098] Step 502, matching the queue according to the target of the message and reading the queue information.
[0099] The message component core matches the local target message queue corresponding to the message according to the target of the message, and obtains the configuration information of the local target message queue.
[0100] Step 503: call the save method of the roller of the queue for persistence.
[0101] According to the persistence configuration of the local target message queue, the message persister (MessageRoller) of the queue is called to save the message to the local database.
[0102] Step 504: If there is a current transaction, register the listener of the current transaction after the save method writes the table.
[0103] If there is a transaction currently, the transaction is the aforementioned local transaction, and the listener of the current transaction is registered after the save method writes the table (writes to the local database). The message persister will automatically register a transaction listener with the current local transaction to listen for the transaction submission success event.
[0104] Step 505: triggering a transaction submission success event.
[0105] The business application commits the current local transaction, triggering a transaction commit success event.
[0106] Step 506, queue the message.
[0107] In step 504, the registered transaction listener puts the message into the local target message queue in the memory.
[0108] Step 507, sorting to a recipient queue according to the recipient.
[0109] The processing program of the local target message queue will sort the message into the corresponding local recipient queue according to the recipient information of the message (ie, the indication information of the message).
[0110] Step 508, send.
[0111] The handler of the local target message queue can either send the message directly or call the message middleware adapter interface to send the message through the message middleware adapter.
[0112] It should be noted that for messages that fail to be sent, they will be retrieved from the local database by the message recovery device (MessageRecover) and re-added to the local target message queue. Figure 5 As shown, MessageRecover will poll the timeout and unsent messages and re-queue them.
[0113] In summary, by persisting messages in the local database, the messages are committed together with the local transactions of the business. Through the local transaction listener, after the local transaction is successfully committed, the message is queued and sent. If the sending fails, the message component is responsible for requeuing the message that needs to be retried from the local database. In this way, the processing logic of transactional message sending is simplified, the processing performance is improved, and the transaction consistency between the business processing of the message producer and the message sending is guaranteed.
[0114] The above embodiments are described from the perspective of the message production end. In order to more clearly illustrate the message processing process, the embodiments of the present disclosure provide a possible implementation of a message processing method described from the perspective of the message consumption end. Figure 6 It is a flowchart of the message processing method provided by the fifth embodiment of the present disclosure.
[0115] It should be noted that the message processing method of the embodiment of the present disclosure is applied to the message consumer. In the present disclosure, if the message consumer processes the message synchronously, the local message processing module is directly exposed as a message processing service. If the message is processed asynchronously, the message processing method of the embodiment of the present disclosure is adopted.
[0116] like Figure 6 As shown, the message processing method includes the following steps:
[0117] Step 601, receiving a message sent by a message production end, and uploading the message to a local message processing component.
[0118] The number of messages sent by the message producer can be one or more.
[0119] In order to simplify the processing logic of message sending, in the present disclosure, a local message processing component is also deployed on the message consumer side, for example, a message component core (MessageCore), so as to process and send the received messages on the message consumer side through the component.
[0120] As a possible implementation method, after receiving the message, the message adapter at the message consumer end calls the message receiving method of the message component core (MessageCore) to upload the message to the local message processing component.
[0121] Step 602: Determine the local target message queue corresponding to the message through the local message processing component.
[0122] It should be noted that multiple local message queues are also deployed in the message consumer end. After the message consumer end uploads one or more received messages to the local message processing component, the local message processing component can determine the local target message queue corresponding to each message from multiple local message queues.
[0123] As a possible implementation method, each message carries target message queue information, such as target, so that the message consumer can determine the local target message queue corresponding to the message from multiple local message queues through a local message processing component according to the target message queue information carried in the message.
[0124] Step 603: Call the message persister of the local target message queue to save the message into the local database.
[0125] In order to ensure business continuity and data integrity, in the embodiment of the present disclosure, after determining the local target message queue corresponding to the message, the message consumer can call the message persister of the local target message queue to save the message to the local database. Thus, by persisting the message to the local database, even if there is a temporary network failure or the message queue service is unavailable, the message will not be lost, and after the failure is recovered, the message can be reloaded from the local database to continue processing the unfinished business logic, thereby reducing the risk of business interruption.
[0126] In the present disclosure, using a local database to persist messages can alleviate the problems of system resource occupation and high disk IO performance requirements caused by log file management and cleaning to a certain extent.
[0127] As an example, the save method of the message persister (MessageRoller) of the local target message queue may be called to persist the message.
[0128] As a possible implementation manner, the message consumer may obtain the configuration information of the local target message queue through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
[0129] As an example, a local message processing component, such as a message component core (MessageCore), can match the local target message queue corresponding to the message according to the target of the message, and read the configuration information of the local target message queue. Then, according to the persistence configuration information of the target message queue, the message persister of the local target message queue is called.
[0130] Step 604: Add the message to the local target message queue to send the message to the local message processing module through the local target message queue.
[0131] In summary, by using the local components and local message queues on the message consumer side (when processing messages asynchronously) to send messages to the local message processing module for processing (after receiving the message, first persist the message, then queue it and send it to the local message processing module for processing), the processing logic of transactional message reception is simplified and the processing performance is improved.
[0132] The message processing method of the embodiment of the present disclosure receives the message sent by the message production end, and uploads the message to the local message processing component; determines the local target message queue corresponding to the message through the local message processing component; calls the message persister of the local target message queue to save the message in the local database; adds the message to the local target message queue to send the message to the local message processing module through the local target message queue. Therefore, by uploading the received message to the local message processing component and taking a series of measures to send the message to the local message processing module for processing, the processing logic of transactional message reception is simplified, the processing performance is improved, and the transaction consistency of message reception and business processing at the message consumer end is ensured.
[0133] In order to clearly explain how to send messages to the local message processing module through the local target message queue in the above embodiment, the present disclosure proposes another message processing method.
[0134] Figure 7 It is a flowchart of the message processing method shown in the sixth embodiment of the present disclosure.
[0135] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message consumption end.
[0136] Step 701, receiving a message sent by a message production end, and uploading the message to a local message processing component.
[0137] Step 702: Determine the local target message queue corresponding to the message through the local message processing component.
[0138] Step 703: Call the message persister of the local target message queue to save the message into the local database.
[0139] Step 704: Add the message to the local target message queue.
[0140] Step 705: sort the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue.
[0141] The indication information of the message is used to indicate the basis for sorting the message, for example, it may be the message type, label, content, etc.
[0142] In the disclosed embodiment, the message consumer can sort the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue. For example, the message with message tag flag=A is sent to the local recipient message queue 1 for processing, and the message with message tag flag=B is sent to the local recipient message queue 2 for processing.
[0143] Step 706: Send the message to the local message processing module through the local target recipient message queue.
[0144] In the present disclosure, at the message consumption end, the message can be directly sent to the local message processing module through the local target recipient message queue.
[0145] It should be noted that the execution process of steps 701 to 704 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0146] The message processing method of the disclosed embodiment receives the message sent by the message production end, uploads the message to the local message processing component; determines the local target message queue corresponding to the message through the local message processing component; calls the message persister of the local target message queue to save the message to the local database; adds the message to the local target message queue, so as to send the message to the local message processing module through the local target message queue, thereby different messages can be sorted into different queues according to their content or type, thereby realizing classified processing. Moreover, messages sorted into specific recipient queues can be processed in parallel, reducing the waiting time for message processing.
[0147] Similar to the message production end, on the message consumption end, if the message fails to be sent or times out, it can also be re-added to the local target message queue. Figure 8 This process is described below.
[0148] Figure 8 It is a flowchart of the message processing method shown in the seventh embodiment of the present disclosure.
[0149] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message consumption end.
[0150] like Figure 8 As shown, the message processing method includes the following steps:
[0151] Step 801, receiving a message sent by a message production end, and uploading the message to a local message processing component.
[0152] Step 802: Determine the local target message queue corresponding to the message through the local message processing component.
[0153] Step 803: Call the message persister of the local target message queue to save the message into the local database.
[0154] Step 804: Add the message to the local target message queue to send the message to the local message processing module through the local target message queue.
[0155] Step 805: When the message fails to be sent or is not sent due to timeout, the local message restorer is called to query the message from the local database and add the message to the local target message queue again.
[0156] In the present disclosure, messages that fail to be sent or are not sent due to timeout will be retrieved from the local database by a message recoverer (MessageRecover) and re-added to the local target message queue, thereby reducing instability and data inconsistency caused by message loss or processing failure.
[0157] The message processing method of the embodiment of the present invention, when a message fails to be sent or times out, calls a local message recoverer to query the message from the local database and re-adds the message to the local target message queue. Thus, through the local message recoverer, when a message fails to be sent or times out, the message can be recovered from the local database and put back into the queue, thereby ensuring that the message will not be lost.
[0158] In order to explain the above embodiment clearly, an example is given for illustration.
[0159] Fig. 9 It is a schematic diagram of the principle of the message processing method shown in the eighth embodiment of the present disclosure.
[0160] Similarly, the message processing method of the embodiment of the present disclosure is applied to the message consumption end.
[0161] like Fig. 9 As shown in the figure, after receiving the message, if the message is processed synchronously, the message adapter of the message consumer directly exposes the local message processing module as a message processing service. If the message is processed asynchronously, the message is sent to the local message processing module through the following steps:
[0162] Step 901, calling the put method to send the message to the actual message processing module via the local queue.
[0163] After receiving the message, the message adapter calls the message receiving method of the message component core (MessageCore).
[0164] Step 902, matching the queue according to the target of the message and reading the queue information.
[0165] The message component core matches the local target message queue corresponding to the message according to the target of the message, and obtains the configuration information of the local target message queue.
[0166] Step 903: call the save method of the roller of the queue for persistence.
[0167] According to the persistence configuration of the local target message queue, the message persister (MessageRoller) of the queue is called to save the message to the local database.
[0168] Step 904, queue the message.
[0169] After persistence is successful, the message is placed in the local target message queue in memory.
[0170] Step 905, sorting to a recipient queue according to the recipient.
[0171] The processing program of the local target message queue will sort the message into the corresponding local recipient queue according to the recipient information of the message (ie, the indication information of the message).
[0172] Step 906, send.
[0173] The processing program of the local target message queue sends the message to the message processing module (ie, the local message processing module).
[0174] It should be noted that for messages that are not successfully processed, they will be retrieved from the local database by the message recovery device (MessageRecover) and re-added to the local target message queue. Fig. 9 As shown, MessageRecover will poll the timeout and unsent messages and re-queue them.
[0175] In summary, after receiving a message, the message consumer first persists the message and then puts it into the local queue for processing. For messages that cannot be processed, the message restorer is responsible for recovering them from the local database and re-putting them into the target message queue. This simplifies the processing logic of transactional message reception, improves processing performance, and ensures the transactional consistency of message consumer message reception and business processing.
[0176] With the above Figures 2 to 5 Corresponding to the message processing method provided in the embodiment, the present disclosure also provides another message processing device. Figures 2 to 5 The message processing method provided in the embodiment corresponds to the embodiment, so the implementation of the message processing method is also applicable to the message processing device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.
[0177] Fig.10 It is a structural diagram of a message processing device shown in the ninth embodiment of the present disclosure.
[0178] like Fig.10 As shown, the message processing device 1000 includes: an uploading module 1010 , a first determining module 1020 , a first calling module 1030 and a first processing module 1040 .
[0179] Among them, the upload module 1010 is used to upload the message generated during the execution of the local transaction to the local message processing component before the local transaction is committed; the first determination module 1020 is used to determine the local target message queue corresponding to the message through the local message processing component; the first calling module 1030 is used to call the message persister of the local target message queue to save the message to the local database and register the transaction listener of the local transaction; the first processing module 1040 is used to add the message to the local target message queue when the transaction listener detects that the local transaction is successfully committed, so as to send the message to the message consumer through the local target message queue.
[0180] As a possible implementation method of the embodiment of the present disclosure, the first processing module 1040 is also used to: sort the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue; send the message to the message consumer end through the local target recipient message queue, or call a third-party message adapter through the local target recipient message queue to send the message to the message consumer end.
[0181] As a possible implementation of the embodiment of the present disclosure, the first determination module 1020 is further used to: determine, through a local message processing component, a local target message queue corresponding to a message from multiple local message queues according to target message queue information carried in the message.
[0182] As a possible implementation method of the embodiment of the present disclosure, the above-mentioned device also includes: a first acquisition module, which is used to obtain the configuration information of the local target message queue through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
[0183] As a possible implementation method of the embodiment of the present disclosure, the above-mentioned device also includes: a second processing module, which is used to call the local message restorer to query the message from the local database and add the message back to the local target message queue when the message fails to be sent or is not sent due to timeout.
[0184] The message processing device of the embodiment of the present disclosure uploads the message generated during the execution of the local transaction to the local message processing component before the local transaction is committed; determines the local target message queue corresponding to the message through the local message processing component; calls the message persister of the local target message queue to save the message to the local database and the transaction listener that registers the local transaction; when the transaction listener detects that the local transaction is successfully committed, adds the message to the local target message queue to send the message to the message consumer end through the local target message queue. Therefore, by uploading the message to the local message processing component before the local transaction is committed and taking a series of measures to complete the sending of the message, the processing logic of transactional message sending is simplified, the processing performance is improved, and the transaction consistency between the business processing of the message production end and the message sending is ensured.
[0185] With the above Figures 6 to 9 Corresponding to the message processing method provided in the embodiment, the present disclosure also provides another message processing device. Figures 6 to 9 The message processing method provided in the embodiment corresponds to the embodiment, so the implementation of the message processing method is also applicable to the message processing device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.
[0186] Fig.11 It is a structural diagram of a message processing device shown in the tenth embodiment of the present disclosure.
[0187] like Fig.11 As shown, the message processing device 1100 includes: a third processing module 1110 , a second determining module 1120 , a second calling module 1130 and a fourth processing module 1140 .
[0188] Among them, the third processing module 1110 is used to receive messages sent by the message production end and upload the messages to the local message processing component; the second determination module 1120 is used to determine the local target message queue corresponding to the message through the local message processing component; the second calling module 1130 is used to call the message persister of the local target message queue to save the message to the local database; the fourth processing module 1140 is used to add the message to the local target message queue to send the message to the local message processing module through the local target message queue.
[0189] As a possible implementation method of the embodiment of the present disclosure, the fourth processing module 1140 is also used to: sort the message into the local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue; and send the message to the local message processing module through the local target recipient message queue.
[0190] As a possible implementation of the embodiment of the present disclosure, the second determination module 1120 is further used to: determine, through a local message processing component, a local target message queue corresponding to a message from multiple local message queues according to target message queue information carried in the message.
[0191] As a possible implementation method of the embodiment of the present disclosure, the above-mentioned device also includes: a second acquisition module, which is used to obtain the configuration information of the local target message queue through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
[0192] As a possible implementation method of the embodiment of the present disclosure, the above-mentioned device also includes: a fifth processing module, which is used to call the local message restorer to query the message from the local database and add the message back to the local target message queue when the message fails to be sent or is not sent due to timeout.
[0193] The message processing device of the embodiment of the present disclosure receives the message sent by the message production end, uploads the message to the local message processing component; determines the local target message queue corresponding to the message through the local message processing component; calls the message persister of the local target message queue to save the message in the local database; adds the message to the local target message queue to send the message to the local message processing module through the local target message queue. Therefore, by uploading the received message to the local message processing component and taking a series of measures to send the message to the local message processing module for processing, the processing logic of transactional message reception is simplified, the processing performance is improved, and the transaction consistency of message reception and business processing at the message consumer end is ensured.
[0194] In an exemplary embodiment, an electronic device is also provided.
[0195] Among them, electronic equipment includes:
[0196] processor;
[0197] a memory for storing processor-executable instructions;
[0198] The processor is configured to execute instructions to implement the message processing method proposed in any of the aforementioned embodiments.
[0199] As an example, Fig.12 is a schematic diagram of the structure of an electronic device 1200 shown in an exemplary embodiment of the present disclosure, such as Fig.12 As shown, the electronic device 1200 may further include:
[0200] The memory 1210 and the processor 1220, a bus 1230 connecting different components (including the memory 1210 and the processor 1220), the memory 1210 stores a computer program, and when the processor 1220 executes the program, the message processing method described in the embodiment of the present disclosure is implemented.
[0201] Bus 1230 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0202] The electronic device 1200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 1200, including volatile and non-volatile media, removable and non-removable media.
[0203] The memory 1210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1240 and / or cache memory 1250. The server 1200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1260 may be used to read and write non-removable, non-volatile magnetic media ( Fig.12 not shown, often referred to as a "hard drive packager"). Although Fig.12 Not shown, a disk driver package for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk driver package for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each driver package can be connected to the bus 1230 via one or more data medium interfaces. The memory 1210 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.
[0204] A program / utility 1280 having a set (at least one) of program modules 1270 may be stored, for example, in the memory 1210, such program modules 1270 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 1270 generally perform the functions and / or methods of the embodiments described in the present disclosure.
[0205] The electronic device 1200 may also communicate with one or more external devices 1290 (e.g., keyboard, pointing device, display 1291, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or communicate with any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 1292. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 1293. As shown, the network adapter 1293 communicates with other modules of the electronic device 1200 through a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device driver package, redundant processing unit, external disk driver package array, RAID system, tape driver package, and data backup storage system, etc.
[0206] The processor 1220 executes various functional applications and data processing by running the programs stored in the memory 1210 .
[0207] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the message processing method of the embodiment of the present disclosure, and will not be repeated here.
[0208] In an exemplary embodiment, a computer-readable storage medium including instructions is also proposed, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the message processing method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0209] In an exemplary embodiment, a computer program product is also proposed, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the message processing method proposed in any of the above embodiments is implemented.
[0210] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0211] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A message processing method, characterized in that: Applied to the message production end, including: Before the local transaction is committed, the messages generated during the execution of the local transaction are uploaded to the local message processing component; Determine, by the local message processing component, a local target message queue corresponding to the message; Calling the message persister of the local target message queue to save the message into the local database and registering the transaction listener of the local transaction; When the transaction listener detects that the local transaction is successfully submitted, the message is added to the local target message queue, so as to send the message to the message consumer through the local target message queue.
2. The method according to claim 1, characterized in that The sending the message to the message consumer end through the local target message queue includes: sorting the message into a local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue; The message is sent to the message consumer via the local target recipient message queue, or the message is sent to the message consumer via a third-party message adapter called via the local target recipient message queue.
3. The method according to claim 1, characterized in that The determining, by the local message processing component, a local target message queue corresponding to the message includes: The local message processing component determines the local target message queue corresponding to the message from multiple local message queues according to the target message queue information carried in the message.
4. The method according to claim 1, characterized in that Before calling the message persister of the local target message queue to save the message into the local database and registering the transaction listener of the local transaction, the method further includes: The configuration information of the local target message queue is obtained through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In the case where the message fails to be sent or is not sent due to timeout, the local message restorer is called to query the message from the local database and add the message to the local target message queue again.
6. A message processing method, characterized in that: Applied to message consumption end, including: Receive messages sent by the message production end, and upload the messages to the local message processing component; Determine, by the local message processing component, a local target message queue corresponding to the message; Calling the message persister of the local target message queue to save the message into the local database; The message is added to the local target message queue, so as to send the message to the local message processing module through the local target message queue.
7. The method according to claim 6, characterized in that The sending the message to the local message processing module through the local target message queue includes: sorting the message into a local target recipient message queue corresponding to the message according to the indication information of the message through the local target message queue; The message is sent to the local message processing module through the local target recipient message queue.
8. The method according to claim 6, characterized in that The determining, by the local message processing component, a local target message queue corresponding to the message includes: The local message processing component determines the local target message queue corresponding to the message from multiple local message queues according to the target message queue information carried in the message.
9. The method according to claim 6, characterized in that Before calling the message persister of the local target message queue to save the message in the local database, the method further includes: The configuration information of the local target message queue is obtained through the local message processing component, so as to call the message persister of the local target message queue according to the configuration information of the local target message queue.
10. The method according to any one of claims 6 to 9, characterized in that: The method further comprises: In the case where the message fails to be sent or is not sent due to timeout, the local message restorer is called to query the message from the local database and add the message to the local target message queue again.
11. A message processing device, characterized in that: Applied to the message production end, including: The upload module is used to upload the messages generated during the execution of the local transaction to the local message processing component before the local transaction is committed; A first determination module, configured to determine a local target message queue corresponding to the message through the local message processing component; A first calling module, used for calling the message persister of the local target message queue to save the message into the local database and registering the transaction listener of the local transaction; The first processing module is used to add the message to the local target message queue when the transaction listener monitors that the local transaction is successfully submitted, so as to send the message to the message consumer through the local target message queue.
12. A message processing device, characterized in that: Applied to message consumption end, including: The third processing module is used to receive the message sent by the message production end and upload the message to the local message processing component; A second determination module, configured to determine a local target message queue corresponding to the message through the local message processing component; A second calling module is used to call the message persister of the local target message queue to save the message into a local database; The fourth processing module is used to add the message to the local target message queue, so as to send the message to the local message processing module through the local target message queue.