A message processing method and related apparatus

Abstract

This application discloses a message processing method and related apparatus, applicable to the financial field. A monitoring application retrieves a first message from a message queue and performs first business logic processing. This first business logic processing includes m business logic operations, where m is a positive integer less than M and M is a positive integer greater than 1. The first processing result is obtained. If the first processing result is a failure, the first message is added to a delayed queue. The monitoring application continues to listen to the message queue and retrieve a second message for first business logic processing. A daemon thread application performs second business logic processing on the first message in the delayed queue, reprocessing the failed first message to obtain a second processing result. This second business logic processing includes M-m business logic operations. This enables concurrent processing of multiple messages, improves message processing efficiency, avoids message congestion, increases resource utilization, and significantly reduces the risk of bank system crashes.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a message processing method and related apparatus. Background Technology

[0002] Message Queues (MQ) are middleware used for message transmission between banking systems, and are among the most important middleware components. Message consumption based on MQ is one common message consumption method.

[0003] Currently, the solution for message consumption based on MQ is to read messages from the MQ one by one and insert them into the database, process them in a loop for 5 times, and exit if the message is successfully processed; if all 5 attempts fail, the message consumption is considered to have failed.

[0004] However, as business grows and data volume increases, this method can lead to slow message consumption, message congestion in the message queue (MQ), and resource waste due to underutilization of resources; in severe cases, it may even cause the banking system to crash. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a message processing method and related apparatus that eliminates the impact of reprocessing failed messages on the monitoring application, greatly improves the efficiency of the monitoring application in obtaining messages from the message queue, enables concurrent processing of multiple messages, significantly improves message processing efficiency, avoids message congestion, increases resource utilization, and greatly reduces the risk of bank system crashes.

[0006] In a first aspect, embodiments of this application provide a message processing method, the method comprising:

[0007] The application retrieves the first message from the message queue and performs the first business logic processing. The first business processing logic includes m business logic processing steps, where m is a positive integer less than M and M is a positive integer greater than 1.

[0008] Obtain the first processing result of the first business logic processing;

[0009] If the first processing result is a processing failure, the first message is added to the delayed queue, and the listening application continues to listen to and obtain the second message from the message queue to perform the first business logic processing. The second message is the message in the message queue that is located after the first message.

[0010] The first message in the delayed queue is processed by a daemon thread application to obtain a second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

[0011] In one possible implementation, the method further includes:

[0012] Insert the first message into the database;

[0013] Before adding the first message to the delayed queue, the method further includes:

[0014] The status of the first message in the database is marked as pending retry.

[0015] In one possible implementation, adding the first message to the delayed queue includes:

[0016] The first message with the status "Pending Retry" is added to the delay queue, and the status of the first message in the database is changed from "Pending Retry" to "Retrying".

[0017] In one possible implementation, adding the first message to the delayed queue includes:

[0018] Get the number of messages already in the delay queue;

[0019] If the number of existing messages is less than a predetermined threshold, the first message is added to the delay queue.

[0020] In one possible implementation, if the first message is inserted into the database and the status of the first message is pending retry, the method further includes:

[0021] If the number of existing messages reaches the specified threshold, the status of the first message in the database is changed from pending retry to failed pending retry.

[0022] In one possible implementation, the method further includes:

[0023] According to a set time, n messages with a status of failure and pending retry are extracted from the database and added to the delay queue. The first message is included among the n messages with a status of failure and pending retry.

[0024] In one possible implementation, if the first message is inserted into the database, the method further includes:

[0025] If the second processing result is processing failure, the status of the first message in the database is changed to "retry count full".

[0026] Secondly, embodiments of this application provide a message processing apparatus, the apparatus comprising:

[0027] The processing unit is used to obtain the first message from the message queue through the listening application and perform the first business logic processing. The first business processing logic includes m business logic processing operations, where m is a positive integer less than M and M is a positive integer greater than 1.

[0028] The acquisition unit is used to acquire the first processing result of the first business logic processing;

[0029] An adding unit is used to add the first message to a delayed queue if the first processing result is a processing failure.

[0030] The processing unit is further configured to continue listening to and obtaining a second message from the message queue through the listening application to perform the first business logic processing, wherein the second message is a message in the message queue that is located after the first message;

[0031] The processing unit is further configured to perform second business logic processing on the first message in the delayed queue through a daemon thread application to obtain a second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

[0032] In one possible implementation, the device further includes an insertion unit and a labeling unit:

[0033] The insertion unit is used to insert the first message into the database;

[0034] The labeling unit is used to label the status of the first message in the database as pending retry before adding the first message to the delay queue.

[0035] In one possible implementation, the adding unit is specifically used for:

[0036] The first message with the status "Pending Retry" is added to the delay queue, and the status of the first message in the database is changed from "Pending Retry" to "Retrying".

[0037] In one possible implementation, the adding unit is specifically used for:

[0038] Get the number of messages already in the delay queue;

[0039] If the number of existing messages is less than a predetermined threshold, the first message is added to the delay queue.

[0040] In one possible implementation, if the first message is inserted into the database and the status of the first message is pending retry, the annotation unit is specifically used for:

[0041] If the number of existing messages reaches the specified threshold, the status of the first message in the database is changed from pending retry to failed pending retry.

[0042] In one possible implementation, the device further includes an extraction unit:

[0043] The extraction unit is used to extract n messages with a status of failure and awaiting retry from the database according to a set time and add them to the delay queue. The first message is included among the n messages with a status of failure and awaiting retry.

[0044] In one possible implementation, if the first message is inserted into the database, the annotation unit is further configured to:

[0045] If the second processing result is processing failure, the status of the first message in the database is changed to "retry count full".

[0046] As can be seen from the above technical solution, the core of this solution for message consumption based on MQ is the separation of message acquisition and processing from asynchronous re-consumption of failed messages. Messages are acquired and processed through a listening application, while failed messages are asynchronously reprocessed through a daemon thread application. Specifically, the listening application retrieves the first message from the message queue and performs the first business logic processing. This first business logic processing includes m business logic operations, where m is a positive integer less than M and M is a positive integer greater than 1. Then, the first processing result is obtained. If the first processing result is a failure, the first message is added to a delayed queue. The listening application no longer reprocesses failed messages, such as the first message, but instead continues to listen to and retrieve the second message from the message queue for the first business logic processing. The second message is the message following the first message in the message queue. The daemon thread application then performs the second business logic processing on the first message in the delayed queue, completing the reprocessing of the failed first message and obtaining the second processing result. This second business logic processing includes Mm business logic operations. This solution uses a listening application to handle message retrieval and processing, while a daemon thread application handles asynchronous reprocessing of failed messages. The two applications operate independently and do not interfere with each other. There is no need to wait for the listening application to complete all message processing cycles before retrieving new messages for processing, thus eliminating the impact of failed message reprocessing on the listening application. This greatly improves the efficiency of the listening application in retrieving messages from the message queue, enables concurrent processing of multiple messages, significantly enhances message processing efficiency, avoids message congestion, improves resource utilization, and greatly reduces the risk of bank system crashes. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A framework diagram of a message processing method provided for related technologies;

[0049] Figure 2 A flowchart illustrating a message processing method provided in an embodiment of this application;

[0050] Figure 3 A framework diagram of a message processing method provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram illustrating the principle of a timed data extraction application provided in this embodiment of the application;

[0052] Figure 5 A logical architecture diagram for message processing provided in an embodiment of this application;

[0053] Figure 6 This is a structural diagram of a message processing device provided in an embodiment of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0055] Currently, the message consumption solution based on MQ involves the listening application retrieving messages from the MQ one by one, inserting the message into the database, and then repeatedly calling the business logic of Dubbo (an open-source distributed service architecture) for processing, limited to 5 times. If processing is successful, the process ends and exits, continuing to listen for and consume the next MQ message; if all 5 message processing attempts fail, the message is considered to have failed, processing is stopped, and the application exits to continue listening for and consuming the next message. Figure 1 As shown.

[0056] The aforementioned technology consumes messages sequentially within the message queue (MQ). Initially, with small business volumes and low data rates, the banking system could quickly process all messages in the MQ. However, as business volume and data volume grow rapidly, the drawbacks of this single-threaded sequential execution approach become apparent. A large number of messages and numerous message consumption failures significantly slow down message processing, causing message congestion in the MQ, reducing the banking system's operational efficiency, and resulting in a poor user experience. Re-consuming failed messages consumes the MQ's new message processing resources, leading to message backlog. Furthermore, due to limitations in the message processing mechanism, server resources are not fully utilized, resulting in significant waste of server resources.

[0057] To address the aforementioned technical issues, this application provides a message processing method. When consuming messages based on a message queue (MQ), the core of this solution is the separation of message acquisition and processing from asynchronous re-consumption of failed messages. Messages are acquired and processed through a listening application, while failed messages are asynchronously reprocessed through a daemon thread application. Specifically, the listening application retrieves the first message from the message queue and performs first business logic processing. This first business logic processing includes m business logic operations, where m is a positive integer less than M and M is a positive integer greater than 1. Then, the first processing result is obtained. If the first processing result is a failure, the first message is added to a delayed queue. The listening application no longer reprocesses failed messages, such as the first message, but instead continues to listen to and retrieve a second message from the message queue for first business logic processing. The second message is the message following the first message in the message queue. The daemon thread application performs second business logic processing on the first message in the delayed queue, completing the reprocessing of the failed first message and obtaining the second processing result of the first message. This second business logic processing includes Mm business logic operations. This solution uses a listening application to handle message retrieval and processing, while a daemon thread application handles asynchronous reprocessing of failed messages. The two applications operate independently and do not interfere with each other. There is no need to wait for the listening application to complete all message processing cycles before retrieving new messages for processing, thus eliminating the impact of failed message reprocessing on the listening application. This greatly improves the efficiency of the listening application in retrieving messages from the message queue, enables concurrent processing of multiple messages, significantly enhances message processing efficiency, avoids message congestion, improves resource utilization, and greatly reduces the risk of bank system crashes.

[0058] The method provided in this application can be applied to a computer device that can run a banking system. The computer device can be, for example, a terminal device, such as a smart terminal, computer, personal digital assistant (PDA), tablet computer, etc.

[0059] It should be noted that the message processing method and related apparatus provided by this invention can be used in the financial field, such as message consumption scenarios based on message queues (MQ) in the financial sector. Of course, they can also be used in other fields; the above are merely examples and do not limit the application areas of the message processing method and related apparatus provided in this application.

[0060] The message processing method provided in the embodiments of this application will now be described in conjunction with the accompanying drawings. See also... Figure 2 As shown, the method includes:

[0061] S201. Obtain the first message from the message queue through the listening application and perform the first business logic processing. The first business processing logic includes m business logic processing steps, where m is a positive integer less than M and M is a positive integer greater than 1.

[0062] In this embodiment of the application, when consuming messages based on MQ, message acquisition and processing are separated from asynchronous re-consumption of failed messages. Messages are acquired and processed through a listening application, while failed messages are asynchronously reprocessed through a daemon thread application. See also Figure 3 As shown, the application listens for new messages, such as the first message, and performs initial business logic processing on it to obtain the first processing result. Then, a daemon thread application asynchronously reprocesses messages that failed to be processed (e.g., subsequent second business logic processing). The initial business logic processing of the first message by the listening application can be a logic process executed by the Dubbo business processing application, which is called by the listening application.

[0063] Message queues are used for message transmission between banking systems. Different message queues transmit different types of messages. Message types can be categorized according to business meaning or channel. Developers need to communicate with the other bank's systems and business personnel to understand the relevant business knowledge, be familiar with the specific meaning of the messages, and categorize the messages accordingly. For example, they can define categories based on different products, channels, or business content, and define different listening applications. Each listening application listens for and processes messages of a specific type; that is, different listening applications listen to messages from different message queues. In this case, the first message received by the listening application in S201 comes from the specific type of message queue it is responsible for listening to.

[0064] The first business logic processing is the partial logic processing that needs to be performed on the first message. The first business logic processing includes m business logic processing operations, where m is a positive integer less than M and M is a positive integer greater than 1. That is to say, in this embodiment of the application, if the first message needs to be processed multiple times, the listening application does not need to complete all the processing of the first message, but only needs to complete part of the processing. The remaining processing is completely handed over to the daemon thread application, thereby eliminating the impact of reprocessing failed messages on the listening application, greatly improving the efficiency of the listening application in obtaining messages in the message queue, enabling concurrent processing of multiple messages, greatly improving the message processing efficiency, avoiding message blocking, and improving resource utilization.

[0065] Where M represents the total number of times the first message is processed, and m can take any value less than M. Generally, the smaller m is, the sooner the monitoring application can obtain other new messages, which is more effective in improving message processing efficiency. In this embodiment, M can be 5, m can be 1, or other values ​​are not limited in this embodiment.

[0066] S202. Obtain the first processing result of the first business logic processing.

[0067] S203. If the first processing result is processing failure, the first message is added to the delay queue, and the listening application continues to listen to and obtain the second message from the message queue to perform the first business logic processing. The second message is the message in the message queue that is located after the first message.

[0068] After completing the first business logic processing of the first message, the first processing result can be obtained. If the first processing result is successful, there is no need to process the first message again. If the first processing result is a failure, the first message is added to a delayed queue, and the listening application continues to listen for and retrieve new messages (i.e., the second message) from the message queue for the first business logic processing. The second message is the message in the message queue that follows the first message. Therefore, once the listening application retrieves the first message and performs the first business logic processing, regardless of whether the first message processing fails or not, the listening application no longer intervenes but continues to listen for new messages (such as the second message).

[0069] In one possible implementation, the first message can be inserted into the database (DB) simultaneously with its acquisition. To facilitate understanding of the message status in the database and thus the message processing progress, preventing some messages from being missed or processed repeatedly, the message status in the database can be labeled. Message statuses can include various states, such as initialization, success, pending retry, failure pending retry, pending retry, and retries exhausted.

[0070] When inserting a message (such as the first message) into the database, the status of the first message can be marked as initialized.

[0071] If the first processing result is successful, there is no need to process the first message again; simply change the status of the first message in the database to successful. If the first processing result is unsuccessful, mark the status of the first message in the database as pending retry before adding it to the delay queue.

[0072] When adding the first message to the delay queue, the first message with the status of "pending retry" can be added to the delay queue, and the status of the first message in the database can be changed from "pending retry" to "retrying", indicating that it has entered the delay queue for retry, that is, the second business logic processing can be performed.

[0073] Understandably, the space of the delayed queue is not infinite; there is a limit (i.e., a threshold) to the number of messages it can hold. In other words, not all messages can be added to the delayed queue at any given time. Therefore, the way to add the first message to the delayed queue is to obtain the number of messages already in the queue, determine if the number of messages has reached the threshold, and if the number of messages is less than the threshold (meaning the delayed queue is not full), then the first message can be added to the delayed queue.

[0074] If the number of messages reaches the limit, the delay queue will be full, and the first message will no longer be added to the delay queue. At this point, if the first message is inserted into the database and its status is "pending retry," the status of the first message in the database can be changed from "pending retry" to "failed and pending retry."

[0075] S204. The first message in the delay queue is processed by the daemon thread application to obtain the second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

[0076] The application can retrieve messages from the delayed queue through a daemon thread and perform secondary business logic processing. For example, when the first message in the delayed queue is retrieved, the second business logic processing is performed on the first message to obtain the second processing result, which can be used as the final processing result of the first message. When M is 5 and m is 1, the second business logic processing can be performed 4 times, that is, the first message in the delayed queue is processed 4 times in a loop to obtain the second processing result.

[0077] In this case, the second business logic processing of the first message by the daemon thread application can be the logic processing executed by the Dubbo business processing application called by the daemon thread application.

[0078] If the first message is inserted into the database, and the second processing result is successful, the status of the first message in the database can be changed to successful. If the second processing result is unsuccessful, it can be determined that the first message processing failed, and the status of the first message in the database can be changed to "retry count reached".

[0079] As messages in the delay queue are continuously processed, idle space becomes available. Messages with a "failed and awaiting retry" status can be added to the delay queue for further retries. In one possible implementation, n messages with a "failed and awaiting retry" status can be retrieved from the database at set intervals and added to the delay queue. These n messages include the first message. The retrieval of these n messages from the database and their addition to the delay queue at set intervals can be performed by a timed data retrieval application.

[0080] One possible implementation is to use forkjoin multithreading to quickly add n messages to a delayed queue. To do this, developers need to work with business personnel to assess the peak volume of business data and the resources required by the system. Based on this, they should determine the maximum value of the delayed queue (i.e., the threshold), and reasonably plan the amount of data retrieved each time by the data extraction timer and the forkjoin distribution rate, while also determining the maximum number of threads for the daemon thread application.

[0081] In this embodiment, the listening application, the daemon thread application, and forkjoin all employ thread pool technology, enabling concurrent processing of multiple messages and significantly improving the processing efficiency of messages in the message queue. In particular, the data extraction timed application utilizes forkJoin technology, which greatly improves the loading of failed retry information into the delayed queue, adapting to the processing efficiency of the daemon thread application and further enhancing the efficiency of failed message reprocessing. The principle of the data extraction timed application is as follows, see [link to application details]. Figure 4 .

[0082] The process involves a single-threaded timer task periodically extracting 4n failure messages (messages in the "failed and awaiting retry" state). These messages are then aligned and forked to create two subtasks that execute in parallel, each containing 2n failure messages. For each set of 2n failure messages, another fork is performed to create multiple subtasks that execute in parallel, resulting in four subtasks. Each subtask sequentially inserts n failure messages into a delay queue. The execution results of these subtasks are then merged (joined) to obtain two execution results, each containing 2n failure messages inserted into the delay queue. Finally, these two results are merged (joined) to obtain the final execution result: 4n failure messages inserted into the delay queue.

[0083] As can be seen from the above technical solution, the core of this solution for message consumption based on MQ is the separation of message acquisition and processing from asynchronous re-consumption of failed messages. Messages are acquired and processed through a listening application, while failed messages are asynchronously reprocessed through a daemon thread application. Specifically, the listening application retrieves the first message from the message queue and performs the first business logic processing. This first business logic processing includes m business logic operations, where m is a positive integer less than M and M is a positive integer greater than 1. Then, the first processing result is obtained. If the first processing result is a failure, the first message is added to a delayed queue. The listening application no longer reprocesses failed messages, such as the first message, but instead continues to listen to and retrieve the second message from the message queue for the first business logic processing. The second message is the message following the first message in the message queue. The daemon thread application then performs the second business logic processing on the first message in the delayed queue, completing the reprocessing of the failed first message and obtaining the second processing result. This second business logic processing includes Mm business logic operations. This solution uses a listening application to handle message retrieval and processing, while a daemon thread application handles asynchronous reprocessing of failed messages. The two applications operate independently and do not interfere with each other. There is no need to wait for the listening application to complete all message processing cycles before retrieving new messages for processing, thus eliminating the impact of failed message reprocessing on the listening application. This greatly improves the efficiency of the listening application in retrieving messages from the message queue, enables concurrent processing of multiple messages, significantly enhances message processing efficiency, avoids message congestion, improves resource utilization, and greatly reduces the risk of bank system crashes.

[0084] Based on the above description of the message processing method, this application embodiment also provides a logical architecture diagram for message processing, see below. Figure 5 As shown, it mainly includes an MQ cluster (i.e., a message queue cluster), a listening application, a Dubbo business processing application, a database, a data extraction timed application, a delayed queue, and a daemon thread application.

[0085] MQ clusters are used for message transmission between banking systems. Different message queues transmit different types of message content, and message types can be divided according to business meaning or channel.

[0086] The listening application can define different listeners based on the message queue division. Each listener is responsible for listening to messages in its respective message queue, inserting them into the database, marking them as initialized, and then calling the downstream Dubbo business processing application to perform business logic processing (which can be called the first business logic processing). If processing is successful, the status is marked as successful; if processing fails, the status is marked as pending retry. If the delay queue is full, the status is marked as failed pending retry. Regardless of whether message processing fails or not, the listening application continues to listen for and process new MQ messages.

[0087] The database is used to store messages and record the current status and retry count for each message. Message status can be divided into 6 types: initialization, success, failure pending retry, pending retry, retrying, and retry count exhausted.

[0088] Dubbo business processing applications are used to process messages using business logic. Messages in different types of message queues call different Dubbo business logic for processing.

[0089] The data extraction timed application is used to periodically retrieve n failed messages pending retry from the database, mark them as pending retry, and load them into a delay queue to await business logic processing (which can be called the second business logic processing).

[0090] The delay queue is used to store messages that are in a pending retry state.

[0091] The daemon thread application is used to retrieve messages to be retried from the delayed queue, mark the status as retrying, and call the Dubbo business processing application to perform business logic processing; if the processing is successful, it is marked as successful and exits; if the processing fails 4 times in a loop, it is marked as the retry count has been reached.

[0092] Based on the above logical architecture, the message processing method based on MQ is as follows: Different listening applications obtain messages (e.g., the first message) from specific types of message queues in real time and insert them into the database, marking the status as initialized. Then, the Dubbo business processing application is called to perform the first business logic processing. The first business logic processing can be a business logic processing that is limited to one attempt. If the processing is successful, the status is changed to success. If the processing fails, the message is added to the delayed queue, and the status of the message in the database is changed to retrying. If the number of messages in the delayed queue has reached the limit threshold, the message will no longer be added to the delayed queue, and the status of the message in the database will be changed to failed and awaiting retry. Regardless of whether message processing fails, the listening application no longer intervenes but continues to listen for messages in the new, specific type of message queue; the daemon thread application obtains messages from the delayed queue in real time and repeatedly calls the Dubbo business processing application to perform secondary business logic processing on the messages. The secondary business logic processing can be limited to 4 times. If the business logic processing is successful, it exits and the status is changed to success. If it fails to process 4 times in a row, it is determined that the message processing has failed, and the status of the message in the database is changed to "retry count full"; the data extraction timer application periodically extracts messages with the status "failed and awaiting retry" from the database. Each time, the amount of data obtained is n, and forkjoin multi-threading is used to quickly add n messages to the delayed queue.

[0093] MQ listening applications, daemon thread applications, and forkjoin all use thread pool technology, which enables concurrent processing of multiple messages and greatly improves the processing efficiency of MQ messages.

[0094] Based on the message processing method provided in the foregoing embodiments, this application also provides a message processing apparatus, see [link to relevant documentation]. Figure 6 The device includes:

[0095] The processing unit 601 is used to obtain a first message from the message queue through the listening application and perform first business logic processing. The first business processing logic includes m business logic processing steps, where m is a positive integer less than M and M is a positive integer greater than 1.

[0096] Acquisition unit 602 is used to acquire the first processing result of the first business logic processing;

[0097] Adding unit 603 is used to add the first message to the delay queue if the first processing result is a processing failure;

[0098] The processing unit 601 is further configured to continue listening to and obtaining a second message from the message queue through the listening application to perform the first business logic processing, wherein the second message is a message in the message queue that is located after the first message;

[0099] The processing unit 601 is further configured to perform second business logic processing on the first message in the delayed queue through a daemon thread application to obtain a second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

[0100] In one possible implementation, the device further includes an insertion unit and a labeling unit:

[0101] The insertion unit is used to insert the first message into the database;

[0102] The labeling unit is used to label the status of the first message in the database as pending retry before adding the first message to the delay queue.

[0103] In one possible implementation, the adding unit is specifically used for:

[0104] The first message with the status "Pending Retry" is added to the delay queue, and the status of the first message in the database is changed from "Pending Retry" to "Retrying".

[0105] In one possible implementation, the adding unit is specifically used for:

[0106] Get the number of messages already in the delay queue;

[0107] If the number of existing messages is less than a predetermined threshold, the first message is added to the delay queue.

[0108] In one possible implementation, if the first message is inserted into the database and the status of the first message is pending retry, the annotation unit is specifically used for:

[0109] If the number of existing messages reaches the specified threshold, the status of the first message in the database is changed from pending retry to failed pending retry.

[0110] In one possible implementation, the device further includes an extraction unit:

[0111] The extraction unit is used to extract n messages with a status of failure and awaiting retry from the database according to a set time and add them to the delay queue. The first message is included among the n messages with a status of failure and awaiting retry.

[0112] In one possible implementation, if the first message is inserted into the database, the annotation unit is further configured to:

[0113] If the second processing result is processing failure, the status of the first message in the database is changed to "retry count full".

[0114] As can be seen from the above technical solution, the core of this solution for message consumption based on MQ is the separation of message acquisition and processing from asynchronous re-consumption of failed messages. Messages are acquired and processed through a listening application, while failed messages are asynchronously reprocessed through a daemon thread application. Specifically, the listening application retrieves the first message from the message queue and performs the first business logic processing. This first business logic processing includes m business logic operations, where m is a positive integer less than M and M is a positive integer greater than 1. Then, the first processing result is obtained. If the first processing result is a failure, the first message is added to a delayed queue. The listening application no longer reprocesses failed messages, such as the first message, but instead continues to listen to and retrieve the second message from the message queue for the first business logic processing. The second message is the message following the first message in the message queue. The daemon thread application then performs the second business logic processing on the first message in the delayed queue, completing the reprocessing of the failed first message and obtaining the second processing result. This second business logic processing includes Mm business logic operations. This solution uses a listening application to handle message retrieval and processing, while a daemon thread application handles asynchronous reprocessing of failed messages. The two applications operate independently and do not interfere with each other. There is no need to wait for the listening application to complete all message processing cycles before retrieving new messages for processing, thus eliminating the impact of failed message reprocessing on the listening application. This greatly improves the efficiency of the listening application in retrieving messages from the message queue, enables concurrent processing of multiple messages, significantly enhances message processing efficiency, avoids message congestion, improves resource utilization, and greatly reduces the risk of bank system crashes.

[0115] This application provides a computer device including a memory and a processor; wherein, the memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computer device performs any of the methods described in the foregoing embodiments.

[0116] This application provides a computer storage medium including computer instructions, which, when executed on a computer device, cause the computer device to perform any of the methods described in the foregoing embodiments.

[0117] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0118] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0119] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A message processing method characterized by, The method includes: The application retrieves the first message from the message queue and performs the first business logic processing. The first business processing logic includes m business logic processing operations, where m is a positive integer less than M and M is a positive integer greater than 1. M represents the total number of processing operations performed on the first message. Obtain the first processing result of the first business logic processing; If the first processing result is a processing failure, the first message is added to the delayed queue, and the listening application continues to listen to and obtain the second message from the message queue to perform the first business logic processing. The second message is the message in the message queue that is located after the first message. The first message in the delayed queue is processed by a daemon thread application to obtain a second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

2. The method according to claim 1, characterized in that, The method further includes: Insert the first message into the database; Before adding the first message to the delayed queue, the method further includes: The status of the first message in the database is marked as pending retry.

3. The method according to claim 2, characterized in that, Adding the first message to the delayed queue includes: The first message with the status "Pending Retry" is added to the delay queue, and the status of the first message in the database is changed from "Pending Retry" to "Retrying".

4. The method according to claim 1, characterized in that, Adding the first message to the delayed queue includes: Get the number of messages already in the delay queue; If the number of existing messages is less than a predetermined threshold, the first message is added to the delay queue.

5. The method according to claim 4, characterized in that, If the first message is inserted into the database and the status of the first message is pending retry, the method further includes: If the number of existing messages reaches the specified threshold, the status of the first message in the database is changed from pending retry to failed pending retry.

6. The method according to claim 5, characterized in that, The method further includes: According to a set time, n messages with a status of failure and pending retry are extracted from the database and added to the delay queue. The first message is included among the n messages with a status of failure and pending retry.

7. The method according to any one of claims 1-6, characterized in that, If the first message is inserted into the database, the method further includes: If the second processing result is processing failure, the status of the first message in the database is changed to "retry count full".

8. A message processing device, characterized in that, The device includes: The processing unit is used to obtain a first message from the message queue through the listening application and perform first business logic processing. The first business processing logic includes m business logic processing operations, where m is a positive integer less than M and M is a positive integer greater than 1, and M is the total number of processing operations on the first message. The acquisition unit is used to acquire the first processing result of the first business logic processing; An adding unit is used to add the first message to a delayed queue if the first processing result is a processing failure. The processing unit is further configured to continue listening to and obtaining a second message from the message queue through the listening application to perform the first business logic processing, wherein the second message is a message in the message queue that is located after the first message; The processing unit is further configured to perform second business logic processing on the first message in the delayed queue through a daemon thread application to obtain a second processing result of the first message. The second business logic processing includes Mm business logic processing operations.

9. The apparatus according to claim 8, characterized in that, The device also includes an insertion unit and a labeling unit: The insertion unit is used to insert the first message into the database; The labeling unit is used to label the status of the first message in the database as pending retry before adding the first message to the delay queue.

10. The apparatus according to claim 9, characterized in that, The adding unit is specifically used for: The first message with the status "Pending Retry" is added to the delay queue, and the status of the first message in the database is changed from "Pending Retry" to "Retrying".

Images