Message processing method and device, equipment and storage medium

Abstract

The message processing method provided by the invention comprises the following steps: firstly, receiving message information from each producer, and sequentially inputting each message into a message queue corresponding to the producer; then, sequentially taking out the message information from each message queue in a polling mode, and putting the message information into a message candidate queue; the process ensures that the messages of all producers can be efficiently processed, and the situation that some messages wait for a long time is avoided, so that the response speed of the system is improved. The message information in the message candidate queue is sequentially taken out and sent to a consumer for processing operation. In the processing process, the message information is stored in a cache queue in a token form, and the token state is updated according to a processing result. The cache queues are in one-to-one correspondence with the consumers, so that the reliability and consistency of message processing are ensured. And the message of each producer is processed in a balanced manner, so that each producer is responded in time, and the response speed is further improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and more specifically, to a message processing method, apparatus, related equipment and readable storage medium. Background Art

[0002] The design of distributed systems is becoming increasingly complex, mainly because they need to handle large amounts of data and high concurrent requests. In order to meet performance requirements such as high throughput, low latency, and high availability, the design of distributed system components has also become complex. However, this complexity also brings higher resource consumption and difficulty in system maintenance.

[0003] In contrast, the microservices architecture emphasizes lightweight communication and fast response between services. In this architecture, message passing between services does not need to pursue extreme high performance like large-scale distributed systems. If complex, high-performance components designed for distributed systems are directly applied to the microservices architecture, it may lead to unnecessary waste of resources and reduce the response speed of the system. Therefore, using overly complex components will not only fail to improve the response speed of the microservices architecture, but may also aggravate performance problems due to its complexity and resource consumption. Summary of the invention

[0004] In view of the above problems, this application is proposed to provide a message processing method, apparatus, related equipment and readable storage medium to improve the performance of microservice architecture. The specific solution is as follows:

[0005] In a first aspect, a message processing method is provided, comprising:

[0006] Receive message information sent by each producer, and input each message information into the message queue corresponding to the producer in turn, the message queue is used to store all message information sent by the corresponding producer, and the number of the producers is the same as the number of the message queues;

[0007] Use polling to retrieve message information from each message queue in turn, and put them into the message candidate queue in turn;

[0008] Take out each message information of the message candidate queue in turn, send the message information to the consumer for processing, and store the message information in the form of a token into the cache queue, and update the token according to the result of the processing operation. The processing operation includes reading processing and executing processing. The token includes the message information and processing status. The processing status includes read, execution failure and execution success. The cache queue corresponds to the consumer.

[0009] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0010] Determine whether the processing status of the token is execution failure, and if so, determine whether the number of current execution failures reaches a preset number;

[0011] If the number of current execution failures reaches the preset number, a rollback operation is performed on the consumer.

[0012] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0013] If the current number of execution failures does not reach the preset number, the consumer is used to read and execute the message information in the token again, and the token is updated according to the execution result.

[0014] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method of sequentially taking out message information from each message queue in a polling manner and sequentially placing the message information into the message candidate queue includes:

[0015] Call the pre-configured class to determine whether the message thread is alive. If it is alive, start the message thread and instruct the message thread to take out the message information from each message queue in turn and put it into the message candidate queue in turn.

[0016] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the consumers include more than two, and the cache queues include multiple ones, respectively corresponding to each consumer;

[0017] The process of sending the message information to the consumer for processing and storing the message information in the form of a token into a cache queue includes:

[0018] The message information is sent to a target consumer corresponding to the message information for processing, and the message information is stored in a target cache queue corresponding to the target consumer in the form of a token.

[0019] In a possible design, in another implementation of the first aspect of the embodiment of the present application, before receiving the message information sent by each producer, the method further includes:

[0020] Determine whether the length of the message queue reaches the preset length;

[0021] If so, the message queue that has reached the preset length refuses to receive the message information sent by the corresponding producer, and sends a message to the corresponding producer indicating that the message queue is full.

[0022] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0023] Receive query instructions, obtain the processing status of the token in the cache queue, and send the processing status of the token to the client, so that the user can view the processing status corresponding to the message information through the client.

[0024] In a second aspect, a message processing device is provided, including:

[0025] A message information acquisition unit, used to receive message information sent by each producer, and input each message information into a message queue corresponding to the producer in sequence, wherein the message queue is used to store all message information sent by the corresponding producer, and the number of the producers is the same as the number of the message queues;

[0026] The message information output unit is used to sequentially take out the message information in each message queue in a polling manner and sequentially put it into the message candidate queue;

[0027] A message information processing unit is used to sequentially retrieve each message information of the message candidate queue, send the message information to the consumer for processing, and store the message information in the form of a token into a cache queue, and update the token according to the result of the processing operation. The processing operation includes reading processing and executing processing. The token includes the message information and a processing status. The processing status includes read, execution failure and execution success. The cache queue corresponds to the consumer.

[0028] In a third aspect, an electronic device is provided, comprising: a memory and a processor;

[0029] The memory is used to store programs;

[0030] The processor is used to execute the program to implement the various steps of the message processing method described in any one of the first aspects of the present application.

[0031] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the message processing method described in any one of the first aspects of the present application are implemented.

[0032] By means of the above technical scheme, the message processing method proposed in the present application first receives message information from each producer, and inputs each message into the message queue corresponding to the producer in turn. By allocating a dedicated message queue to each producer, the orderly storage and management of messages are ensured. Then, the message information is taken out from each message queue in turn by polling, and put into the message candidate queue. This process ensures that the message information of all producers can be processed efficiently, avoiding the situation where some messages wait for a long time, thereby improving the response speed of the system. The message information in the message candidate queue will be taken out in turn and sent to the consumer for processing. During the processing, the message information is stored in the cache queue in the form of a token, and the token status is updated according to the processing result. The cache queue corresponds to the consumer one by one, ensuring the reliability and consistency of message processing. By balancing the messages of each producer, each producer can be responded to in time, further improving the response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0034] Figure 1 A schematic diagram of a message processing method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of another message processing method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a message processing structure provided in an embodiment of the present application;

[0037] Figure 4 Another message processing structure diagram provided in the embodiment of the present application;

[0038] Figure 5 A schematic diagram of the structure of a message processing device provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Before introducing this application, the English terms involved in this article are explained first:

[0041] Microservices Architecture is a modern software development approach that decomposes an application into a series of small, independent services, each built around a specific business function and running in its own process. These services interact with each other through lightweight communication protocols (usually RESTful APIs). The core advantage of this architectural style is its high degree of modularity, which allows each service to be developed, deployed, and expanded independently without affecting the operation of other services.

[0042] Message Queue refers to the use of highly reliable message delivery mechanisms for platform-independent data interaction and the integration of distributed systems based on data communication. Through message delivery and message queuing, it can provide application decoupling, elastic scaling, redundant storage, traffic peak shaving, asynchronous communication, data synchronization and other functions in a distributed environment.

[0043] Producer: In a computer system, a producer refers to a component that generates data or messages. It is responsible for creating information and sending it to the target unit. Producers are usually closely related to business logic and generate messages according to business needs. These messages can be user behaviors, transaction records, system logs, etc. In a distributed system, the key to the design of producers is to ensure the reliability and order of data so that consumers can process the data correctly.

[0044] Consumer: A consumer is a receiver and processor of messages. In a microservice system, consumers are responsible for taking out messages and processing them.

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The present application provides a message processing method, which sets up a dedicated message queue for each producer, and uses a polling mechanism to evenly extract messages from each queue and put them into a message candidate queue, thereby ensuring timely processing of messages, avoiding long waiting times, and improving the system response speed. The information in the message candidate queue is then sent to the consumer for processing, and is stored in the cache queue corresponding to the consumer in the form of a token, and the status is updated according to the processing results, further improving the system's responsiveness.

[0047] The present application solution can be implemented based on a terminal with data processing capabilities, which can be a mobile phone, computer, learning machine, server, etc.

[0048] Next, see Figure 1 , a flow chart of a message processing method provided by the present application, the message processing method of the present application can be implemented by a message processing system deployed on a terminal, as described below, specifically including the following steps:

[0049] Step S100: Receive message information sent by each producer, and input each message information into the message queue corresponding to the producer in sequence.

[0050] Specifically, when each producer generates a message, the producer will start an asynchronous task to send the message to the system. This is done by implementing a customized asynchronous task class, which encapsulates the corresponding parameters. The system will then receive these message messages. When the message is successfully received, the system can input each message into the message queue corresponding to the corresponding producer in turn according to the producer identification information contained in the message or the preset rules. This process ensures the orderliness and consistency of the messages. Each producer has a dedicated message queue, which is specifically used to store all messages sent by the producer.

[0051] Among them, the implementation of the message queue can be based on the list data structure. The advantage of this structure is that it can quickly add and remove message information at the head and tail of the queue. Specifically, the system can add message information to the tail of the message queue. The time complexity of this operation is O(1), that is, no matter how many message information there are in the queue, the system can complete the message enqueue operation in an almost constant time. Similarly, consumers can also take out message information from the head of the queue for processing in an almost constant time. This efficient enqueue and dequeue operation ensures high efficiency of message processing and ensures that the system can quickly respond to the producer's message sending request.

[0052] In one possible implementation, before receiving a message from a producer, the system will first check whether the length of the message queue has reached the preset maximum value. This preset length can be set according to system performance and resource constraints to prevent queue overload and ensure stable system operation. If it is found that the length of a producer's message queue has reached this limit, the system will take measures to refuse to receive new messages sent by the producer and actively feedback to the producer that the message queue is full. This mechanism can not only ensure the processing efficiency of the message queue and prevent the performance of the entire system from being reduced due to a large number of messages from a single producer, but also provide producers with instant feedback, enabling them to adjust their message sending strategies in a timely manner, such as implementing message sending current limiting, thereby achieving more balanced and efficient message processing in the entire system.

[0053] Step S110: Take out the message information in each message queue in turn by polling, and put them into the message candidate queue in turn.

[0054] Specifically, in order to ensure that messages can be processed evenly and efficiently, the system uses a polling mechanism to manage the message queues of multiple producers. The system periodically traverses the exclusive message queue of each producer. This polling method ensures that each queue has a chance to be accessed, thereby ensuring the fairness of message processing. During the polling process, the system will take out message information from the head of each message queue in turn. This operation is performed sequentially, ensuring that the order in which the messages are processed is consistent with the order in which they are placed in the queue. The taken message information will then be placed in a unified message candidate queue. This message candidate queue acts as a transit station, which collects messages from different producer queues in preparation for the next step of message consumption.

[0055] Step S120: Take out each message information of the message candidate queue in turn, send the message information to the consumer for processing, store the message information in the form of a token in the cache queue, and update the token according to the result of the processing operation.

[0056] The processing operation includes a read process and an execute process. The token includes the message information and a processing state, the processing state includes read, execution failure and execution success, and the cache queue corresponds to the consumer.

[0057] Specifically, the message candidate queue stores message information from different producers, waiting to be processed by consumers. The system can retrieve each message information from the message candidate queue in a predetermined order. After the message information is retrieved, it will be sent to the consumer for processing.

[0058] The processing operation here is divided into two stages: the first is the reading process, in which the consumer performs necessary parsing and preparation after receiving the message; the second is the execution process, in which the consumer executes specific business logic according to the message content. After each message is read and processed by the consumer, it will be stored in the cache queue in the form of a token. This token contains the message information itself and the processing status related to it. There are three possible processing statuses: read, execution failed, and execution successful. These statuses reflect the processing status of the message information on the consumer side. The cache queue corresponds to the consumer, which means that each consumer has a dedicated cache queue to store the message tokens that it is processing or has processed. When the consumer completes the processing operation on the message, regardless of whether the result is success or failure, the system will update the corresponding token in the cache queue according to the processing result. This update mechanism ensures that the system can monitor the processing progress of each message in real time, and perform corresponding compensation operations or rollback operations when necessary, thereby ensuring the data consistency and stability of the entire system.

[0059] One possible implementation method is that if the content obtained from the message candidate queue is empty when consuming a message, the system will perform a waiting operation on the consumer until a new message arrives in the queue.

[0060] The message processing method proposed in the embodiment of the present application first receives message information from each producer, and inputs each message into the message queue corresponding to the producer in turn. By allocating a dedicated message queue to each producer, the orderly storage and management of messages is ensured. Then, the message information is taken out from each message queue in turn by polling, and put into the message candidate queue. This process ensures that the message information of all producers can be processed efficiently, avoiding the situation where some messages wait for a long time, thereby improving the response speed of the system. The message information in the message candidate queue will be taken out in turn and sent to the consumer for processing. During the processing, the message information is stored in the cache queue in the form of a token, and the token status is updated according to the processing result. The cache queue corresponds to the consumer one by one, ensuring the reliability and consistency of message processing. By balancing the messages of each producer, each producer can be responded to in time, further improving the response speed.

[0061] Further, in some embodiments of the present application, it may also include determining the processing status of the token, referring to Figure 2 , Figure 2 A schematic diagram of another message processing method provided in an embodiment of the present application, the specific process is as follows:

[0062] Step S200: Receive message information sent by each producer, and input each message information into the message queue corresponding to the producer in sequence.

[0063] Step S210: Take out the message information in each message queue in turn by polling, and put them into the message candidate queue in turn.

[0064] Step S220: Take out each message information of the message candidate queue in turn, send the message information to the consumer for processing, store the message information in the form of a token in the cache queue, and update the token according to the result of the processing operation.

[0065] Specifically, the above steps S200-S220 correspond one-to-one to the steps S100-S120 in the aforementioned embodiment, and please refer to the above introduction for details, which will not be repeated here.

[0066] Step S230: determine whether the processing status of the token is execution failure. If the processing status of the token is execution failure, execute step S240.

[0067] Specifically, the system will determine whether the processing status of the token is "failed to execute". If the processing status of the token is displayed as failed to execute, it means that the consumer encountered a problem when trying to process the message information. In this case, the system will not abandon the message information immediately, but will further determine whether the current number of failed executions has reached the preset limit.

[0068] Step S240, determine whether the number of current execution failures reaches a preset number, if it reaches the preset number, execute step S250; if it does not reach the preset number, execute step S260.

[0069] Specifically, when the processing status of the token is "execution failed", the system will further determine whether the number of current execution failures has reached the preset number limit. This preset number can be pre-set by the system based on business requirements and fault tolerance strategies, in order to avoid infinite attempts to process a message that may not be successfully executed, thereby wasting system resources.

[0070] Step S250: Perform a rollback operation on the consumer.

[0071] Specifically, when the system detects that the processing status of a token for a message is "failed to execute" and the current number of failed executions has reached a preset threshold, the system will perform a rollback operation on the consumer. When performing a rollback operation, the system will undo all processing performed by the consumer on the message information and restore the system state to the state before these operations were performed. In this way, the system can ensure that no inconsistent state or data is left in the event of a processing failure. After the rollback operation is completed, the system can record relevant error information for subsequent analysis and debugging.

[0072] Step S260: Use the consumer to read and execute the message information in the token again, and update the token according to the execution result.

[0073] Specifically, when the system detects that the processing status of a token is "failed", and the number of current execution failures has not reached the preset threshold, the system will not abandon the message immediately, but will start a compensation mechanism. The purpose of this retry is to give the system a chance to correct the processing failure that may be caused by temporary problems or accidental errors.

[0074] After the consumer tries to process the message again, the system will update the token based on the result of this execution. If the message information is processed successfully, the status of the token will be updated to "execution successful", indicating that the message has been properly processed. If the message processing still fails after another attempt, the status of the token will remain "execution failed", and the system will record this failed attempt and increase the number of execution failures.

[0075] In this embodiment, first, the message information from each producer is received, and each message is input into the message queue corresponding to the producer in turn. By allocating a dedicated message queue to each producer, the orderly storage and management of messages is ensured. Then, the message information is taken out from each message queue in turn by polling and put into the message candidate queue. This process ensures that the message information of all producers can be processed efficiently, avoiding the situation where some messages wait for a long time, thereby improving the response speed of the system. The message information in the message candidate queue will be taken out in turn and sent to the consumer for processing. During the processing, the message information is stored in the cache queue in the form of a token, and the token status is updated according to the processing result. The cache queue corresponds to the consumer one by one, ensuring the reliability and consistency of message processing. By balancing the messages of each producer, each producer can be responded to in time, further improving the response speed.

[0076] In addition, the system will determine whether the processing status of the token is "execution failure". If the token status is found to be execution failure, the system will enter step S240 to further determine whether the current number of execution failures has reached the preset limit. This avoids unnecessary repeated attempts on messages that may not be successfully executed, thereby saving system resources. If the number of execution failures reaches the preset threshold, the system will execute step S250 to roll back the consumer, and restore the system state to the state before the operation by undoing all processing, thereby ensuring data consistency and system reliability. On the contrary, if the number of execution failures does not reach the preset threshold, the system will execute step S260 to use the consumer to try to read and execute the message information in the token again. By trying to process the message again, the system has the opportunity to correct the processing failure caused by temporary problems or accidental errors, thereby improving the success rate of message processing and the fault tolerance of the system. After the consumer tries to process the message again, the system will update the token according to the execution result. This update mechanism ensures that the system can monitor the processing progress of each message in real time. It achieves efficient and reliable message processing while ensuring system stability and data integrity. It is suitable for high-concurrency and complex business scenarios, and can provide effective recovery and compensation mechanisms in the face of failures and exceptions, ensuring the reliability of the entire system.

[0077] In some embodiments of the present application, for the process of "taking out message information from each message queue in turn by polling and putting it into the message candidate queue in turn" in the aforementioned embodiment, a message thread may be used. The process of obtaining message information by using a message thread is described below:

[0078] Optionally, the system can use a thread management mechanism to obtain message information. This mechanism involves calling a pre-configured class that is specifically used to manage and monitor the status of message threads. First, this class will be used to determine whether the message thread is alive and whether the thread can be started and run normally.

[0079] The system will then instruct the surviving message thread to start its message processing task. The task of the message thread is to access the message queues of each producer in turn. This is a polling process that ensures that each queue has a chance to be processed, thereby achieving balanced consumption of message information. The message thread will take out the message information from the head of each queue. This operation is performed sequentially, ensuring that the order in which the messages are processed is consistent with the order in which they are placed in the queue. The taken-out message information will then be placed in a unified message candidate queue, which acts as a transit station, collecting messages from different producer queues in preparation for the next step of message consumption.

[0080] In this embodiment, the system can effectively improve efficiency by managing and monitoring message threads through pre-configured classes. This thread usage strategy not only improves the efficiency of resource utilization, but also enhances the response speed of the system. By polling the message queues of each producer through the message thread, balanced message consumption is achieved, ensuring that the messages of each producer can be processed fairly and in a timely manner. This balancing mechanism avoids excessive delays in the messages of some producers, thereby improving the processing efficiency and fairness of the entire system.

[0081] In some embodiments of the present application, another method for querying message information is provided. This method can support users to view the processing status of tokens on the client, so as to timely understand the processing progress of message information. The process is introduced below.

[0082] Optionally, when the user needs to know the processing status of the message information, this can be achieved by sending a query instruction. After receiving this query instruction, the system will access the cache queue, which is a queue specifically used to store message tokens and their processing status. The system retrieves the token from the cache queue and obtains its processing status. This processing status can include information such as read, successful execution, or failed execution. Each status clearly indicates the current position or final result of the message in the processing flow. After obtaining this status information, the system sends it to the client, which can be any user interface that interacts with the system, such as a web page, mobile application, or other form of user interface.

[0083] In this embodiment, this design not only allows users to understand the progress of message processing in a timely manner, but also allows them to quickly respond when message processing fails. In addition, this query mechanism also facilitates system monitoring and management, allowing users to easily track message processing, thereby improving system manageability.

[0084] See also Figure 3 , Figure 3 This is a message processing structure disclosed in this embodiment. Figure 3 To introduce:

[0085] The figure includes key components such as producers, consumers, message queues, message candidate queues and cache queues, among which the message queues, message candidate queues and cache queues can constitute a message queue unit.

[0086] Optionally, in the message processing system, the producer is responsible for generating message information and sending the message information to the message queue according to the system design. The system supports a single producer or multiple producers. When there is only one producer, the producer can directly send the message information to the message candidate queue without going through an additional polling mechanism. Figure 3In the example, there are more than two producers. In this case, each producer has its own business, and they send message information to consumers through their own message queues. These message queues are exclusive to producers and are used to store all messages sent by producers. There are message threads in the message candidate queue and message queue. The message thread is responsible for taking messages from each producer's message queue in turn by polling and putting these messages into the message candidate queue.

[0087] Then the message information in the message candidate queue will be processed by the consumer according to the first-in-first-out principle. After processing the message, the consumer will store the message in the cache queue in the form of a token. These tokens contain the message information and processing status, where the processing status includes read, execution failure, and execution success. If the consumer encounters an execution failure when processing a message, the system will determine whether to roll back the producer based on the number of failures. If the number of execution failures does not reach the preset number, the consumer will try to read and execute the message information in the token again until it succeeds or reaches the preset number of failures.

[0088] Allows users to query the processing status of the token corresponding to the message information through the client, providing a way to monitor and track the progress of message processing. Users can view the processing status corresponding to the message information through the client to understand whether the message has been successfully processed or whether there are any issues that need attention.

[0089] In some embodiments of the present application, a message processing method based on multiple consumers is further provided.

[0090] See also Figure 4 , Figure 4 Another message processing structure disclosed in this embodiment is a structure of a message processing method based on multiple consumers, wherein a message queue, a message candidate queue, and a cache queue can constitute a message queue unit.

[0091] For the process of "sending the message information to the consumer for processing, and storing the message information in the form of a token into the cache queue" in the aforementioned embodiment, it can also support the use of more than two consumers to process the message information. Next, the message processing flow adopted by the message processing method of the present application when facing multiple consumers will be introduced in detail.

[0092] Optionally, in order to improve the processing power and flexibility of the system, the system includes more than two consumers. Each consumer can be responsible for processing different or the same type of messages. This design allows the system to process messages in parallel, improving the overall processing efficiency.

[0093] Corresponding to the number of consumers, the system also includes multiple cache queues. These cache queues correspond to each consumer and are responsible for storing the tokens of the message information processed by the corresponding consumer, ensuring that the message information can be correctly allocated and stored.

[0094] When a message is sent to a consumer for processing, the process involves sending the message to the target consumer corresponding to the message. Once the message is sent to the target consumer, the consumer will perform the processing. After the processing is completed, the message will be stored in the target cache queue corresponding to the target consumer in the form of a token. This token contains not only the message itself, but also detailed information about the message processing status, i.e., consumed, executed successfully, or executed failed.

[0095] The embodiment of the present application significantly improves the processing power and flexibility of the message processing system by introducing the design of multiple consumers and multiple cache queues. The system includes more than two consumers, and each consumer is responsible for processing messages of different or the same type. Corresponding to the number of consumers, the system also includes multiple cache queues, each cache queue corresponds to a consumer, ensuring that the message information can be correctly allocated and stored. This design allows the system to process messages in parallel, improves the overall processing efficiency, ensures the consistency and integrity of the data, and facilitates the monitoring and troubleshooting of the system. Each cache queue serves as an independent storage unit, responsible for keeping the message tokens processed by the corresponding consumer, which contain the message information and its processing status, such as errors that have been successfully processed or encountered during the processing process. Through the design of multiple consumers and multiple cache queues, efficient parallel processing of messages is achieved, and the overall performance and reliability of the system are improved.

[0096] The message processing method of the present application will have a significant effect. The following Table 1 shows the performance comparison of the microservice system when the method of the present application is used and when the method of the present application is not used. Table 2 shows the operation of the Pod when the method of the present application is used and when the method of the present application is not used. Next, these data will be analyzed in detail to demonstrate the superiority of the message processing method of the present application.

[0097] Table 1

[0098]

[0099] Table 1 is a data analysis of the performance of the microservice system before and after using the message processing method of the present application. By comparing the two sets of data, it can be clearly seen that after using the message processing method of the present application, the microservice system has significantly improved in multiple key performance indicators.

[0100] First, from the perspective of average throughput, when the message processing method of the present application is not used, the throughput of the microservice system is 2, and after using the message processing method of the present application, the throughput is greatly improved to 215. This shows that the message processing method of the present application greatly improves the processing capacity of the microservice system, enabling the microservice system to process more requests more efficiently.

[0101] Secondly, the average throughput of successful requests also increased from 2.0 to 215, indicating the effectiveness of the message processing method of the present application in improving the processing efficiency of the microservice system. In terms of response time, after using the message processing method of the present application, the average response time was reduced from 9.7917 seconds to 0.4917 seconds, which means that the microservice system has a faster response speed and the user experience has been greatly improved. In addition, the maximum response time was also significantly reduced from 300.858 seconds to 8.4680 seconds, which shows that the performance of the microservice system has also been significantly improved when processing the most complex or time-consuming requests.

[0102] Furthermore, the improvement of the request response code success rate and the verification point success rate also reflects the advantages of the message processing method of the present application. When the message processing method of the present application is not used, the success rates are 82.91% and 82.91% respectively, and after using the message processing method of the present application, both are increased to 99.87%. This shows that the message processing method of the present application not only improves the processing speed of the microservice system, but also improves the stability and reliability of the microservice system.

[0103] In summary, by introducing the message processing method of the present application, the system has significantly improved multiple key performance indicators such as throughput, response time and success rate.

[0104] Table 2

[0105]

[0106] Table 2 is a data analysis of resource utilization of the microservice system without and with the message processing method of the present application. By comparing the two sets of data, it can be clearly seen that after using the message processing method of the present application, the microservice system has significantly optimized CPU and memory utilization.

[0107] First, from the perspective of average memory utilization, when the message processing method of the present application is not used, the memory utilization of the microservice system is 50%, while after using the message processing method of the present application, the memory utilization is reduced to 30%. This significant decrease shows that the microservice system using the message processing method of the present application is more efficient in memory management, which can reduce the occupation of memory resources when processing messages, thereby improving the efficiency of memory use.

[0108] Secondly, the average CPU utilization rate also dropped from 1.8% to 1.5%. This shows that the message processing method of the present application not only optimizes the message processing process, but also reduces the demand for CPU resources, allowing the microservice system to run more efficiently.

[0109] In summary, the microservice system that introduces the message processing method of the present application has significantly improved multiple key performance indicators such as throughput, response time, success rate, and resource utilization. These improvements not only improve the overall performance of the system, but also improve the user experience, while also providing a guarantee for the stable operation of the system. Therefore, it can be concluded that the message processing method of the present application has obvious advantages in improving the communication efficiency between services in the microservice system and the system response rate.

[0110] The message processing device provided in an embodiment of the present application is described below. The message processing device described below and the message processing method described above can be referenced to each other.

[0111] See also Figure 5 , Figure 5 A schematic diagram of the structure of a message processing device disclosed in an embodiment of the present application.

[0112] like Figure 5 As shown, the device may include:

[0113] The message information acquisition unit 11 is used to receive the message information sent by each producer, and input each message information into the message queue corresponding to the producer in sequence, and the message queue is used to store all the message information sent by the corresponding producer. The number of the producers is the same as the number of the message queues;

[0114] The message information output unit 12 is used to sequentially take out the message information in each message queue in a polling manner and sequentially put it into the message candidate queue;

[0115] The message information processing unit 13 is used to take out each message information of the message candidate queue in turn, send the message information to the consumer for processing, and store the message information in the form of a token into the cache queue, and update the token according to the result of the processing operation. The processing operation includes reading processing and executing processing. The token includes the message information and the processing status. The processing status includes read, execution failure and execution success. The cache queue corresponds to the consumer.

[0116] Optionally, a message processing device in an embodiment of the present application further includes:

[0117] A status determination unit, configured to determine whether the processing status of the token is execution failure, and if so, to determine whether the number of current execution failures has reached a preset number;

[0118] The rollback operation unit is used to perform a rollback operation on the consumer when the number of current execution failures reaches a preset number.

[0119] Optionally, a message processing device in an embodiment of the present application further includes:

[0120] The reprocessing unit is used to use the consumer to read and execute the message information in the token again when the number of current execution failures does not reach the preset number, and update the token according to the execution result.

[0121] In one possible implementation, the message information output unit 12 uses a polling method to sequentially take out the message information from each message queue and sequentially put it into the message candidate queue. The process includes: calling a pre-configured class to determine whether the message thread is alive. If it is alive, the message thread is started, and the message thread is instructing to sequentially take out the message information from each message queue and sequentially put it into the message candidate queue.

[0122] In a possible implementation, the process in which the message information processing unit 13 sends the message information to the consumer for processing and stores the message information in the form of a token into the cache queue includes:

[0123] The message information is sent to a target consumer corresponding to the message information for processing, and the message information is stored in a target cache queue corresponding to the target consumer in the form of a token.

[0124] Optionally, a message processing device in an embodiment of the present application further includes:

[0125] A length determination unit, used to determine whether the length of the message queue reaches a preset length;

[0126] The message blocking unit is used to refuse to receive the message information sent by the corresponding producer when the length of the message queue reaches a preset length, and send a message to the corresponding producer that the message queue is full.

[0127] Optionally, a message processing device in an embodiment of the present application further includes:

[0128] The token query unit is used to receive the query instruction, obtain the processing status of the token in the cache queue, and send the processing status of the token to the client, so that the user can view the processing status corresponding to the message information through the client.

[0129] The present application also provides an electronic device in an embodiment. Figure 6As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, teaching large screens, wearable devices, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0130] like Figure 6 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the message processing method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0131] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0132] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the message processing method of the aforementioned embodiment of the present application.

[0133] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0134] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0135] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0136] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0137] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A message processing method, characterized in that: include: Receive message information sent by each producer, and input each message information into the message queue corresponding to the producer in turn, the message queue is used to store all message information sent by the corresponding producer, and the number of the producers is the same as the number of the message queues; Use polling to retrieve message information from each message queue in turn, and put them into the message candidate queue in turn; Take out each message information of the message candidate queue in turn, send the message information to the consumer for processing, and store the message information in the form of a token into the cache queue, and update the token according to the result of the processing operation. The processing operation includes reading processing and executing processing. The token includes the message information and processing status. The processing status includes read, execution failure and execution success. The cache queue corresponds to the consumer.

2. The method according to claim 1, characterized in that Also includes: Determine whether the processing status of the token is execution failure, and if so, determine whether the number of current execution failures reaches a preset number; If the number of current execution failures reaches the preset number, a rollback operation is performed on the consumer.

3. The method according to claim 2, characterized in that Also includes: If the current number of execution failures does not reach the preset number, the consumer is used to read and execute the message information in the token again, and the token is updated according to the execution result.

4. The method according to claim 1, characterized in that: The method of taking out the message information from each message queue in turn by polling and putting them into the message candidate queue in turn includes: Call the pre-configured class to determine whether the message thread is alive. If it is alive, start the message thread and instruct the message thread to take out the message information from each message queue in turn and put it into the message candidate queue in turn.

5. The method according to claim 1, characterized in that: The consumers include more than two, and the cache queues include multiple, corresponding to each consumer respectively; The process of sending the message information to the consumer for processing and storing the message information in the form of a token into a cache queue includes: The message information is sent to a target consumer corresponding to the message information for processing, and the message information is stored in a target cache queue corresponding to the target consumer in the form of a token.

6. The method according to claim 1, characterized in that Before receiving the message information sent by each producer, it also includes: Determine whether the length of the message queue reaches the preset length; If so, the message queue that has reached the preset length refuses to receive the message information sent by the corresponding producer, and sends a message to the corresponding producer indicating that the message queue is full.

7. The method according to claim 1, characterized in that Also includes: Receive query instructions, obtain the processing status of the token in the cache queue, and send the processing status of the token to the client, so that the user can view the processing status corresponding to the message information through the client.

8. A message processing device, characterized in that: include: A message information acquisition unit, used to receive message information sent by each producer, and input each message information into a message queue corresponding to the producer in sequence, wherein the message queue is used to store all message information sent by the corresponding producer, and the number of the producers is the same as the number of the message queues; The message information output unit is used to sequentially take out the message information in each message queue in a polling manner and sequentially put it into the message candidate queue; A message information processing unit is used to sequentially retrieve each message information of the message candidate queue, send the message information to the consumer for processing, and store the message information in the form of a token into a cache queue, and update the token according to the result of the processing operation. The processing operation includes reading processing and executing processing. The token includes the message information and a processing status. The processing status includes read, execution failure and execution success. The cache queue corresponds to the consumer.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the message processing method as described in claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the message processing method as described in claims 1 to 7 is implemented.

Images