Message processing method and device, computer equipment and storage medium

Abstract

The invention relates to a message processing method and device, computer equipment and a storage medium. The message processing method comprises the following steps: in response to a processing request of messages in a message queue, determining that a pre-configured cache space exists in the message queue, aggregating the messages in the message queue through the cache space, and obtaining the current aggregation frequency, the current message sending rate and / or the current CPU utilization rate, and determining the current maximum aggregation frequency according to the current message sending rate and / or the current CPU utilization rate, and when the current aggregation frequency is greater than or equal to the current maximum aggregation frequency, sending the aggregated messages in the cache space. By adopting the method, the timeliness of message processing can be improved under the condition that the hardware cost is not increased.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a message processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] A normal storage system write (Input / Output) involves numerous message types, including cache mirroring messages, metadata synchronization messages, and RAID (redundant array of independent disks) synchronization messages. Each message must be processed by the communication module, submitted to the driver, and ultimately sent through the interconnect card. Due to the excessive number of communication messages being processed, the CPU (Central Processing Unit) core bound to the driver is overloaded with processing tasks, resulting in high CPU utilization and message congestion, ultimately affecting the storage system's write bandwidth and write latency.

[0003] Currently, a common approach to addressing communication latency is to use hardware multi-queues. Hardware multi-queues essentially allocate a CPU core to each hardware queue without expanding the hardware. This allows interconnected boards to have more CPU cores processing communication messages, thereby improving the communication module's processing power.

[0004] However, CPU core resources are limited and valuable for the entire storage system, as each business module requires a portion. Allocating too many CPU cores to interconnect boards alone would result in excessive hardware costs. Summary of the Invention

[0005] Based on this, it is necessary to provide a message processing method, apparatus, computer equipment and storage medium that can improve the timeliness of message processing without increasing hardware costs in order to address the above technical problems.

[0006] In a first aspect, the present application provides a message processing method, comprising:

[0007] In response to a request to process a message in a message queue, determining that a preconfigured buffer space exists in the message queue;

[0008] Aggregate messages in the message queue through the cache space and obtain the current aggregation count;

[0009] Obtain the current message sending rate and / or the current CPU utilization, and determine the current maximum number of aggregations based on the current message sending rate and / or the current CPU utilization;

[0010] When the current aggregation times is greater than or equal to the current maximum aggregation times, the aggregated messages in the cache space are sent.

[0011] In one embodiment, messages in a message queue are aggregated using a cache space, and a current aggregation count is obtained, including:

[0012] Starting from the head of the message list in the message queue, each message is traversed in sequence and copied to the cache space in sequence for aggregation;

[0013] The current aggregation count is determined based on the number of scan starts.

[0014] In one embodiment, the message processing method further includes:

[0015] When no message is taken out from the message list, the number of invalid aggregations is recorded;

[0016] The maximum number of aggregation attempts is determined based on the current message sending rate and / or current CPU utilization, including:

[0017] When the current message sending rate and / or current CPU utilization is greater than a preset value, the aggregation times range configured in advance for the message queue is obtained. The aggregation times range is configured according to the priority of the message queue. The higher the priority of the message queue, the smaller the aggregation times range corresponding to the message queue.

[0018] Determine the current aggregation efficiency based on the current aggregation times and the invalid aggregation times;

[0019] Get the time when the message enters the message queue and determine the timeout of the message based on the time when the message enters the queue;

[0020] The current maximum number of aggregations is determined based on the current aggregation efficiency, the range of aggregation times, and the message timeout.

[0021] In one embodiment, the message processing method further includes:

[0022] When the cache space is full, the messages in the cache space are sent;

[0023] When the current aggregation count is greater than or equal to the current maximum aggregation count, the messages in the cache space are sent, including:

[0024] When the cache space is not full and the current aggregation count is greater than or equal to the current maximum aggregation count, the messages in the cache space are sent.

[0025] In one embodiment, the message processing method further includes:

[0026] When there is no cache space in the message queue, new cache space is obtained from the cache resource pool. The cache resource pool has a cache linked list, and the new cache space is obtained from the head of the cache linked list.

[0027] Aggregate the messages in the message queue based on the new cache space.

[0028] In one embodiment, the message processing method further includes:

[0029] When the new cache space is not full and the current aggregation count is less than the current maximum aggregation count, the new cache space is saved in the message queue.

[0030] In one embodiment, after sending the message in the cache space, the message processing method further includes:

[0031] Release the cache space and add it to the end of the cache list.

[0032] In a second aspect, a message processing device is provided, including:

[0033] a determination module, configured to determine, in response to a request for processing a message in the message queue, whether a preconfigured buffer space exists in the message queue;

[0034] Aggregation module, used to aggregate messages in the message queue through cache space and obtain the current aggregation count;

[0035] An acquisition module is used to obtain a current message sending rate and / or a current CPU utilization rate, and determine a current maximum number of aggregation times based on the current message sending rate and / or the current CPU utilization rate;

[0036] The sending module is used to send the aggregated messages in the cache space when the current aggregation number is greater than or equal to the current maximum aggregation number.

[0037] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the message processing method provided in any one of the embodiments of the present application in the first aspect are implemented.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the message processing method provided in any embodiment of the first aspect of the present application.

[0039] The above-mentioned message processing method, device, computer equipment and storage medium reduce the number of messages processed by the CPU by aggregating messages. The CPU originally needed to process multiple small-block message messages separately, but after aggregation, it only needs to process fewer large-block message messages, thereby reducing the processing burden of the CPU. At the same time, the current maximum number of aggregations is dynamically determined according to the current message sending rate or CPU utilization, and the current maximum number of aggregations is adaptively adjusted. The degree of aggregation is controlled according to the current maximum number of aggregations. This can avoid excessive aggregation leading to long message waiting times, improve the timeliness of message processing, and improve the aggregation effect as much as possible when system resources permit. It can be seen that the present application can reduce the CPU core utilization rate of the interconnected board by aggregating messages and adaptively adjusting the aggregation capability. While meeting the same business processing requirements, the number of CPU cores required is reduced, saving hardware CPU cores, and improving the timeliness of message processing without increasing hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A diagram of an application environment of a message processing method in some embodiments;

[0041] Figure 2 Schematic diagram of a message processing method in some embodiments;

[0042] Figure 3 Schematic diagram of a flow chart of a message processing method in some other embodiments;

[0043] Figure 4 is a structural block diagram of a message processing device in some embodiments;

[0044] Figure 5 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] Please refer to Figure 1 , Figure 1 This is a schematic diagram of an application environment of a message processing method provided by an exemplary embodiment of the present application. Figure 1 As shown, the application environment includes a storage system 100 , which is provided with a business module 101 , a message receiving module 102 , a message aggregation module 103 , an aggregation calculation module 104 , a message sending module 105 and a driver layer 106 .

[0047] The service module 101 is used to send a message to the message queue in the message receiving module 102 and notify the message receiving module 102 to process the message.

[0048] The message receiving module 102 is used to store messages and notify the message aggregation module 103 to aggregate the messages.

[0049] The message aggregation module 103 is configured to respond to a request for processing a message in a message queue, obtain a pre-configured buffer space in the message queue, and aggregate the messages in the message queue using the buffer space.

[0050] The aggregation calculation module 104 is used to obtain the current number of aggregations, the current message sending rate and / or the current CPU utilization, and determine the current maximum number of aggregations based on the current message sending rate and / or the current CPU utilization. When the current number of aggregations is greater than or equal to the current maximum number of aggregations, the driver layer 106 is notified to send out the aggregated messages in the cache space.

[0051] In a first aspect, the present application provides a message processing method, such as Figure 2 As shown, this method is applied to Figure 1 The following steps are used as an example to illustrate the storage system in Figure 1:

[0052] Step S21 : In response to a request to process a message in a message queue, determining whether there is pre-configured buffer space in the message queue.

[0053] A storage system is a system composed of hardware and software used to store, manage, and retrieve data. Its main function is to store data reliably over a long period of time and provide fast and accurate access to data when needed.

[0054] A message queue is an asynchronous communication mechanism used to pass messages between different components or systems. It works on a first-in-first-out principle, just like a queuing system, where the first message to enter the queue is processed first.

[0055] A buffer is a pre-allocated memory area in a message queue used to store aggregated, large messages. A large message is a message aggregated from multiple small messages. Specifically, the present application may include multiple buffers, each consisting of a preset number (e.g., 8) physical pages, each of which is used to store data.

[0056] Specifically, the present application may first check whether there is a pre-configured cache space in the message queue. If so, it may further determine whether there is any remaining space in the cache space. If so, the messages in the message queue may be aggregated through the cache space.

[0057] Step S22: Aggregate the messages in the message queue using the cache space and obtain the current aggregation times.

[0058] The current aggregation count refers to the number of aggregation operations performed on messages in the message queue from the start of the aggregation operation to the current moment. It is a dynamically changing value that is continuously updated as the message aggregation process progresses, and is used to record the number of message aggregation operations that have been performed.

[0059] Specifically, the application uses the variable cur_coalesce_count to count the current number of coalescences. Each time the message list is traversed sequentially from the head (i.e., the operation of copying the message into the cache space is started), cur_coalesce_count is increased by 1, so that the value of cur_coalesce_count represents the number of coalescences that have been performed so far.

[0060] Message aggregation refers to the process of combining multiple, dispersed, smaller messages into a larger, more comprehensive message. Specifically, this application aggregates small messages into a larger message block and stores it in a cache. The cache space is the maximum buffer size for each large message block. If the cache space is full, aggregation cannot continue and the large message block must be sent.

[0061] Specifically, the present application can use the cache space it has to process the messages stored in the message queue. Small blocks of messages are taken out from the message queue one by one to determine whether these small blocks of messages can be combined and placed in the cache space. If so, these small blocks of messages are aggregated into large blocks of messages according to certain rules (such as the order in which they are received) and stored in the cache space. The present application can add 1 to the number of aggregations when completing such an operation of aggregating small blocks of messages into a large block of messages to obtain the current number of aggregations.

[0062] Furthermore, continue to take messages from the message queue and repeat the above judgment and aggregation operations, and update the current aggregation count each time the aggregation is performed until all the messages in the message queue are processed or other aggregation termination conditions are met.

[0063] Step S23: Acquire the current message sending rate and / or the current CPU utilization, and determine the current maximum number of aggregation times according to the current message sending rate and / or the current CPU utilization.

[0064] The current maximum number of aggregations is a threshold determined based on the current message sending rate and / or CPU utilization. This threshold indicates the maximum number of times the message aggregation operation can be performed in the cache space to ensure system performance and message processing efficiency under the current system state. When the current number of aggregations reaches or exceeds this threshold, the aggregated messages in the cache space must be sent. The current maximum number of aggregations is also a dynamic value that can be adjusted dynamically based on the current message sending rate and / or CPU utilization.

[0065] The current message sending rate refers to the number of messages sent per unit time by the system at the current moment or within the current time period. It reflects the speed and frequency of the system's message sending.

[0066] The current CPU utilization indicates the percentage of total CPU time used at the current moment. It measures how busy the CPU is; a higher value indicates a busier CPU.

[0067] The current maximum aggregation count is the maximum number of times messages can be aggregated to achieve optimal system performance or meet specific resource usage requirements, based on factors such as the current message sending rate or CPU utilization.

[0068] Among them, the current maximum number of aggregations in this application will change. Each time an aggregation occurs, the current maximum number of aggregations will be calculated based on the current message sending rate and / or the current CPU utilization, and the current maximum number of aggregations will be adjusted in time.

[0069] Specifically, the present application can monitor the current message sending rate or the current CPU utilization. For example, the message sending rate can be calculated by recording the number of messages sent per unit time; for CPU utilization, the operating system usually provides a corresponding interface to obtain this data.

[0070] Furthermore, the present application may pre-set certain rules or algorithms to calculate the current maximum number of aggregations based on the current message sending rate and / or current CPU utilization. For example, if the current message sending rate is low, it indicates that the system has sufficient processing power, and more aggregations may be allowed to improve efficiency; if the CPU utilization is already high, the maximum number of aggregations may be reduced to avoid system overload, thereby reducing the additional burden of processing message aggregations.

[0071] Step S24: When the current aggregation number is greater than or equal to the current maximum aggregation number, the aggregated messages in the cache space are sent.

[0072] Specifically, the present application can check the current aggregation times regularly or at a specific time point and compare it with the current maximum aggregation times. This checking process can be performed after each message aggregation operation is completed, or it can be polled and checked at a certain time interval.

[0073] Furthermore, when it is determined that the current aggregation count is greater than or equal to the current maximum aggregation count, the message sending mechanism is triggered to send out the messages that have been aggregated in the cache space. The sending process can be performed through the driver layer.

[0074] In this application, by limiting the maximum number of aggregations, we avoid excessive aggregation, which causes messages to stay in the cache space for too long, ensuring that messages can be sent in a timely manner. At the same time, we also prevent excessive occupation of system resources such as CPU and memory due to too frequent aggregation operations, affecting the overall performance of the system.

[0075] It can be seen that this application can ensure that messages will not be delayed due to indefinite waiting for more message aggregation, ensuring that messages can be delivered to the recipient in a timely manner within a certain period of time, meeting the business requirements for real-time messages.

[0076] In one embodiment, messages in a message queue are aggregated through a cache space, and the current aggregation count is obtained, including: traversing each message in sequence from the head of a message linked list in the message queue, and copying each message in sequence to the cache space for aggregation, and determining the current aggregation count based on the number of times the scan is started.

[0077] Among them, the message linked list is a data structure used to organize messages in the message queue. It consists of a series of nodes, each node stores a message, and each node contains a pointer to the next node (in a one-way linked list) or contains pointers to the previous and next nodes (in a two-way linked list). In this application, the messages in the message queue are organized using a linked list structure, and the latest sent message will be hung at the end of the linked list, which is convenient for adding and traversing messages. Through the linked list structure, the messages in the message queue can be dynamically managed, and message nodes can be flexibly inserted and deleted.

[0078] Specifically, the present application can start a traversal operation from the head of the message linked list in the message queue when receiving a message processing request. Take out a message from the head of the linked list each time. Copy the taken messages to the cache space in sequence to achieve message aggregation. Every time a new traversal scan is started from the head of the message linked list in the message queue, the current aggregation count is increased by 1. The present application continues to perform the above-mentioned traversal, copying and updating of the current aggregation count. Among them, the number of times the scan is started corresponds to the current aggregation count. The aggregation count increases once the scan is performed, and the number of aggregation operations currently performed is counted by recording the number of scans. In this way, it can be clearly known how many times the message has been traversed and aggregated starting from the head of the linked list.

[0079] Furthermore, the present application can periodically check the relationship between the current number of aggregations and the current maximum number of aggregations. When the current number of aggregations is greater than or equal to the current maximum number of aggregations, it indicates that the aggregation operation has reached the set upper limit. At this time, the aggregation operation is terminated and the large block of messages that have been aggregated in the cache space are sent out.

[0080] The beneficial effect of this embodiment is that: under high-load IOPS (Input / Output Operations Per Second), the transmission of a large number of small-block message messages will occupy more system resources and bandwidth. Therefore, the present application reduces the number of message transmissions by aggregating these small-block messages into large-block message messages, so that more data can be transmitted per unit time, thereby improving the communication bandwidth. In addition, by aggregating messages, the number and time of message processing by the storage system are reduced, avoiding the additional delay caused by frequent processing of small-block messages, so that data can be transmitted and processed more quickly under high-load IOPS.

[0081] In one embodiment, the message processing method also includes: when the message is not taken out from the message list, recording the invalid aggregation number, determining the current maximum aggregation number based on the current message sending rate and / or the current CPU utilization, including: when the current message sending rate and / or the current CPU utilization is greater than a preset value, obtaining the range of aggregation times pre-configured for the message queue, the range of aggregation times is configured according to the priority of the message queue, wherein the higher the priority of the message queue, the smaller the range of aggregation times corresponding to the message queue, determining the current aggregation efficiency based on the current aggregation number and the invalid aggregation number, obtaining the enqueue time of the message in the message queue, and determining the message timeout time based on the enqueue time, and determining the current maximum aggregation number based on the current aggregation efficiency, the range of aggregation times and the message timeout time.

[0082] The number of invalid aggregations refers to the number of times when no message was retrieved while attempting to retrieve a message from the message list for aggregation. It reflects the cumulative number of times no message was retrieved during the aggregation process.

[0083] Specifically, the present application performs a traversal operation on the message list to perform the message aggregation task. Each time an attempt is made to retrieve a message from the message list, a judgment will be made to check whether the message is successfully retrieved. If no message is retrieved from the message list in this attempt (i.e., the message list is empty or there is no valid message at the current traversal position), this aggregation is determined to be an invalid aggregation, and the number of invalid aggregations is increased by 1 to count the number of invalid aggregations. If the message is successfully retrieved from the message list during this aggregation process, this aggregation is determined to be a valid aggregation, and the number of valid aggregations is increased by 1 to count the number of valid aggregations for subsequent calls.

[0084] The priority of a message queue refers to setting different priority levels for different messages in the message queue. It determines the order in which messages are processed.

[0085] The aggregation count range is the range of allowed values for the aggregation count. A higher-priority message queue has a smaller aggregation count range. This means that a higher-priority message queue has a smaller range of aggregation counts, meaning the difference between the upper and lower limits of the range is smaller.

[0086] In this application, the priority of each message queue is different, and a different aggregation number range is set for the message queue of each priority, and the higher the priority of the message queue, the smaller the aggregation number range is set. For example, the three message queues are message queue 1, message queue 2 and message queue 3 in order from high to low priority. When configuring the aggregation number range, they can be configured as (10, 20), (20, 40) and (40, 70) respectively. Message queue 1 has the highest priority, so the configured range size is the smallest, which is 20-10=10. Message queue 3 has the highest priority, so the configured range size is the largest, which is 70-40=30.

[0087] Specifically, this application can configure the aggregation times range of different message queues in the following ways:

[0088] First, the higher the priority of the message queue, the smaller the range of aggregation times is configured.

[0089] Secondly, the higher the priority of the message queue, the smaller the range of aggregation times is configured, and the lower limit of the range is also configured to be smaller.

[0090] Third, the higher the priority of the message queue, the smaller the range of aggregation times is configured, and the upper and lower limits of the range are set to be smaller.

[0091] The enqueue time of a message in a message queue refers to the time each message is added to the queue. The message timeout is the maximum processing time set for each message in the message queue to prevent messages from consuming resources if they remain unprocessed for extended periods. If a message remains in the queue for longer than this set time, it is considered to have timed out. The message timeout is calculated as: timeout = message enqueue time + set timeout threshold - current time. If the result is less than 0, the message has timed out. For example, if a message is enqueued at 10:00, the set timeout threshold is 30 minutes, and the current time is 10:40, then the timeout = 10:00 + 30 minutes - 10:40 = -10 minutes, indicating that the message has timed out for 10 minutes.

[0092] Furthermore, the step of calculating the current maximum number of aggregations in this application may include:

[0093] Calculate the timeout of each message, and then calculate the average timeout of all messages based on the timeout of each message. Delay impact factor is determined based on the average timeout. For example, if the average timeout is T1 and the maximum timeout is T2, then the delay impact factor f = min(1, T1 / T2).

[0094] The polymerization efficiency is calculated based on the effective polymerization times and the ineffective polymerization times, wherein polymerization efficiency = effective polymerization times / (effective polymerization times + ineffective polymerization times).

[0095] The current maximum number of aggregations is calculated based on the delay impact factor and aggregation efficiency. The calculation formula is as follows:

[0096]

[0097] Among them, C max-cur represents the current maximum number of aggregations, E is the aggregation efficiency, C min and C max They refer to the lower limit and upper limit of the range of polymerization times respectively, f refers to the delay impact factor, a and b are set coefficients, and in one embodiment, a and b can be set to 30% and 70 respectively.

[0098] In addition, in some embodiments, the present application can also adaptively adjust the current maximum number of aggregations based on the current aggregation efficiency. For example, if the current aggregation efficiency has reached 90%, it means that almost every aggregation can retrieve messages from the queue, and the maximum number of aggregations can be further increased. If the current aggregation efficiency can only reach 10%, it means that most aggregations will not retrieve messages from the queue and these aggregations are useless, and the maximum number of aggregations can be reduced.

[0099] The beneficial effects of this embodiment are: the present application can preliminarily determine the current maximum aggregation count range based on the current message sending rate and CPU utilization, and determine the current maximum aggregation count based on the current aggregation efficiency and message timeout period. This enables the system to dynamically adjust the message aggregation strategy based on actual operating conditions, thereby improving resource utilization efficiency. Furthermore, the present application sets different aggregation count ranges based on the priority of each message queue. The higher the message queue priority, the smaller the corresponding aggregation count range is configured, thereby reducing the calculated maximum aggregation count, thereby reducing the queue time in the message queue and sending messages in a timely manner.

[0100] In one embodiment, the message processing method may further include: when the cache space is full, sending the message in the cache space; when the current aggregation number is greater than or equal to the current maximum aggregation number, sending the message in the cache space, including:

[0101] When the cache space is not full and the current aggregation count is greater than or equal to the current maximum aggregation count, the messages in the cache space are sent.

[0102] The cache is a specific storage area used to temporarily store messages, and it has a certain capacity limit. When the number of messages stored in the cache, or the storage space occupied by the messages, reaches the maximum capacity set for the cache, the cache is considered full. At this point, no new messages can be stored in the cache, and the messages in the cache must be sent to make room for subsequent messages.

[0103] Specifically, the present application can continuously monitor the status of the cache space (whether it is full) and the relationship between the current aggregation number and the current maximum aggregation number. When the cache space is full, the operation of sending the messages in the cache space is triggered.

[0104] When the cache space is not full, the current aggregation count and the current maximum aggregation count are further checked. When the current aggregation count is greater than or equal to the current maximum aggregation count, the operation of sending the messages in the cache space is triggered.

[0105] The beneficial effects of this embodiment are: reasonable use of cache space to avoid waste of cache space, and at the same time, by controlling the number of aggregations and sending messages when the number of aggregations reaches the maximum, it can ensure that messages will not be excessively backlogged, thereby ensuring the timeliness of data processing within a certain period of time.

[0106] In one embodiment, the message processing method may further include: when there is no cache space in the message queue, obtaining new cache space from the cache resource pool, the cache resource pool is provided with a cache linked list, the new cache space is obtained from the head of the cache linked list, and the messages in the message queue are aggregated according to the new cache space.

[0107] The cache resource pool is a collection of maintained resources consisting of multiple cache spaces. This resource pool is used to manage and provide storage space.

[0108] A cache linked list connects multiple cache spaces in a specific order, forming a linked list structure. Each cache space in this linked list is a node, and nodes are connected to each other through pointers and other methods. This facilitates the management and operation of cache spaces, such as accessing a cache space from the head of the linked list (because the head of the linked list is the first accessible resource).

[0109] Specifically, the present application can create and maintain a cache resource pool, organize multiple cache spaces, and assemble them into a linked list, and the head and tail of the linked list are recorded and managed accordingly.

[0110] When there is no buffer space in the message queue, new buffer space is obtained from the head of the buffer list in the buffer resource pool. Using the new buffer space, the messages in the message queue are aggregated and related messages are grouped together according to certain rules.

[0111] This embodiment has the beneficial effect of managing cache space through a large cache resource pool and linked lists, achieving efficient cache space management. When a message queue needs cache space, it can quickly obtain it from the head of the linked list, ensuring timely message processing and avoiding processing delays caused by waiting for cache space.

[0112] In one embodiment, the message processing method may further include: when the new buffer space is not full and the current aggregation number is less than the current maximum aggregation number, saving the new buffer space into the message queue.

[0113] Among them, the present application can continuously monitor the filling status of the newly acquired cache space to check whether the cache space has been filled with messages; at the same time, record and check the current number of aggregation operations in real time, and compare it with the pre-set current maximum number of aggregation operations.

[0114] When it is found that the new cache space has not been filled and the current aggregation number is less than the current maximum aggregation number, it indicates that the new cache space still needs to continue message aggregation operations. At this time, the new cache space is saved to the message queue so that messages can be added to it and aggregated later.

[0115] The beneficial effect of this embodiment is that it ensures that messages are continuously aggregated when the cache space is capable and the aggregation operation has not reached the limit, so that the message aggregation is more sufficient, which helps to reduce the number of messages sent and improve data transmission efficiency.

[0116] In one embodiment, after sending the message in the cache space, the message processing method may further include: releasing the cache space and adding it to the end of the cache linked list.

[0117] In this application, when the conditions for sending messages in the cache space are met (for example, the cache space is full or the current aggregation times reaches the current maximum aggregation times, etc.), all messages in the cache space are sent and delivered to the target location through the network or other transmission methods.

[0118] Furthermore, after the message is successfully sent, the cache space is released, that is, the cache space is released from occupation so that it can be reused, for example, by clearing the remaining data in the cache space and restoring the cache space to its initial state.

[0119] Furthermore, the released cache space is added to the tail of the cache linked list. Specifically, the tail node of the cache linked list is found, and the node corresponding to the cache space is connected to the tail of the linked list by modifying the pointer of the linked list node. This allows the cache space to be used again in the order it was obtained from the head of the linked list when new cache space is needed.

[0120] The beneficial effect of this embodiment is that: when obtaining cache space, the present application always starts from the head, releases the cache space after use and adds the cache space to the tail of the cache linked list, thereby realizing the recycling and orderly management of cache space resources, avoiding the resource consumption caused by repeatedly creating new cache space, and enabling the system to process messages more efficiently, especially in scenarios with high concurrency or large-scale message processing, and being able to better respond to resource requirements and ensure stable operation of the system.

[0121] To further explain the message processing method of this application, please refer to Figure 3 ,like Figure 3As shown, the message processing method of the present application may include: the business module sends a message, the message is added to the message queue according to the priority, the message queue is notified to process the message, and it is determined whether there is cache space in the message queue. When there is cache space in the message queue, the messages in the message queue are traversed to determine whether the cache space is full. If the cache space is full, the aggregated messages in the cache space are sent. If the cache space is not full, it is further determined whether the current aggregation number is greater than or equal to the current maximum aggregation number. When the current aggregation number is greater than or equal to the current maximum aggregation number, the aggregated messages in the cache space are sent. If there is no cache space in the message queue, new cache space is applied for from the cache resource pool, and the new cache space is used to aggregate messages.

[0122] In a second aspect, the present application provides a message processing device, such as Figure 4 As shown, the message processing device includes: a determination module 41, an aggregation module 42, an acquisition module 43 and a sending module 44, wherein:

[0123] a determination module 41 for determining, in response to a request for processing a message in a message queue, whether a pre-configured buffer space exists in the message queue;

[0124] Aggregation module 42, used to aggregate messages in the message queue through the cache space and obtain the current aggregation count;

[0125] An acquisition module 43 is configured to acquire a current message sending rate and / or a current CPU utilization rate, and determine a current maximum number of aggregation times based on the current message sending rate and / or the current CPU utilization rate;

[0126] The sending module 44 is configured to send the aggregated messages in the cache space when the current aggregation times is greater than or equal to the current maximum aggregation times.

[0127] In one embodiment, the aggregation module 42 may traverse each message in sequence starting from the head of the message linked list in the message queue, and copy each message to the cache space in sequence for aggregation, and determine the current aggregation number according to the number of times the scan is started.

[0128] In one embodiment, the acquisition module 43 can record the number of invalid aggregations when no message is taken out from the message list, and obtain the range of aggregation times pre-configured for the message queue when the current message sending rate and / or the current CPU utilization is greater than the preset value. The range of aggregation times is configured according to the priority of the message queue, wherein the higher the priority of the message queue, the smaller the range of aggregation times corresponding to the message queue. The current aggregation efficiency is determined based on the current aggregation times and the invalid aggregation times, the enqueuing time of the message in the message queue is obtained, and the timeout time of the message is determined based on the enqueuing time, and the current maximum aggregation times is determined based on the current aggregation efficiency, the range of aggregation times and the timeout time of the message.

[0129] In one embodiment, the sending module 44 may send the messages in the cache space when the cache space is full, and send the messages in the cache space when the cache space is not full and the current aggregation number is greater than or equal to the current maximum aggregation number.

[0130] In one embodiment, the sending module 44 can also obtain new cache space from the cache resource pool when there is no cache space in the message queue. The cache resource pool is provided with a cache linked list. The new cache space is obtained from the head of the cache linked list, and the messages in the message queue are aggregated according to the new cache space.

[0131] In one embodiment, the sending module 44 may further save the new buffer space into the message queue when the new buffer space is not full and the current aggregation number is less than the current maximum aggregation number.

[0132] In one embodiment, after sending the message in the cache space, the sending module 44 may further release the cache space and add it to the end of the cache linked list.

[0133] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the message processing method provided in any one of the embodiments of the present application in the first aspect are implemented.

[0134] In one embodiment, the internal structure diagram of the computer device can be as follows: Figure 5As shown. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements the message processing method.

[0135] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the message processing method provided in any embodiment of the first aspect of the present application.

[0136] The computer readable storage medium may be Figure 5 A computer-readable storage medium in the computer device shown.

[0137] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0138] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A message processing method, characterized in that: The method comprises: In response to a request to process a message in a message queue, determining that a pre-configured buffer space exists in the message queue; aggregating the messages in the message queue using the buffer space, and obtaining a current aggregation count; Obtaining a current message sending rate and / or a current CPU utilization, and determining a current maximum number of aggregations based on the current message sending rate and / or the current CPU utilization; When the current aggregation number is greater than or equal to the current maximum aggregation number, the aggregated messages in the cache space are sent.

2. The method according to claim 1, characterized in that The aggregating the messages in the message queue according to the cache space and obtaining the current aggregation count includes: Starting from the head of the message linked list in the message queue, traversing each message in sequence, and copying each message in sequence to the cache space for aggregation; The current aggregation number is determined according to the number of times the scan is started.

3. The method according to claim 2, characterized in that The method further comprises: When no message is taken out from the message chain list, the number of invalid aggregations is recorded; The determining the current maximum number of aggregation times according to the current message sending rate and / or the current CPU utilization includes: when the current message sending rate and / or the current CPU utilization is greater than a preset value, obtaining a range of aggregation times pre-configured for the message queue, wherein the range of aggregation times is configured according to the priority of the message queue, wherein a higher priority message queue corresponds to a smaller range of aggregation times; Determine a current polymerization efficiency according to the current polymerization number and the invalid polymerization number; Obtain the time when the message in the message queue is enqueued, and determine the timeout of the message according to the time when the message is enqueued; determine the current maximum number of aggregations according to the current aggregation efficiency, the range of the number of aggregations, and the timeout of the message.

4. The method according to claim 1, wherein The method further comprises: When the cache space is full, sending the messages in the cache space; When the current aggregation number is greater than or equal to the current maximum aggregation number, sending the message in the cache space includes: When the cache space is not full and the current aggregation number is greater than or equal to the current maximum aggregation number, the messages in the cache space are sent.

5. The method according to claim 1, wherein The method further comprises: When the cache space does not exist in the message queue, new cache space is obtained from a cache resource pool, wherein a cache linked list is provided in the cache resource pool, and the new cache space is obtained from the head of the cache linked list; Messages in the message queue are aggregated according to the new cache space.

6. The method according to claim 5, characterized in that The method further comprises: When the new cache space is not full and the current aggregation number is less than the current maximum aggregation number, the new cache space is saved in the message queue.

7. The method according to claim 5, characterized in that After sending the message in the cache space, the method further includes: The cache space is released and added to the tail of the cache linked list.

8. A message processing device, characterized in that: The device comprises: a determining module, configured to determine, in response to a request for processing a message in a message queue, whether a preconfigured buffer space exists in the message queue; an aggregation module, configured to aggregate messages in the message queue through the cache space and obtain a current aggregation count; an acquisition module, configured to acquire a current message sending rate and / or a current CPU utilization rate, and determine a current maximum number of aggregation times based on the current message sending rate and / or the current CPU utilization rate; The sending module is configured to send the aggregated messages in the cache space when the current aggregation times is greater than or equal to the current maximum aggregation times.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images