Multi-queue management method, device, medium and program product for multi-process communication

Abstract

The application provides a multi-queue management method and device for multi-process communication, which comprises the following steps: acquiring a preset token simultaneous processing quantity and a preset single data reading quantity; polling and identifying data reading tokens in a message queue; based on the polling and identified data reading tokens, the token simultaneous processing quantity and the single data reading quantity, reading and processing the to-be-processed data in a data queue; updating the data information in the data reading tokens; and removing the data reading tokens when the number of times that the data information in the data reading tokens is detected as empty is a preset number of times. It can be seen that the method can effectively manage multiple message queues in a priority manner, avoid polling of a no-data queue, save the loss of CPU resources, and improve overall performance.

Description

Technical Field

[0001] The present application relates to the field of communication data processing, and in particular to a multi-queue management method for multi-process communication, an electronic device, a readable storage medium, and a computer program product. Background Art

[0002] In network communications and network security systems, multiple processes often need to collaborate to process protocols and data. For example, a data communication engine, an application protocol processing engine, and a virus engine collaborate to complete network data communication and content virus detection. This collaboration requires the use of inter-process communication technologies, including shared memory, message queues, and sockets.

[0003] To achieve high-speed data exchange between processes, a common approach is to establish a lock-free circular queue (referred to as a communication queue) and a shared memory pool in shared memory. The two processes store data in the shared memory and transmit the shared memory address to each other through the queue. However, in practice, it has been found that when the number of queues is large, a large number of empty queues will be polled, resulting in high CPU resource consumption and poor performance. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a multi-queue management method and device for multi-process communication, which can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0005] The first aspect of the present application provides a multi-queue management method for multi-process communication, comprising:

[0006] Obtaining a preset number of tokens that can be processed simultaneously and a preset number of single data reads; wherein the number of tokens that can be processed simultaneously corresponds to a plurality of preset priorities, and the number of single data reads corresponds to the plurality of priorities;

[0007] Polling to identify the data read token in the message queue;

[0008] Based on the data reading token identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it;

[0009] Updating the data information in the data reading token;

[0010] When it is detected that the data information in the data read token is empty for a predetermined number of times, the data read token is removed.

[0011] In the above implementation, this method can avoid the overhead of polling empty queues by using priority queues and token transmission. Specifically, the sender adds queue information containing data to the message queue, and the receiver quickly retrieves the queue information based on the message and reads the queue data. Furthermore, after the queue data is read, the token is removed, avoiding the overhead of polling empty queues.

[0012] Furthermore, before obtaining the preset number of tokens processed simultaneously and the preset number of single data reads, the method further includes:

[0013] Initialize multiple message queues corresponding to multiple priorities;

[0014] Receive a data queue application token and create multiple data queues corresponding to the multiple priorities; wherein each priority corresponds to at least one data queue, and the multiple data queues are mutually exclusive.

[0015] In the above implementation process, the method can build a data-exclusive data queue based on queue application, thereby ensuring the correlation between data and data queue on the basis, and can avoid the emergence of data queues with no data to a large extent, thereby avoiding polling of empty queues.

[0016] Furthermore, before polling and identifying the data read token in the message queue, the method further includes:

[0017] receiving a data read token and data to be written; wherein the data read token includes at least queue information, priority information and data information corresponding to the data to be written;

[0018] The data read token is placed in a message queue corresponding to the priority information, and the data to be written is written into a data queue corresponding to the queue information.

[0019] In the above implementation process, this method can ensure that the message queue accurately processes the data through the relationship between the data reading token and the data to be written. At the same time, when the data reading token is updated to no data information, the corresponding data queue can be promptly eliminated, thereby avoiding the generation of empty queues and avoiding the impact of empty queues on polling.

[0020] Furthermore, the step of reading and processing the data to be processed in the data queue based on the data read token identified by polling, the number of simultaneous processing of the token, and the number of single data reads includes:

[0021] In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined;

[0022] determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding to the plurality of target tokens;

[0023] read and process the to-be-processed data in the plurality of target data queues based on the single data reading quantity.

[0024] In the implementation process, the method can determine the position and quantity of the to-be-processed data based on the priority of the message queue, thereby facilitating accurate processing of the data, ensuring that the data is processed when there is data and no processing when there is no data, avoiding invalid processing flow, and avoiding polling of empty queues.

[0025] Further, when reading and processing the to-be-processed data in the data queue, the method comprises:

[0026] determining whether the processing speed of the to-be-processed data reaches a preset processing speed threshold;

[0027] when the processing speed of the to-be-processed data reaches the preset processing speed threshold, suspending data processing and recording the remaining duration of the current speed calculation period;

[0028] obtaining unprocessed data in the to-be-processed data and moving the unprocessed data from the corresponding message queue to a time wheel;

[0029] calculating an overtime based on the remaining duration and a preset percentage threshold;

[0030] when the overtime is reached, placing the unprocessed data in the time wheel into a message queue of a previous priority for processing.

[0031] In the implementation process, the method can adjust the state of the data queue according to the flow threshold, add excess data to the time wheel, and reprocess the excess data after the timeout by increasing the priority, thereby achieving the effect of regulating the flow of the data queue.

[0032] Further, when reading and processing the to-be-processed data in the data queue, the method further comprises:

[0033] sampling a first data queue that has not been read and / or a second data queue whose data storage exceeds a preset storage threshold within a preset time interval;

[0034] temporarily upgrading the priority of the first data queue and / or the second data queue by one level, and placing a temporary token generated based on the data reading token into a message queue of a previous priority.

[0035] Further, when reading and processing the to-be-processed data in the data queue, the method further comprises:

[0036] A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval;

[0037] The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in a message queue of the next higher priority level.

[0038] In the above implementation process, the method can dynamically adjust the priority of the data queue according to the sampling period and the specified sampling conditions, thereby ensuring the probability that the data in the data queue can be processed, thereby ensuring the efficiency and stability of data processing.

[0039] A second aspect of the present application provides a multi-queue management device for multi-process communication, the multi-queue management device for multi-process communication comprising:

[0040] An acquisition unit, configured to acquire a preset number of tokens that can be processed simultaneously and a preset number of single data reads; wherein the number of tokens that can be processed simultaneously corresponds to a plurality of preset priorities, and the number of single data reads corresponds to the plurality of priorities;

[0041] A polling unit, used for polling and identifying data reading tokens in a message queue;

[0042] a processing unit, configured to read and process the data to be processed in the data queue based on the data reading token identified by polling, the number of simultaneous processing of the token, and the number of single data reads;

[0043] An updating unit, configured to update the data information in the data reading token;

[0044] The removing unit is configured to remove the data reading token when it is detected that the number of times that the data information in the data reading token is empty is a preset number.

[0045] Furthermore, the multi-queue management device for multi-process communication further includes:

[0046] an initialization unit, configured to initialize a plurality of message queues corresponding to a plurality of priorities one by one before the acquisition unit acquires a preset number of tokens to be processed simultaneously and a preset number of data to be read at a single time;

[0047] The creation unit is configured to receive a data queue application token and create a plurality of data queues corresponding to the plurality of priorities; wherein each priority corresponds to at least one data queue, and the plurality of data queues are mutually exclusive.

[0048] Furthermore, the multi-queue management device for multi-process communication further includes:

[0049] a receiving unit, configured to receive a data read token and data to be written before the polling unit polls and identifies the data read token in the message queue; the data read token includes at least queue information, priority information, and data information corresponding to the data to be written;

[0050] The first placing unit is configured to place the data read token into a message queue corresponding to the priority information, and write the data to be written into a data queue corresponding to the queue information.

[0051] Furthermore, the processing unit includes:

[0052] a determination subunit, configured to determine, in a single message queue, a plurality of target tokens corresponding to a number of tokens to be processed simultaneously based on the data read tokens identified by polling;

[0053] The determining subunit is further configured to determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding one-to-one to the plurality of target tokens;

[0054] The processing subunit is configured to read and process the data to be processed in the plurality of target data queues based on the single data reading quantity.

[0055] Furthermore, the multi-queue management device for multi-process communication further includes:

[0056] a judging unit, configured to judge whether a processing speed of the data to be processed reaches a preset processing speed threshold when the data to be processed in the data queue is read and processed;

[0057] a pausing unit, configured to suspend data processing when the processing speed of the data to be processed reaches the preset processing speed threshold, and record the remaining time of the current speed calculation cycle;

[0058] A transfer unit, configured to obtain unprocessed data from the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel;

[0059] a calculation unit, configured to calculate a timeout period based on the remaining time and a preset percentage threshold;

[0060] The second placing unit is configured to place the unprocessed data in the time wheel into a message queue of a previous priority for processing when a timeout period is reached.

[0061] Furthermore, the multi-queue management device for multi-process communication further includes:

[0062] A sampling unit is configured to sample a first data queue that has not been read and / or a second data queue whose data storage volume exceeds a preset storage threshold within a preset time interval when the data to be processed in the data queue is read and processed;

[0063] The adjustment unit is configured to temporarily increase the priority of the first data queue and / or the second data queue by one level, and place a temporary token generated based on the data read token into a message queue of a previous priority level.

[0064] Furthermore, the multi-queue management device for multi-process communication further includes:

[0065] A sampling unit is configured to sample a third data queue whose data storage volume is smaller than the single data reading volume within a preset time interval when the data to be processed in the read data queue is processed;

[0066] The adjustment unit is configured to temporarily reduce the priority of the third data queue by one level, and place a temporary token generated based on the data read token into a message queue of a lower priority level.

[0067] A third aspect of the present application provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication described in any one of the first aspects of the present application.

[0068] In a fourth aspect, the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication described in any one of the first aspects of the present application is executed.

[0069] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it executes the multi-queue management method for multi-process communication described in any one of the first aspects of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 A flowchart of a multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0072] Figure 2 A flowchart of another multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram illustrating an example flow of a multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the structure of a multi-queue management device for multi-process communication provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the structure of another multi-queue management device for multi-process communication provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0077] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0078] Example 1

[0079] Please see Figure 1 , Figure 1 The following is a flow chart of a multi-queue management method for multi-process communication provided in this embodiment. The multi-queue management method for multi-process communication includes:

[0080] S101. Obtain a preset number of tokens that can be processed simultaneously and a preset number of single data reads; wherein the number of tokens that can be processed simultaneously corresponds to a plurality of preset priorities, and the number of single data reads corresponds to a plurality of priorities.

[0081] S102: Poll and identify the data read token in the message queue.

[0082] S103 : Based on the data reading token identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it.

[0083] As an optional implementation, when reading and processing the data to be processed in the data queue, the method includes:

[0084] Determine whether the processing speed of the data to be processed reaches a preset processing speed threshold;

[0085] When the processing speed of the data to be processed reaches the preset processing speed threshold, data processing is paused and the remaining time of the current speed calculation cycle is recorded;

[0086] Get the unprocessed data in the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel;

[0087] Calculate the timeout period based on the remaining time and the preset percentage threshold;

[0088] When the timeout period is reached, the unprocessed data in the time wheel is placed in the message queue of the previous priority for processing.

[0089] In this embodiment, when reading the data to be processed in the data queue, the method can dynamically adjust the reading speed of the queue within the speed range corresponding to the threshold x (assuming that the above unit time is 1 second, then the speed range is less than 1 second x data) according to the traffic threshold x (i.e., the speed of reading the data in the data queue) within the unit time of each data stream (for example, 1 second or 100 milliseconds).

[0090] In this embodiment, when the amount of data read from the data queue within a unit time is greater than or equal to a threshold value x, data processing is suspended and the unprocessed data is moved from the message queue to the time wheel. Simultaneously, the remaining time within the unit time is calculated and multiplied by a percentage y (e.g., 80%) to obtain a timeout period. When the timeout period is reached, the data in the time wheel is added to a message queue with a higher priority for processing.

[0091] In this embodiment, the method may perform a detection once per unit time and continuously adjust the priority of data processing until the priority is the highest.

[0092] As an optional implementation, when reading and processing the data to be processed in the data queue, the method further includes:

[0093] Within a preset time interval, sampling a first data queue that has not been read and / or a second data queue whose data storage amount exceeds a preset storage threshold;

[0094] The priority of the first data queue and / or the second data queue is temporarily increased by one level, and a temporary token generated based on the data read token is placed in a message queue of the previous priority level.

[0095] As an optional implementation, when reading and processing the data to be processed in the data queue, the method further includes:

[0096] A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval;

[0097] The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in the message queue of the next priority level.

[0098] In this embodiment, the method proposes a method for performing periodic sampling based on a preset time interval t and dynamically adjusting the priority of the data queue based on the sampling result. The sampling object does not include the data in the time wheel.

[0099] In this embodiment, the method can set various sampling conditions according to business characteristics. Specifically, the objects sampled by the method are the reading status of the data queue and the amount of data in the data queue.

[0100] In this embodiment, the method can temporarily upgrade the priority of a data queue by one level when the data queue has not been read within time t or the amount of data packets stored in the data queue exceeds the storage threshold v. If the data queue is already at the highest level, the upgrade process will be terminated.

[0101] In this embodiment, if the amount of data in the high-priority data queue is less than p, the amount of data in the medium-priority data queue is less than q, or the amount of data in the low-priority data queue is less than r, its priority will be temporarily downgraded by one level. If the level is the lowest level, the downgrading process will be terminated.

[0102] In this embodiment, when a data queue is upgraded or downgraded, the method can generate a temporary token (with temporary priority information written in the temporary token) and place it in the message queue of the corresponding priority. The data queue with temporarily adjusted priority still processes data according to the single data read quantity corresponding to the original priority.

[0103] In this embodiment, when there are two tokens corresponding to the same data queue, the token read first shall prevail, and the data in the data queue shall be processed according to the corresponding priority, and the token read later shall not be processed; or the temporary token shall prevail, and the data in the data queue shall be processed according to the temporary priority, and no further processing shall be performed when the original token is read.

[0104] In this embodiment, the priority of the data queue can be increased or decreased by only one level.

[0105] In this embodiment, during the process of sampling and adjusting priorities within a preset time interval, the method can perform priority restoration processing (i.e., cancel the temporary priority) on data queues that no longer meet the upgrade or downgrade conditions, and return the corresponding tokens.

[0106] S104: Update the data information in the data reading token.

[0107] S105 : When it is detected that the data information in the data read token is empty for a predetermined number of times, the data read token is removed.

[0108] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and this is not limited in this embodiment.

[0109] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0110] It can be seen that the multi-queue management method for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0111] Example 2

[0112] Please see Figure 2 , Figure 2 The following is a flow chart of a multi-queue management method for multi-process communication provided in this embodiment. The multi-queue management method for multi-process communication includes:

[0113] S201: Initialize multiple message queues corresponding to multiple priorities one by one.

[0114] S202: Receive a data queue application token and create multiple data queues corresponding to multiple priorities; wherein each priority corresponds to at least one data queue, and the multiple data queues are mutually exclusive.

[0115] In this embodiment, when transferring business data and message data between processes, a data transmission queue needs to be established for each business data flow (e.g., a network connection). This queue is called a data queue. The data queue can be set to high, medium, or low levels, respectively, for transmitting business data of different levels.

[0116] In this embodiment, the method also establishes three queues of corresponding levels to transmit high-, medium-, and low-level messages. These queues are referred to as message queues. The message queues are implemented as lock-free circular queues, supporting bidirectional communication. Each queue is assigned an ID, and a hash table using the ID as the key is used to manage the queue using a linked list approach. The queue's data structure stores information such as status, number of messages, lifecycle counter, and timeout threshold.

[0117] In this embodiment, the methods for applying for and using the data queue and the message queue may include:

[0118] (1) One process applies for something, another process receives the application, and then uses it;

[0119] (2) Store the hash table in shared memory and set up a mutual exclusion mechanism process, and then use it based on this mechanism.

[0120] In this embodiment, the method uses a dual-process model to illustrate the data transmission process between the processes. Specifically, this example operates in a CS model (client-server model). The C side (i.e., client) requests a data queue, and the S side (i.e., server) accepts the request and establishes a large number of high, medium, and low priority data queues.

[0121] In this embodiment, the method for sending data from the C end to the S end and the method for sending data from the S end to the C end are the same, and the only difference is the setting or the corresponding variables of the S end and the C end. Among them, the variables can be a preset number of tokens to be processed simultaneously and a preset number of single data reads.

[0122] S203, obtaining a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one-to-one, and the number of single data reads corresponds to a plurality of priorities one-to-one.

[0123] In this embodiment, the number of tokens processed simultaneously refers to the number of tokens read by the S-end or C-end each time it polls a message queue. Specifically, the S-end or C-end can read m, n, or o tokens and obtain the corresponding data for processing each time it polls the three high, medium, and low priority message queues.

[0124] In this embodiment, the token is used to record exchanged messages and processing results between the client and the server. Specifically, the messages recorded by the token typically include data queue requests, queue destruction requests, and data read requests. The message queue can transmit messages using the token as a carrier, and the token can be pre-stored in the token queue as a resource.

[0125] In this embodiment, the single data read quantity refers to the number of data packets that can be read based on high, medium, and low priority when polling the three priority message queues on the S side and determining the data queue to be read based on the information in the data read token. Specifically, for the high-priority data queue, p data packets can be read, for the medium-priority data queue, q data packets can be read, and for the low-priority data queue, r data packets can be read.

[0126] S204 , receiving a data read token and data to be written; the data read token at least includes queue information, priority information, and data information corresponding to the data to be written.

[0127] S205 : placing the data read token into the message queue corresponding to the priority information, and writing the data to be written into the data queue corresponding to the queue information.

[0128] In this embodiment, after client C writes a large amount of data simultaneously to multiple data queues on client S, client C can retrieve a token from the token queue and write the data details to a data read token, which records information such as the amount and location of the data written and retains the token. Subsequently, the method can check the message queue to determine whether the token exists in the message queue. If not, the token is written to the message queue of the corresponding priority. If so, the token is updated synchronously in real time.

[0129] S206: Poll and identify the data reading token in the message queue.

[0130] S207 . In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined.

[0131] S208 . Determine multiple target data queues based on multiple queue information of the to-be-processed data corresponding one-to-one to the multiple target tokens.

[0132] S209: Based on the single data reading quantity, read the data to be processed in multiple target data queues and process them.

[0133] S210: Update the data information in the data reading token.

[0134] In this embodiment, the method can update the data read token after processing the data, and synchronously update the data read token of the C end; or, the C end rewrites a new data read token to the message queue of the corresponding priority to update the data read token.

[0135] In this embodiment, after completing the update of the data read token, the method repeats the above steps to continue reading the data until the corresponding data queue is empty.

[0136] S211 : When it is detected that the data information in the data read token is empty for a preset number of times, remove the data read token.

[0137] In this embodiment, when the corresponding data queue is empty, the method still attempts to read data from the data queue based on the data read token, and after retrying w times, if no result contrary to "data queue is empty" occurs, the data read token is removed from the message queue.

[0138] Please see Figure 3 , Figure 3This example shows a multi-queue management method for multi-process communication. This method is applicable to 1:1 or 1:many scenarios in the CS model, but this example uses a 1:1 scenario as an example to specifically describe the process from the C-end to the S-end (the process from the S-end to the C-end is similar, with only the variable names being different). The example process is as follows:

[0139] (1) The S side initializes three message queues with different priorities: high, medium, and low (with IDs of 0, 1, and 2 respectively);

[0140] The C-side initiates a request to the S-side to create a data queue;

[0141] The S side receives the data queue creation application and creates the data queue (the application can be multi-round);

[0142] At this time, there are high-priority message queues with ID 0, medium-priority message queues with ID 1, low-priority message queues with ID 2, high-priority data queues with IDs 3-10 (number of queues is 8), medium-priority data queues with IDs 11-34 (number of queues is 24), and low-priority data queues with IDs 35-66 (number of queues is 32).

[0143] (2) Parameter settings:

[0144] The number of tokens processed simultaneously by the high-priority message queue is m=3;

[0145] The number of tokens processed simultaneously by the medium priority message queue is n = 2;

[0146] The number of tokens processed simultaneously by the low-priority message queue is o=1;

[0147] The number of single data reads in the high-priority data queue p = 10;

[0148] The number of single data reads in the medium priority data queue is q=5;

[0149] The number of single data reads of the low priority data queue r=2;

[0150] The number of times the data information is empty required to remove the data read token is w=3;

[0151] The preset storage threshold v=900 (accounting for 90% of the total queue data, which can be adjusted according to the number of queue data);

[0152] The preset sampling time interval t = 100ms;

[0153] The preset flow threshold when the unit time is 1 second (the value range is 100 milliseconds to 1 second, the smaller the value, the higher the accuracy) is x;

[0154] Preset percentage threshold y=80% (the value range is 1% to 100%, the smaller the value, the higher the accuracy);

[0155] u=5。

[0156] (3) The C end writes 100 data packets to queues 3 to 5 and queues 11 to 13 respectively, and generates corresponding tokens; at this time, it checks whether the tokens are in the corresponding message queues (message queue 0 and message queue 1). If not, it generates the same token in the message queue and performs synchronous updates.

[0157] (4) The S end continuously reads m tokens from message queue 0, reads p data packets from queues 3 to 5, processes them, and then updates the m tokens (the update is to increase the count of the S end reading on queues 3 to 5 respectively), and the token held by the C end is synchronously updated (it can also be that the C end resends a new token to message queue 0 after the update).

[0158] (5) The S end continuously reads n tokens from message queue 1, reads q data packets from queues 11 and 12, processes them, and then updates the n tokens (the update is to increase the count of the S end reading on queues 11 and 12 respectively), and the token held by the C end is synchronously updated (it can also be that the C end resends a new token to message queue 1 after the update).

[0159] (6) Loop: The S end loops through the processes (4) and (5); after processing of queue 11 or queue 12, the data of queue 13 is automatically looped through.

[0160] (7) When the data in queues 3 to 5 and queues 11 to 13 are all empty and the data queue is determined to be empty after retrying w times, write the flag of processing completion (or there is no data to be processed in the message queue) in the token and remove the token from the message queue.

[0161] The process related to the above example process is as follows:

[0162] (8) In period t, assuming that the number of packets stored in queue 6 is ≤ p, it is temporarily downgraded, the corresponding temporary token is obtained, a temporary message is generated in the token, and then it is added to queue 1.

[0163] (9) When the S end reads the token and temporary token of queue 6 in queue 0 and queue 1, the token read first (i.e. the first one) is the main one, p data packets are read on queue 6 and the token is processed, and the second token read is not processed.

[0164] (10) During the t period, the data queue is upgraded when it meets the upgrade conditions. The process is similar to (8) (9), which increases the probability of the data queue's token and message being processed.

[0165] (11) If the conditions for upgrading or downgrading are not met, the priority will be restored and the corresponding token will be returned.

[0166] (12) When the S side processes the data packet, if the flow control conditions are met and flow control is implemented, the token and message will be removed from the message queue as required and added to the time wheel. After the timeout occurs, it will be added to the high-priority message queue.

[0167] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and this is not limited in this embodiment.

[0168] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0169] It can be seen that the multi-queue management method for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0170] Example 3

[0171] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a multi-queue management device for multi-process communication provided by this embodiment. Figure 4 As shown, the multi-queue management device for multi-process communication includes:

[0172] An acquisition unit 310 is configured to acquire a preset number of tokens that can be processed simultaneously and a preset number of data reads at a time; wherein the number of tokens that can be processed simultaneously corresponds to a plurality of preset priorities, and the number of data reads at a time corresponds to a plurality of priorities;

[0173] A polling unit 320 is configured to poll and identify a data read token in a message queue;

[0174] The processing unit 330 is used to read and process the data to be processed in the data queue based on the data reading token, the number of tokens processed simultaneously, and the number of single data reads identified by polling;

[0175] An updating unit 340 is used to update the data information in the data reading token;

[0176] The removing unit 350 is configured to remove the data read token when it is detected that the data information in the data read token is empty for a preset number of times.

[0177] In this embodiment, the explanation of the multi-queue management device for multi-process communication can refer to the description in Embodiment 1 or Embodiment 2, and the same will not be repeated in this embodiment.

[0178] It can be seen that the multi-queue management device for multi-process communication described in this embodiment can effectively manage multiple message queues based on priority, avoid polling of no-data queues, save CPU resource consumption, and improve overall performance.

[0179] Embodiment 4

[0180] Please refer to Figure 5 , Figure 5 A structure diagram of a multi-queue management device for multi-process communication provided in this embodiment. As shown in Figure 5 , the multi-queue management device for multi-process communication comprises:

[0181] The acquisition unit 310 is configured to acquire a preset token simultaneous processing quantity and a preset single data reading quantity. The token simultaneous processing quantity is one-to-one corresponding to a plurality of priorities, and the single data reading quantity is one-to-one corresponding to the plurality of priorities.

[0182] The polling unit 320 is configured to poll and identify a data reading token in a message queue.

[0183] The processing unit 330 is configured to read and process the data to be processed in the data queue based on the polling-identified data reading token, the token simultaneous processing quantity, and the single data reading quantity.

[0184] The update unit 340 is configured to update the data information in the data reading token.

[0185] The removal unit 350 is configured to remove the data reading token when it is detected that the number of times of empty data information in the data reading token is a preset number of times.

[0186] As an optional implementation, the multi-queue management device for multi-process communication further comprises:

[0187] The initialization unit 360 is configured to initialize a plurality of message queues corresponding to a plurality of priorities before the acquisition unit 310 acquires the preset token simultaneous processing quantity and the preset single data reading quantity.

[0188] The creation unit 370 is configured to receive a data queue application token and create a plurality of data queues corresponding to the plurality of priorities. Each priority corresponds to at least one data queue, and the plurality of data queues are mutually exclusive.

[0189] As an optional implementation, the multi-queue management device for multi-process communication further comprises:

[0190] The receiving unit 380 is configured to receive the data read token and the to-be-written data before the polling unit 320 polls the data read token in the identification message queue; the data read token at least includes queue information, priority information and data information corresponding to the to-be-written data;

[0191] The first placing unit 390 is configured to place the data read token in the message queue corresponding to the priority information, and write the to-be-written data in the data queue corresponding to the queue information.

[0192] As an optional implementation, the processing unit 330 includes:

[0193] The determining sub-unit 331 is configured to determine, in the single message queue, a plurality of target tokens corresponding to the number of simultaneous processing based on the data read token identified by polling;

[0194] The determining sub-unit 331 is further configured to determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding to the plurality of target tokens one by one.

[0195] The processing sub-unit 332 is configured to read and process the to-be-processed data in the plurality of target data queues based on the number of single data reading.

[0196] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0197] The judging unit 400 is configured to judge whether the processing speed of the to-be-processed data reaches a preset processing speed threshold when the to-be-processed data in the data queue is read and processed;

[0198] The pausing unit 410 is configured to pause the data processing when the processing speed of the to-be-processed data reaches the preset processing speed threshold, and record the remaining duration of the current speed calculation period;

[0199] The transferring unit 420 is configured to obtain unprocessed data in the to-be-processed data, and move the unprocessed data from the corresponding message queue to the time wheel;

[0200] The calculating unit 430 is configured to calculate the timeout time based on the remaining duration and a preset percentage threshold;

[0201] The second placing unit 440 is configured to place the unprocessed data in the time wheel in the message queue of the previous priority for processing when the timeout time is reached.

[0202] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0203] The sampling unit 450 is configured to sample, within a preset time interval, a first data queue that has not been read and / or a second data queue whose data storage volume exceeds a preset storage threshold when reading and processing the data to be processed in the data queue;

[0204] The adjustment unit 460 is configured to temporarily increase the priority of the first data queue and / or the second data queue by one level, and place a temporary token generated based on the data read token into a message queue of the previous priority level.

[0205] As an optional implementation manner, the multi-queue management device for multi-process communication further includes:

[0206] The sampling unit 450 is configured to sample the third data queue whose data storage volume is smaller than the single data reading volume within a preset time interval when reading and processing the data to be processed in the data queue;

[0207] The adjusting unit 460 is configured to temporarily reduce the priority of the third data queue by one level, and place the temporary token generated based on the data read token into a message queue of a lower priority level.

[0208] In this embodiment, the explanation of the multi-queue management device for multi-process communication can refer to the description in Example 1 or Example 2, and will not be further elaborated in this embodiment.

[0209] It can be seen that the multi-queue management device for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0210] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication in embodiment 1 or embodiment 2 of the present application.

[0211] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication in embodiment 1 or embodiment 2 of the present application is executed.

[0212] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0213] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0214] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0216] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0217] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A multi-queue management method for multi-process communication, characterized in that: include: Obtaining a preset number of tokens that can be processed simultaneously and a preset number of single data reads; wherein the number of tokens that can be processed simultaneously corresponds to a plurality of preset priorities, and the number of single data reads corresponds to the plurality of priorities; Polling to identify the data read token in the message queue; Based on the data reading token identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it; Updating the data information in the data reading token; When it is detected that the data information in the data read token is empty for a predetermined number of times, the data read token is removed.

2. The multi-queue management method for multi-process communication according to claim 1, characterized in that: Before obtaining the preset token simultaneous processing quantity and the preset single data reading quantity, the method further includes: Initialize multiple message queues corresponding to multiple priorities; Receive a data queue application token and create multiple data queues corresponding to the multiple priorities; wherein each priority corresponds to at least one data queue, and the multiple data queues are mutually exclusive.

3. The multi-queue management method for multi-process communication according to claim 1, characterized in that: Before polling and identifying the data read token in the message queue, the method further includes: receiving a data read token and data to be written; wherein the data read token includes at least queue information, priority information and data information corresponding to the data to be written; The data read token is placed in a message queue corresponding to the priority information, and the data to be written is written into a data queue corresponding to the queue information.

4. The multi-queue management method for multi-process communication according to claim 1, characterized in that: The method of reading and processing the data to be processed in the data queue based on the data read token identified by polling, the number of simultaneous processing of the token, and the number of single data reads, includes: In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined; Determining a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding one-to-one to the plurality of target tokens; Based on the single data read quantity, the data to be processed are read from the multiple target data queues and processed.

5. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method includes: Determining whether the processing speed of the data to be processed reaches a preset processing speed threshold; When the processing speed of the data to be processed reaches the preset processing speed threshold, data processing is paused and the remaining time of the current speed calculation cycle is recorded; Acquire unprocessed data from the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel; Calculating a timeout period based on the remaining duration and a preset percentage threshold; When the timeout period is reached, the unprocessed data in the time wheel is placed into the message queue of the previous priority for processing.

6. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method further includes: Within a preset time interval, sampling a first data queue that has not been read and / or a second data queue whose data storage amount exceeds a preset storage threshold; The priority of the first data queue and / or the second data queue is temporarily increased by one level, and a temporary token generated based on the data read token is placed in a message queue of the previous priority level.

7. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method further includes: A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval; The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in a message queue of the next higher priority level.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication according to any one of claims 1 to 7.

9. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication according to any one of claims 1 to 7 is executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program executes the multi-queue management method for multi-process communication according to any one of claims 1 to 7.

Images