Multi-queue management method and device for multi-process communication, medium and program product

Abstract

The invention provides a multi-queue management method and device for multi-process communication. The method comprises the steps of obtaining a preset token simultaneous processing number and a preset single data reading number; polling and identifying the data reading token in the message queue; reading the data to be processed in the data queue and processing the data to be processed based on the data read tokens identified by polling, the token simultaneous processing quantity and the single data reading quantity; updating the data information in the data reading token; and when it is detected that the number of times that the data information in the data reading token is empty is a preset number of times, the data reading token is removed. Visibly, the method can perform effective data management on the plurality of message queues based on a priority mode, and avoids polling without data queues, so that the loss of CPU resources can be saved, and the overall performance is improved.

Description

Technical Field

[0001] The present application relates to the field of communication data processing, and in particular to a multi-queue management method for multi-process communication, an electronic device, a readable storage medium, and a computer program product. Background Art

[0002] In network communication and network security systems, it is usually necessary for multiple processes to collaboratively process protocols and data. For example, the data communication engine, application protocol processing engine, and virus engine collaborate to complete network data communication and content virus inspection. In the collaborative process, inter-process communication technologies are required, including shared memory, message queues, Sockets, etc.

[0003] In order to achieve high-speed data exchange between processes, the usual practice is to establish a lock-free circular queue (referred to as a communication queue) and a shared memory pool on the shared memory. The two processes store data in the shared memory and transmit the shared memory address to each other through the queue. However, in practice, it is found that when the number of queues is large, a large number of empty queues will be polled, resulting in high CPU resource consumption, which in turn affects performance. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a multi-queue management method and device for multi-process communication, which can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0005] The first aspect of the present application provides a multi-queue management method for multi-process communication, comprising:

[0006] Obtaining a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to the plurality of priorities one by one;

[0007] Polling identifies the data read token in the message queue;

[0008] Based on the data read token identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it;

[0009] Updating the data information in the data reading token;

[0010] When it is detected that the data information in the data read token is empty for a preset number of times, the data read token is removed.

[0011] In the above implementation process, the method can avoid the loss of empty queue polling based on priority queue and token transmission message. That is, the sender adds the queue information with data to the message queue, and the receiver quickly obtains the queue information with data according to the information and reads the queue data; at the same time, the token is removed after the queue data is read, avoiding the loss caused by the previous empty queue polling.

[0012] Furthermore, before obtaining the preset number of tokens processed simultaneously and the preset number of single data reads, the method further includes:

[0013] Initialize multiple message queues corresponding to multiple priorities;

[0014] A data queue application token is received, and a plurality of data queues corresponding to the plurality of priorities are created; wherein each priority corresponds to at least one data queue, and the plurality of data queues are mutually exclusive.

[0015] In the above implementation process, the method can build a data-exclusive data queue based on queue application, thereby ensuring the correlation between data and data queue on the basis, and can avoid the appearance of data queue with no data to a large extent, thereby avoiding polling of empty queue.

[0016] Furthermore, before polling to identify the data read token in the message queue, the method further includes:

[0017] Receiving a data read token and data to be written; the data read token at least includes queue information, priority information and data information corresponding to the data to be written;

[0018] The data read token is placed in a message queue corresponding to the priority information, and the data to be written is written into a data queue corresponding to the queue information.

[0019] In the above implementation process, the method can ensure that the message queue accurately processes the data through the relationship between the data read token and the data to be written. At the same time, when the data read token is updated to no data information, the corresponding data queue can be deleted in time, thereby avoiding the generation of empty queues and avoiding the impact of empty queues on polling.

[0020] Further, based on the data read token identified by polling, the number of tokens processed simultaneously and the number of single data reads, reading the data to be processed in the data queue and processing it includes:

[0021] In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined;

[0022] Determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding one-to-one to the plurality of target tokens;

[0023] Based on the single data read quantity, the data to be processed are read from the multiple target data queues and processed.

[0024] In the above implementation process, the method can determine the location and quantity of the data to be processed based on the priority of the message queue, which is conducive to the accurate processing of the data, and further ensures the working state of processing data when there is data and not processing data when there is no data, avoiding invalid processing processes and avoiding polling of empty queues.

[0025] Further, when reading the data to be processed in the data queue and processing it, the method includes:

[0026] Determining whether the processing speed of the data to be processed reaches a preset processing speed threshold;

[0027] When the processing speed of the data to be processed reaches the preset processing speed threshold, data processing is suspended and the remaining time of the current speed calculation cycle is recorded;

[0028] Acquire unprocessed data from the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel;

[0029] Calculate the timeout period based on the remaining time and a preset percentage threshold;

[0030] When the timeout period is reached, the unprocessed data in the time wheel is placed into the message queue of the previous priority for processing.

[0031] In the above implementation process, the method can adjust the state of the data queue according to the flow threshold, and add excess data to the time wheel. After the timeout, the priority is increased to reprocess the excess data, thereby achieving the effect of regulating the flow of the data queue.

[0032] Furthermore, when reading the data to be processed in the data queue and processing it, the method further includes:

[0033] Within a preset time interval, sampling a first data queue that has not been read and / or a second data queue whose data storage amount exceeds a preset storage threshold;

[0034] The priority of the first data queue and / or the second data queue is temporarily increased by one level, and a temporary token generated based on the data read token is placed in a message queue of the previous priority level.

[0035] Furthermore, when reading the data to be processed in the data queue and processing it, the method further includes:

[0036] A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval;

[0037] The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in a message queue of the next priority level.

[0038] In the above implementation process, the method can dynamically adjust the priority of the data queue according to the sampling period and the specified sampling conditions, thereby ensuring the probability that the data in the data queue can be processed, thereby ensuring the efficiency and stability of data processing.

[0039] A second aspect of the present application provides a multi-queue management device for multi-process communication, the multi-queue management device for multi-process communication comprising:

[0040] An acquisition unit, used for acquiring a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to the plurality of priorities one by one;

[0041] A polling unit, used for polling a data reading token in an identification message queue;

[0042] A processing unit, configured to read and process the data to be processed in the data queue based on the data reading token identified by polling, the number of tokens processed simultaneously and the number of single data reads;

[0043] An updating unit, used for updating the data information in the data reading token;

[0044] The removing unit is used to remove the data reading token when it is detected that the number of times that the data information in the data reading token is empty reaches a preset number.

[0045] Furthermore, the multi-queue management device for multi-process communication also includes:

[0046] An initialization unit, used for initializing a plurality of message queues corresponding to a plurality of priorities one by one before the acquisition unit acquires a preset number of tokens to be processed simultaneously and a preset number of data to be read at a single time;

[0047] The creation unit is used to receive a data queue application token and create a plurality of data queues corresponding to the plurality of priorities; wherein each priority corresponds to at least one data queue, and the plurality of data queues are mutually exclusive.

[0048] Furthermore, the multi-queue management device for multi-process communication also includes:

[0049] A receiving unit, used for receiving a data read token and data to be written before the polling unit polls and identifies the data read token in the message queue; the data read token includes at least queue information, priority information and data information corresponding to the data to be written;

[0050] The first placing unit is used to place the data read token into a message queue corresponding to the priority information, and write the to-be-written data into a data queue corresponding to the queue information.

[0051] Furthermore, the processing unit includes:

[0052] A determination subunit, configured to determine, in a single message queue, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously based on the data read tokens identified by polling;

[0053] The determining subunit is further configured to determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding one-to-one to the plurality of target tokens;

[0054] The processing subunit is used to read and process the data to be processed in the multiple target data queues based on the single data reading quantity.

[0055] Furthermore, the multi-queue management device for multi-process communication also includes:

[0056] A judging unit, configured to judge whether a processing speed of the data to be processed reaches a preset processing speed threshold when the data to be processed in the data queue is read and processed;

[0057] A pause unit, used to pause data processing when the processing speed of the data to be processed reaches the preset processing speed threshold, and record the remaining time of the current speed calculation cycle;

[0058] A transfer unit, used for acquiring unprocessed data in the data to be processed, and moving the unprocessed data from the corresponding message queue to the time wheel;

[0059] A calculation unit, configured to calculate a timeout period based on the remaining time and a preset percentage threshold;

[0060] The second placing unit is used to place the unprocessed data in the time wheel into the message queue of the previous priority for processing when the timeout time is reached.

[0061] Furthermore, the multi-queue management device for multi-process communication also includes:

[0062] A sampling unit, configured to sample a first data queue that has not been read and / or a second data queue whose data storage volume exceeds a preset storage threshold within a preset time interval when the data to be processed in the read data queue is processed;

[0063] The adjustment unit is used to temporarily increase the priority of the first data queue and / or the second data queue by one level, and place the temporary token generated based on the data read token into the message queue of the previous priority level.

[0064] Furthermore, the multi-queue management device for multi-process communication also includes:

[0065] A sampling unit, configured to sample a third data queue whose data storage volume is smaller than the single data reading volume within a preset time interval when the to-be-processed data in the read data queue is processed;

[0066] The adjustment unit is used to temporarily reduce the priority of the third data queue by one level, and place the temporary token generated based on the data read token into a message queue of the next priority level.

[0067] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication described in any one of the first aspect of the present application.

[0068] The fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication described in any one of the first aspect of the present application is executed.

[0069] A fifth aspect of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it executes the multi-queue management method for multi-process communication described in any one of the first aspect of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 A flowchart of a multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0072] Figure 2 A flowchart of another multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of an example flow chart of a multi-queue management method for multi-process communication provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the structure of a multi-queue management device for multi-process communication provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the structure of another multi-queue management device for multi-process communication provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0077] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0078] Example 1

[0079] Please see Figure 1 , Figure 1 A flowchart of a multi-queue management method for multi-process communication provided in this embodiment. The multi-queue management method for multi-process communication includes:

[0080] S101. Obtain a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to a plurality of priorities one by one.

[0081] S102: Poll and identify the data reading token in the message queue.

[0082] S103 . Based on the data reading tokens identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it.

[0083] As an optional implementation, when reading the data to be processed in the data queue and processing it, the method includes:

[0084] Determine whether the processing speed of the data to be processed reaches a preset processing speed threshold;

[0085] When the processing speed of the data to be processed reaches the preset processing speed threshold, the data processing is suspended and the remaining time of the current speed calculation cycle is recorded;

[0086] Obtain unprocessed data from the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel;

[0087] Calculate the timeout period based on the remaining time and the preset percentage threshold;

[0088] When the timeout period is reached, the unprocessed data in the time wheel is placed in the message queue of the previous priority for processing.

[0089] In this embodiment, when reading the data to be processed in the data queue, the method can dynamically adjust the reading speed of the queue within the speed range corresponding to the threshold x (assuming that the above unit time is 1 second, then the speed range is less than 1 second x data) according to the flow threshold x (i.e., the speed of reading the data in the data queue) within the unit time of each data flow (for example, 1 second or 100 milliseconds).

[0090] In this embodiment, when the data read from the data queue within a unit time is greater than or equal to the threshold value x, the processing of the data is suspended, and the unprocessed data is moved from the message queue to the time wheel; at the same time, the remaining time length within the unit time is calculated, and the remaining time length is multiplied by the percentage y (for example, 80%) to obtain the timeout time. When the above timeout time is reached, the data in the time wheel is added to the message queue with a higher priority for processing.

[0091] In this embodiment, the method can perform a detection once per unit time and continuously adjust the priority of data processing until the priority is the highest.

[0092] As an optional implementation, when reading and processing the data to be processed in the data queue, the method further includes:

[0093] Within a preset time interval, sampling a first data queue that has not been read and / or a second data queue whose data storage amount exceeds a preset storage threshold;

[0094] The priority of the first data queue and / or the second data queue is temporarily increased by one level, and a temporary token generated based on the data read token is placed in a message queue of the previous priority level.

[0095] As an optional implementation, when reading and processing the data to be processed in the data queue, the method further includes:

[0096] A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval;

[0097] The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in the message queue of the next priority level.

[0098] In this embodiment, the method proposes a method for periodically sampling based on a preset time interval t and dynamically adjusting the priority of the data queue based on the sampling result, wherein the sampling object does not include the data in the time wheel.

[0099] In this embodiment, the method can set various sampling conditions according to business characteristics. Specifically, the objects sampled by the method are the reading status of the data queue and the amount of data in the data queue.

[0100] In this embodiment, the method can temporarily upgrade the priority of a data queue by one level when the data queue has not been read within time t or the amount of data packets stored in the data queue exceeds the storage threshold v. If the data queue is already at the highest level, the upgrade process will be terminated.

[0101] In this embodiment, if the amount of data in the high priority data queue is less than p, the amount of data in the medium priority data queue is less than q, or the amount of data in the low priority data queue is less than r, its priority will be temporarily downgraded by one level. If the level is the lowest level, the downgrading process will be terminated.

[0102] In this embodiment, when the data queue is upgraded or downgraded, the method can generate a temporary token (temporary priority information is written in the temporary token) and place it in the message queue of the corresponding priority. The data queue with temporarily adjusted priority still processes the data according to the single data reading quantity corresponding to the original priority.

[0103] In this embodiment, when there are two tokens corresponding to the same data queue, the token read first shall prevail, and the data in the data queue shall be processed according to the corresponding priority, and the token read later shall not be processed; or the temporary token shall prevail, and the data in the data queue shall be processed according to the temporary priority, and the original token shall not be processed when it is read.

[0104] In this embodiment, the priority of the data queue can be increased or decreased by only one level.

[0105] In this embodiment, during the process of sampling and adjusting the priority within a preset time interval, the method can perform priority restoration processing (i.e., cancel the temporary priority) on the data queue that no longer meets the upgrade condition or downgrade condition, and return the corresponding token.

[0106] S104: Update the data information in the data reading token.

[0107] S105: When it is detected that the data information in the data read token is empty for a preset number of times, remove the data read token.

[0108] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0109] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0110] It can be seen that the multi-queue management method for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0111] Example 2

[0112] Please see Figure 2 , Figure 2 A flowchart of a multi-queue management method for multi-process communication provided in this embodiment. The multi-queue management method for multi-process communication includes:

[0113] S201, initializing a plurality of message queues corresponding to a plurality of priorities one by one.

[0114] S202, receiving a data queue application token, and creating multiple data queues corresponding to multiple priorities; wherein each priority corresponds to at least one data queue, and the multiple data queues are mutually exclusive.

[0115] In this embodiment, when transmitting business data and message data between processes, a data transmission queue needs to be established for each business data stream (such as a network connection), and the queue is called a data queue. The data queue can be set to high, medium, and low levels, respectively, for transmitting business data of different levels.

[0116] In this embodiment, the method also establishes three corresponding levels of queues to transmit high, medium and low level messages, which are called message queues. Among them, the message queue is implemented by a lock-free circular queue and supports two-way communication; the queue is assigned an ID, and the hash table with the ID as the key is combined with a linked list method to manage the queue. The data structure of the queue is extended to store information such as status, quantity, life cycle counter, timeout threshold, etc.

[0117] In this embodiment, the method of applying for and using the two queues, the data queue and the message queue, may include:

[0118] (1) One process applies for the application, another process receives the application, and then uses it;

[0119] (2) Store the hash table in shared memory and set up a mutual exclusion mechanism process, and then use it based on this mechanism.

[0120] In this embodiment, the method is based on a dual process as an example to illustrate the data transmission process between the processes. Specifically, the example is run in a CS model (client-server model), the C end (i.e., the client) applies for a data queue, the S end (i.e., the server) accepts the application, and establishes a large number of high, medium, and low priority data queues.

[0121] In this embodiment, the method of sending data from the C end to the S end and the method of sending data from the S end to the C end are the same, and only the variables corresponding to the S end and the C end are set or based on different variables. Among them, the variables can be the preset number of tokens processed simultaneously and the preset number of single data reads.

[0122] S203, obtaining a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to a plurality of priorities one by one.

[0123] In this embodiment, the number of tokens processed simultaneously refers to the number of tokens read by the S or C end each time the message queue is polled. Specifically, the S or C end can read m, n, or o tokens and obtain corresponding data for processing each time the S or C end polls three high, medium, and low priority message queues.

[0124] In this embodiment, the token is used to record the exchange messages and processing results between the C end and the S end. Specifically, the messages that the token can record usually include data queue application, queue destruction application, data reading application, etc. Among them, the message queue can transmit messages with the token as a carrier, and the token can be pre-stored in the token queue as a resource.

[0125] In this embodiment, the single data read quantity refers to: when polling the three priority message queues at the S end and determining the data queue of the data to be read based on the information of the data read token, the number of data that can be read based on high, medium, and low priorities. Specifically, p data packets can be read for the high priority data queue, q data packets can be read for the medium priority data queue, and r data packets can be read for the low priority data queue.

[0126] S204, receiving a data read token and data to be written; the data read token at least includes queue information, priority information and data information corresponding to the data to be written.

[0127] S205 , placing the data read token into the message queue corresponding to the priority information, and writing the data to be written into the data queue corresponding to the queue information.

[0128] In this embodiment, after the C end writes a large amount of data to multiple data queues of the S end at the same time, the C end can obtain a token from the token queue and write the data situation into the data read token, so that it records the amount of data written and the location of the data written, and always holds this token. After that, the method can check the message queue to determine whether there is a token in the message queue. If not, the token is written into the message queue of the corresponding priority; if it exists, it is updated synchronously in real time.

[0129] S206: Poll and identify the data reading token in the message queue.

[0130] S207. In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined.

[0131] S208. Determine multiple target data queues based on multiple queue information of the data to be processed corresponding one-to-one to the multiple target tokens.

[0132] S209: Based on the single data read quantity, read the data to be processed in multiple target data queues and process them.

[0133] S210: Update the data information in the data reading token.

[0134] In this embodiment, the method can update the data read token after processing the data, and synchronously update the data read token of the C end; or, the C end rewrites a new data read token to the message queue of the corresponding priority to update the data read token.

[0135] In this embodiment, after completing the update of the data read token, the method repeats the above steps to continue reading the data until the corresponding data queue is empty.

[0136] S211 , when it is detected that the data information in the data read token is empty for a preset number of times, remove the data read token.

[0137] In this embodiment, when the corresponding data queue is empty, the method still attempts to read data from the data queue based on the data read token, and removes the data read token from the message queue when no result contrary to "data queue is empty" occurs after retrying w times.

[0138] Please see Figure 3 , Figure 3The following is a flow chart of an example of a multi-queue management method for multi-process communication. This method is applicable to the 1-to-1 or 1-to-many scenarios of the CS model, but this example takes the 1-to-1 scenario as an example to specifically describe the process from the C end to the S end (the process from the S end to the C end is similar, only the variable names involved are different). The process of this example is as follows:

[0139] (1) The S end initializes three message queues with different priorities: high, medium, and low (with IDs of 0, 1, and 2, respectively);

[0140] The C end initiates a request to create a data queue to the S end;

[0141] The S end receives the data queue creation application and creates the data queue (the application can be multiple rounds);

[0142] At this time, there is a high priority message queue with ID 0, a medium priority message queue with ID 1, a low priority message queue with ID 2, high priority data queues with IDs 3-10 (the number of queues is 8), medium priority data queues with IDs 11-34 (the number of queues is 24), and low priority data queues with IDs 35-66 (the number of queues is 32).

[0143] (2) Parameter settings:

[0144] The number of tokens that are processed simultaneously in the high-priority message queue is m=3;

[0145] The number of tokens that are processed simultaneously in the medium priority message queue is n=2;

[0146] The number of tokens processed simultaneously by the low-priority message queue is o=1;

[0147] The number of single data reads of the high priority data queue p = 10;

[0148] The number of single data reads of the medium priority data queue is q=5;

[0149] The number of single data reads of the low priority data queue r = 2;

[0150] The number of times the data information is empty required to remove the data read token is w=3;

[0151] The preset storage threshold v=900 (accounting for 90% of the total queue data, which can be adjusted according to the number of queue data);

[0152] The preset time interval of sampling is t=100ms;

[0153] The preset flow threshold when the unit time is 1 second (the value range is 100 milliseconds to 1 second, the smaller the value, the higher the accuracy) is x;

[0154] Preset percentage threshold y=80% (the value range is 1% to 100%, the smaller the value, the higher the accuracy);

[0155] u=5.

[0156] (3) The C end writes 100 data packets to queues 3 to 5 and queues 11 to 13 respectively, and generates corresponding tokens; at this time, it checks whether the tokens are in the corresponding message queues (message queue 0 and message queue 1). If not, it generates the same token in the message queue and performs synchronous update.

[0157] (4) The S end continuously reads m tokens from message queue 0, reads p data packets from queues 3 to 5 in succession, and then updates the m tokens (the update is to increase the counts read by the S end on queues 3 to 5 respectively). The token held by the C end is also updated synchronously (it is also possible for the C end to resend a new token to message queue 0 after the update).

[0158] (5) The S end continuously reads n tokens from message queue 1, reads q data packets from queues 11 to 12 for processing, and then updates the n tokens (the update is to increase the counts read by the S end on queues 11 to 12 respectively), and the token held by the C end is synchronously updated (it can also be that the C end resends a new token to message queue 1 after the update).

[0159] (6) Loop: The S end loops through the processes (4) and (5); after queue 11 or queue 12 is processed, the data in queue 13 is automatically looped through.

[0160] (7) When the data in queues 3 to 5 and queues 11 to 13 are all empty and the data queue is determined to be empty after retrying w times, an indication of processing completion (or no data to be processed in the message queue) is written into the token and the token is removed from the message queue.

[0161] The relevant process to the above example process is as follows:

[0162] (8) In period t, assuming that the amount of data packets stored in queue 6 is ≤ p, it is temporarily downgraded, the corresponding temporary token is obtained, a temporary message is generated in the token, and then it is added to queue 1.

[0163] (9) When the S end reads the token and temporary token of queue 6 in queue 0 and queue 1, the token read first (i.e. the first one) is used as the main token, p data packets are read on queue 6 and the token is processed, and the second token read is not processed.

[0164] (10) Within the t period, the data queue is upgraded when it meets the upgrade conditions. The process is similar to (8) and (9), which increases the probability of the data queue's token and message being processed.

[0165] (11) If the conditions for upgrading or downgrading are not met, the priority will be restored and the corresponding token will be returned.

[0166] (12) When the S end processes a data packet, if the flow control conditions are met and flow control is implemented, the token and message will be removed from the message queue as required and added to the time wheel. After the timeout occurs, they will be added to the high-priority message queue.

[0167] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0168] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0169] It can be seen that the multi-queue management method for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0170] Example 3

[0171] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a multi-queue management device for multi-process communication provided by this embodiment. Figure 4 As shown, the multi-queue management device for multi-process communication includes:

[0172] The acquisition unit 310 is used to acquire a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to a plurality of priorities one by one;

[0173] A polling unit 320, configured to poll and identify a data read token in a message queue;

[0174] The processing unit 330 is used to read and process the data to be processed in the data queue based on the data reading token, the number of tokens processed simultaneously and the number of single data reads identified by polling;

[0175] An updating unit 340, used to update the data information in the data reading token;

[0176] The removing unit 350 is configured to remove the data read token when it is detected that the number of times that the data information in the data read token is empty reaches a preset number.

[0177] In this embodiment, the explanation of the multi-queue management device for multi-process communication can refer to the description in Embodiment 1 or Embodiment 2, which will not be further elaborated in this embodiment.

[0178] It can be seen that the multi-queue management device for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0179] Example 4

[0180] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a multi-queue management device for multi-process communication provided by this embodiment. Figure 5 As shown, the multi-queue management device for multi-process communication includes:

[0181] The acquisition unit 310 is used to acquire a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to a plurality of priorities one by one;

[0182] A polling unit 320, configured to poll and identify a data read token in a message queue;

[0183] The processing unit 330 is used to read and process the data to be processed in the data queue based on the data reading token, the number of tokens processed simultaneously and the number of single data reads identified by polling;

[0184] An updating unit 340, used to update the data information in the data reading token;

[0185] The removing unit 350 is configured to remove the data read token when it is detected that the number of times that the data information in the data read token is empty reaches a preset number.

[0186] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0187] The initialization unit 360 is used to initialize a plurality of message queues corresponding to a plurality of priorities one by one before the acquisition unit 310 acquires a preset number of tokens to be processed simultaneously and a preset number of data to be read at a single time;

[0188] The creation unit 370 is used to receive a data queue application token and create multiple data queues corresponding to multiple priorities; wherein each priority corresponds to at least one data queue, and the multiple data queues are mutually exclusive.

[0189] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0190] The receiving unit 380 is used to receive the data read token and the data to be written before the polling unit 320 polls the data read token in the identification message queue; the data read token includes at least queue information, priority information and data information corresponding to the data to be written;

[0191] The first placing unit 390 is used to place the data read token into the message queue corresponding to the priority information, and write the data to be written into the data queue corresponding to the queue information.

[0192] As an optional implementation, the processing unit 330 includes:

[0193] A determination subunit 331 is used to determine, in a single message queue, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously based on the data read tokens identified by polling;

[0194] The determination subunit 331 is further used to determine multiple target data queues based on multiple queue information of the to-be-processed data corresponding one-to-one to the multiple target tokens;

[0195] The processing subunit 332 is used to read and process the data to be processed in multiple target data queues based on the single data reading quantity.

[0196] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0197] The judging unit 400 is used to judge whether the processing speed of the data to be processed reaches a preset processing speed threshold when reading and processing the data to be processed in the data queue;

[0198] The pause unit 410 is used to pause data processing and record the remaining time of the current speed calculation cycle when the processing speed of the data to be processed reaches a preset processing speed threshold;

[0199] The transfer unit 420 is used to obtain unprocessed data in the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel;

[0200] A calculation unit 430, configured to calculate a timeout period based on a remaining time and a preset percentage threshold;

[0201] The second placing unit 440 is used to place the unprocessed data in the time wheel into the message queue of the previous priority for processing when the timeout time is reached.

[0202] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0203] The sampling unit 450 is used to sample the first data queue that has not been read and / or the second data queue whose data storage amount exceeds a preset storage threshold within a preset time interval when reading and processing the data to be processed in the data queue;

[0204] The adjustment unit 460 is used to temporarily increase the priority of the first data queue and / or the second data queue by one level, and place the temporary token generated based on the data read token into the message queue of the previous priority level.

[0205] As an optional implementation, the multi-queue management device for multi-process communication further includes:

[0206] The sampling unit 450 is used to sample the third data queue whose data storage volume is less than the single data reading volume within a preset time interval when reading and processing the data to be processed in the data queue;

[0207] The adjustment unit 460 is used to temporarily reduce the priority of the third data queue by one level, and place the temporary token generated based on the data read token into a message queue of the next priority level.

[0208] In this embodiment, the explanation of the multi-queue management device for multi-process communication can refer to the description in Embodiment 1 or Embodiment 2, which will not be further elaborated in this embodiment.

[0209] It can be seen that the multi-queue management device for multi-process communication described in this embodiment can effectively manage data of multiple message queues based on priority, avoid polling of queues with no data, thereby saving CPU resource loss and improving overall performance.

[0210] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication in embodiment 1 or embodiment 2 of the present application.

[0211] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication in Embodiment 1 or Embodiment 2 of the present application is executed.

[0212] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0213] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0214] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0216] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0217] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A multi-queue management method for multi-process communication, characterized in that: include: Obtaining a preset number of tokens to be processed simultaneously and a preset number of single data reads; wherein the number of tokens to be processed simultaneously corresponds to a plurality of preset priorities one by one, and the number of single data reads corresponds to the plurality of priorities one by one; Polling identifies the data read token in the message queue; Based on the data read token identified by polling, the number of tokens processed simultaneously and the number of single data reads, read the data to be processed in the data queue and process it; Updating the data information in the data reading token; When it is detected that the data information in the data read token is empty for a preset number of times, the data read token is removed.

2. The multi-queue management method for multi-process communication according to claim 1, characterized in that: Before obtaining the preset number of tokens processed simultaneously and the preset number of single data reads, the method further includes: Initialize multiple message queues corresponding to multiple priorities one by one; A data queue application token is received, and a plurality of data queues corresponding to the plurality of priorities are created; wherein each priority corresponds to at least one data queue, and the plurality of data queues are mutually exclusive.

3. The multi-queue management method for multi-process communication according to claim 1, characterized in that: Before the polling identification message queue data read token, the method further includes: Receiving a data read token and data to be written; the data read token at least includes queue information, priority information and data information corresponding to the data to be written; The data read token is placed in a message queue corresponding to the priority information, and the data to be written is written into a data queue corresponding to the queue information.

4. The multi-queue management method for multi-process communication according to claim 1, characterized in that: The method of reading and processing the data to be processed in the data queue based on the data read token identified by polling, the number of tokens processed simultaneously, and the number of single data reads, includes: In a single message queue, based on the data read token identified by polling, a plurality of target tokens corresponding to the number of tokens to be processed simultaneously are determined; Determine a plurality of target data queues based on a plurality of queue information of the to-be-processed data corresponding one-to-one to the plurality of target tokens; Based on the single data read quantity, the data to be processed are read from the multiple target data queues and processed.

5. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method includes: Determining whether the processing speed of the data to be processed reaches a preset processing speed threshold; When the processing speed of the data to be processed reaches the preset processing speed threshold, data processing is suspended and the remaining time of the current speed calculation cycle is recorded; Acquire unprocessed data from the data to be processed, and move the unprocessed data from the corresponding message queue to the time wheel; Calculate the timeout period based on the remaining time and a preset percentage threshold; When the timeout period is reached, the unprocessed data in the time wheel is placed into the message queue of the previous priority for processing.

6. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method further includes: Within a preset time interval, sampling a first data queue that has not been read and / or a second data queue whose data storage amount exceeds a preset storage threshold; The priority of the first data queue and / or the second data queue is temporarily increased by one level, and a temporary token generated based on the data read token is placed in a message queue of the previous priority level.

7. The multi-queue management method for multi-process communication according to claim 1, characterized in that: When reading the data to be processed in the data queue and processing it, the method further includes: A third data queue in which the amount of sampled data stored is less than the amount of single data reading within a preset time interval; The priority of the third data queue is temporarily reduced by one level, and the temporary token generated based on the data read token is placed in a message queue of the next priority level.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multi-queue management method for multi-process communication according to any one of claims 1 to 7.

9. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the multi-queue management method for multi-process communication according to any one of claims 1 to 7 is executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the multi-queue management method for multi-process communication according to any one of claims 1 to 7 is executed.

Images