Network data processing optimization method based on multiple queues and multiple threads
By adopting multi-queue and multi-threading methods in network data processing and processing according to data priority, the problem that traditional methods cannot flexibly adjust processing priority is solved, and the efficiency and effectiveness of network data processing are improved.
Patent Information
- Application Number
- CN202411758127.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-25
AI Technical Summary
Traditional network data processing methods cannot flexibly adjust processing priorities based on data characteristics, resulting in untimely processing of important or emergency data, affecting system performance and user experience.
Using a network data processing optimization method based on multi-queue and multi-threads, high, medium and low priority queues are created, corresponding queues are inserted according to the priority of the data, and parallel processing is carried out through multiple threads.
It realizes dynamic adjustment of priority according to data characteristics and timely processing of network data, improving the timeliness and effectiveness of network data processing.
Smart Images
Figure CN119690702A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a network data processing optimization method based on multiple queues and multiple threads. Background Art
[0002] With the continuous advancement of network communication technology, various network applications and services are emerging in an endless stream. The scale of network data processing is constantly expanding and the complexity is also increasing. How to process network data efficiently and accurately has become an urgent problem to be solved.
[0003] Traditional network data processing methods generally use a single queue and single thread mechanism, that is, a single thread processes the data in serial order according to the order in which the data arrives. However, the importance and urgency of different data vary. Traditional methods cannot flexibly adjust the priority of data processing according to the characteristics of the data, and the data processing capacity of a single thread is also weak. When a large amount of data pours in, it cannot guarantee that important or urgent data will be processed in a timely and effective manner, which has a significant impact on system performance and user experience. Summary of the invention
[0004] The purpose of the present invention is to provide a network data processing optimization method based on multiple queues and multiple threads, aiming to solve the problem that traditional network data processing methods cannot flexibly adjust the priority of data processing according to data characteristics.
[0005] To achieve the above object, the present invention provides a network data processing optimization method based on multiple queues and multiple threads, comprising the following steps:
[0006] Create three queue data structures of high, medium and low priority;
[0007] Receive network data in steps;
[0008] Insert the network data into the tail of the corresponding receive buffer queue according to the priority of the network data;
[0009] Data is taken out from the corresponding receive buffer queue in the order of high, medium, and low priority, and processed in parallel through multiple threads.
[0010] Among them, in “inserting the network data into the tail of the corresponding receiving buffer queue according to the priority of the network data”, it includes:
[0011] Taking the instruction type contained in the network data as the characteristic data, first read the predefined instruction type and priority mapping relationship configuration table;
[0012] Perform table lookup operations based on instruction types to determine the processing priority of network data;
[0013] Insert high priority network data into the tail of the high priority receive buffer queue, insert medium priority network data into the tail of the medium priority receive buffer queue, and insert low priority network data into the tail of the low priority receive buffer queue.
[0014] Among them, in "performing a table lookup operation according to the instruction type to determine the processing priority of the network data", the instruction types set in the configuration table are divided into three categories: alarm, setting and query, and correspond to high, medium and low priorities respectively.
[0015] Among them, "taking data from the corresponding receive buffer queue in the order of high, medium and low priority, and processing in parallel through multiple threads" includes:
[0016] Determine whether there is data in the high priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data. If not, proceed to the next step.
[0017] Determine whether there is data in the medium priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data. If not, proceed to the next step.
[0018] Determine whether there is data in the low-priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data in sequence.
[0019] The network data processing thread class is instantiated into one or more thread objects according to actual needs, and multiple thread objects can process network data in parallel.
[0020] The present invention provides a network data processing optimization method based on multiple queues and multiple threads, comprising the following steps: creating three queue data structures of high, medium and low priority; receiving network data in steps; inserting the network data into the tail of the corresponding receiving buffer queue according to the priority of the network data; taking out the data from the corresponding receiving buffer queue in the order of high, medium and low priority, and processing it in parallel through multiple threads. The present invention combines the three strategies of multi-queue management, priority adjustment and multi-thread processing. By introducing corresponding algorithms and mechanisms, it is possible to dynamically adjust the priority according to the characteristics of the data and process it in time, thereby optimizing the network data processing method and improving the timeliness and effectiveness of the network data processing. It solves the problem that the traditional network data processing method cannot flexibly adjust the priority of data processing according to the characteristics of the data. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0022] Figure 1 It is a schematic diagram of the network data receiving and processing architecture.
[0023] Figure 2 It is a flow chart of network data reception and processing.
[0024] Figure 3 It is a flow chart of a network data processing optimization method based on multiple queues and multiple threads provided by the present invention.
[0025] Figure 4 It is a flowchart for inserting network data into the tail of the corresponding receive buffer queue according to the priority of the network data.
[0026] Figure 5 It is a flowchart of taking data out from the corresponding receive buffer queue in the order of high, medium and low priority, and processing it in parallel through multiple threads. DETAILED DESCRIPTION
[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0028] See also Figures 1 to 5 The present invention provides a network data processing optimization method based on multiple queues and multiple threads, comprising the following steps:
[0029] S1 creates three queue data structures with high, medium and low priority;
[0030] Specifically, three queue objects can be declared by using the built-in FIFO (first-in-first-out) queue container class of the programming language, such as Figure 1 , used as high, medium, and low priority network data receiving buffers respectively. If the programming language does not have a built-in queue container class, you can use containers such as arrays or linked lists to achieve equivalent implementations.
[0031] S2 receives network data in steps;
[0032] Specifically, receive a network data packet length of four bytes. If the length is less than four bytes, repeatedly try to read the missing bytes until the four bytes of data are completely read; convert the four bytes of data into integer data in big-endian mode, and then try to receive the corresponding number of bytes according to the length value. If the length is less than expected, repeatedly try to read the missing bytes until the data packet is completely read.
[0033] S3 inserts the network data into the tail of the corresponding receiving buffer queue according to the priority of the network data;
[0034] S31 uses the instruction type contained in the network data as characteristic data and first reads a predefined instruction type and priority mapping relationship configuration table;
[0035] Specifically, taking the instruction type contained in the network data as the characteristic data, first read the predefined instruction type and priority mapping relationship configuration table. The instruction types set in the configuration table are divided into three categories: alarm, setting and query, and correspond to high, medium and low priorities respectively.
[0036] S32 performs a table lookup operation according to the instruction type to determine the processing priority of the network data;
[0037] Specifically, a table lookup operation is performed according to the instruction type to determine the processing priority of the network data.
[0038] S33 inserts the high priority network data into the tail of the high priority receiving buffer queue, the medium priority network data into the tail of the medium priority receiving buffer queue, and the low priority network data into the tail of the low priority receiving buffer queue.
[0039] Specifically, high priority network data is inserted into the tail of the high priority receiving buffer queue, medium priority network data is inserted into the tail of the medium priority receiving buffer queue, and low priority network data is inserted into the tail of the low priority receiving buffer queue.
[0040] S4 takes data from the corresponding receive buffer queue in the order of high, medium, and low priority, and processes it in parallel through multiple threads.
[0041] Define a network data processing thread class, and implement steps S41 to S43 in the thread body.
[0042] S41 determines whether there is data in the high priority receive buffer queue, if so, locks the queue, takes out the data, unlocks the queue and processes the data in sequence, if not, executes the next step;
[0043] Specifically, determine whether there is data in the high priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data in sequence. If not, execute the next step.
[0044] S42 determines whether there is data in the medium priority receiving buffer queue, if so, locks the queue, takes out the data, unlocks the queue and processes the data in sequence, if not, executes the next step;
[0045] Specifically, it is determined whether there is data in the medium priority receiving buffer queue. If so, the queue is locked, data is retrieved, the queue is unlocked, and data processing operations are performed in sequence. If not, the next step is executed.
[0046] S43 determines whether there is data in the low priority receive buffer queue, and if so, sequentially performs operations of locking the queue, retrieving the data, unlocking the queue, and processing the data;
[0047] Specifically, it is determined whether there is data in the low priority receiving buffer queue, and if so, the queue is locked, data is retrieved, the queue is unlocked, and data processing operations are performed in sequence.
[0048] S44 instantiates the network data processing thread class into one or more thread objects according to actual needs, and the multiple thread objects can process the network data in parallel.
[0049] Specifically, the network data processing thread class is instantiated into one or more thread objects according to actual needs, and the multiple thread objects can process the network data in parallel.
[0050] Beneficial effects:
[0051] The present invention combines three strategies of multi-queue management, priority adjustment and multi-threaded processing. By introducing corresponding algorithms and mechanisms, it is possible to dynamically adjust the priority according to data characteristics and process it in a timely manner, thereby optimizing the network data processing method and further improving the timeliness and effectiveness of network data processing.
[0052] What is disclosed above is only a preferred embodiment of the network data processing optimization method based on multiple queues and multiple threads of the present invention. Of course, it cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.
Claims
1. A network data processing optimization method based on multiple queues and multiple threads, characterized in that: The following steps are involved: Create three queue data structures of high, medium and low priority; Receive network data in steps; Insert the network data into the tail of the corresponding receive buffer queue according to the priority of the network data; Data is taken out from the corresponding receive buffer queue in the order of high, medium, and low priority, and processed in parallel through multiple threads.
2. The network data processing optimization method based on multiple queues and multiple threads as claimed in claim 1, characterized in that: In "Inserting network data into the tail of the corresponding receive buffer queue according to the priority of the network data", it includes: Taking the instruction type contained in the network data as the characteristic data, first read the predefined instruction type and priority mapping relationship configuration table; Perform table lookup operations based on instruction types to determine the processing priority of network data; Insert high priority network data into the tail of the high priority receive buffer queue, insert medium priority network data into the tail of the medium priority receive buffer queue, and insert low priority network data into the tail of the low priority receive buffer queue.
3. The network data processing optimization method based on multiple queues and multiple threads as claimed in claim 2, characterized in that: In "performing a table lookup operation according to the instruction type to determine the processing priority of network data", the instruction types set in the configuration table are divided into three categories: alarm, setting and query, and correspond to high, medium and low priorities respectively.
4. The network data processing optimization method based on multiple queues and multiple threads as claimed in claim 1, characterized in that: In "taking data from the corresponding receive buffer queue in the order of high, medium and low priority, and processing it in parallel through multiple threads", it includes: Determine whether there is data in the high priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data. If not, proceed to the next step. Determine whether there is data in the medium priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data. If not, proceed to the next step. Determine whether there is data in the low-priority receive buffer queue. If so, lock the queue, retrieve the data, unlock the queue, and process the data in sequence. The network data processing thread class is instantiated into one or more thread objects according to actual needs, and multiple thread objects can process network data in parallel.