QUIC protocol safe and reliable transmission method based on adaptive congestion control
By introducing adaptive congestion control and Q-Learning deep learning load balancing algorithms into the QUIC protocol, the problem that the QUIC protocol cannot effectively handle network congestion in complex network environments is solved, and more secure and reliable data transmission and more balanced network load are achieved.
Patent Information
- Application Number
- CN202411683706.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-11
AI Technical Summary
The existing QUIC protocol has limitations in complex network environments and cannot effectively deal with network congestion and multiple stream interactions, resulting in data loss and network load imbalance.
Adaptive congestion control is adopted to enhance the adaptability of the QUIC protocol, select appropriate transmission paths, and reduce network congestion by adding node blocking parameters, network usage prediction and Q-Learning-based deep learning load balancing algorithm.
It realizes safer and more reliable data transmission in complex network environments, reduces network congestion, improves network load balancing, and ensures the secure transmission of key information.
Smart Images
Figure CN119603223A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a QUIC protocol secure and reliable transmission method based on adaptive congestion control, and belongs to the field of secure transmission. Background Art
[0002] In today's era of rapid development of digitalization and informatization, the security and reliability of data transmission have become the core issues of network communication. Especially for the transmission of critical information, such as financial transactions, medical data and personal privacy information, it is even more necessary to ensure that the data is not affected by packet loss, delay and security attacks during transmission. QUIC (Quick UDP Internet Connections), as a new transmission protocol, has attracted widespread attention due to its advantages of low latency and fast connection establishment. However, the existing QUIC protocol still has certain limitations when dealing with complex network environments. In recent years, machine learning algorithms have made significant progress in solving network problems. Deep learning algorithms based on Q-Learnig have been widely used in network congestion control. However, existing machine learning algorithms still have shortcomings. For example, when observing network conditions, current machine learning algorithms only observe broadband usage, which will result in some nodes with large available bandwidth, but due to congestion in the transmission nodes connected to them, the buffer is full and cannot receive data. Since the network cannot make a judgment on this situation, data loss occurs. The current congestion control algorithm based on machine learning also involves the problem of multiple flow interactions. If the flow is regarded as an intelligent control body, there are multiple intelligent bodies, and other flows are in an unknown state. The network status is partially observable, and it is impossible to accurately judge the network status. Summary of the invention
[0003] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and provide a QUIC protocol secure and reliable transmission method based on adaptive congestion control, which enhances the adaptability of the QUIC protocol in complex network environments and reduces network congestion by adding node blocking parameters, network usage prediction and a deep learning load balancing algorithm based on Q-Learing, thereby achieving secure and reliable transmission of key information.
[0004] The technical solution of the present invention is: a QUIC protocol secure and reliable transmission method based on adaptive congestion control, comprising:
[0005] Record the bandwidth and node buffer size of each transmission path in the network; set the data transmission path and delay from each sending node to each receiving node under the current network conditions in the central server;
[0006] When the sending node sends a data stream to the receiving node, the elephant flow identification algorithm is used to identify whether the sent data stream is an elephant flow:
[0007] If the data stream to be sent is not a large stream, it is directly sent to the receiving node to complete the data pre-sending;
[0008] If the data stream to be sent is an elephant stream, the corresponding information needs to be passed to the central server. The central server obtains the network status, calculates the bandwidth utilization and node blocking rate, and re-estimates the network status based on the records maintained by the central server. It inputs the information into the trained Q-Learning deep learning module to select a suitable transmission path. After the central server selects the transmission path, it calculates the time delay for the data to reach each node in the transmission path according to the selected transmission path and stores it, returns the record of the transmission path in the central server, updates the record of the network status, and completes the network transmission.
[0009] Preferably, when the transmitted data stream is an elephant stream, the corresponding information transmitted to the central server includes: data size and data transmission destination.
[0010] Preferably, the Q-Learning deep learning module includes a Q-Learning agent, an environment, an action, a state, a reward, and a reward function, wherein:
[0011] Q-learning agent: plays the role of learner and decision maker, responsible for selecting actions;
[0012] Environment: The network environment that the agent interacts with;
[0013] Status: Information obtained by the Q-Learning agent from the network, including link status, bandwidth usage, and node congestion;
[0014] Action: The algorithm needs to select a transmission path for information transmission from the sending node to the receiving node. The action of the algorithm is the node that sends the information and the path of information transmission;
[0015] Reward: The environment’s feedback on the action selected by the agent. After the action is selected, the network transmission environment will change to a certain extent.
[0016] Reward function: By obtaining the changes in network conditions before and after path selection, the Q-Learning agent is rewarded, so that the Q-Learning agent can adjust its action selection strategy and continuously learn and train the link allocation according to the network status to achieve continuous optimization of path allocation.
[0017] Preferably, in the process of training the Q-Learning deep learning module and using the Q-Learning deep learning module to select a path, the root mean square error of bandwidth utilization and node blocking rate is used to measure the current link resource utilization of the network, and it is used as a reward for the Q-Learning deep learning algorithm in the Q-Learning deep learning module to optimize path allocation in real time.
[0018] Preferably, the information included in the network status acquired and estimated by the central server is:
[0019] The used bandwidth of each path, the transmission bandwidth of each path, the node buffer occupancy size, and the node buffer size.
[0020] Preferably, the central server obtains the network transmission path bandwidth and the used bandwidth, the node buffer size and the node buffer used size, and calculates the bandwidth utilization and the node blocking rate in the following method:
[0021]
[0022] Preferably, the network status is adjusted according to the transmission network status estimated by the central server, specifically:
[0023] The central server estimates the transmission delay of data from the sender to each node on the selected transmission path; the central server maintains a timer for each path, the bandwidth occupied by the sending node's transmission information, the node ID, and an array of the delay from the sending node to each node. The value recorded in the array is: the total delay from the sending node to the destination minus the delay from the sending node to each node in the path; the initial value of the timer is: the total delay from the sending node to the destination;
[0024] When the sending node sends information, it sends a signal to the central server. The central server maintains multiple records. The timer of each record counts down according to the time, decreasing by 1 per second until the timer reaches -1, and the corresponding record is deleted.
[0025] Preferably, when the sending node sends a new request to transmit information to the central server, the central server first obtains the network bandwidth and node buffer usage; then the central server reads the records in order, and for each read record, the central server reads the timer size in the record, and compares the timer in the record with the corresponding delay array in the record to obtain the transmission status of the previously transmitted data; the central server updates the bandwidth corresponding to the path that has not been transmitted in each record to obtain a new bandwidth, and the calculation method is: path bandwidth = path bandwidth + record bandwidth;
[0026] Traverse all records to get the adjusted bandwidth, use the adjusted bandwidth as the input of the agent, and calculate the path.
[0027] Preferably, in the transmission of the elephant flow, the delay is the sum of the queuing delay and the propagation delay; wherein:
[0028] The queuing delay is caused by the buffer queuing of the selected node and is calculated based on the node buffer usage size and node processing speed;
[0029] The propagation delay is calculated from the data length and the transmission speed.
[0030] Compared with the prior art, the present invention has the following advantages:
[0031] (1) The present invention aims at the network congestion caused by node selection conflicts due to transmission delay in the network. The transmission delay is estimated by using a delay calculation formula, and the detected network information is updated in combination with clock information to obtain the network status when data is transmitted to all path nodes. The path for data transmission of the new transmission node is selected according to the new network status. This method fundamentally solves the problem of network congestion caused by excessive task load of some nodes due to repeated node selection caused by network transmission delay.
[0032] (2) The present invention combines the machine learning algorithm with the QUIC transmission protocol, uses the Q-Learning deep learning algorithm to select paths, uses bandwidth utilization and node blocking rate to evaluate network conditions, and uses the root mean square error of bandwidth utilization and node blocking rate to measure the current link resource utilization of the network, and uses it to measure the reward of the Q-Learning deep learning algorithm, so as to achieve continuous optimization of path allocation and ensure a more balanced network load;
[0033] (3) The present invention aims at the problem that the current congestion control algorithm based on machine learning can only partially observe the network status. This paper establishes a central server and proposes that when the data flow is a large flow, the data to be sent by the sender is simplified into information such as data volume and receiving node by using pre-sent data packets and sent to the central server. The central server will schedule them uniformly and combine timers and request numbers to prevent outdated replies. This can more accurately observe the entire network status and improve the accuracy of machine learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the method of the present invention;
[0035] Figure 2 Schematic diagram of the Q-Learing deep learning algorithm. DETAILED DESCRIPTION
[0036] In order to address the shortcomings of the prior art, the present invention proposes an improved QUIC protocol congestion control method, which enhances the adaptability of the QUIC protocol in complex network environments and reduces network congestion by adding node blocking parameters, network usage prediction and a deep learning load balancing algorithm based on Q-Learing, thereby achieving secure and reliable transmission of key information.
[0037] A QUIC protocol secure and reliable transmission method based on adaptive congestion control, the specific process is:
[0038] The first step is to describe the network status. Describe the network as an undirected weighted graph, and record the bandwidth of each transmission link and the size of the node buffer. The central server records the data transmission path and delay from each sending node to each receiving node under the current network conditions.
[0039] The second step is data pre-sending. When the sending node wants to send data to the receiving node, it uses the elephant flow identification module to identify whether it is an elephant flow. If it is an elephant flow, the corresponding information needs to be passed to the central server.
[0040] The third step is to obtain the network status. The central server obtains the network status, calculates the bandwidth utilization and node blocking rate, and estimates the network bandwidth usage during the data transmission based on the maintained transmission timer and node delay clock, updates the obtained network bandwidth, and forms a new network status.
[0041] The fourth step is to select the transmission path. The central server uses the estimated network status as input and selects the transmission path using the Q-Learning deep learning algorithm. The Q-Learning deep learning algorithm includes state space, action space and reward function. The reward function rewards the agent by obtaining the changes in network status before and after path selection, so that the agent can continuously learn and train the link allocation according to the network status, and realize the continuous optimization of path allocation.
[0042] 1. Further, the specific process of step 1 of the present invention is:
[0043] (1) First, the network is described as an undirected weighted graph, G = (V, E, U, B), where:
[0044] V={v1,v2,v3,…v n} represents the set of nodes n is the number of nodes;
[0045] The set of edges also represents the path in the network, e ij represents the path between node i and node j. If e ij =-1 means that there is no connection between node i and node j, 1≤e,j≤n;
[0046] is the collection of network bandwidth usage, u ij Indicates how much bandwidth is used between node i and node j. If u ij =0 means there is no communication between node i and point j, where 1≤i,j≤n.
[0047] B={b1,b2,b3,…b n Each component in} represents the blocking status of each node.
[0048] (2) The data transmission path and delay from each sending node to each receiving node under the current network conditions are pre-set in the central server.
[0049] 2. Further, the specific process of the second step of the present invention is:
[0050] (1) Before sending a data stream, the sending node uses the elephant flow identification algorithm to identify the data stream to be sent.
[0051] (2) If the data stream being sent is a large data stream, the data stream size and data transmission destination information are sent to the central server.
[0052] (3) If the data stream being sent is not a large data stream, it is sent directly to the receiving end.
[0053] 3. Further, the specific process of the third step of the present invention is:
[0054] (1) Before path selection, the central server obtains the network transmission link bandwidth and used bandwidth, node buffer size and node buffer used size, calculates bandwidth utilization and node blocking rate, and calculates the sum of their root mean square errors as an indicator for evaluating network conditions.
[0055] (2) The central server estimates the network bandwidth usage during the data transmission based on the transmission timer and node delay clock maintained, and updates the obtained network bandwidth to form a new network status. Based on the records maintained by the central server, the transmission network status is estimated and the network status is adjusted.
[0056] Read the records maintained by the central server in order, and read the nodes {N0, N1, N2, ..., N m} and read the timer t size. If t = -1, it means that it has been transmitted to all nodes, and delete the record; if t ≠ -1, compare t and {t0, t1, t2, ... t m} until t>t i Indicates that the data has not yet been transmitted to N i Node, that is, the data has not yet been transmitted to N i To Ni+1 And the paths after that. In order to avoid node selection blocking caused by transmission delay, the bandwidth of the paths that have not been transmitted in the current network situation is added to the corresponding bandwidth, and the network situation when the information is transmitted to all nodes is estimated until all records are traversed.
[0057] The central server estimates the transmission delay of data from the sender to each node on the selected transmission path. The central server maintains an array of timers, transmission bandwidth of the sending node, and delay from the sending node to each node. The value recorded in the array is: the total delay from the sending node to the destination minus the delay from the sending node to each node in the path. The value of the timer is: the total delay from the sending node to the destination.
[0058] In the transmission of elephant flows, the delay mainly considers queuing delay and propagation delay. Queuing delay is mainly caused by the buffer queue of the selected node, which can be calculated based on the node buffer usage size and node processing speed. Propagation delay can be calculated based on data length and transmission speed.
[0059] Delay = propagation delay + queuing delay
[0060] When the sending node sends information, it sends a signal to the central server, which maintains multiple records. The timer of each record counts down according to the time, decreasing by 1 per second. When the timer reaches -1, the corresponding record is deleted.
[0061] When the sending node sends a new request to transmit information to the central server, the central server first obtains the network bandwidth and node buffer usage. After that, the central server reads the records in order. For each read record, the central server performs the following operations: reads the timer size in the record, and compares the timer in the record with the corresponding delay array in the record to obtain the transmission status of the previously transmitted data, including: the nodes and paths transmitted, and the nodes and paths not transmitted. The central node updates the bandwidth corresponding to the paths not transmitted in each record to obtain a new bandwidth, calculated as: path bandwidth = path bandwidth + record bandwidth. Traverse all records to obtain the adjusted bandwidth. Use the adjusted bandwidth as the input of the intelligent agent to calculate the path.
[0062] 4. Further, the specific process of the fourth step of the present invention is:
[0063] (1) The central server uses the estimated network status as input and selects the transmission path using the Q-Learning deep learning algorithm. The Q-Learning deep learning algorithm includes state space, action space and reward function. The reward function rewards the Q-learning agent by obtaining the changes in network status before and after path selection, so that the agent can continuously learn and train the link allocation according to the network status, and realize the continuous optimization of path allocation.
[0064] The Q-Learning deep learning algorithm includes Q-Learning agent, environment, action, state, and reward:
[0065] Q-learning agent: plays the role of learner and decision maker, responsible for selecting actions.
[0066] Environment: The network environment that the agent interacts with.
[0067] Status: Information obtained by the Q-Learning agent from the network, including link status, bandwidth usage, and node congestion.
[0068] Action: The algorithm needs to select a transmission path for information transmission from a sending node to a receiving node. The action of the algorithm in the present invention is the node that sends the information and the path for information transmission.
[0069] Reward: The environment’s feedback on the action selected by the agent. After the action is selected, the network transmission environment will change to a certain extent.
[0070] In order to update the action selection strategy of the Q-Learning agent, a reward function is designed. The reward function rewards the Q-Learning agent by obtaining the changes in network conditions before and after path selection, so that the agent can adjust its action selection strategy and continuously learn and train the link allocation according to the network status to achieve continuous optimization of path allocation.
[0071] The training process of the Q-Learning deep learning algorithm is:
[0072] S1, initialize the mapping relationship Q between state s and action a, and set the initial value to 0 or a random decimal value;
[0073] S2, the Q-Learning agent determines its current state s;
[0074] S3, select an action a according to a certain strategy;
[0075] S4, execute action a and obtain reward r and next state: After the agent executes action a, the environment will give the agent a reward r and make the agent enter the next state;
[0076] S5. Use the Q-Learning update formula (reward function) to update the value Q(s, a) corresponding to Q;
[0077] S6. Repeat S2-S5 until the value in Q converges (the value basically no longer changes or the change meets the indicator requirements). At this time, the agent can choose the optimal action based on the value in Q.
[0078] In the process of training the Q-Learning deep learning algorithm and using it to select paths, bandwidth utilization and node blocking rate are used to evaluate the network situation. The root mean square error of bandwidth utilization and node blocking rate is used to measure the current link resource utilization of the network, and it is used as a reward for the Q-Learning deep learning algorithm to achieve continuous optimization of path allocation and ensure a more balanced network load.
[0079] (2) After the central server selects the transmission path, it calculates the time delay for the data to reach each node in the path based on the selected path, and stores it together with the occupied bandwidth and transmission path of the node sending the data to form a storage record.
[0080] Mainly consider the propagation delay and queuing delay. The delay calculation formula is as follows:
[0081] Total delay = propagation delay + queuing delay
[0082] The central server estimates the delay t from the sending node to the receiving node and the delay from the sending node to each transmission node, and uses t-the delay from the sending node to each transmission node as t i Recorded in array form as {t0,t1,t2,…t n}, timer t starts timing when data transmission starts.
[0083] Calculate the additional traffic caused by the information being delivered and obtain the bandwidth used:
[0084] Bandwidth (w) = data transmission rate / number of bits transmitted
[0085] The central server maintains a record for each transmission, including the record number, transmission path, timer, delay of each node, and occupied bandwidth.
[0086] When the timer t=-1, the delay array is cleared, indicating that the initial information has reached all the routing nodes to be reached, and the record can be deleted.
[0087] (3) After completing the path allocation, the central server sends the transmission path to the data sending node, and the data sending node performs data transmission.
[0088] (4) After path selection, the central server obtains the network transmission link bandwidth and used bandwidth, node buffer size and node buffer used size, calculates bandwidth utilization and node blocking rate, and calculates the sum of their root mean square errors. The root mean square error after path selection is subtracted from the root mean square error before path selection as feedback for the agent, and the Q-Learning agent action selection strategy is adjusted to achieve continuous optimization.
[0089] The present invention uses delay calculation and timers to estimate the network status during transmission, fundamentally solving the problem of network congestion caused by excessive task load of some nodes due to repeated node selection caused by the inconsistency between the network status during transmission and the network status during observation caused by network transmission delay, thereby reducing network congestion. The Q-Learning deep learning algorithm is used to select the path, and the node blocking rate is added as an indicator for evaluating node blocking when evaluating the network status, reducing node blocking during path selection, and designing a reward algorithm to achieve continuous optimization of path allocation and ensure a more balanced network load. In response to the problem that the current congestion control algorithm based on machine learning is partially observable for the network status, this paper establishes a central server and proposes to use pre-sent data packets to compress the data to be sent by the sender, and only send information related to transmission blocking, thereby improving the accuracy and convergence of machine learning. The proposed method has certain reference significance for network flow control and avoiding network congestion.
[0090] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.
Claims
1. A QUIC protocol secure and reliable transmission method based on adaptive congestion control, characterized in that include: Record the bandwidth and node buffer size of each transmission path in the network; The data transmission path and delay from each sending node to each receiving node under the current network conditions are set in the central server; When the sending node sends a data stream to the receiving node, the elephant flow identification algorithm is used to identify whether the sent data stream is an elephant flow: If the data stream to be sent is not a large stream, it is directly sent to the receiving node to complete the data pre-sending; If the data stream to be sent is an elephant stream, the corresponding information needs to be passed to the central server. The central server obtains the network status, calculates the bandwidth utilization and node blocking rate, and re-estimates the network status based on the records maintained by the central server. It inputs the information into the trained Q-Learning deep learning module to select a suitable transmission path. After the central server selects the transmission path, it calculates the time delay for the data to reach each node in the transmission path according to the selected transmission path and stores it, returns the record of the transmission path in the central server, updates the record of the network status, and completes the network transmission.
2. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: When the data stream sent is an elephant stream, the corresponding information transmitted to the central server includes: data size and data transmission destination.
3. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: The Q-Learning deep learning module includes the Q-Learning agent, environment, action, state, reward, and reward function, among which: Q-learning agent: plays the role of learner and decision maker, responsible for selecting actions; Environment: The network environment that the agent interacts with; Status: Information obtained by the Q-Learning agent from the network, including link status, bandwidth usage, and node congestion; Action: The algorithm needs to select a transmission path for information transmission from the sending node to the receiving node. The action of the algorithm is the node that sends the information and the path of information transmission; Reward: The environment’s feedback on the action selected by the agent. After the action is selected, the network transmission environment will change to a certain extent. Reward function: By obtaining the changes in network conditions before and after path selection, the Q-Learning agent is rewarded, so that the Q-Learning agent can adjust its action selection strategy and continuously learn and train the link allocation according to the network status to achieve continuous optimization of path allocation.
4. According to claim 3, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: In the process of training the Q-Learning deep learning module and using the Q-Learning deep learning module to select paths, the root mean square error of bandwidth utilization and node blocking rate is used to measure the current link resource utilization of the network, and it is used as a reward for the Q-Learning deep learning algorithm in the Q-Learning deep learning module to optimize path allocation in real time.
5. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: The network status information obtained and estimated by the central server includes: The used bandwidth of each path, the transmission bandwidth of each path, the node buffer occupancy size, and the node buffer size.
6. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: The central server obtains the network transmission path bandwidth and used bandwidth, node buffer size and node buffer used size, and calculates the bandwidth utilization and node blocking rate as follows:
7. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: According to the transmission network status estimated by the central server, the network status is adjusted, specifically: The central server estimates the transmission delay of data from the sender to each node on the selected transmission path; the central server maintains a timer for each path, the bandwidth occupied by the sending node's transmission information, the node ID, and an array of the delay from the sending node to each node. The value recorded in the array is: the total delay from the sending node to the destination minus the delay from the sending node to each node in the path; the initial value of the timer is: the total delay from the sending node to the destination; When the sending node sends information, it sends a signal to the central server. The central server maintains multiple records. The timer of each record counts down according to the time, decreasing by 1 per second until the timer reaches -1, and the corresponding record is deleted.
8. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: When the sending node sends a new request to transmit information to the central server, the central server first obtains the network bandwidth and node buffer usage; then the central server reads the records in order, and for each record read, the central server reads the timer size in the record, and compares the timer in the record with the corresponding delay array in the record to obtain the transmission status of the previously transmitted data; the central server updates the bandwidth corresponding to the path that has not been transmitted in each record to obtain a new bandwidth, which is calculated as follows: path bandwidth = path bandwidth + record bandwidth; Traverse all records to get the adjusted bandwidth, use the adjusted bandwidth as the input of the agent, and calculate the path.
9. According to claim 1, a QUIC protocol secure and reliable transmission method based on adaptive congestion control is characterized in that: In the transmission of elephant flows, the delay is the sum of the queuing delay and the propagation delay; where: The queuing delay is caused by the buffer queuing of the selected node and is calculated based on the node buffer usage size and node processing speed; The propagation delay is calculated from the data length and the transmission speed.