Lossless-ethernet flow control method and apparatus

Abstract

Provided in the present invention are a lossless-Ethernet flow control method and apparatus. A designed lossless-Ethernet flow control algorithm based on deep learning is combined with the idea of reinforcement learning and a convolutional neural network model, which can provide a dynamic threshold adjustment mechanism for complex and changeable network environments, so that the cache space between an ECN threshold and a PFC threshold can accommodate traffic transmitted during the period from ECN congestion marking to a speed reduction at a source end, the triggering of network PFC is avoided as much as possible, and the performance of PFC and ECN in RDMA is maximized, thereby realizing anti-packet-loss, high bandwidth utilization, and low-delay transmission of lossless Ethernet in the complex and changeable network environments.

Description

A lossless Ethernet flow control method and device Technical Field

[0001] The present invention belongs to the technical field of aircraft antenna design, and in particular relates to a lossless Ethernet flow control method and device. Background Art

[0002] Lossless Ethernet enhances traditional Ethernet by leveraging flow control technologies to prevent packet loss, significantly improving Ethernet's real-time performance and reliability. The two primary technologies are Priority-Based Flow Control (PFC) and Quantized Congestion Notification (QCN). PFC implements port-based flow control with coarse granularity and is often used as a supplement to QCN. QCN uses the Explicit Congestion Notification (ECN) mechanism for flow control, which can limit the rate of individual flows. Typical algorithms using this mechanism include DCTCP and DCQCN.

[0003] The traditional ECN flow control mechanism is a static method that requires users to manually set ECN threshold parameters. Congestion is detected using pre-set thresholds, which can provide a certain degree of flow control. However, selecting ECN parameter values ​​requires comprehensive consideration of multiple complex network factors, making parameter selection difficult. Furthermore, with the ever-increasing volume of data, static ECN thresholds are no longer adaptable to scenarios such as bursty traffic and concurrent traffic of varying sizes. Specifically, for large data flows, higher ECN thresholds are required to ensure transmission bandwidth. However, this results in larger buffers and deeper queues for small data flows, leading to higher latency. For smaller data flows, lower ECN thresholds can ensure lower latency, but cannot guarantee the high throughput requirements for large data flows. In summary, static ECN thresholds clearly cannot address mixed scenarios of small and large data flows and are no longer suitable for today's complex and volatile network environments.

[0004] With the gradual maturity of intelligent technology, intelligent algorithms for dynamically adjusting thresholds have become a hot research direction. There are currently two main dynamic ECN adjustment strategies. The first dynamic ECN strategy mainly uses statistical mathematics such as multi-queue weight allocation to better utilize resources. Typical algorithms include MQ-ECN, DemePro, and DEMT. The second dynamic ECN strategy uses traditional machine learning algorithms (Bayesian networks, Gaussian regression models) to effectively use traffic characteristics such as transmission bandwidth and queue length to comprehensively predict ECN thresholds. Typical algorithms include DC-ECN. In fact, the above algorithm models can indeed dynamically change ECN threshold information, but there are still some problems. (1) Because traditional machine learning relies heavily on manual design for feature selection and representation, it does not consider the connection between thresholds and dynamic environmental changes, resulting in the model being unable to extract deep relationships between different traffic features. (2) For tens of thousands of traffic features generated by large-scale data, traditional machine learning can no longer perform fast and efficient learning. The real-time performance and accuracy of the algorithm are difficult to guarantee, and it will also cause large computational overhead.

[0005] Summary of the Invention

[0006] The present invention provides a lossless Ethernet flow control method and device, which combines the concept of reinforcement learning with a convolutional neural network model. The method and device can provide a dynamic threshold adjustment mechanism for complex and changeable network environments. This allows the buffer space between the ECN threshold and the PFC threshold to accommodate traffic sent between the time when ECN congestion is marked and the time when the source end slows down. This avoids triggering network PFC flow control as much as possible, maximizes the effectiveness of PFC and ECN in RDMA, and achieves lossless Ethernet with anti-packet loss, high bandwidth utilization, and low-latency transmission in complex and changeable network environments.

[0007] A first aspect of the present invention provides a lossless Ethernet flow control method, comprising:

[0008] S1. Obtain the simulated Ethernet environment at time T;

[0009] S2, obtaining the data packet characteristic information of the Ethernet traffic in the simulated Ethernet environment at time T; the data packet characteristic information includes: traffic characteristic state information;

[0010] S3. Construct a shallow convolutional neural network with a residual structure as the policy generation network. Input the traffic feature state information into the policy generation network model for feature extraction to obtain an encoded feature vector. The encoded feature vector includes multiple threshold information decision results with different probabilities.

[0011] S4. Input the encoded feature vector traffic feature state information into the DQN model, use the ε-greedy strategy of the DQN model to obtain the optimal lossless Ethernet flow control dynamic threshold information, and input it into the simulated Ethernet environment to update the traffic feature state information.

[0012] Optionally, the lossless Ethernet flow control method further includes:

[0013] Construct a target policy network, which is a shallow convolutional neural network with a residual structure;

[0014] After generating the optimal dynamic threshold information, enter the simulated Ethernet environment for execution, and use the following reward function to obtain the reward value W based on the updated traffic feature status information;

[0015] The reward value and updated traffic feature state information are input into the loss functions of the strategy generation network and the target strategy network respectively, completing a training update of the shallow convolutional neural network model with a residual structure.

[0016] Among them, K1, K2, K3 are the reward coefficients of average transmission delay, average throughput, and packet loss rate respectively, ρ represents the discount factor, g represents the reward threshold, τ represents the preset maximum average transmission delay, η represents the preset minimum average throughput, δ represents the preset maximum packet loss rate, and D t is the average transmission delay, T t is the average throughput, L t is the packet loss rate.

[0017] Optionally, the lossless Ethernet flow control method further includes:

[0018] A random data source is used to test in a simulated network environment. The effectiveness of the dynamic threshold information of lossless Ethernet flow control is verified by detecting whether there is packet loss.

[0019] Optionally, obtain the simulated Ethernet environment at time T, including:

[0020] Build a simulated Ethernet environment and configure the preset maximum average transmission delay, preset minimum average throughput, and preset maximum packet loss rate;

[0021] The simulated Ethernet environment is a many-to-one Ethernet simulation environment; the Ethernet simulation environment deploys PFC and ECN basic transmission strategies.

[0022] Optionally, the packet characteristics information includes

[0023] in, represents the traffic characteristic status information of all ports n at the sending and receiving ends in period t, Fbandwidth Indicates the bandwidth of the current link between the sender and the receiver, F delay represents the delay, F packets Indicates the number of packets, f demand Indicates the current cumulative data traffic size, F Qdepth Indicates the queue depth, F Incast Indicates a value of one more.

[0024] Optionally, the encoded feature vector is

[0025] Among them, p buffer Indicates the buffer size in the current congested area, p xon and p xoff Indicates the threshold for starting and stopping data transmission in the congestion control area, e buffer Indicates the buffer size in the entry area, e xon and e xoff They represent the thresholds for increasing and slowing down data transmission within the ingress control area, respectively.

[0026] A second aspect of the present invention provides a lossless Ethernet flow control device, comprising:

[0027] The simulated Ethernet environment acquisition module is used to obtain the simulated Ethernet environment at time T;

[0028] The data packet characteristic information acquisition module is used to obtain the data packet characteristic information of the Ethernet traffic in the simulated Ethernet environment at time T; the data packet characteristic information includes: traffic characteristic state information;

[0029] The encoded feature vector acquisition module is used to construct a shallow convolutional neural network with a residual structure as a policy generation network. The traffic feature state information is input into the policy generation network model for feature extraction to obtain the encoded feature vector. The encoded feature vector includes multiple threshold information decision results with different probabilities.

[0030] The lossless Ethernet flow control dynamic threshold information acquisition module is used to input the encoded feature vector flow feature state information into the DQN model, use the DQN model's ε-greedy strategy to obtain the optimal lossless Ethernet flow control dynamic threshold information, and input it into the simulated Ethernet environment to update the flow feature state information.

[0031] Optionally, the lossless Ethernet flow control device also includes:

[0032] The network construction module is used to build the target policy network, which is a shallow convolutional neural network with a residual structure;

[0033] The reward acquisition module is used to generate the optimal dynamic threshold information, input it into the simulated Ethernet environment for execution, and obtain the reward value W using the following reward function based on the updated traffic feature state information;

[0034] The training module is used to input the reward value and the updated traffic feature state information into the loss function of the strategy generation network and the target strategy network respectively, completing a training update of the shallow convolutional neural network model with a residual structure;

[0035] Among them, K1, K2, K3 are the reward coefficients of average transmission delay, average throughput, and packet loss rate respectively, ρ represents the discount factor, g represents the reward threshold, τ represents the preset maximum average transmission delay, η represents the preset minimum average throughput, δ represents the preset maximum packet loss rate, and D t is the average transmission delay, T t is the average throughput, L t is the packet loss rate.

[0036] Beneficial effects of the present invention:

[0037] Based on the transmission characteristics of Ethernet, the present invention combines the idea of ​​reinforcement learning with the convolutional neural network (CNN) model to design a set of lossless Ethernet flow control methods and devices. The main beneficial effects of the present invention are:

[0038] 1. This invention mainly applies reinforcement learning and convolutional neural network technology to congestion control in lossless Ethernet. It can adjust the ECN threshold in real time according to the transmission traffic situation, thereby achieving a lower packet loss rate, higher bandwidth utilization and lower latency. It improves the congestion control technology based on traditional machine learning and is more compatible with existing dynamic ECN strategies.

[0039] 2. The present invention utilizes the efficient feature extraction and modeling capabilities of deep convolutional neural networks, which can not only dynamically adjust the ECN threshold, but also has high real-time performance and accuracy.

[0040] 3. The present invention uses a reinforcement learning framework to train convolutional neural networks. Reinforcement learning is different from supervised learning. It can perform online learning and intelligent decision-making in unknown environments without the need for pre-labeled data and prior knowledge.

[0041] 4. The present invention proposes a lightweight residual network structure model, which can not only increase feature reuse, make the model converge quickly and lightweight network, but also can respond quickly according to the current traffic transmission characteristics to achieve adaptive adjustment of the ECN threshold.

[0042] 5. The present invention can not only realize automatic real-time ECN prediction of the network, but also predict the future ECN waterline based on historical traffic information, thereby providing a more efficient network transmission environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1: Structure of the flow control algorithm based on reinforcement learning and convolutional neural network;

[0044] Figure 2 CNN feature extraction model structure;

[0045] Figure 3 implements an example emulated Ethernet environment. DETAILED DESCRIPTION

[0046] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0047] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed in order to provide a comprehensive understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is in no way limited to any specific arrangement and method proposed below, but rather encompasses any improvements, replacements, and modifications to structures, methods, and devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, well-known structures and techniques are not shown to avoid unnecessary ambiguity in the present invention.

[0048] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0049] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0050] The present invention will be further described below with reference to the accompanying drawings:

[0051] The present invention provides a lossless Ethernet flow control method and device, the technical contents of which include:

[0052] 1. Build a simulated Ethernet environment. First, build an N-to-1 Ethernet simulation environment based on remote memory access (RDMA) technology to implement Ethernet data collection, recording, and lossless testing. Deploy basic transmission strategies such as PFC and ECN, and set a set of traffic characteristic information benchmarks as a reference for evaluating and rewarding lossless Ethernet. The traffic characteristic information F includes: average transmission delay D, average throughput T, and packet loss rate L. (Benchmark values: average transmission delay less than τ, average throughput greater than η, and packet loss rate less than δ. τ, η, and δ are all configurable parameters.)

[0053] 2. Ethernet traffic characteristic information perception. Obtain the data packet characteristic status information of the current network data port from the simulated network environment in, represents the traffic characteristic status information of all ports n at the sending and receiving ends in period t, F bandwidth Indicates the bandwidth of the current link between the sender and the receiver, F delay represents the delay, F packets Indicates the number of packets, f demand Indicates the current cumulative data traffic size, F Qdepth Indicates the queue depth, F Incast Indicates a value of one more.

[0054] 3. Data information feature extraction. Based on the state vector, CNN is used to integrate the link information of each port. It is sent to CNN for feature encoding to learn the complex relationship between features, and the encoded features are output to help the intelligent agent learn appropriate threshold information features.

[0055] The neural network CNN predicts multiple threshold information decision results with different probabilities Threshold Information Decision In, p buffer Indicates the buffer size in the current congested area, p xon and p xoff Indicates the threshold for starting and stopping data transmission in the congestion control area, e buffer Indicates the buffer size in the entry area, e xon and e xoff Indicates the thresholds for increasing and slowing data transmission within the entry control area. Furthermore, to ensure the model's learning capability and lightweightness, and facilitate rapid and accurate prediction results, a shallow convolutional neural network model with a residual structure is designed for feature extraction, enabling better feature reuse and avoiding loss of feature information.

[0056] 4. Threshold intelligent decision-making. The agent makes the best decision through the DQN model based on the multiple decision results output by the CNN. Specifically, the ε-greedy strategy of the DQN model is used to determine the threshold information with the highest final benefit. Setting decision. The decision-making method of the ε-greedy strategy is shown in formula (1).

[0057] in, Indicates a feature status information Take action The expected value of return, represents the final threshold information decision made by CNN at period port n in period t, r is a random number in the range [0,1], and ε is a variable that controls the probability of random exploration with an initial value of 1. Its value gradually decreases with the increase of the number of iterations. When r>ε, the action with the largest expected value is selected; when r≤ε, a predefined feasible action is randomly selected. The significance of this method is that in the early stages of iteration, the model is allowed to perform random exploration with a high probability, exploring all possible scenarios. As the number of iterations increases, the probability of selecting the action with the highest expected value increases. Its decay function ε is shown in formula (2).

[0058] Among them, ε start is the initial value of ε at the beginning of the iteration, ε end represents the minimum threshold of ε, ε decay is the attenuation factor. As the number of iterations I increases, the value of ε gradually decreases and gradually converges to ε end , achieving a stable strategy. ε-greedy is the most common learning strategy in the DQN model. By setting a variable ε value, this invention enables the DQN model to explore all situations in the early stages of training and learn more efficiently in the later stages of training, selecting the global optimal solution from all situations and making high-quality threshold information decisions.

[0059] 5. Reward generation. Dynamic threshold information for each time After inputting the simulated Ethernet environment and executing it, the reward value is given according to the designed reward function, and the reward value is positively correlated with the throughput and negatively correlated with the delay and packet loss rate, reflecting the quality of this decision. In the method proposed by the present invention, the reward is calculated by obtaining the updated traffic feature information F, which includes: average transmission delay D t , average throughput T t , packet loss rate L t , the reward function W formula is as shown in formula (3).

[0060] Here, K1, K2, and K3 are reward coefficients, and K1 + K2 + K3 = 1, indicating the importance of different traffic feature information. ρ is the discount factor, indicating the degree of attenuation of future rewards. g represents the reward threshold. When the network simultaneously satisfies the conditions that latency and packet loss rate are below or within the benchmark value, and throughput is above or within the benchmark value, the deep reinforcement learning model is given a positive reward, indicating that the threshold information decision can achieve lossless transmission. Otherwise, a certain penalty value is imposed. The reward value is calculated based on the state information obtained from the simulated network environment, and the traffic feature state, action, and reward information are input into the agent for model training.

[0061] 6. Model training. Past decisions are stored as samples in the training data buffer for model training. To improve the stability of neural network updates, two lightweight neural networks with identical structures but different parameters are used to enhance training stability. Loss is calculated by interactively outputting the two network models, and network parameters are then backpropagated and updated.

[0062] Exemplarily, two lightweight neural networks with identical structures but different parameters are used to improve training stability. Specifically, two neural networks with identical structures are used: a policy generation network and a target policy network. The policy generation network participates in model training, while the target policy network integrates all historical parameters of the policy generation network and includes stable model parameters. Stability constraints are provided for the policy generation network. Therefore, the final loss function of the present invention consists of a reward function W and a stability constraint term, as specifically defined in Formula (4).

[0063] in, represents the expected value of the return obtained by the target policy network, represents the expected value of the return obtained by the strategy generation network. E(·) represents the expected calculation. In addition, Denotes CNN f ω (·)In input state The following output. Represents CNN In input state The output under ω - and ω are the learnable parameters of the two CNN models. The parameters of the policy generation network are continuously updated during training, while the parameters of the target policy network are based on a more stable version of the past. During training, the policy generation network periodically updates the parameters of the target policy network when positive rewards are received, so that training gradually converges to a fixed target.

[0064] 7. Network lossless testing: Using random data sources, tests were conducted in a simulated network environment to verify that the lossless Ethernet flow control algorithm designed in this invention can dynamically adjust the PFC and ECN thresholds to achieve Ethernet high-throughput, low-latency, and zero-packet-loss network communication capabilities.

[0065] The deep learning-based lossless Ethernet flow control algorithm designed in this paper combines reinforcement learning ideas with convolutional neural network models. It can provide a dynamic threshold information adjustment mechanism for complex and changing network environments. This allows the buffer space between the ECN threshold and the PFC threshold to accommodate traffic sent between ECN congestion marking and source speed reduction, minimizing the triggering of network PFC flow control. This maximizes the effectiveness of PFC and ECN in Ethernet traffic transmission, achieving lossless Ethernet with anti-packet loss, high bandwidth utilization, and low-latency transmission in complex and changing network environments.

[0066] The above is only a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solutions and concepts of the present invention within the scope disclosed by the present invention, which fall within the scope of protection of the present invention.

Claims

1. A lossless Ethernet flow control method, characterized in that: include: S1. Obtain the simulated Ethernet environment at time T; S2, obtaining data packet characteristic information of Ethernet traffic in the simulated Ethernet environment at time T; The data packet characteristic information includes: flow characteristic status information; S3. Construct a shallow convolutional neural network with a residual structure as a strategy generation network, input the traffic feature state information into the strategy generation network model for feature extraction, and obtain an encoded feature vector. The encoded feature vector includes multiple threshold information decision results with different probabilities. S4. Input the encoded feature vector traffic feature state information into the DQN model, use the ε-greedy strategy of the DQN model to obtain the optimal lossless Ethernet flow control dynamic threshold information, and input it into the simulated Ethernet environment to update the traffic feature state information.

2. The method according to claim 1, characterized in that The method further comprises: Construct a target policy network, which is a shallow convolutional neural network with a residual structure; After the optimal dynamic threshold information is generated, it is input into the simulated Ethernet environment for execution, and the reward value W is obtained using the following reward function according to the updated traffic feature state information; The reward value and the updated traffic feature state information are input into the loss functions of the strategy generation network and the target strategy network respectively, completing a training update of the shallow convolutional neural network model with a residual structure; Among them, K1, K2, K3 are the reward coefficients of average transmission delay, average throughput, and packet loss rate, ρ represents the discount factor, g represents the reward threshold, τ represents the preset maximum average transmission delay, η represents the preset minimum average throughput, δ represents the preset maximum packet loss rate, and D t is the average transmission delay, T t is the average throughput, L t is the packet loss rate.

3. The method according to claim 1, characterized in that The method further comprises: A random data source is used to test in a simulated network environment to verify the effectiveness of the dynamic threshold information of lossless Ethernet flow control by detecting whether there is packet loss.

4. The method according to claim 1, characterized in that: Get the simulated Ethernet environment at time T, including: Build a simulated Ethernet environment, configure the preset maximum average transmission delay, preset minimum average throughput, and preset maximum packet loss rate; The simulated Ethernet environment is a many-to-one Ethernet simulation environment; the Ethernet simulation environment deploys PFC and ECN basic transmission strategies.

5. The method according to claim 1, characterized in that Packet characteristic information includes in, represents the traffic characteristic status information of all ports n at the sending and receiving ends in period t, F bandwidth Indicates the bandwidth of the current link between the sender and the receiver, F delay Indicates the delay, F packets Indicates the number of packets, f demand Indicates the current accumulated data flow size, F Qdepth Indicates the queue depth, F Incast Indicates a value of one more.

6. The method according to claim 1, characterized in that The encoded feature vector is Among them, p buffer Indicates the buffer size in the current congested area, p xon and p xoff Indicates the threshold for starting and stopping data transmission in the congestion control area, e buffer Indicates the buffer size in the entry area, e xon and e xoff They represent the thresholds for increasing and slowing down data transmission in the ingress control area respectively.

7. A lossless Ethernet flow control device, characterized in that: include: A simulated Ethernet environment acquisition module is used to acquire the simulated Ethernet environment at time T; The data packet characteristic information acquisition module is used to acquire the data packet characteristic information of the Ethernet traffic in the simulated Ethernet environment at time T; The data packet characteristic information includes: flow characteristic status information; The encoded feature vector acquisition module is used to construct a shallow convolutional neural network with a residual structure as a policy generation network, input the traffic feature state information into the policy generation network model for feature extraction, and obtain the encoded feature vector. The encoded feature vector includes multiple threshold information decision results with different probabilities; The lossless Ethernet flow control dynamic threshold information acquisition module is used to input the encoded feature vector flow feature state information into the DQN model, use the ε-greedy strategy of the DQN model to obtain the optimal lossless Ethernet flow control dynamic threshold information, and input it into the simulated Ethernet environment to update the flow feature state information.

8. The device according to claim 7, characterized in that Also includes: The network construction module is used to construct the target policy network, which is a shallow convolutional neural network with a residual structure. A reward acquisition module is used to input the simulated Ethernet environment for execution after generating the optimal dynamic threshold information, and to obtain a reward value W using the following reward function according to the updated traffic characteristic state information; The training module is used to input the reward value and the updated traffic feature state information into the loss functions of the strategy generation network and the target strategy network respectively, and complete a training update of the shallow convolutional neural network model with a residual structure; Among them, K1, K2, K3 are the reward coefficients of average transmission delay, average throughput, and packet loss rate, ρ represents the discount factor, g represents the reward threshold, τ represents the preset maximum average transmission delay, η represents the preset minimum average throughput, δ represents the preset maximum packet loss rate, and D t is the average transmission delay, T t is the average throughput, L t is the packet loss rate.

Images