A load balancing scheduling method for data center network

Abstract

The application discloses a load balancing scheduling method for a data center network and relates to the technical field of data center networks, and comprises the following steps: S1, constructing a multimode path state perception module at a sending end; the multimode path state perception module simultaneously adopts an active detection mechanism and a passive monitoring mechanism to collect transmission characteristic data of each path; the active detection mechanism periodically sends a detection packet with a time stamp to obtain a round-trip delay detection value of each link; and the passive monitoring mechanism analyzes real-time delay information carried by an acknowledgement character packet returned in a data flow transmission process to obtain a round-trip delay observation value of each link. The load balancing scheduling method for the data center network solves the problem of information lag of a single detection mode in traffic mutation, forms a complete closed loop of perception, decision, execution and re-perception, and thus maintains the stability of overall transmission performance in path state fluctuation.

Description

Technical Field

[0001] This invention relates to the field of data center network technology, specifically to a load balancing scheduling method for data center networks. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, data center networks are expanding in scale and carrying increasingly complex business types. Modern data center networks typically employ multi-rooted tree topologies, providing high-bandwidth transmission capabilities through numerous parallel paths. However, in real-world production environments, path diversity issues caused by dynamic traffic changes, link failures, and heterogeneous switching equipment are widespread, posing a significant challenge to multi-path load balancing methods. How to fully utilize network bandwidth while effectively avoiding out-of-order packet delivery and reducing flow completion time has become a research hotspot in the field of data center networks.

[0003] Existing data center network load balancing solutions are mainly divided into two categories: fine-grained and coarse-grained solutions. Fine-grained solutions, such as Random Packet Spreading (RPS) and Presto, can fully utilize multiple available paths in the network to achieve high link utilization. However, when facing asymmetric topologies, the significant differences in transmission latency between different paths can easily lead to severe packet out-of-order problems, causing the TCP congestion control mechanism to frequently trigger window halving, thus reducing transmission performance. Coarse-grained solutions, such as Equal Cost Multipath Routing (ECMP) and LetFlow, effectively avoid packet out-of-order issues by fixing all packets in the same flow to the same path. However, because they cannot dynamically adapt to changes in path status, they can easily cause uneven load distribution among multiple parallel paths, resulting in low overall link utilization.

[0004] To address the aforementioned issues, Chinese patent application CN114553783A discloses a "load balancing method for adaptively adjusting flow cell granularity in data center networks." This method acquires round-trip time (RTT) sample information for each link by periodically sending timestamped detection packets, performs path asymmetry estimation to distinguish between congested and non-congested paths, calculates the optimal flow cell granularity, and transmits flow cells on all paths using a round-robin approach to balance out-of-order rate and link utilization. This method employs a TCP congestion control mechanism, requires no modification to the protocol stack of the peer host or switch, and has good deployment compatibility. However, this scheme still has the following shortcomings in practical applications: First, the optimization calculation of flow cell granularity depends on the periodic detection of network status, and the parameter adjustment is lagging in the scenario of sudden traffic changes; Second, the method mainly estimates the path asymmetry based on the static threshold judgment of RTT, which is difficult to accurately characterize the complex coupling relationship of the dynamic change of delay in multi-path transmission; Third, when the load is high or the proportion of congested paths changes rapidly, the convergence speed of the adaptive adjustment mechanism of flow cell granularity needs to be further improved, and it may still lead to increased out-of-order recovery overhead or decreased bandwidth utilization due to improper granularity selection.

[0005] Therefore, how to design a load balancing scheduling method that can quickly respond to changes in path status, accurately control the risk of out-of-order packets, and maintain high link utilization remains a pressing technical problem to be solved in the current data center network field. Summary of the Invention

[0006] The purpose of this invention is to provide a load balancing scheduling method for data center networks to solve the problems mentioned in the background art.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a load balancing scheduling method for data center networks, comprising the following steps: S1: Construct a multi-mode path state awareness module at the sending end. The multi-mode path state awareness module simultaneously uses both active detection and passive listening mechanisms to collect transmission characteristic data of each path. The active detection mechanism periodically sends detection packets with timestamps to obtain the round-trip delay detection value of each link. The passive listening mechanism parses the real-time delay information carried by the acknowledgment character packets returned during the data stream transmission process to obtain the round-trip delay observation value of each link. S2: Input the round-trip delay detection value and round-trip delay observation value obtained in S1 into the cross-validation and confidence evaluation module. The cross-validation and confidence evaluation module performs dynamic weighted fusion of the round-trip delay data of the two paths and calculates the state confidence of each path based on the fused data. The state confidence is used to characterize the degree of certainty of the sender's perception of the true state of the path. S3: Input the state confidence and path state probability distribution obtained in S2 into the iterative flow cell granularity optimization module. The iterative flow cell granularity optimization module constructs an iterative optimization model between flow cell granularity and flow completion time, and determines the optimal flow cell granularity under the current network state through multiple rounds of iterative calculation and gradient approximation algorithm. S4: Input the optimal flow cell granularity determined in S3 into the dynamic polling scheduling module. The dynamic polling scheduling module cuts the data stream into flow cells according to the optimal flow cell granularity, and performs weighted polling allocation of the paths based on the state confidence and congestion probability of each path, and transmits the flow cells on each path according to the weight ratio.

[0008] Preferably, the cross-validation and confidence assessment module introduces time decay weights during the dynamic weight fusion process, calculates short-term weighted averages for active detection values ​​and passive observation values ​​respectively, and calculates the state confidence of each path based on the difference between the two weighted averages; when the state confidence is lower than a preset threshold, the cross-validation and confidence assessment module triggers a secondary verification mechanism, temporarily increases the active detection frequency and suspends the use of this path for data transmission, until the difference between the two weighted averages converges to a preset range.

[0009] Preferably, the cross-validation and confidence assessment module calculates the probability that each path is in a congested state based on the distribution characteristics of the round-trip delay of each path within the historical time window. The congestion probability is calculated using an adaptive threshold coefficient, which is dynamically adjusted according to the path misjudgment rate. The path state probability distribution includes the congestion probability of each path.

[0010] Preferably, the iterative flow cell granularity optimization module uses a multi-dimensional temporal feature fusion and path state field modeling method to determine the optimal flow cell granularity under the current network state, specifically including: The first sub-step involves extracting multi-dimensional time-series features from the round-trip delay sequences of each path within a preset time window. These multi-dimensional time-series features include fluctuation features, trend features, periodic features, and burst features. The second sub-step involves modeling the data center network topology as a graph structure, where nodes are switches and edges are paths. The multi-dimensional temporal features of each path are used as the initial feature vectors of the edges. The feature vectors are then spatiotemporally modeled using a temporal graph convolutional network to output the state embedding vectors of each path. The third sub-step involves constructing a path state field matrix from the state embedding vectors of all paths, and then performing online clustering on the path state field matrix to automatically divide the paths into stable good path clusters, stable bad path clusters, and oscillating clusters. The fourth sub-step is to determine the optimal flow cell size in the current network state based on the clustering results of the path state field.

[0011] Preferably, the iterative flow cell granularity optimization module introduces a dynamic game-theoretic path state evaluation mechanism in the process of determining the optimal flow cell granularity. It constructs a two-layer game model between the sender and the path set, and generates the state confidence of each path in the game equilibrium. In the two-layer game model, the sender's strategy is to select a set of probe weight distributions, and the path set's strategy is to select to represent the true state or a masquerade state. The sender updates its posterior belief in the true state of the path based on historical observations. At the mixed strategy Nash equilibrium point, the sender allocates more probe resources to the path with high belief uncertainty. The state confidence of each path is defined by the inverse of the variance of the sender's posterior belief in the true state of that path.

[0012] Preferably, the iterative flow cell granularity optimization module employs a generative adversarial network (GAN) for flow cell granularity optimization. The GAN includes a granularity generator and a flow completion time discriminator. The granularity generator takes the current path state field matrix and the previous optimal flow cell granularity as input and outputs a candidate granularity distribution. The flow completion time discriminator takes the path state field matrix and the flow cell granularity as input and outputs the predicted flow completion time at that granularity. The granularity generator and the flow completion time discriminator are iteratively optimized through adversarial training. The goal of the granularity generator is to generate a granularity distribution that minimizes the flow completion time predicted by the flow completion time discriminator. The goal of the flow completion time discriminator is to accurately predict the flow completion time at different granularities. The training data comes from historical scheduling records and small-volume measured data.

[0013] Preferably, the iterative flow cell granularity optimization module employs a self-supervised adversarial mechanism during the generative adversarial network training process. In each round of adversarial training, the flow completion time discriminator is first fixed to train the granularity generator, so that the weighted average granularity of the granularity distribution output by the granularity generator obtains the minimum predicted flow completion time on the flow completion time discriminator. Then, the granularity generator is fixed, and the newly generated granularity is tested in a real environment with a small flow rate. The tested data is added to the training set to update the flow completion time discriminator. The above process is repeated until the granularity generated by the granularity generator converges to a stable value in both the flow completion time discriminator prediction and the actual test.

[0014] Preferably, the dynamic round-robin scheduling module adopts an adaptive reconstruction mechanism of the transmission strategy. The adaptive reconstruction mechanism includes a predefined scheduling strategy base function library and a meta-learner. The scheduling strategy base function library contains multiple scheduling strategy base functions. Each scheduling strategy base function maps the state confidence, congestion probability, current flow cell granularity, and flow remaining size of each path to a scheduling weight vector. The meta-learner takes the recent path state field sequence as input and outputs the combination coefficients of each scheduling strategy base function. The dynamic round-robin scheduling module performs a weighted summation of the outputs of multiple scheduling strategy base functions based on the combination coefficients to obtain the final scheduling weight of each path.

[0015] Preferably, the meta-learner adopts a neural network structure combining a long short-term memory network and an attention mechanism. The long short-term memory network captures the temporal evolution pattern of the path state field sequence, and the attention mechanism selects a scheduling strategy basis function combination that is suitable for the current pattern. The meta-learner is pre-trained on a simulation environment or historical traffic logs through reinforcement learning. After actual deployment, after each flow transmission is completed, the path state field sequence, the selected basis function combination, and the transmission performance are recorded as new samples to update the meta-learner online, thereby realizing the continuous evolution of the scheduling strategy.

[0016] Preferably, during the weighted round-robin allocation process, when the congestion probability of each path changes or the state confidence level is lower than a preset threshold, the dynamic round-robin scheduling module feeds back the scheduling result to the cross-validation and confidence evaluation module. The cross-validation and confidence evaluation module corrects the time decay coefficient in the dynamic weight fusion and the adaptive threshold coefficient in the congestion probability calculation based on the out-of-order occurrence of flow cells on each path after actual scheduling and the throughput data, forming a closed-loop optimization link of perception, decision-making, execution, and re-perception.

[0017] This invention provides a load balancing scheduling method for data center networks. It has the following beneficial effects: By constructing a multi-mode path state awareness and cross-validation mechanism, dynamic weighted fusion of active detection and passive monitoring data solves the problem of information lag during sudden traffic changes in single detection methods. Simultaneously, a joint representation of state confidence and congestion probability is introduced, transforming path state evaluation from static binary judgment to probabilistic modeling incorporating uncertainty metrics. Building upon this, the combined use of a cyclic iterative flow cell granularity optimization module and a generative adversarial network upgrades flow cell granularity adjustment from open-loop formula calculation to closed-loop iterative optimization, reducing the number of rounds required for convergence to the optimal granularity and enhancing the stability of granularity selection during rapid path state changes.

[0018] By employing dynamic game-theoretic path state evaluation and meta-learning scheduling strategy reconstruction, the information interaction between the sender and the path set is modeled as a game equilibrium solution process. This allows the generation of state confidence to simultaneously reflect the dynamic balance of strategy interactions. Furthermore, the meta-learner adaptively combines multiple scheduling basis functions based on the path state field sequence, enabling the scheduling strategy to be continuously reconstructed as network characteristics evolve. The closed-loop feedback mechanism of the transmission strategy feeds the scheduling results back to the perception module, continuously correcting key parameters in dynamic weight fusion and congestion probability calculation. This creates a complete closed loop of perception, decision-making, execution, and re-perception, thereby maintaining the stability of overall transmission performance even when path states fluctuate. Attached Figure Description

[0019] Figure 1This is a flowchart illustrating a load balancing scheduling method for data center networks according to the present invention. Figure 2 This is an inter-module data flow diagram of a load balancing scheduling method for data center networks according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 and Figure 2 This invention provides a technical solution: a load balancing scheduling method for data center networks, comprising the following steps: S1: Construct a multi-mode path state awareness module at the sending end. The multi-mode path state awareness module simultaneously uses both active detection and passive listening mechanisms to collect transmission characteristic data of each path. The active detection mechanism periodically sends detection packets with timestamps to obtain the round-trip delay detection value of each link. The passive listening mechanism parses the real-time delay information carried by the acknowledgment character packets returned during the data stream transmission process to obtain the round-trip delay observation value of each link. S2: Input the round-trip delay probe value and round-trip delay observation value obtained in S1 into the cross-validation and confidence evaluation module. The cross-validation and confidence evaluation module performs dynamic weighted fusion of the round-trip delay data of the two paths, and calculates the state confidence of each path based on the fused data. The state confidence is used to characterize the degree of certainty of the sender's perception of the true state of the path. S3: Input the state confidence and path state probability distribution obtained from S2 into the iterative flow cell granularity optimization module. The iterative flow cell granularity optimization module constructs an iterative optimization model between flow cell granularity and flow completion time. Through multiple rounds of iterative calculation and gradient approximation algorithm, the optimal flow cell granularity under the current network state is determined. Flow cell granularity refers to the number of data packets or the length of data bytes contained in each transmission unit when the data flow is cut into independent transmission units. It is the core parameter controlling the granularity of data flow transmission. S4: Input the optimal flow cell granularity determined in S3 into the dynamic polling scheduling module. The dynamic polling scheduling module cuts the data stream into flow cells according to the optimal flow cell granularity, and performs weighted polling allocation of the paths based on the state confidence and congestion probability of each path, and transmits the flow cells on each path according to the weight ratio.

[0022] It should be further explained that in S1, the sending end deploys a multi-mode path state awareness module, which runs two acquisition links in parallel: active probing and passive monitoring. The active probing mechanism generates detection packets carrying transmission timestamps according to a preset probing period. These detection packets are sent to the receiving end via each path. Upon receiving the packet, the receiving end immediately returns a response packet carrying the original timestamp. The sending end calculates the round-trip delay detection value for each path based on the difference between the transmission time of the detection packet and the reception time of the response packet. During normal data transmission, the passive monitoring mechanism extracts the transmission timestamp pre-written by the sending end from the header of the acknowledgment packet returned by the receiving end. This timestamp is subtracted from the arrival time of the acknowledgment packet to obtain the round-trip delay observation value for each path.

[0023] In S2, the round-trip delay data obtained from the two acquisition links are input into the cross-validation and confidence assessment module. This module first maintains two sliding windows for each path, storing the most recent consecutive active detection values ​​and passive observation values ​​respectively. The data within the window are assigned exponential decay weights according to their time proximity, and the decay coefficient is dynamically adjusted according to the variance of the data within the window. The weighted average of active detection and the weighted average of passive observation are calculated respectively. Then, the relative deviation of the two weighted averages is calculated, and the complement of the relative deviation is used as the initial value of the state confidence of the path. Then, the initial value of the state confidence of the path within the most recent ten time windows is weighted and moved averaged to obtain the state confidence at the current time.

[0024] When the state confidence level is lower than the preset admission threshold, the module triggers a secondary verification mechanism: the active detection frequency is increased to three times the normal frequency, and the scheduling weight of the path is temporarily set to zero. Only the detection flow cells are retained to maintain state awareness, and new active detection values ​​and passive observation values ​​are continuously collected until the relative deviation of the two weighted average values ​​within three consecutive time windows is lower than the preset convergence threshold. Only then is the path reinstated into the scheduling set.

[0025] Under the premise that the state confidence level is consistently higher than the admission threshold, the module calculates the congestion probability based on the distribution characteristics of the round-trip delay data within the historical time window of the path: using the mean and standard deviation of the historical round-trip delay of the path as a benchmark, a dynamic threshold coefficient is set. This coefficient is adaptively adjusted according to the degree of consistency between the past ten congestion judgment results of the path and the actual occurrence of out-of-order events. The ratio of the number of sampling points in the statistical window whose round-trip delay exceeds the sum of the mean and the dynamic threshold coefficient times the standard deviation to the total number of sampling points is used as the congestion probability of the path at the current moment.

[0026] In S3, the iterative flow cell granularity optimization module receives the congestion probability and state confidence of each path, and constructs an iterative optimization model between flow cell granularity and flow completion time. This model takes the distribution vector of the congestion probability of all current paths as input, the range of flow cell granularity values ​​as the search space, and uses a gradient approximation algorithm for iteration: First, the search direction is determined by the gradient direction of the current flow completion time prediction function at the optimal granularity in the previous round. Candidate granularities are generated in the search direction with a dynamic step size. The step size is positively correlated with the variance of the current congestion probability distribution; the larger the variance, the larger the step size. Then, for each candidate granularity, the congestion window evolution trajectory required to transmit the entire flow at that granularity is predicted using the congestion probability vector and the state confidence vector. After simulating multiple rounds of transmission to reach the flow completion time, the candidate granularity with the smallest predicted flow completion time is selected as the output of this round. After the granularity output in this round is used for actual transmission, the actual flow completion time is recorded. The deviation between the predicted value and the measured value is used as a feedback signal to correct the parameters of the congestion window evolution model in the next iteration.

[0027] In S4, the dynamic round-robin scheduling module segments the data stream using the optimal flow cell granularity determined in S3. Each segmented flow cell is treated as an independent scheduling unit. For each path, a scheduling weight is calculated based on its congestion probability and state confidence. The scheduling weight is positively correlated with the state confidence and negatively correlated with the congestion probability. The specific calculation formula is: weight equals state confidence multiplied by one minus the congestion probability. Then, the weights of each path are normalized to obtain a weight distribution vector. The scheduling module performs probabilistic round-robin allocation for each path according to the weight distribution vector. That is, when transmitting a flow cell, a path is randomly selected for transmission based on the weight of each path. The path with a higher weight has a higher probability of being selected. Furthermore, all data packets within the same flow cell are sent sequentially on the same path to ensure internal ordering.

[0028] The cross-validation and confidence assessment module introduces time decay weights during the dynamic weight fusion process. It calculates short-term weighted averages for active probes and passive observations, and calculates the state confidence of each path based on the difference between the two weighted averages. When the state confidence is lower than a preset threshold, the cross-validation and confidence assessment module triggers a secondary verification mechanism, temporarily increases the frequency of active probes and suspends the use of that path for data transmission, until the difference between the two weighted averages converges to a preset range.

[0029] It should be further explained that in S2, the cross-validation and confidence assessment module maintains two independent circular buffers for each path. One buffer stores the round-trip delay probe values ​​obtained from the most recent thirty active probes, and the other buffer stores the round-trip delay observation values ​​obtained from the most recent thirty passive monitoring probes. The data in each buffer are arranged in order of sampling time. Each data point in the buffer is assigned a time decay weight, which is determined by a decay coefficient. This decay coefficient is not a fixed constant, but is dynamically calculated based on the fluctuation of the difference between the active probe values ​​and passive observation values ​​in the most recent ten time windows of the path. When the fluctuation of the difference increases, the decay coefficient increases accordingly, so that recent data occupies a higher proportion in the weighted average. Conversely, the decay coefficient decreases to smooth short-term fluctuations. For the active probe buffer, each probe value is multiplied by its corresponding time decay weight, summed, and then divided by the total weight to obtain the active probe weighted average. For the passive observation buffer, the same calculation method is used to obtain the passive observation weighted average.

[0030] Subsequently, the ratio of the absolute difference between the weighted average of active detection and the weighted average of passive observation to the larger of the two values ​​is calculated. This ratio is used as the original deviation, and the value minus the original deviation is used as the initial value of the path's state confidence. After the initial value of the state confidence is calculated, it is stored in the confidence history queue of the path. The length of the confidence history queue is set to ten time windows. A weighted moving average is calculated on the ten initial values ​​in the queue, with the weight of each initial value decreasing linearly according to the time proximity, to obtain the state confidence of the path at the current time.

[0031] When the cross-validation and confidence assessment module performs dynamic weight fusion, it first constructs time decay weights for the active detection values ​​and passive observation values ​​of each path. The time decay weights are dynamically allocated according to the time of data sampling. The closer the sampled data is to the current time, the higher the weight is assigned, and the farther the sampled data is to the current time, the lower the weight is assigned. The decay coefficient is adaptively determined according to the fluctuation range of the path round-trip delay data. The greater the data fluctuation, the smaller the decay coefficient value, and the smaller the data fluctuation, the larger the decay coefficient value. After assigning corresponding time decay weights to the round-trip delay data at different times, the short-term weighted average of the active detection data and passive observation data is calculated by summing them sequentially. Then, the state confidence is calculated by comparing the deviation of the two sets of weighted averages. The smaller the deviation, the higher the state confidence, and the larger the deviation, the lower the state confidence. The preset threshold for state confidence is set to 0.6. When the calculated state confidence is lower than this threshold, the secondary verification mechanism is immediately triggered.

[0032] When the state confidence level is lower than the preset admission threshold, the module initiates a secondary verification process: the sending frequency of active probing is adjusted from once every 50 milliseconds to once every 16 milliseconds, and the scheduling weight of the path is reset to zero. The path no longer receives any data stream cells, but only receives active detection packets and passive listening packets to maintain state awareness. In the secondary verification state, the module continuously monitors the relative deviation between the weighted average of active probing and the weighted average of passive observation within the next three consecutive time windows. If the relative deviation of all three windows is lower than the preset convergence threshold, it is determined that the dual-source data has reached consensus. The state confidence level of the path is reset to the average of the complements of the relative deviations of the three windows, the probing frequency is restored to the normal value, and the path is reinstated into the scheduling set. If the relative deviation of any one of the three windows is not lower than the convergence threshold, the secondary verification state is maintained, and high-frequency probing continues until the convergence condition is met for all three consecutive windows.

[0033] The cross-validation and confidence assessment module calculates the probability that each path is in a congested state based on the distribution characteristics of the round-trip delay of each path within the historical time window. The congestion probability is calculated using an adaptive threshold coefficient, which is dynamically adjusted according to the path misclassification rate. The path state probability distribution includes the congestion probability of each path.

[0034] It should be further explained that in S2, the cross-validation and confidence assessment module maintains a historical round-trip delay window for each path. The window length is fixed at two hundred sampling points, and both active probe values ​​and passive observation values ​​are stored within the window. Each time a new sampling point is acquired, the oldest sampling point in the window is replaced according to the first-in, first-out (FIFO) principle. Based on all round-trip delay data within this window, the module calculates the historical mean and historical standard deviation for the path. The update of the mean and standard deviation uses an incremental calculation method; each time the window slides, only the contribution of old points is removed and the contribution of new points is added to reduce computational overhead.

[0035] The congestion probability is calculated using an adaptive threshold coefficient, initially set to two. After each congestion probability calculation, the module compares the events that were determined to be congested on the path within the past ten time windows with the actual out-of-order data packet events. If the path is determined to be congested within a certain time window but no out-of-order data packets occur in the subsequent flow cell transmission within the corresponding window, this determination is recorded as a misjudgment. The number of misjudgments within the ten windows is counted, and the misjudgment rate is defined as the ratio of the number of misjudgments to the total number of determinations. The adaptive threshold coefficient is adjusted according to the change in the misjudgment rate. When the misjudgment rate is higher than 10%, the adaptive threshold coefficient increases by 0.1; when the misjudgment rate is lower than 3%, the adaptive threshold coefficient decreases by 0.1. The value of the adaptive threshold coefficient is limited to the range of 1.5 to 3.

[0036] The module then iterates through all round-trip delay data points within the historical window, counting the number of data points whose round-trip delay values ​​exceed the sum of the mean and the adaptive threshold coefficient times the standard deviation. The ratio of this number to the total number of sampling points in the window is used as the initial congestion probability of that path at the current moment. To reduce the impact of instantaneous fluctuations on the congestion probability, the module further performs exponential weighted smoothing on this initial congestion probability value, weighting the congestion probability of the previous moment and the initial congestion probability value of the current moment with a 7:3 weight to obtain the congestion probability of the current moment.

[0037] The congestion probability calculated by the module, together with the aforementioned state confidence, constitutes the path state probability distribution. In this distribution, each path corresponds to a set of congestion probability values ​​and confidence values. The congestion probability is used to characterize the possibility that the path is currently in a congested state, and the confidence is used to characterize the reliability of the congestion probability estimate.

[0038] The iterative flow cell granularity optimization module uses a multi-dimensional temporal feature fusion and path state field modeling method to determine the optimal flow cell granularity under the current network state, specifically including: The first sub-step is to extract multi-dimensional time-series features from the round-trip delay sequences of each path within a preset time window. The multi-dimensional time-series features include fluctuation features, trend features, periodic features, and burst features. The second sub-step involves modeling the data center network topology as a graph structure, where nodes are switches and edges are paths. The multi-dimensional temporal features of each path are used as the initial feature vectors of the edges. The feature vectors are then spatiotemporally modeled using a temporal graph convolutional network to output the state embedding vectors of each path. The third sub-step involves constructing a path state field matrix from the state embedding vectors of all paths, and then performing online clustering on the path state field matrix to automatically divide the paths into stable good path clusters, stable bad path clusters, and oscillating clusters. The fourth sub-step is to determine the optimal flow cell granularity in the current network state based on the clustering results of the path state field. The path state field refers to the feature matrix formed by integrating the state characteristics of all paths within a data center network, which can comprehensively represent the current path state distribution of the network.

[0039] It should be further explained that in S3, the iterative flow cell granularity optimization module executes the first sub-step, extracting multi-dimensional temporal features from the round-trip delay sequences of each path within a preset time window. The preset time window length is fixed at one hundred continuous sampling points, and the feature values ​​are updated every ten sampling points. For each path, the module extracts fluctuation features from the round-trip delay sequences within the window, including the standard deviation, skewness, and kurtosis of the sequence. The standard deviation reflects the dispersion of the delay, the skewness reflects the asymmetry of the delay distribution, and the kurtosis reflects the tail risk of the delay distribution. It also extracts trend features, namely the slope obtained by linear regression calculation of the sequence within the window, which reflects the monotonic change direction and intensity of the delay within the time window. Furthermore, it extracts periodic features, namely, by performing a fast Fourier transform on the sequence within the window and extracting the amplitudes corresponding to the two frequency components with the largest amplitudes after the transform. Finally, it extracts burst features, namely, by calculating the squared deviation of each sampling point within the window from the mean of the sequence, and only accumulating the squared deviation values ​​corresponding to sampling points with deviations exceeding three times the standard deviation, using the accumulated result as the short-time energy value. The module concatenates the above eight feature values ​​to form an eight-dimensional feature vector of the path at the current time.

[0040] The module executes the second sub-step, modeling the data center network topology as a graph structure. Nodes in the graph correspond to switches, edges correspond to links connecting the switches, and each edge is associated with a feature vector. The module constructs a temporal graph convolutional network, which contains two graph convolutional layers and one gated recurrent unit layer. In the graph convolutional layers, for each edge, the feature vectors of its neighboring edges sharing the same node are aggregated. Aggregation is performed using a weighted summation of a normalized adjacency matrix, and the aggregation result is linearly transformed using a learnable weight matrix before being output as intermediate features via an activation function. The gated recurrent unit layer takes the graph convolutional outputs of each edge at the current time step as input, combines them with the hidden states of the gated recurrent unit layer from the previous time step, and outputs the state embedding vector of each edge at the current time step through update and reset gate operations. The state embedding vector has a 16-dimensional dimension.

[0041] The temporal graph convolutional network used in the iterative flow cell granularity optimization module is configured with two graph convolutional layers and one gated recurrent unit layer. The graph convolutional layers use 1×1 convolutional kernels and the ReLU activation function. The path feature vector is output as a 16-dimensional state embedding vector after convolution. The hidden layer dimension of the gated recurrent unit layer is set to 64 dimensions. When performing spatiotemporal joint modeling of path features, the feature information of adjacent paths is aggregated in turn and combined with the temporal change law to complete the feature update. Finally, a stable state embedding vector for each path is output.

[0042] The module executes the third sub-step, arranging the state embedding vectors of all paths in path order to form a state field matrix with sixteen rows equal to the total number of paths. The module then performs online clustering on the state field matrix using an adaptive density clustering algorithm. This algorithm uses the Euclidean distance between the row vectors in the state field matrix as a similarity measure, automatically determining the neighborhood radius and the minimum number of neighborhood points based on the data point distribution density. Points with tightly clustered state embedding vectors in the vector space are grouped into the same cluster, while discrete points are classified as noise. After clustering, the module identifies three types of path clusters: the first type is stable good path clusters, where the state embedding vectors of each path within the cluster do not undergo inter-cluster migration within five consecutive time windows, and the Euclidean distance between the cluster center and the global center of the state field matrix is ​​less than 0.6 times the average distance between the global center and each cluster center; the second type is stable bad path clusters, where the state embedding vectors of each path within the cluster do not undergo inter-cluster migration within five consecutive time windows, and the Euclidean distance between the cluster center and the global center of the state field matrix is ​​greater than 0.6 times the average distance between the global center and each cluster center; the third type is oscillating clusters, where the state embedding vectors of each path within the cluster undergo at least two inter-cluster migrations within five consecutive time windows.

[0043] The module executes the fourth sub-step, determining the optimal flow cell granularity for the current network state based on the clustering results of the path state field. When the number of paths contained in a stable path cluster exceeds 60% of the total number of paths, the module sets the flow cell granularity to 44; when the number of paths contained in a stable bad path cluster exceeds 30% of the total number of paths, the module sets the flow cell granularity to 8; when the number of paths contained in an oscillating cluster exceeds 20% of the total number of paths, the module sets the flow cell granularity to 16; when none of the above conditions are met, the module uses the proportion of paths within a stable path cluster as a weight to perform a weighted average of the 44, 16, and 8 granularities, and takes the result as the optimal flow cell granularity.

[0044] The iterative flow cell granularity optimization module introduces a dynamic game-theoretic path state evaluation mechanism in the process of determining the optimal flow cell granularity. It constructs a two-layer game model between the sender and the path set, and generates the state confidence of each path in the game equilibrium. In the two-layer game model, the sender's strategy is to select a set of probe weight distributions, and the path set's strategy is to select to represent the true state or a masquerade state. The sender updates its posterior belief in the true state of the path based on historical observations. At the mixed-strategy Nash equilibrium point, the sender allocates more probe resources to the path with high belief uncertainty. The state confidence of each path is defined by the inverse of the variance of the sender's posterior belief in the true state of that path.

[0045] It should be further explained that in S3, the iterative flow cell granularity optimization module introduces a dynamic game-like path state evaluation mechanism, constructing a two-layer game model between the sender and the path set. In this model, the two parties participating in the game are the sender and the path set consisting of all paths. The sender acts as the evaluator, and the path set acts as the evaluated party. The sender's policy space is a vector of all possible probe weight distributions, the dimension of which is equal to the total number of paths. Each component represents the proportion of detection packets sent to the corresponding path, and the sum of all components equals one. The path set's policy space is the performance mode chosen by each path. Each path can choose between a real mode and a masquerading mode. In the real mode, the path truthfully reflects its round-trip delay characteristics. In the masquerading mode, the path actively introduces additional delays in the acknowledgment packet to interfere with the sender's measurement. The sender's payoff function is defined as the difference between the state estimation accuracy and the probe cost. The state estimation accuracy is determined by the degree of matching between the sender's estimate of the path's true state and the path's true state; the higher the matching degree, the greater the payoff. The probe cost is positively correlated with the sum of the probe weights assigned to each path. The revenue function of the path set is defined as the difference between the misjudgment loss and the masquerade cost, where the misjudgment loss represents the revenue gained by the path when the sender's estimate of the path state deviates from the true state, and the masquerade cost represents the additional resource consumption required when the path selects the masquerade mode.

[0046] At the start of each decision cycle, the module initializes its posterior beliefs about the state of each path at the end of the previous cycle. These posterior beliefs are represented as probability vectors, with each component representing the probability that the sender considers the corresponding path to be congested. All components are initially set to 0.5. During the cycle, the sender transmits detection packets to each path according to the current probe weight distribution vector and updates its posterior beliefs about the state of each path based on the returned measurement results. The update uses Bayes' theorem, calculating the posterior probability conditioned on the measurement results exhibited by the path under the current policy.

[0047] In the two-level game model, the sender's payoff function is based on the accuracy of path state estimation and the overhead of sending probe packets. The path set's payoff function is based on the transmission advantage gained from the sender's misjudgment of the state and the resource consumption caused by the path's masquerading as a state. The Nash equilibrium is solved through multiple rounds of iterative exploration. In each round, the sender adjusts the probe weight distribution based on the current probe results, and the path set adjusts its state behavior pattern based on the sender's probe behavior, until the payoffs of both sides reach a stable level and no longer change. The posterior belief update is based on the path state observation results of each round, gradually correcting the probability of judging the true state of the path. When the observation results are consistent with the prediction, the belief probability of the corresponding state is increased; when they are inconsistent, the belief probability of the corresponding state is decreased.

[0048] At the end of each cycle, the module calculates the expected return of the transmitter under the current probe weight distribution and the expected return of the path set under the current performance mode. By iteratively solving the hybrid strategy Nash equilibrium, the transmitter gradually shifts its probe resources from paths with high posterior belief certainty to paths with low posterior belief certainty during the iteration process. The path set gradually converges to the performance mode combination that maximizes its own expected return.

[0049] The module uses the sender's posterior belief in the state of each path at the game equilibrium point as the final evaluation result. For each path, it extracts the congestion probability estimate from the posterior belief probability vector and calculates the uncertainty measure of this estimate, defined as the variance of the posterior belief probability vector on that path component. The reciprocal of the uncertainty measure is used as the state confidence of the path. When the posterior belief variance approaches zero, the state confidence approaches infinity; when the posterior belief variance approaches one, the state confidence approaches one. The module replaces the previously directly calculated congestion probability and state confidence with the congestion probability and state confidence obtained from the game equilibrium and inputs them into the subsequent flow cell granularity optimization process, so that the evaluation result of the path state simultaneously reflects the equilibrium state of the policy interaction between the sender and the path set.

[0050] The iterative flow cell granularity optimization module employs a generative adversarial network (GAN) to optimize flow cell granularity. The GAN includes a granularity generator and a flow completion time discriminator. The granularity generator takes the current path state field matrix and the previous optimal flow cell granularity as input and outputs a candidate granularity distribution. The flow completion time discriminator takes the path state field matrix and the flow cell granularity as input and outputs the predicted flow completion time at that granularity. The granularity generator and the flow completion time discriminator are iteratively optimized through adversarial training. The goal of the granularity generator is to generate a granularity distribution that minimizes the flow completion time predicted by the flow completion time discriminator. The goal of the flow completion time discriminator is to accurately predict the flow completion time at different granularities. The training data comes from historical scheduling records and small-volume measured data.

[0051] It should be further explained that in S3, the iterative flow cell granularity optimization module uses a generative adversarial network (GAN) to optimize flow cell granularity. This GAN consists of two parts: a granularity generator and a flow completion time discriminator. The granularity generator adopts a three-layer fully connected neural network structure. The input layer receives the current path state field matrix and the previous round's optimal flow cell granularity. The current path state field matrix is ​​a real matrix with 16 rows equal to the total number of paths and 16 columns. The previous round's optimal flow cell granularity is an integer between one and forty-four. The input layer flattens the state field matrix into a one-dimensional vector and concatenates it with the previous round's optimal flow cell granularity. The intermediate layer contains 128 neurons, and the output layer contains 44 neurons. The output layer uses a flexible maximum activation function and outputs a probability distribution vector with 44 components, each component corresponding to the selection probability of a candidate granularity.

[0052] The flow completion time discriminator adopts a five-layer fully connected neural network structure, with residual connections introduced between each layer. The input layer receives the concatenation vector of the current path state field matrix and the candidate granularity. The middle three layers each contain 256 neurons, and the output layer is a real number representing the predicted flow completion time under the input conditions.

[0053] The generative adversarial network is iteratively optimized using adversarial training. The training data comes from historical scheduling records and small-volume measured data. The historical scheduling records store triplet data of the state field matrix, the actual flow cell granularity used, and the measured flow completion time in past transmissions.

[0054] In the generative adversarial network, the granularity generator is set as a three-layer fully connected neural network, and the flow completion time discriminator is set as a five-layer fully connected neural network with residual connections. The loss function takes the deviation between the predicted and measured flow completion times as the core optimization objective. During training, the learning rate is set to 0.001 and the batch size is set to 64. The model parameters are updated once after each round of adversarial training until the flow completion time prediction deviation is stable within 1% of the measured value. The training data adopts a combination of historical scheduling records and small-volume measured data to continuously expand the model training sample library.

[0055] In the first phase of adversarial training, the network parameters of the flow completion time discriminator are fixed. Candidate granularities are sampled from the probability distribution output by the granularity generator. These candidate granularities, along with the state field matrix, are input into the flow completion time discriminator to obtain the predicted flow completion time. The predicted flow completion time is used as the loss signal, and the network parameters of the granularity generator are updated through backpropagation, causing the probability distribution output by the granularity generator to shift towards a smaller predicted flow completion time. In the second phase of adversarial training, the network parameters of the granularity generator are fixed. The three granularities with the highest probability values ​​in the current output probability distribution of the granularity generator are selected as candidates. One-two-hundredth of the total traffic is allocated as test traffic in the actual network environment. Test traffic is transmitted with these three granularities, and the measured flow completion time is recorded. The state field matrix, test granularities, and measured flow completion times are combined to form a new training sample and added to the historical scheduling record. The updated historical scheduling record is used to train the flow completion time discriminator for one round, updating its network parameters.

[0056] The first and second phases of adversarial training are executed alternately. After each round of alternation, the module checks whether the granularity corresponding to the maximum probability value in the probability distribution output by the granularity generator is the same as that in the previous round and has not changed for three consecutive rounds. At the same time, it checks whether the average absolute deviation between the predicted value and the measured value of the flow completion time discriminator on the newly added test samples in the most recent round is less than one percent of the measured value of the flow completion time. When both conditions are met, the generative adversarial network is considered to have converged, and the granularity corresponding to the maximum probability value in the probability distribution output by the granularity generator is taken as the optimal flow cell granularity in the current network state.

[0057] The iterative flow cell granularity optimization module employs a self-supervised adversarial mechanism during the training of the generative adversarial network. In each round of adversarial training, the flow completion time discriminator is first fixed to train the granularity generator. This ensures that the weighted average granularity of the granularity distribution output by the granularity generator reaches the minimum predicted flow completion time on the flow completion time discriminator. Then, the granularity generator is fixed, and the newly generated granularity is tested in a real environment with a small flow rate. The tested data is added to the training set to update the flow completion time discriminator. This process is repeated until the granularity generated by the granularity generator converges to a stable value in both the flow completion time discriminator prediction and the actual test.

[0058] Self-supervised adversarial mechanism refers to the process in which, during the training of generative adversarial networks, no manual data labeling is required. Instead, the model generates its own candidate granularity and forms a supervisory signal with the measured transmission results, thereby completing the alternating optimization of the generator and discriminator.

[0059] It should be further explained that in S3, the iterative flow cell granularity optimization module employs a self-supervised adversarial mechanism during the generative adversarial network (GAN) training process. This mechanism divides the adversarial training between the granularity generator and the flow completion time discriminator into several rounds, each round containing a generator update phase and a discriminator update phase. In the generator update phase, the module fixes all network parameters of the flow completion time discriminator, extracts input data from the current path state field matrix and the optimal flow cell granularity of the previous round, inputs it into the granularity generator to obtain a candidate granularity probability distribution with forty-four components, samples three candidate granularities from this distribution according to their probability values, combines the three candidate granularities with the state field matrix respectively, and inputs them into the flow completion time discriminator to obtain three predicted flow completion times. The weighted sum of the three predicted flow completion times is used as the loss value, where the weights are proportional to the probability values ​​of the candidate granularities. The network parameters of the granularity generator are updated through the backpropagation algorithm, causing the probability distribution output by the granularity generator to shift towards a smaller predicted flow completion time. The generator update phase executes five iterations before entering the discriminator update phase.

[0060] During the discriminator update phase, the module fixes all network parameters of the granularity generator, selects the granularity with the highest probability value in the current output probability distribution of the granularity generator as the primary candidate granularity, and then selects the two granularities with the second and third highest probability values ​​as secondary candidate granularities. In the actual network environment, 0.5% of the total traffic of the data stream to be transmitted is allocated as test traffic. The test traffic is divided into three equal parts by the primary candidate granularity and the two secondary candidate granularities. During the test traffic transmission, the actual flow completion time corresponding to each test granularity is recorded. The state field matrix is ​​then compared with the primary candidate granularity and its... The measured flow completion time forms a training sample. The state field matrix, each auxiliary candidate granularity, and the measured flow completion time each form a training sample. These three training samples are added to the historical scheduling record. The module randomly selects sixty-four historical samples from the historical scheduling record that are similar in features to the current state field matrix. Together with the three newly added samples, they form the training set for the current batch. The flow completion time discriminator is updated once using the training set. After the update, the granularity corresponding to the maximum probability value in the probability distribution output by the granularity generator is recorded as the output granularity of the current round.

[0061] The generator update phase and the discriminator update phase are executed alternately. After each complete adversarial round, the module checks whether the output granularity of the current round is the same as that of the previous round. If they are the same, the counter is incremented; otherwise, the counter is reset to zero. When the counter reaches three consecutive rounds, the module further checks the average absolute deviation between the predicted flow completion time and the measured flow completion time of the newly added test samples in the last three discriminator update phases. If the average absolute deviation is less than two percent of the corresponding measured flow completion time, the generative adversarial network is considered to have converged, and the output granularity of the current round is taken as the optimal flow cell granularity. If the average absolute deviation is greater than or equal to two percent, the next adversarial round continues until both convergence conditions are met simultaneously.

[0062] The dynamic round-robin scheduling module adopts an adaptive reconstruction mechanism based on transmission strategies. This mechanism includes a predefined scheduling strategy base function library and a meta-learner. The scheduling strategy base function library contains various scheduling strategy base functions. Each scheduling strategy base function maps the state confidence, congestion probability, current flow cell granularity, and flow remaining size of each path to a scheduling weight vector. The meta-learner takes the recent path state field sequence as input and outputs the combination coefficients of each scheduling strategy base function. The dynamic round-robin scheduling module performs a weighted summation of the outputs of multiple scheduling strategy base functions based on the combination coefficients to obtain the final scheduling weights of each path.

[0063] It should be further explained that in S4, the dynamic round-robin scheduling module adopts an adaptive reconstruction mechanism based on transmission strategies. This mechanism consists of two parts: a predefined scheduling strategy base function library and a meta-learner. The scheduling strategy base function library predefines four scheduling strategy base functions. Each base function takes the state confidence, congestion probability, current flow cell granularity, and flow remaining size of each path as input and outputs a weight vector with a dimension equal to the total number of paths.

[0064] The first type of basis function is proportional fairness. Each component of its weight vector equals the product of the path's state confidence and a factor minus the difference in congestion probability, divided by the sum of these products across all paths. This results in paths with high state confidence and low congestion probability receiving higher weights. The second type of basis function is minimum delay priority. Each component of its weight vector equals the product of the inverse of the path's round-trip delay and its state confidence, divided by the sum of these products across all paths. This results in paths with low round-trip delay and high state confidence receiving higher weights. The third type of basis function is load balancing. Each component of its weight vector is obtained by solving a convex optimization problem that minimizes the variance of the expected queue length for each path. This optimization problem uses the congestion probability of each path as the queue length increment prediction and the state confidence as the constraint weight. The fourth type of basis function is exploration and utilization. Five percent of the weights in its weight vector are evenly distributed to paths with state confidence below 0.7, and the remaining 95% of the weights are distributed among the other paths using the method of the first type of basis function.

[0065] The meta-learner employs a neural network structure combining a Long Short-Term Memory (LSTM) network and an attention mechanism. The LSM network consists of two layers, each with a hidden state dimension of 64. The attention mechanism calculates attention weights for each time step on the output sequence of the LSM network. The input to the meta-learner is a sequence of recent path state fields, which is ten time windows in length. Each time window corresponds to a state field matrix, which is output by the cross-validation and confidence evaluation module in each scheduling cycle. The meta-learner inputs the ten state field matrices into the LSM network in chronological order to obtain the hidden states at ten time steps. The attention mechanism performs a weighted summation of the ten hidden states to obtain a context vector. This context vector is passed through a fully connected layer to output a combination coefficient of four components, each corresponding to a weight of a basis function for one of the four scheduling strategies. The fully connected layer uses a flexible maximum activation function to ensure that the sum of the combination coefficients is one. At the beginning of each scheduling cycle, the dynamic polling scheduling module inputs the current cycle's state confidence vector, congestion probability vector, current flow cell granularity, and flow remaining size into the four scheduling strategy basis functions to obtain four weight vectors. These four weight vectors are then weighted and summed according to the combination coefficients output by the meta-learner to obtain the final path scheduling weight vector. This vector is used as the basis for path allocation of flow cell transmission in the current cycle.

[0066] The scheduling strategy basis functions in the dynamic round-robin scheduling module take path state confidence, congestion probability, current flow cell granularity, and flow remaining size as input parameters. The scheduling weight vector is obtained by linear mapping according to the actual value of the parameters. The better the parameter value, the higher the corresponding weight. The attention mechanism of the meta-learner first scores the features of each time step of the path state field sequence. The time step features with higher scores are assigned greater attention weights. Then, the fused context features are obtained by weighted summation. Based on the context features, the combination coefficients of each scheduling strategy basis function are determined. The magnitude of the combination coefficients is positively correlated with the degree of fit of the corresponding basis functions.

[0067] The meta-learner employs a neural network structure combining a long short-term memory network and an attention mechanism. It captures the temporal evolution pattern of the path state field sequence through the long short-term memory network and selects a scheduling strategy basis function combination that is suitable for the current pattern through the attention mechanism. The meta-learner is pre-trained on a simulation environment or historical traffic logs through reinforcement learning. After actual deployment, after each flow transmission is completed, the path state field sequence, the selected basis function combination, and the transmission performance are recorded as new samples to update the meta-learner online, thereby realizing the continuous evolution of the scheduling strategy.

[0068] It should be further explained that in S4, the meta-learner adopts a neural network structure combining a Long Short-Term Memory (LSTM) network and an attention mechanism. The LSM network consists of two layers, with each layer having a hidden state dimension of 64. The input sequence length is ten time windows, each time window corresponding to a state field matrix. The dimension of the state field matrix is ​​the total number of paths multiplied by 16. Before input, the state field matrix is ​​flattened into a one-dimensional vector. The LSM network processes the ten input vectors sequentially in time order, outputting the hidden states at ten time steps. The attention mechanism receives these ten hidden states and calculates an attention weight for each hidden state. The attention weight is calculated by mapping the hidden state to a scalar score through a feedforward network, then performing flexible maximum normalization on the ten scalar scores to obtain ten attention weights that sum to a certain value. The ten hidden states are then weighted and summed according to their corresponding attention weights to obtain a 64-dimensional context vector. The context vector is input to a fully connected layer, which outputs four-dimensional combination coefficients. The fully connected layer uses a flexible maximum activation function to make the sum of the four combination coefficients a certain value. The four combination coefficients correspond to the weights of the basis functions of the four scheduling strategies.

[0069] The meta-learner is pre-trained using reinforcement learning on a simulation environment or historical traffic logs. During pre-training, multiple meta-tasks are constructed, each corresponding to a network traffic pattern. Each meta-task includes a support set and a query set. The support set contains a small number of state field sequence samples within a specific time window of the traffic pattern, while the query set contains another set of state field sequence samples within the same time window. In each meta-task, the meta-learner first learns from the support set, updating its parameters using gradient descent with samples from the support set. Then, it calculates the loss on the query set, defined as the difference between the scheduled flow completion time and the baseline flow completion time. The loss on the query set is used as the performance metric for the meta-learner. The goal of pre-training is to minimize the average loss of the meta-learner across multiple meta-tasks. A meta-optimization algorithm updates the initial parameters of the meta-learner, ensuring that after a small number of gradient steps on the support set, a low loss is achieved on the query set.

[0070] After deployment, the meta-learner is continuously updated online. Upon completion of each stream transmission, the module records the path state sequence during that transmission, the combination coefficients output by the meta-learner within that period, and the actual performance metrics of the stream transmission, including stream completion time and the number of out-of-order events. These data are grouped into an experience sample and stored in an experience replay buffer, with a buffer size of one thousand samples. When the experience replay buffer accumulates one hundred new samples, the module randomly selects sixty-four samples from the buffer to form a training batch. Using the path state sequence from the samples as input, the combination coefficients from the samples as the target output, and the actual performance metrics from the samples as weights, the loss is weighted and the meta-learner is updated using gradient descent. During the online update process, the learning rate of the meta-learner is set to one-tenth of the learning rate during the pre-training phase to adapt to new network dynamics while maintaining existing knowledge.

[0071] During the weighted round-robin allocation process, when the congestion probability of each path changes or the state confidence level falls below a preset threshold, the dynamic round-robin scheduling module feeds back the scheduling results to the cross-validation and confidence evaluation module. Based on the out-of-order occurrence of flow cells and throughput data on each path after actual scheduling, the cross-validation and confidence evaluation module corrects the time decay coefficient in dynamic weight fusion and the adaptive threshold coefficient in congestion probability calculation, forming a closed-loop optimization link of perception, decision-making, execution, and re-perception.

[0072] A closed-loop optimization link refers to the interconnection of four links: path status perception, scheduling decision, flow cell transmission, and transmission result feedback. The perception and decision parameters are corrected in reverse by the transmission results, forming a complete transmission control link with cyclic optimization.

[0073] It should be further explained that in S4, after the dynamic polling scheduling module completes each round of flow cell transmission, it continuously monitors the congestion probability and state confidence of each path. When the change in the congestion probability of any path exceeds 0.15 within three consecutive time windows, or the state confidence of any path drops below 0.6, the dynamic polling scheduling module triggers a closed-loop feedback mechanism. This mechanism uses the ratio of the number of out-of-order occurrences of flow cells on each path observed in that round of transmission to the total number of transmission flow cells as the actual out-of-order rate, and the ratio of the number of bytes successfully transmitted on each path to the transmission duration as the actual throughput. The actual out-of-order rate and throughput data are packaged into a scheduling result feedback packet and sent to the cross-checking and confidence assessment module through the control channel.

[0074] After receiving the scheduling result feedback packet, the cross-validation and confidence assessment module compares the actual out-of-order rate with the congestion probability prediction value calculated in S2 based on the current time decay coefficient and adaptive threshold coefficient for each path. For each path, the absolute deviation between the actual out-of-order rate and the congestion probability prediction value is calculated, and this deviation is divided by the actual out-of-order rate to obtain the relative deviation. The module maintains a relative deviation sliding window with a length of twenty time windows. When the average relative deviation within the sliding window exceeds 0.12, the module adjusts the time decay coefficient in the dynamic weight fusion by 5% of the current time decay coefficient. The adjustment direction is positively or negatively correlated with the average relative deviation value; that is, when the congestion probability prediction value is systematically higher than the actual out-of-order rate, the time decay coefficient is decreased, and vice versa. The adjusted time decay coefficient is limited to between 0.02 and 0.2.

[0075] The module simultaneously adjusts the adaptive threshold coefficient in the congestion probability calculation based on the actual throughput data in the scheduling result feedback packet. For each path, the module calculates the consistency between the congestion probability judgment result of that path in S2 and the actual out-of-order events. Specifically, if the congestion probability of the path is greater than 0.5 and the actual out-of-order rate is greater than 0.1, it is judged as a correct judgment; if the congestion probability is greater than 0.5 but the actual out-of-order rate is less than 0.05, it is judged as a misjudgment; if the congestion probability is less than 0.5 but the actual out-of-order rate is greater than 0.1, it is judged as a missed judgment. The module calculates the ratio of the sum of the number of misjudgments and the number of missed judgments in the most recent ten time windows to the total number of judgments as the comprehensive misjudgment rate. When the comprehensive misjudgment rate exceeds 8%, the adaptive threshold coefficient is increased by 0.1; when the comprehensive misjudgment rate is less than 2%, the adaptive threshold coefficient is decreased by 0.1. The adjusted adaptive threshold coefficient is limited to the range of 1.5 to 3.

[0076] After the cross-validation and confidence assessment module corrects the time decay coefficient and adaptive threshold coefficient, it applies the corrected parameters to the next round of state perception and assessment. Simultaneously, it returns the corrected parameter values ​​to the dynamic polling scheduling module via the control channel, which records the parameter corrections. This closed-loop chain of perception, decision-making, execution, and re-perception continues to operate during data transmission. Each completed flow triggers a round of closed-loop feedback, enabling the module parameters to continuously self-adjust as the network state changes.

[0077] In this technical solution, all key parameters are set with specific values ​​and applicable ranges. The entry threshold for state confidence is set to 0.6, the convergence threshold is set to 0.1, the historical time window length is fixed at 200 sampling points, the preset feature extraction time window length is 100 consecutive sampling points, the normal active detection period is set to 50 milliseconds, and the detection period is adjusted to 16 milliseconds during secondary verification. The flow cell granularity thresholds are set to 8, 16, and 44 respectively. The initial value of the adaptive threshold coefficient is set to 2, and its value range is limited to between 1.5 and 3. The time decay coefficient is limited to between 0.02 and 0.2.

[0078] This system constructs a multi-mode path state awareness and cross-validation mechanism, dynamically weighting and fusing active detection and passive monitoring data from two sources. This solves the problem of information lag when traffic changes suddenly occur in a single detection method. At the same time, it introduces a joint representation of state confidence and congestion probability, transforming path state evaluation from static binary judgment to probabilistic modeling that includes uncertainty measurement.

[0079] Based on this, by combining the iterative flow cell granularity optimization module with the generative adversarial network, the adjustment of flow cell granularity is upgraded from open-loop formula calculation to closed-loop iterative optimization, which reduces the number of rounds required for convergence of the optimal granularity and enhances the stability of granularity selection when the path state changes rapidly.

[0080] This system further models the information interaction between the sender and the path set as a game equilibrium solution process through dynamic game-theoretic path state evaluation and meta-learning scheduling strategy reconstruction. This allows the generation of state confidence to reflect the dynamic balance of strategy interaction. At the same time, the meta-learner adaptively combines multiple scheduling basis functions according to the path state field sequence, enabling the scheduling strategy to be continuously reconstructed as network characteristics evolve.

[0081] The closed-loop feedback mechanism of the transmission strategy sends the scheduling results back to the perception module, continuously correcting the key parameters in dynamic weight fusion and congestion probability calculation, so that perception, decision-making, execution and re-perception form a complete closed loop, thereby maintaining the stability of overall transmission performance when path status fluctuates.

[0082] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0083] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A load balancing scheduling method for data center networks, characterized in that, Includes the following steps: S1: Construct a multi-mode path state awareness module at the sending end. The multi-mode path state awareness module simultaneously uses both active detection and passive listening mechanisms to collect transmission characteristic data of each path. The active detection mechanism periodically sends detection packets with timestamps to obtain the round-trip delay detection value of each link. The passive listening mechanism parses the real-time delay information carried by the acknowledgment character packets returned during the data stream transmission process to obtain the round-trip delay observation value of each link. S2: Input the round-trip delay detection value and round-trip delay observation value obtained in S1 into the cross-validation and confidence evaluation module. The cross-validation and confidence evaluation module performs dynamic weighted fusion of the round-trip delay data of the two paths and calculates the state confidence of each path based on the fused data. The state confidence is used to characterize the degree of certainty of the sender's perception of the true state of the path. S3: Input the state confidence and path state probability distribution obtained in S2 into the iterative flow cell granularity optimization module. The iterative flow cell granularity optimization module constructs an iterative optimization model between flow cell granularity and flow completion time, and determines the optimal flow cell granularity under the current network state through multiple rounds of iterative calculation and gradient approximation algorithm. S4: Input the optimal flow cell granularity determined in S3 into the dynamic polling scheduling module. The dynamic polling scheduling module cuts the data stream into flow cells according to the optimal flow cell granularity, and performs weighted polling allocation of the paths based on the state confidence and congestion probability of each path, and transmits the flow cells on each path according to the weight ratio.

2. The load balancing scheduling method for data center networks according to claim 1, characterized in that: The cross-validation and confidence assessment module introduces time decay weights during the dynamic weight fusion process, calculates short-term weighted averages for active detection values ​​and passive observation values ​​respectively, and calculates the state confidence of each path based on the difference between the two weighted averages. When the state confidence is lower than a preset threshold, the cross-validation and confidence assessment module triggers a secondary verification mechanism, temporarily increases the active detection frequency and suspends the use of this path for data transmission until the difference between the two weighted averages converges to a preset range.

3. The load balancing scheduling method for data center networks according to claim 1, characterized in that: The cross-validation and confidence assessment module calculates the probability that each path is in a congested state based on the distribution characteristics of the round-trip delay of each path within the historical time window. The congestion probability is calculated using an adaptive threshold coefficient, which is dynamically adjusted according to the path misjudgment rate. The path state probability distribution includes the congestion probability of each path.

4. The load balancing scheduling method for data center networks according to claim 1, characterized in that: The iterative flow cell granularity optimization module uses a multi-dimensional temporal feature fusion and path state field modeling method to determine the optimal flow cell granularity under the current network state, specifically including: The first sub-step involves extracting multi-dimensional time-series features from the round-trip delay sequences of each path within a preset time window. These multi-dimensional time-series features include fluctuation features, trend features, periodic features, and burst features. The second sub-step involves modeling the data center network topology as a graph structure, where nodes are switches and edges are paths. The multi-dimensional temporal features of each path are used as the initial feature vectors of the edges. The feature vectors are then spatiotemporally modeled using a temporal graph convolutional network to output the state embedding vectors of each path. The third sub-step involves constructing a path state field matrix from the state embedding vectors of all paths, and then performing online clustering on the path state field matrix to automatically divide the paths into stable good path clusters, stable bad path clusters, and oscillating clusters. The fourth sub-step is to determine the optimal flow cell size in the current network state based on the clustering results of the path state field.

5. A load balancing scheduling method for data center networks according to claim 4, characterized in that: The iterative flow cell granularity optimization module introduces a dynamic game-theoretic path state evaluation mechanism in the process of determining the optimal flow cell granularity. It constructs a two-layer game model between the sender and the path set, and generates the state confidence of each path in the game equilibrium. In the two-layer game model, the sender's strategy is to select a set of probe weight distributions, and the path set's strategy is to select to represent the true state or a masquerade state. The sender updates its posterior belief in the true state of the path based on historical observations. At the mixed strategy Nash equilibrium point, the sender allocates more probe resources to the path with high belief uncertainty. The state confidence of each path is defined by the inverse of the variance of the sender's posterior belief in the true state of that path.

6. The load balancing scheduling method for data center networks according to claim 4, characterized in that: The iterative flow cell granularity optimization module uses a generative adversarial network (GAN) to optimize the flow cell granularity. The GAN includes a granularity generator and a flow completion time discriminator. The granularity generator takes the current path state field matrix and the previous optimal flow cell granularity as input and outputs a candidate granularity distribution. The flow completion time discriminator takes the path state field matrix and the flow cell granularity as input and outputs the predicted flow completion time at that granularity. The granularity generator and the flow completion time discriminator are iteratively optimized through adversarial training. The goal of the granularity generator is to generate a granularity distribution that makes the flow completion time predicted by the flow completion time discriminator as small as possible. The goal of the flow completion time discriminator is to accurately predict the flow completion time under different granularities. The training data comes from historical scheduling records and small-volume test data.

7. A load balancing scheduling method for data center networks according to claim 6, characterized in that: The iterative flow cell granularity optimization module employs a self-supervised adversarial mechanism during the training of the generative adversarial network. In each round of adversarial training, the flow completion time discriminator is first fixed to train the granularity generator, ensuring that the weighted average granularity of the granularity distribution output by the granularity generator reaches the minimum predicted flow completion time on the flow completion time discriminator. Then, the granularity generator is fixed, and the newly generated granularity is tested in a real environment with a small flow rate. The tested data is added to the training set to update the flow completion time discriminator. This process is repeated until the granularity generated by the granularity generator converges to a stable value in both the flow completion time discriminator prediction and the actual test.

8. A load balancing scheduling method for data center networks according to claim 1, characterized in that: The dynamic round-robin scheduling module employs an adaptive reconstruction mechanism based on transmission strategies. This mechanism includes a predefined library of scheduling strategy base functions and a meta-learner. The library contains various scheduling strategy base functions, each mapping the state confidence, congestion probability, current flow cell granularity, and flow remaining size of each path to a scheduling weight vector. The meta-learner takes the recent path state field sequence as input and outputs the combination coefficients of each scheduling strategy base function. The dynamic round-robin scheduling module performs a weighted summation of the outputs of multiple scheduling strategy base functions based on the combination coefficients to obtain the final scheduling weights for each path.

9. A load balancing scheduling method for data center networks according to claim 8, characterized in that: The meta-learner employs a neural network structure combining a long short-term memory network and an attention mechanism. It captures the temporal evolution pattern of the path state field sequence through the long short-term memory network and selects a scheduling strategy basis function combination that matches the current pattern through the attention mechanism. The meta-learner is pre-trained on a simulation environment or historical traffic logs through reinforcement learning. After actual deployment, after each flow transmission is completed, the path state field sequence, the selected basis function combination, and the transmission performance are recorded as new samples to update the meta-learner online, thereby realizing the continuous evolution of the scheduling strategy.

10. A load balancing scheduling method for data center networks according to claim 1, characterized in that: During the weighted round-robin allocation process, when the congestion probability of each path changes or the state confidence level falls below a preset threshold, the dynamic round-robin scheduling module feeds back the scheduling results to the cross-validation and confidence evaluation module. Based on the out-of-order occurrence of flow cells and throughput data on each path after actual scheduling, the cross-validation and confidence evaluation module corrects the time decay coefficient in dynamic weight fusion and the adaptive threshold coefficient in congestion probability calculation, forming a closed-loop optimization link of perception, decision-making, execution, and re-perception.

Images