A method and system for dynamic reconfiguration of a data center optoelectronic hybrid switching network

Abstract

The application discloses a kind of data center photoelectric hybrid exchange network dynamic reconfigurable method and system, belong to data center network technical field.The application includes historical traffic data collection module, traffic size prediction module, traffic filtering module, traffic scheduling module.The application collects the traffic load information of each network node, trains long short-term memory network to obtain the traffic prediction value of next time node, and the prediction value is provided as the state information of long short-term memory network to shunt training to obtain distributed dynamic threshold, and is divided into arrival flow category and transmission switch by dynamic threshold, in the traffic data transmission stage, the property of k-double random matrix is used, the working time of single transmission circuit is maximized, and the optimal scheduling of traffic transmission is completed;Even if there is sudden traffic in the data transmission process, traffic offloading is quickly completed by updating network state information, efficient network resource allocation is realized, traffic transmission completion time is minimized, and optical link utilization is improved.

Description

Technical Field

[0001] This invention belongs to the field of data center network technology, and in particular relates to a method for realizing dynamic reconfigurability of data center optoelectronic hybrid switching networks. Background Technology

[0002] Data centers not only serve as the supporting platform for internal network architecture but also as the business operation platform supporting internet applications. With the rapid development of information technologies such as cloud computing, big data, and artificial intelligence, as well as the digital transformation of traditional industries, data centers are developing towards ultra-large scales, and correspondingly, they will face increasing traffic transmission pressure. How to improve the transmission efficiency of data center networks is currently a key research focus. Optical interconnect networks based on optical circuit switches (OCS) combine the advantages of low latency and high throughput of electrical switching with the advantages of high bandwidth and low energy consumption of optical switching, and offer flexible configuration.

[0003] Within a data center network (DCN), flows with low bandwidth requirements and short durations are called "mouse flows" (non-dominant data flows), while flows with high bandwidth requirements and long durations are called "elephant flows" (dominant data flows). To fully leverage the high bandwidth advantage of the OCS-based architecture, and also to avoid the impact of optical circuit switch reconfiguration delays on mouse flow transmission, dominant data flows are assigned to optical circuit switches for transmission, while non-dominant data flows are assigned to electrical packet switches. Previous work mostly used a fixed threshold to classify flow types, and then scheduled them separately through different switching components. The network topology of a reconfigurable optical switching architecture may change over time, and it is difficult to handle the highly dynamic network conditions in the current DCN by differentiating and scheduling all arriving flows based on a fixed threshold.

[0004] The current high-speed development of the Internet pursues higher bandwidth and lower latency, which requires the support of more switches, network ports, and links, and thus means higher costs. Most network applications concentrate their computation and transmission within a relatively small node range, leading to network underload and a certain degree of network resource waste. While proportionally reducing the bandwidth of the network aggregation and core layers can alleviate underload to some extent, reducing costs and engineering complexity and improving the overall utilization of network resources, it may also significantly degrade network communication performance, thus triggering network overload. To address both underload and overload issues, current optical switching network technology can physically reallocate and schedule network resources according to specific communication needs, allowing for the construction of interconnected networks that flexibly meet user communication requirements with fewer resources.

[0005] Data center network traffic exhibits strong bursts. At the hardware level: current high-speed network interface cards (NICs) typically employ load balancing designs to reduce CPU overhead and ensure extremely high link bandwidth. However, this also results in a large number of data packets being sent in a very short time, creating bursts of traffic. At the application level: data centers generally use a partitioned aggregation concurrent communication model. After a request is sent to a server cluster, the cluster synchronously returns data and aggregates to generate a response, further amplifying the burst intensity of parallel data flows. Traffic routing and scheduling algorithms used on the internet are inefficient in data center networks, leading to wasted bandwidth resources. From a queuing theory perspective, bursts of traffic also increase queuing latency and packet loss rates, reducing network throughput. Summary of the Invention

[0006] To address the problems of low link resource utilization and difficulty in meeting bursty service transmission requirements caused by dynamic network changes in data center hybrid optical-electrical switching networks, this invention aims to provide a method and system for dynamically reconfigurable hybrid optical-electrical switching networks in data centers. The method involves collecting traffic load information from each network node, training a Long Short-Term Memory (LSTM) network to obtain traffic prediction values ​​for the next time node, and then using the LSM network's state information to train a splitter to obtain distributed dynamic thresholds. These dynamic thresholds are used to classify arrival flow categories and transmission switches. During the traffic data transmission phase, the properties of k-double random matrices are utilized to maximize the single transmission circuit's operating time, achieving optimal traffic transmission scheduling. Even if bursty traffic occurs during data transmission, timely updates to network state information can quickly offload the traffic, achieving efficient network resource allocation, minimizing traffic transmission completion time, and improving optical link utilization.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] In a first aspect, the present invention discloses a method for dynamically reconfigurable optoelectronic hybrid switching networks in data centers, comprising the following steps:

[0009] Step 1: Collect historical traffic data. The data types of historical traffic data include various service information in the data center network. This historical traffic data is used to characterize the actual traffic load at each historical time point. Before training, the historical traffic data is preprocessed, including data normalization and data dimensionality enhancement. By adding Long Short-Term Memory (LSTM) networks to each transmission link in the data center network, traffic prediction is performed using the trained LSM networks to predict relevant information about traffic about to enter the data center network for transmission. This relevant information includes the path information and size of the traffic to be transmitted. Collect the current data network topology information; this topology information includes the establishment and dismantling information of each link in the optoelectronic hybrid switching network. Collect historical traffic data for each link and node in the virtual network.

[0010] A long short-term memory (LSTM) network model is trained and constructed using deep learning strategies. This model then predicts the traffic load information of each network node at the next time point. The actual traffic load information of each network node at multiple historical time points is used as input, and the traffic load information at the next time point is used as output, thus training the entire LSM network model.

[0011] After each iteration of deep learning, the Long Short-Term Memory (LSTM) network model obtains the predicted traffic load of each network node. The predicted value is compared with the actual traffic load, and the error between the predicted value and the actual traffic load is calculated. The parameters of the LSM network model are corrected based on the error value until the error of the trained LSM network model meets the preset accuracy requirements. The trained LSM network model is then obtained, and the traffic load information of each network node is predicted again based on the trained LSM network model.

[0012] Based on the predicted traffic load information of each network node, a load information table of traffic to be transmitted is established, which facilitates more efficient traffic scheduling, allocation and establishment of relevant links in the data center network, thereby improving the utilization of transmission links and data transmission efficiency.

[0013] Step Two: Introducing a dynamic threshold into data center traffic scheduling within a hybrid optoelectronic switching network architecture. A distributed dynamic threshold is obtained by training a traffic splitter using a deep deterministic gradient strategy (DDPG). This dynamic threshold in traffic scheduling means that after traffic reaches the top switch, the threshold is used as a standard to divide the arriving flow into "elephant flows" and "mouse flows." "Mouse flows" are assigned to electrical data packet switches for transmission, while "elephant flows" are assigned to optical switches for transmission. This minimizes the data transmission completion time, thereby improving link utilization and traffic transmission efficiency. In this invention, the dynamic threshold is a distributed dynamic threshold.

[0014] Based on the distributed dynamic threshold, a flow splitter is trained using the Deep Deterministic Gradient Policy (DDPG) to obtain the distributed dynamic threshold, mapping the collected network state information to the threshold for splitting arriving flows. The threshold decision problem is constructed as a Deep Deterministic Gradient Policy model, and specific states, actions, and rewards are designed for this model. The details are as follows:

[0015] The state of the deep deterministic gradient strategy model originates from the source top switch and the target top switch. The state includes the maximum flow capacity and total demand through the electrical switches, the maximum flow capacity and total demand through the optical switches, and the current total traffic demand of the entire data center network. This state information reflects the bottleneck information of each optoelectronic switch device and the current load capacity of the data center network.

[0016] The action of the deep deterministic gradient policy model represents the distributed threshold of each traffic between the current source top switch and the target top switch.

[0017] The reward of the deep deterministic gradient policy model is defined as minimizing the average time to complete traffic transmission.

[0018] Based on the results of training the deep deterministic gradient strategy, the threshold table in each top switch is updated. When traffic arrives, the top switch classifies the type of the current arriving traffic according to the arrival flow load information obtained in the aforementioned traffic prediction process and the stored threshold table. The "mouse flow" is assigned to the electrical packet switch for transmission, and the "elephant flow" is assigned to the optical switch for transmission, thereby improving link utilization and traffic transmission efficiency.

[0019] Step 3: Based on the distributed dynamic threshold results trained using the deep deterministic gradient policy model in Step 2, the arriving flows are first divided into corresponding switches. Then, combined with the arriving flow load information obtained after traffic prediction in Step 1, the optimal traffic scheduling strategy within the switch is constructed.

[0020] The optimized traffic scheduling strategy transforms the information of arriving flows within each switch into a traffic demand matrix. This matrix is ​​then filled to form a k-double random matrix (where the sum of the row and column elements is a constant k). Finally, it is greedily sliced ​​to maximize the amount of transmitted traffic, maximize the duration of a single transmission circuit configuration, and minimize the delay of transmission circuit reconfiguration, thus achieving optimal traffic scheduling at the switch level. This optimizes traffic transmission completion time, improves the transmission efficiency of arriving flows within the switch, and further enhances link utilization.

[0021] Step 4: In the network topology, the top switch of the access layer is connected to multiple servers, and the top switch is connected to multiple core layer electrical packet switches (EPS) and optical circuit switches (OCS). A link is established between the source top switch and the destination top switch to complete data transmission. After type classification using the dynamic threshold obtained in Step 2, "mouse flows" are transmitted through electrical packet switches, and "elephant flows" are transmitted through optical packet switches. The same arriving flow cannot be transmitted through two different types of switches simultaneously. Distributed dynamic thresholds support finer-grained flow scheduling, improving transmission efficiency and resource utilization.

[0022] When transmitting data between top switches, the link status of the optical circuit switch determines whether there is an optical path available for data transmission. Similarly, the offloading strategy for burst traffic aims to allow "mouse streams" to be transmitted through electrical packet switches and "elephant streams" to be transmitted through optical circuit switches.

[0023] When burst traffic occurs, the dynamic threshold in step two is used to detect changes in the current network state. At this time, the information about the arriving flow in the top switch is updated. For burst traffic, the path planning method based on the Iterative Deepening Algorithm (IDA) is used to continuously update the path information, calculate the remaining bandwidth and path cost by traversing all nodes in the network, and thus find the optimal path to complete the offloading of burst traffic.

[0024] In another aspect, the present invention also discloses a dynamically reconfigurable system for a data center optoelectronic hybrid switching network, used to implement the method for dynamically reconfiguring the data center optoelectronic hybrid switching network. The dynamically reconfigurable system for a data center optoelectronic hybrid switching network includes a historical traffic data collection module, a traffic size prediction module, a traffic filtering module, and a traffic scheduling module.

[0025] The historical traffic data collection module is used to collect and store historical traffic data of network nodes. The historical traffic data includes the historical traffic of various services in the data center network, which is used to characterize the actual traffic load at multiple historical time points.

[0026] The traffic volume prediction module is used to predict the traffic load information of each network node at the next time node based on the historical traffic data stored in the aforementioned historical traffic data collection module, using a long short-term memory network model and deep learning training.

[0027] The traffic filtering module is used for classifying arriving flows using distributed dynamic thresholds. It includes a network state collector, a network simulator, and a deep learning reinforcement agent. The network state collector collects the state information of the currently running network and then provides this information to the network simulator and the deep learning reinforcement agent. Based on the deep learning results, it generates thresholds corresponding to the arriving flows and then distributes the flows accordingly.

[0028] The traffic scheduling module is used to fill the traffic demand matrix after the arrival flow is divided according to the aforementioned dynamic threshold, to obtain a k-double random demand matrix, and then greedily slice the obtained matrix to obtain the traffic scheduling scheme with the longest circuit configuration time, so as to achieve optimal scheduling for the "mouse flow" assigned to the electrical switch and the "elephant flow" assigned to the optical switch.

[0029] The traffic offloading module is used to handle the offloading of bursty traffic in the network. It uses a path planning method based on the iterative deepening algorithm IDA to continuously update path information, calculate the remaining bandwidth and path cost by traversing all nodes in the network, thereby finding the optimal path and completing the offloading of bursty traffic.

[0030] Beneficial effects:

[0031] 1. The present invention discloses a method and system for dynamic reconfiguration of a data center optoelectronic hybrid switching network. It introduces a long short-term memory network and uses a trained long short-term memory network to predict traffic. The prediction of business resource demand can greatly reduce the waste of network resources. By accurately predicting the traffic size of virtual network links for data center services and the connectivity status of each link, better scheduling and reasonable allocation of network and computing resources can be achieved to improve network link utilization.

[0032] 2. This invention discloses a method and system for dynamically reconfigurable data center optoelectronic hybrid switching networks. It introduces dynamic thresholds: upon arrival of traffic, the top-level switch distinguishes and allocates transmission switches based on the flow size information notified by the application layer and a trained threshold table. "Elephant flows" (flows with high bandwidth requirements and long durations) are allocated to optical circuit switches, while "mouse flows" (flows with low bandwidth requirements, short durations, and latency sensitivity) are allocated to electrical data packet switches, optimizing flow completion time as much as possible. Compared to global thresholds, distributed dynamic thresholds support finer-grained flow scheduling, improving transmission efficiency and resource utilization.

[0033] 3. This invention discloses a method and system for dynamically reconfigurable data center optoelectronic hybrid switching networks, which optimizes traffic scheduling for different types of arriving flows after segmentation. The traffic demand matrix is ​​filled to obtain a k-double-random demand matrix, and then the resulting matrix is ​​greedily sliced ​​to obtain the traffic scheduling scheme with the longest circuit configuration time, achieving optimal traffic scheduling and realizing the goals of shortening traffic transmission completion time, improving transmission link utilization, and increasing transmission efficiency.

[0034] 4. The present invention discloses a method and system for dynamic reconfiguration of a data center optoelectronic hybrid switching network. It utilizes a path planning method based on an iterative deepening algorithm to continuously update path information, calculate remaining bandwidth and path cost by traversing all nodes in the network, thereby finding the optimal path, completing the offloading of bursty traffic, and thus offloading and scheduling bursty traffic in the network, thereby reducing packet loss rate, reducing queuing delay, and improving network resource utilization. Attached Figure Description

[0035] To more clearly and intuitively illustrate the technical solutions in the embodiments of the present invention or the prior art, the example drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 The flowchart is for the traffic prediction algorithm provided by this invention.

[0037] Figure 2 This is a schematic diagram of the traffic filtering module in the distributed dynamic threshold provided by the present invention.

[0038] Figure 3 This is a schematic diagram of the traffic matrix scheduling algorithm selected in this invention.

[0039] Figure 4 This is a schematic diagram of the path planning method based on the iterative deepening algorithm selected in this invention. Detailed Implementation

[0040] To better illustrate the purpose and advantages of the present invention, the invention will be further described below in conjunction with the accompanying drawings and examples.

[0041] Example 1:

[0042] Before optimizing traffic scheduling algorithms and leveraging the reconfigurable flexibility of hybrid optoelectronic switching network structures, it is crucial to consider the dynamic nature of demand in the network environment and to predict resource requirements and traffic load in the network. This can be considered a prerequisite for improving the utilization rate of data center network resources.

[0043] Figure 1 The flowchart of the traffic prediction algorithm provided by this invention is as follows: Figure 1 As shown, traffic prediction is introduced into the data center optoelectronic hybrid switching network. This embodiment discloses a method for dynamically reconfigurable data center optoelectronic hybrid switching networks, and the specific implementation steps are as follows:

[0044] Step 101: Collect historical traffic data from each network node; preprocess the collected historical data, mainly including data normalization and dimensionality increase processing.

[0045] Historical traffic data reflects the historical network status information of a network node, encompassing different services within the data center network. Normalization is performed because historical traffic data may differ in dimensionality and units, hindering deep learning training. Normalizing data from different dimensions to a common value range improves training convergence speed. Dimensionality upscaling is necessary because subsequent training iterations require comparing predicted values ​​with actual values. The original traffic load change is a two-dimensional time-varying sequence; dimensionality upscaling transforms it into a three-dimensional time-series dataset for easier comparison.

[0046] The preprocessing of the aforementioned collected historical traffic data specifically includes:

[0047] The "scaling normalization" method is used to normalize the traffic data of the nodes in the dataset, normalizing the value of each feature to the range [0,1]. The normalized feature value x can be expressed as:

[0048]

[0049] in: Let x be the normalized value. (i) For the raw traffic load data collected from the network, max i (x (i) ) and min i (x (i) ) represent the maximum and minimum values, respectively.

[0050] Step 102: Based on the historical traffic data collected from each network node, train a long short-term memory network model in deep learning to obtain the predicted value of the traffic load information of each network node at the next time node.

[0051] The deep learning training process specifically includes: taking the historical traffic data of each network node collected in step 101 as input, and outputting the traffic load of these network nodes at the next time node, and training through a Long Short-Term Memory model. Figure 1As shown: During each iteration of training, the predicted traffic load information of each network node needs to be compared with the actual traffic load information. Based on the error between the two, the parameters of the Long Short-Term Memory network model used in deep learning are adjusted until the training error requirement is met, and the iteration continues to complete the model training. Finally, the predicted traffic value of each network node is output.

[0052] The Long Short-Term Memory (LSTM) network used in the aforementioned deep learning is a classic and commonly used Recurrent Neural Network (RNN), primarily used to handle continuous time series problems. This is because when an RNN calculates the content of the current time point, information from previous time points is also considered. However, as the number of time points involved in the calculation increases, the relevant features and influences of more distant time points gradually diminish, which is not conducive to finding the optimal solution. By combining it with an LSTM network, relying on its special structure, it can solve many problems that are difficult for RNN learning algorithms to address, especially for predicting the values ​​of events with long time intervals.

[0053] Figure 2 This is a schematic diagram of the traffic filtering module in the distributed dynamic threshold provided by the present invention, as shown below. Figure 2 As shown, the entire traffic filtering module can be viewed as a traffic splitter, with the network state collector serving as the central controller. It consists of a network state collector, a network simulator, and a DRL agent. Within the splitter, the network state collector aims to gather operational status information of the data center network, including current network topology information, and then provides this information to the network simulator and the DRL agent. The core of this process lies in the design of the DRL agent.

[0054] The purpose of the DRL design in the above-mentioned flow splitter is to schedule the arriving flow based on the information collected in the network state collector. Its output is a threshold table of the flow between a pair of top switches (ToR) (source ToR to destination ToR). All thresholds are calculated by a deep reinforcement learning (DRL) agent, which can reduce the difficulty of training the DRL agent and also reduce the complexity of the deep learning model.

[0055] Step 201: The network state collector in the splitter collects the operational status information of the data center network, which also includes the current network topology information, as mentioned above. Figure 1 The status information of each network node is obtained through traffic prediction.

[0056] Step 202: Train the DRL agent to map the network state information collected from the network state collector to the traffic threshold. Considering that we use a distributed dynamic threshold in this invention, which is a continuous variable, we choose Deep Deterministic Policy Gradients (DDPG) as the DRL model to handle this problem. It is a model algorithm that relies on deep neural networks (DNNs) and is well-suited for learning state-action policies in a continuous action space.

[0057] The threshold decision problem is constructed as a DDPG problem, and training states, actions, and rewards are assigned to it.

[0058] Status information includes: the maximum operating traffic and maximum total traffic demand between the source ToR and the target ToR via electrical switches; the maximum operating traffic and maximum total traffic demand via optical links; this can reflect bottleneck information of electrical switches and optical switches; status information also includes the current total traffic demand of the data center network, which can reflect the global statistics of the current network load.

[0059] The action information includes m flow size thresholds obtained by training the DDPG model under a given period P.

[0060] The reward information is defined as: minimizing the average completion time of traffic.

[0061] like Figure 2 As shown: For DDPG training, the Bellman equation can be used to estimate the reward value y for each action-state pair. t for:

[0062] y t =r(s t ,a t )+γQ(s t+1 ,π(s t+1 ;θ π );θ Q )

[0063] Where r(s) t ,a t ) represents the state s t Next, take action a t The actual reward received, γ is the discount factor for future rewards, and θ π and θ Q These are the parameters selected for the DDPG model.

[0064] After training and calculating the reward, state transition samples are stored in different buffers (positive buffer, negative buffer, and normal buffer) according to the reward value. This invention does not use the random sampling method in traditional DDPG during training. Instead, it samples 50%, 25%, and 25% from the positive buffer, negative buffer, and normal buffer, respectively, as the training batch size. In this way, the DRL agent can perform more instructive sampling, thereby training the DDPG model more effectively and ultimately finding the optimal flow partitioning strategy.

[0065] Step 203: After obtaining the threshold table through training, store it in ToR, such as... Figure 2 As shown: After the transmission request of the incoming stream arrives from the server, the incoming stream is divided into groups according to the threshold table in the ToR. The "mouse stream" with a traffic load below the threshold is assigned to the electrical switch for transmission, and the "elephant stream" with a traffic load above the threshold is assigned to the optical switch for transmission.

[0066] Figure 3 This is a schematic diagram of the traffic demand matrix scheduling algorithm selected in this invention, as shown below. Figure 3 As shown: The arrival flow information of the network is represented in the form of a traffic demand matrix. The specific traffic scheduling method is as follows:

[0067] Step 301: Fill the demand matrix. In a real network environment, the traffic demand matrix is ​​not necessarily a k-double random matrix, i.e., a matrix where the sum of its row and column elements is a fixed value k. First, the traffic demand matrix needs to be filled so that the sum of its row and column elements is a fixed value. For example... Figure 3 As shown: In the embodiment, the traffic demand matrix needs to be filled with a double random matrix with a k value of 98, so that the sum of its row and column elements is a constant value of 98.

[0068] Step 302: After filling the demand matrix, it needs to be sliced ​​to obtain the transmission path with the highest efficiency. For the filled k-double random matrix, the highest power of 2 closest to the largest element in the matrix is ​​selected as the slicing threshold, such as... Figure 3 As shown: In this embodiment, before the first slice, the largest power of 2 is 64, so 64 is used as the threshold for the first slice. After slicing, the smaller value of 69 among the elements is selected as the transmission size after the first slice, thus obtaining the transmission path matrix with the largest element, which is the transmission path matrix with the longest circuit configuration duration. This minimizes the adverse effects of the delay caused by the reconfiguration of the circuit on traffic transmission and improves transmission efficiency.

[0069] Step 303: After performing maximum slicing on the filled demand matrix, continue with the slicing method in Step 302 to iteratively slice the remaining flow matrix. For example... Figure 3 The embodiment states that when slicing for the second time, the maximum power of 2 can be 8. Using 8 as the threshold for the second slicing, two sets of transmission path matrices with transmission sizes of 11 and 10 can be sliced ​​out.

[0070] Step 304: As Figure 3 As shown, in the embodiment, after two slicing operations, the elements of the remaining traffic demand matrix are relatively small. When the largest element of the remaining demand matrix approaches ten times the reconfiguration time of the transmission circuit, slicing is stopped, and the remaining traffic demand is handed over to the optical switch for transmission, reducing the delay caused by multiple reconfigurations of the transmission circuit.

[0071] Figure 4 A schematic diagram of the path planning method based on iterative deepening algorithm selected for this invention is shown below: Figure 4 As shown, when burst traffic occurs in the network environment, this algorithm is used to plan the path for the burst traffic, thereby completing the offloading and scheduling of the burst traffic.

[0072] Step 401: Locate the starting point of the burst traffic.

[0073] Step 402: Starting from the starting node, explore its possible transmission paths through depth-first search and store them.

[0074] Step 403: During the search process, based on the path information that has been explored, evaluate whether there is a better path to offload and transmit bursty traffic. The evaluation criteria include the remaining bandwidth of the path and the transmission cost.

[0075] Step 404: Continuously iterate and update the transmission path until the optimal transmission path is obtained, thus completing the offloading and scheduling of bursty traffic.

[0076] This embodiment discloses a dynamically reconfigurable system for a data center optoelectronic hybrid switching network, used to implement the method for dynamically reconfiguring a data center optoelectronic hybrid switching network. The dynamically reconfigurable system for a data center optoelectronic hybrid switching network includes a historical traffic data collection module, a traffic size prediction module, a traffic filtering module, and a traffic scheduling module.

[0077] The historical traffic data collection module is used to collect and store historical traffic data of network nodes. The historical traffic data includes the historical traffic of various services in the data center network, which is used to characterize the actual traffic load at multiple historical time points.

[0078] The traffic volume prediction module is used to predict the traffic load information of each network node at the next time node based on the historical traffic data stored in the aforementioned historical traffic data collection module, using a long short-term memory network model and deep learning training.

[0079] The traffic filtering module is used for classifying arriving flows using distributed dynamic thresholds. It includes a network state collector, a network simulator, and a deep learning reinforcement agent. The network state collector collects the state information of the currently running network and then provides this information to the network simulator and the deep learning reinforcement agent. Based on the deep learning results, it generates thresholds corresponding to the arriving flows and then distributes the flows accordingly.

[0080] The traffic scheduling module is used to populate the traffic demand matrix after the arrival flow is divided according to the aforementioned dynamic threshold, to obtain a k-double random demand matrix. Then, the resulting matrix is ​​greedily sliced ​​to obtain the traffic scheduling scheme with the longest circuit configuration time, dividing the flow into "mouse flows" assigned to electrical switches and "elephant flows" assigned to optical switches.

[0081] Achieve optimal scheduling.

[0082] The traffic offloading module is used to handle the offloading of bursty traffic in the network. It uses a path planning method based on the iterative deepening algorithm IDA to continuously update path information, calculate the remaining bandwidth and path cost by traversing all nodes in the network, thereby finding the optimal path and completing the offloading of bursty traffic.

[0083] This invention provides a method and system for dynamically reconfigurable optoelectronic hybrid switching networks in data centers. It can collect traffic load information from each network node and use this information to train and obtain traffic prediction values ​​for the next time node. This prediction is then used as network state information to train a splitter to obtain distributed dynamic thresholds, classifying incoming flows and setting transmission switches. During the traffic data transmission phase, the properties of k-double random matrices are utilized to maximize the single transmission circuit's operating time, achieving optimal scheduling of traffic transmission. Even if sudden traffic spikes occur during data transmission, timely updates to network state information can quickly offload the traffic, achieving efficient network resource allocation, minimizing traffic transmission completion time, and improving optical link utilization.

[0084] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for dynamically reconfigurable optoelectronic hybrid switching networks in data centers, characterized in that: Includes the following steps, Step 1: Collect historical traffic data. The data types of the historical traffic data include various service information in the data center network. The historical traffic data is used to characterize the actual traffic load at each historical time point. Before training, the historical traffic data is preprocessed, including data normalization and data dimensionality enhancement. By adding Long Short-Term Memory (LSTM) networks to each transmission link of the data center network, and using the trained LSM networks to predict traffic, the relevant information of the traffic about to enter the data center network for transmission is predicted. The relevant information includes the path information and size of the traffic to be transmitted; and the topology information of the current data network is collected. The topology information includes the establishment and dismantling information of each link in the optoelectronic hybrid switching network; Collect historical traffic data for each link and node in the virtual network; Step Two: Introduce dynamic thresholds in data center traffic scheduling within a hybrid optoelectronic switching network architecture. A distributed dynamic threshold is obtained by training a traffic splitter using a deep deterministic gradient strategy (DDPG). Introducing dynamic thresholds in traffic scheduling means that after traffic reaches the top switch, the threshold is used as a criterion to divide the arriving flow into "elephant flows" and... "Mouse stream" is assigned to an electrical data packet switch for transmission, while "elephant stream" is assigned to an optical switch for transmission; the dynamic threshold is a distributed dynamic threshold; Step 3: Using the distributed dynamic threshold results trained by the deep deterministic gradient policy model, the arriving flow is first divided into the corresponding switches. Then, combined with the arriving flow load information obtained after traffic prediction in Step 1, the optimal traffic scheduling strategy within the switch is constructed. Step 4: In the network topology, the top access layer switch is connected to multiple servers, and the top switch is connected to multiple core layer electrical packet switches (EPS) and optical circuit switches (OCS). A link is established between the source top switch and the destination top switch to complete data transmission. After classifying the data type using the dynamic threshold obtained in Step 2... "Mouse streams" are transmitted via electrical packet switches, while "elephant streams" are transmitted via optical packet switches. Distributed dynamic thresholds support finer-grained flow scheduling, improving transmission efficiency and resource utilization.

2. The method for dynamically reconfigurable data center optoelectronic hybrid switching network as described in claim 1, characterized in that: In step one, A long short-term memory network model is trained and constructed using a deep learning strategy. This model predicts the traffic load information of each network node at the next time node. The actual traffic load information of each network node at multiple historical time nodes is used as input, and the traffic load information of the next time node is used as output, thereby training the entire long short-term memory network model. After each iteration of deep learning, the Long Short-Term Memory (LSTM) network model obtains the predicted traffic load of each network node. The predicted value is compared with the actual traffic load, and the error between the predicted value and the actual traffic load is calculated. The parameters of the LSM network model are corrected based on the error value until the error of the trained LSM network model meets the preset accuracy requirements. The trained LSM network model is then obtained, and the traffic load information of each network node is predicted based on the trained LSM network model. Based on the predicted traffic load information of each network node, a load information table of traffic to be transmitted is established, which facilitates more efficient traffic scheduling, allocation and establishment of relevant links in the data center network, thereby improving the utilization of transmission links and data transmission efficiency.

3. The method for dynamically reconfigurable data center optoelectronic hybrid switching network as described in claim 2, characterized in that: In step two, Based on the distributed dynamic threshold, a splitter is trained using the Deep Deterministic Gradient Policy (DDPG) to obtain the distributed dynamic threshold. The collected network state information is then mapped to the threshold for dividing the arriving flows. The threshold decision problem is constructed as a Deep Deterministic Gradient Policy model, and specific states, actions, and rewards are designed for the Deep Deterministic Gradient Policy model. The state of the deep deterministic gradient strategy model comes from the source top switch and the target top switch. The state includes the maximum flow capacity and total demand through the electrical switch, the maximum flow capacity and total demand through the optical switch, and the current total traffic demand of the entire data center network. The state information is used to reflect the bottleneck information of each optoelectronic switch device and the current load capacity of the data center network. The action of the deep deterministic gradient strategy model represents the distributed threshold of each traffic between the current source top switch and the target top switch. The reward of the deep deterministic gradient policy model is defined as minimizing the average time to complete traffic transmission. Based on the results of training the deep deterministic gradient strategy, the threshold table in each top switch is updated. When traffic arrives, the top switch classifies the type of the current arriving traffic according to the arrival flow load information obtained in the aforementioned traffic prediction process and the stored threshold table. The "mouse flow" is assigned to the electrical packet switch for transmission, and the "elephant flow" is assigned to the optical switch for transmission, thereby improving link utilization and traffic transmission efficiency.

4. The method for dynamically reconfigurable data center optoelectronic hybrid switching network as described in claim 3, characterized in that: In step three, The optimized traffic scheduling strategy transforms the information of incoming flows within each switch into a traffic demand matrix. This matrix is ​​then filled to become a k-double random matrix, which is then greedily sliced ​​to maximize the amount of transmitted traffic, maximize the duration of a single transmission circuit configuration, and minimize the delay of transmission circuit reconfiguration. This yields the optimal traffic scheduling, thereby achieving traffic scheduling at the switch level, optimizing the completion time of traffic transmission, improving the transmission efficiency of incoming flows within the switch, and further enhancing link utilization.

5. The method for dynamically reconfigurable data center optoelectronic hybrid switching network as described in claim 4, characterized in that: When burst traffic occurs, the dynamic threshold in step two is used to detect changes in the current network status. At this time, the information about the arriving flow in the top switch is updated. For burst traffic, the path planning method based on the iterative deepening algorithm IDA is used to continuously update the path information by traversing all nodes in the network, calculating the remaining bandwidth and path cost, thereby finding the optimal path and completing the offloading of burst traffic.

6. A dynamically reconfigurable system for a data center optoelectronic hybrid switching network, used to implement the method for dynamically reconfigurable data center optoelectronic hybrid switching network as described in claims 1, 2, 3, 4, or 5, characterized in that: It includes a historical traffic data collection module, a traffic size prediction module, a traffic filtering module, and a traffic scheduling module; The historical traffic data collection module is used to collect and store historical traffic data of network nodes. The historical traffic data includes the historical traffic of various services in the data center network, which is used to characterize the actual traffic load at multiple historical time points. The traffic volume prediction module is used to predict the traffic load information of each network node at the next time node based on the historical traffic data stored in the aforementioned historical traffic data collection module and by using a long short-term memory network model and deep learning training. The traffic filtering module is used to classify arriving flows using distributed dynamic thresholds. The traffic filtering module includes a network state collector, a network simulator, and a deep learning reinforcement agent. The task of the network state collector is to collect the state information of the currently running network, and then provide the network state information to the network simulator and the deep learning reinforcement agent. Based on the results of deep learning, a threshold corresponding to the current arriving flow is generated to divide the arriving flow. The traffic scheduling module is used to fill the traffic demand matrix after the arrival flow is divided according to the aforementioned dynamic threshold, to obtain a k-double random demand matrix, and then greedily slice the obtained matrix to obtain the traffic scheduling scheme with the longest circuit configuration time, and to achieve optimal scheduling for the "mouse flow" assigned to the electrical switch and the "elephant flow" assigned to the optical switch. The traffic offloading module is used to handle the offloading of bursty traffic in the network. It uses a path planning method based on the iterative deepening algorithm IDA to continuously update path information, calculate the remaining bandwidth and path cost by traversing all nodes in the network, thereby finding the optimal path and completing the offloading of bursty traffic.

Images