A multi-wavelength channel dynamic load balancing system and balancing method based on a PPO algorithm
The multi-wavelength channel dynamic load balancing system based on the PPO algorithm achieves adaptive adjustment and real-time feedback of complex network states, solves the problem of mismatch between algorithm decision-making and physical layer switching delay in existing technologies, and realizes efficient multi-wavelength channel dynamic load balancing.
Patent Information
- Application Number
- CN202610341120.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-19
- Publication Date
- 2026-11-03
AI Technical Summary
The existing software-defined network architecture of multi-wavelength channel dynamic load balancing systems cannot achieve adaptive adjustment to complex network conditions, and the routing optimization algorithm cannot provide timely and effective feedback on the decision-making effect, resulting in a time delay mismatch between algorithm decision-making and physical layer switching.
A multi-wavelength channel dynamic load balancing system based on the PPO algorithm is adopted, including a network monitoring layer, an intelligent decision-making layer, and a photonic execution layer. Through the collaborative closed-loop design of traffic monitoring, congestion status determination, PPO algorithm decision-making, and photonic execution layer, the system collects port optical power and buffer frame count in real time, generates optimal migration decisions, and realizes dynamic switching and control of wavelength channels.
It effectively alleviates channel congestion, maximizes bandwidth utilization, enables adaptive adjustment of dynamic traffic, quickly reduces the load on congested ports, avoids communication blockage, improves migration efficiency and accuracy, and supports the efficient operation of data centers and on-chip optical interconnect systems.
Smart Images
Figure CN123035389A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a multi-wavelength channel dynamic load balancing system and method, and more particularly to a multi-wavelength channel dynamic load balancing system and method based on the PPO algorithm. Background Technology
[0002] With the growth of cloud computing and artificial intelligence training tasks, optical interconnect networks have become a critical infrastructure for high-performance data centers. Traditional static wavelength allocation mechanisms are prone to problems such as port congestion, low wavelength utilization, and increased communication latency when facing dynamic traffic fluctuations. In existing multi-wavelength channel dynamic load balancing systems, although the software-defined networking (SDN) architecture can achieve centralized control, it still relies on rule-triggered or threshold-based mechanisms and cannot achieve adaptive adjustment to complex network states. In addition, routing optimization algorithms are mostly used for electronic layer decision-making and fail to form a closed-loop control with photonic hardware micro-ring resonators (MRRs) and Mach-Zehnder interferometer (MZI) switches, which cannot provide timely and effective feedback on decision-making effects, resulting in latency mismatches between algorithm decisions and physical layer switching. In summary, existing multi-wavelength channel dynamic load balancing systems all respond to network changes through preset logic and cannot dynamically adjust strategies based on real-time traffic fluctuations and hardware execution effects. Summary of the Invention
[0003] The purpose of this invention is to solve the technical problem that the software-defined network architecture of existing multi-wavelength channel dynamic load balancing systems cannot achieve adaptive adjustment to complex network states, and its routing optimization algorithm cannot provide timely and effective feedback on decision effects, resulting in a time delay mismatch between algorithm decision and physical layer switching. The invention provides a multi-wavelength channel dynamic load balancing system and balancing method based on the PPO algorithm.
[0004] To achieve the above objectives, the technical solution provided by the present invention is as follows:
[0005] A multi-wavelength channel dynamic load balancing system based on the PPO algorithm is characterized by including a network monitoring layer, an intelligent decision-making layer, and a photonic execution layer.
[0006] The network monitoring layer includes a topology graph module, a traffic monitoring module, and a congestion status determination module; the intelligent decision-making layer includes a PPO algorithm decision-making module and a decision command issuance module; the photonic execution layer includes an MRR-MZI switch array, a thermo-optical tuner, and an electro-optical tuner.
[0007] The MRR-MZI switch array includes multiple input ports and multiple corresponding output ports. Each input port is used to receive one or more optical signals. The MRR-MZI switch array is used to perform phase modulation on optical signals of different wavelengths in separate channels before outputting them.
[0008] The topology association graph module is used to construct an association graph of ports, wavelengths, and links, and to label the port hardware characteristic parameters.
[0009] The first input terminal of the traffic monitoring module is connected to the first output terminal of the topology association map module, and is used to receive port hardware characteristic parameters. Its acquisition port collects the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue in real time, and is used to calculate the bandwidth utilization of each input port of the MRR-MZI switch array in combination with the port hardware characteristic parameters.
[0010] The input terminal of the congestion status determination module is connected to the output terminal of the traffic monitoring module. It is used to determine whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. Its two output terminals are respectively connected to the second input terminal of the traffic monitoring module and the first input terminal of the PPO algorithm decision module. It is used to output the non-congestion status command to the traffic monitoring module for continuous monitoring and output the congestion status command to the PPO algorithm decision module to start the decision process.
[0011] The second input of the PPO algorithm decision module is connected to the second output of the topology association graph module. It is used to receive the association graph, and through the near-end policy update mechanism of the PPO algorithm, combined with the association graph, real-time congestion status and reward mechanism, adjust the policy network parameters to generate the optimal migration decision.
[0012] The input end of the decision instruction issuing module is connected to the output end of the PPO algorithm decision module, and is used to parse the optimal migration decision instruction into a hardware control signal that can be recognized by the photonic execution layer.
[0013] The control ports of the electro-optic tuner and the thermo-optic tuner are connected to the output of the decision command issuing module. They are used to perform thermo-optic tuning and electro-optic switching on the resonant state of the MRR-MZI switch array according to the issued hardware control signal, so as to realize the wavelength shift control of the MRR-MZI switch array.
[0014] Furthermore, the intelligent decision-making layer also includes a feedback optimization module;
[0015] The multiple input terminals of the feedback optimization module are respectively connected to the multiple output ports of the MRR-MZI switch array and the output terminal of the traffic monitoring module, and are used to evaluate the decision-making effect based on the output of the MRR-MZI switch array and the bandwidth utilization of each of its input ports.
[0016] The two outputs of the feedback optimization module are connected to the feedback terminals of the PPO algorithm decision module and the congestion status determination module, respectively, to provide feedback on the decision execution results to both modules. This enables the PPO algorithm decision module to adjust the policy network parameters in real time, while simultaneously enabling the congestion status determination module to update the congestion status of the corresponding input ports of the MRR-MZI switch array in real time.
[0017] Furthermore, the congestion status determination module uses dual thresholds to determine the congestion status, including a bandwidth occupancy rate congestion threshold and a queue length congestion threshold.
[0018] Furthermore, the bandwidth occupancy congestion threshold is 80%, and the queue length congestion threshold is 50 frames.
[0019] Furthermore, the near-end policy update mechanism of the PPO algorithm decision module is implemented through action space design and reward mechanism;
[0020] The action space is designed such that each wavelength of optical signal is independently treated as a migration unit, and the optical signal to be migrated and the target idle port are selected.
[0021] The reward mechanism is as follows: if the utilization rate of the original congested port decreases by more than or equal to the first set threshold after migration, a point reward is given; if the utilization rate of the target idle port is less than or equal to the second set threshold after migration, an additional point reward is given; if the migration time is less than the third set threshold, an additional point reward is given; if the utilization rate of the original congested port decreases by less than the fourth set threshold after migration, a point penalty is given.
[0022] Furthermore, the reward mechanism is as follows: if the utilization rate of the original congested port decreases by ≥30% after migration, 10 points will be awarded; if the utilization rate of the idle port is ≤50% after migration, an additional 5 points will be awarded; if the migration time is <1ms, an additional 3 points will be awarded; if the utilization rate of the original congested port decreases by <10% after migration, 3 points will be deducted.
[0023] Furthermore, the MRR-MZI switch array includes a first multimode interferometer, an MRR-MZI control array, and a second multimode interferometer connected in sequence according to the input-output order;
[0024] The MRR-MZI control array includes multiple sets of microring resonators, each set of microring resonators including two microring resonators with the same wavelength;
[0025] The first multimode interferometer is a beam splitter interferometer, and the second multimode interferometer is a beam combiner interferometer.
[0026] Furthermore, the electro-optic tuner achieves electro-optic switching by embedding a pin diode in a micro-ring resonator;
[0027] The thermo-optical tuner achieves thermo-optical tuning by embedding a silicon resistive microheater.
[0028] In addition, this invention also provides a multi-wavelength channel dynamic load balancing method based on the PPO algorithm, which is characterized by including the following steps:
[0029] Step 1: Build the above-mentioned multi-wavelength channel dynamic load balancing system based on the PPO algorithm;
[0030] Step 2: Multiple optical signals of different wavelengths are input into the MRR-MZI switch array through multiple input ports of the MRR-MZI switch array for channel-specific phase modulation.
[0031] Step 3: The traffic monitoring module collects the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue in real time, and combines it with the port hardware characteristic parameters input by the topology association map module to calculate the bandwidth utilization of each input port of the MRR-MZI switch array.
[0032] Step 4: The congestion status determination module determines whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. If an input port is not congested, its non-congestion status command is output to the traffic monitoring module for continuous monitoring. If an input port is congested, its congestion status command is output to the PPO algorithm decision module.
[0033] Step 5: When the PPO algorithm decision module receives the congestion status instruction, it adjusts the policy network parameters and generates the optimal migration decision by combining the correlation graph and the real-time congestion status through the near-end policy update mechanism.
[0034] Step 6: The decision instruction issuing module parses the optimal migration decision instruction into a hardware control signal that can be recognized by the photonic execution layer, and performs thermo-optical tuning and electro-optical switching on the resonant state of the MRR-MZI switch array to realize wavelength migration control of the MRR-MZI switch array.
[0035] Furthermore, in step 4, the congestion status of the corresponding input port is determined based on the following conditions:
[0036] If the bandwidth utilization rate of an input port is less than or equal to 80% and the queue length is less than or equal to 50 frames, then the input port is determined to be not congested; if the bandwidth utilization rate of an input port is greater than 80% and / or the queue length is greater than 50 frames, then the input port is determined to be congested.
[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0038] 1. This invention provides a multi-wavelength channel dynamic load balancing system based on the PPO algorithm. The system uses a traffic monitoring module to collect and calculate the bandwidth utilization of each input port of the MRR-MZI switch array in real time, thereby determining whether the corresponding input port is congested. When congestion occurs, the near-end policy update mechanism of the PPO algorithm in the PPO algorithm decision module, combined with the correlation graph, real-time congestion status, and reward mechanism, adjusts the policy network parameters to generate the optimal migration decision. Hardware control signals are then sent to the photonic execution layer to control the MRR-MZI switch array to dynamically switch wavelength channels, achieving wavelength migration control of the MRR-MZI switch array. This not only effectively alleviates channel congestion but also maximizes bandwidth utilization. It can be directly applied to different optical communication scenarios such as data centers and on-chip optical interconnect systems without significant hardware architecture adjustments.
[0039] 2. This invention provides a multi-wavelength channel dynamic load balancing system based on the PPO algorithm. Through the collaborative closed-loop design of the network monitoring layer, intelligent decision-making layer, and photonic execution layer, it constructs a full-link closed loop of "congestion monitoring, decision generation, and photonic execution," breaking through the limitations of traditional mechanisms and solving the difficulties of traditional static wavelength allocation, SDN centralized control, and routing optimization algorithms. By collecting port optical power and buffered frame count in real time, it can accurately determine congestion and trigger adaptive adjustment. At the same time, it generates an optimal migration strategy for dynamic traffic. This strategy can be accurate down to a single wavelength channel and can quickly reduce the load of congested ports to a safe threshold, effectively avoiding communication blockage.
[0040] 3. The present invention provides a multi-wavelength channel dynamic load balancing system based on the PPO algorithm, which also includes a feedback optimization module in the intelligent decision layer. This module can optimize the migration strategy in real time based on the migration effect of the feedback, effectively improving the efficiency and accuracy of the migration.
[0041] 4. The present invention provides a multi-wavelength channel dynamic load balancing method based on the PPO algorithm, which is convenient and efficient, realizes dynamic load adaptive adjustment of multi-wavelength channels, and provides technical support for the efficient operation of optical interconnect systems. Attached Figure Description
[0042] Figure 1 This is a schematic block diagram of an embodiment of a multi-wavelength channel dynamic load balancing system based on the PPO algorithm of the present invention. Detailed Implementation
[0043] To make the advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0044] like Figure 1As shown, a multi-wavelength channel dynamic load balancing system based on the PPO algorithm includes a network monitoring layer, an intelligent decision-making layer, and a photonic execution layer. The network monitoring layer includes a topology mapping module, a traffic monitoring module, and a congestion status determination module. The intelligent decision-making layer includes a PPO algorithm decision-making module, a decision command issuance module, and a feedback optimization module. The photonic execution layer includes an MRR-MZI switch array, a thermo-optical tuner, and an electro-optical tuner.
[0045] The MRR-MZI switching array includes a first multimode interferometer (MMI), an MRR-MZI control array, and a second multimode interferometer (MMI) connected in sequence according to the input and output order. The MRR-MZI control array includes N sets of microring resonators, and each set of microring resonators includes two microring resonators (MRR) with the same wavelength. The number N determines the total number of wavelengths that the MRR-MZI control array can process simultaneously. Figure 1 In the diagram, R of the same color represents the same group, and R of different colors represents microring resonators of different wavelengths. The first multimode interferometer is a beam splitter interferometer, and the second multimode interferometer is a beam combiner interferometer.
[0046] In this embodiment, the MRR-MZI switch array has two input ports and two output ports, namely, the first multimode interferometer has two input ports and the second multimode interferometer has two output ports. The two input ports are labeled as input port 1 and input port 2, and the two output ports are labeled as output port 3 and output port 4. Input port 1 and output port 3 form one channel, and input port 2 and output port 4 form another channel.
[0047] Optical signals with wavelengths λ1 and λ2 are input through input port 1, and optical signals with wavelength λ3 are input through input port 2. Taking the optical signal with wavelength λ1 as an example, the first multimode interferometer splits the optical signal with wavelength λ1 into two beams of equal power, which are then fed into the two interferometer arms of the MRR-MZI control array for phase modulation. Finally, the two phase-modulated optical signals are combined into one beam by the second multimode interferometer, which is then output through output port 3. Correspondingly, the optical signal with wavelength λ2 passes through input port 1, then sequentially through the first multimode interferometer, the MRR-MZI control array, and the second multimode interferometer, before being output through output port 3. The optical signal with wavelength λ3 passes through input port 2, then sequentially through the first multimode interferometer, the MRR-MZI control array, and the second multimode interferometer, before being output through output port 4. However, to avoid channel congestion, this invention designs a flow monitoring module, a congestion status determination module, and a PPO algorithm decision module to realize wavelength migration control of the MRR-MZI switch array. That is, if optical signals with wavelengths λ1 and λ2 are input through input port 1 and congestion occurs in the corresponding channel, one of the optical signals will be migrated from the original output port 3 to the output port 4 according to the actual situation.
[0048] The topology mapping module is used to input the connection relationships between the data center ports and the MRR-MZI switch array ports into the system, constructing a port, wavelength, and link association map to identify the target idle output ports that each input port can migrate to. This ensures that the PPO algorithm decision module knows which idle output ports the optical signals carried by congested ports can be migrated to. The topology mapping module is also used to label port hardware characteristic parameters, such as the wavelength range supported by each port (e.g., input port 1 supports wavelengths of 1535-1545nm), maximum allowable optical power, and buffer queue capacity.
[0049] The first input terminal of the traffic monitoring module is connected to the first output terminal of the topology mapping module. It receives port hardware characteristic parameters. Its acquisition port collects in real-time the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue. This data is then used to calculate the bandwidth utilization of each input port of the MRR-MZI switch array based on the port hardware characteristic parameters. In this embodiment, the maximum allowable power for both input port 1 and input port 2 is -5dBm (0.316mW).
[0050] The input of the congestion status determination module is connected to the output of the traffic monitoring module. It is used to determine whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. Its two outputs are connected to the second input of the traffic monitoring module and the first input of the PPO algorithm decision module, respectively. The outputs are used to output the non-congestion status command to the traffic monitoring module for continuous monitoring and the congestion status command to the PPO algorithm decision module to start the decision process. The PPO algorithm decision module selects the optimal migration scheme.
[0051] Preferably, this invention combines actual data and port hardware characteristics to set dual thresholds for congestion status judgment, including a bandwidth utilization congestion threshold and a queue length congestion threshold. In this embodiment, the bandwidth utilization congestion threshold is set to 80%, and the queue length congestion threshold is set to 50 frames. If the bandwidth utilization of a certain input port is less than or equal to 80%, and the queue length is less than or equal to 50 frames, then the input port is determined to be not congested, and its non-congestion status command is output to the traffic monitoring module for continuous monitoring. If the bandwidth utilization of a certain input port is greater than 80% and / or the queue length is greater than 50 frames, then the input port is determined to be congested, and its congestion status command is output to the PPO algorithm decision module.
[0052] The PPO algorithm decision module is integrated into the data center SDN controller. Its second input is connected to the second output of the topology association graph module to receive the association graph. Through the near-end policy update mechanism, combined with the association graph, the real-time congestion status input by the congestion status determination module, and the reward mechanism, it adjusts the policy network parameters to generate the optimal migration decision.
[0053] The proximal policy update mechanism of the PPO algorithm decision module is implemented through action space design and reward mechanism:
[0054] The action space design is used to define the routing switching actions that the near-end policy update mechanism can execute, ensuring that the decision is accurate down to a single channel and the action constraints. Specifically, the action space design is as follows: each wavelength of optical signal is treated as an independent migration unit. The optical signal to be migrated and the target idle port are selected. For example, to migrate an optical signal with wavelength λ1 from a congested channel, the target idle port is selected for output. When selecting the target idle port, the principle is that the migration will not cause congestion at the receiving output port. In this embodiment, output port 4 will not cause congestion after simultaneously carrying optical signals with wavelengths λ1 and λ3; therefore, output port 4 is selected as the target idle port. It is also necessary to ensure that the waveguide of the target idle port matches the resonant wavelength of λ1. In this embodiment, output port 4 supports optical signals with wavelengths between 1535-1545nm, and λ1 is 1540nm, which is supported by hardware.
[0055] The reward mechanism is used to guide the near-end policy update mechanism to prioritize the optimal migration solution. The reward mechanism is as follows: if the utilization rate of the original congested port decreases by ≥30% after migration, that is, the load of the original congested port decreases by ≥30%, indicating that the congestion problem is solved, then 10 points are awarded; if the utilization rate of the idle port is ≤50% after migration, indicating that the migration did not cause new congestion, then an additional 5 points are awarded; if the migration time is <1ms, indicating that the migration did not reduce latency, then an additional 3 points are awarded; if the utilization rate of the original congested port decreases by <10% after migration, that is, the load of the original congested port decreases by <10%, indicating that the congestion problem is not well solved, then 3 points are deducted.
[0056] The input of the decision instruction issuing module is connected to the output of the PPO algorithm decision module, and is used to parse the optimal migration decision instruction into hardware control signals that can be recognized by the photonic execution layer.
[0057] The control ports of the electro-optic tuner and the thermo-optic tuner are connected to the output of the decision command issuing module. They are used to perform thermo-optic tuning and electro-optic switching on the resonant state of the MRR-MZI switch array according to the issued hardware control signals, so as to realize the wavelength shift control of the MRR-MZI switch array.
[0058] The electro-optic tuner achieves high-speed electro-optic switching by embedding pin diodes in micro-ring resonators. Its principle is as follows: Under the control of hardware control signals, the electro-optic tuner energizes the pin diode of one of the green-marked micro-ring resonators, changing the refractive index of that micro-ring resonator. The resonant wavelength of the other green-marked micro-ring resonator corresponding to a light signal with wavelength λ2 is 1537nm. At this time, the phase difference of the light signal with wavelength λ2 in the two interference arms is π. According to the interference principle, the light signal with wavelength λ2 is input from input port 1 and output from output port 3, thus maintaining the "bar" state. Similarly, the electro-optic tuner energizes the pin diode of one of the yellow-marked micro-ring resonators, and the resonant wavelength of the other yellow-marked micro-ring resonator corresponding to a light signal with wavelength λ3 is 1535nm. At this time, the phase difference of the light signal with wavelength λ3 in the two interference arms is π. According to the interference principle, the light signal with wavelength λ3 is input from input port 2 and output from output port 4, also maintaining the "bar" state.
[0059] The thermo-optical tuner achieves low-loss thermo-optical tuning by embedding a silicon resistor micro-heater. Its implementation principle is as follows: Under the control of the hardware control signal, the thermo-optical tuner achieves complete consistency of the resonant wavelengths of the two red-marked micro-ring resonators through micro-heater calibration, both of which are λ1=1540nm. At this time, the phase difference of the light signal with wavelength λ1 in the two interference arms is 0. According to the interference principle, the light signal with wavelength λ1 is input from input port 1 and output from output port 4, that is, it changes from the "bar" state to the "cross" state.
[0060] The multiple inputs of the feedback optimization module are connected to the multiple output ports of the MRR-MZI switch array and the output of the traffic monitoring module, respectively. This allows for evaluation of the decision-making effect based on the outputs of the MRR-MZI switch array and the bandwidth utilization of each input port. In this embodiment, the decision-making effect evaluation is as follows: after migration, output port 3 only outputs optical signals with wavelength λ2, the load drops to a safe threshold, and the bandwidth utilization decreases from 95% to 63%, eliminating congestion. Output port 4 outputs optical signals with wavelengths λ1 and λ3, the load remains within a reasonable range, and its bandwidth utilization increases from 16% to 47%, with no congestion. Therefore, the decision is reasonable. The two outputs of the feedback optimization module are connected to the feedback terminals of the PPO algorithm decision module and the congestion status determination module, respectively. This allows the PPO algorithm decision module to provide feedback on the decision execution results to both modules, enabling the PPO algorithm decision module to adjust the policy network parameters based on the feedback migration effect to optimize subsequent decisions. Simultaneously, the congestion status determination module updates the congestion status of the corresponding input ports of the MRR-MZI switch array in real time.
[0061] A multi-wavelength channel dynamic load balancing method based on the PPO algorithm specifically includes the following steps:
[0062] Step 1: Build the multi-wavelength channel dynamic load balancing system based on the PPO algorithm described above.
[0063] Step 2: Optical signals with wavelengths λ1 and λ2 are input into the MRR-MZI switch array through input port 1, and optical signals with wavelength λ3 are input into the MRR-MZI switch array through input port 2, for channel-specific phase modulation.
[0064] Step 3: The traffic monitoring module collects the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue in real time, and combines it with the port hardware characteristic parameters input by the topology association map module to calculate the bandwidth utilization of each input port of the MRR-MZI switch array.
[0065] When the queue length corresponding to input port 1 is 55 frames at a certain moment, the total optical power it carries reaches -5.23dBm (0.3mW), of which the optical powers of λ1 and λ2 are -10dBm (0.1mW) and -7dBm (0.2mW) respectively. Based on the maximum allowable power of input port 1, the bandwidth utilization of input port 1 is 95% at this time. The queue length corresponding to input port 2 is 20 frames, and the optical power of λ3 is -13dBm (0.05mW). Based on the maximum allowable power of input port 2, the bandwidth utilization of input port 2 is only 16% at this time.
[0066] Step 4: The congestion status determination module determines whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. Since the current bandwidth utilization of input port 2 is 16%, which is much less than the bandwidth utilization congestion threshold of 80%, and the queue length of input port 2 is 20 frames, which is less than the queue length congestion threshold of 50 frames, it is determined that input port 2 is not congested, and the non-congestion status command of input port 2 is output to the traffic monitoring module for continuous monitoring. Since the current bandwidth utilization of input port 1 is 95%, which is greater than the bandwidth utilization congestion threshold of 80%, and the queue length of input port 1 is 55 frames, which is greater than the queue length congestion threshold of 50 frames, it is determined that input port 1 is congested, and the congestion status command of input port 1 is output to the PPO algorithm decision module to start the decision process.
[0067] Step 5: When the PPO algorithm decision module receives the congestion status command from input port 1, it adjusts the policy network parameters and generates the optimal migration decision through the near-end policy update mechanism, combining the correlation graph and the real-time congestion status. In this embodiment, the generated optimal migration decision is: to input the optical signal with wavelength λ1 from input port 1 and output it from output port 4; to input the optical signal with wavelength λ2 from input port 1 and output it from output port 3; and to input the optical signal with wavelength λ3 from input port 2 and output it from output port 4.
[0068] Step 6: The decision instruction issuing module parses the optimal migration decision instruction into a hardware control signal that can be recognized by the photonic execution layer, and performs thermo-optical tuning and electro-optical switching on the resonant state of the MRR-MZI switch array to realize wavelength migration control of the MRR-MZI switch array.
[0069] The above description is only used to illustrate the technical solutions of the present invention, and is not intended to limit them. For those skilled in the art, modifications can be made to the specific technical solutions described in the above embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions protected by the present invention.
Claims
1. A multi-wavelength channel dynamic load balancing system based on the PPO algorithm, characterized in that: It includes a network monitoring layer, an intelligent decision-making layer, and a photonic execution layer; The network monitoring layer includes a topology graph module, a traffic monitoring module, and a congestion status determination module; the intelligent decision-making layer includes a PPO algorithm decision-making module and a decision command issuance module; the photonic execution layer includes an MRR-MZI switch array, a thermo-optical tuner, and an electro-optical tuner. The MRR-MZI switch array includes multiple input ports and multiple corresponding output ports. Each input port is used to receive one or more optical signals. The MRR-MZI switch array is used to perform phase modulation on optical signals of different wavelengths in separate channels before outputting them. The topology association graph module is used to construct an association graph of ports, wavelengths, and links, and to label the port hardware characteristic parameters. The first input terminal of the traffic monitoring module is connected to the first output terminal of the topology association map module, and is used to receive port hardware characteristic parameters. Its acquisition port collects the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue in real time, and is used to calculate the bandwidth utilization of each input port of the MRR-MZI switch array in combination with the port hardware characteristic parameters. The input terminal of the congestion status determination module is connected to the output terminal of the traffic monitoring module. It is used to determine whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. Its two output terminals are respectively connected to the second input terminal of the traffic monitoring module and the first input terminal of the PPO algorithm decision module. It is used to output the non-congestion status command to the traffic monitoring module for continuous monitoring and output the congestion status command to the PPO algorithm decision module to start the decision process. The second input of the PPO algorithm decision module is connected to the second output of the topology association graph module. It is used to receive the association graph, and through the near-end policy update mechanism of the PPO algorithm, combined with the association graph, real-time congestion status and reward mechanism, adjust the policy network parameters to generate the optimal migration decision. The input end of the decision instruction issuing module is connected to the output end of the PPO algorithm decision module, and is used to parse the optimal migration decision instruction into a hardware control signal that can be recognized by the photonic execution layer. The control ports of the electro-optic tuner and the thermo-optic tuner are connected to the output of the decision command issuing module. They are used to perform thermo-optic tuning and electro-optic switching on the resonant state of the MRR-MZI switch array according to the issued hardware control signal, so as to realize the wavelength shift control of the MRR-MZI switch array.
2. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 1, characterized in that: The intelligent decision-making layer also includes a feedback optimization module; The multiple input terminals of the feedback optimization module are respectively connected to the multiple output ports of the MRR-MZI switch array and the output terminal of the traffic monitoring module, and are used to evaluate the decision-making effect based on the output of the MRR-MZI switch array and the bandwidth utilization of each of its input ports. The two outputs of the feedback optimization module are connected to the feedback terminals of the PPO algorithm decision module and the congestion status determination module, respectively, to provide feedback on the decision execution results to both modules. This enables the PPO algorithm decision module to adjust the policy network parameters in real time, while simultaneously enabling the congestion status determination module to update the congestion status of the corresponding input ports of the MRR-MZI switch array in real time.
3. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 2, characterized in that: The congestion status determination module uses dual thresholds to determine the congestion status, including a bandwidth utilization congestion threshold and a queue length congestion threshold.
4. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 3, characterized in that: The bandwidth occupancy congestion threshold is 80%, and the queue length congestion threshold is 50 frames.
5. A multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to any one of claims 1-4, characterized in that: The near-end policy update mechanism of the PPO algorithm decision module is implemented through action space design and reward mechanism; The action space is designed such that each wavelength of optical signal is independently treated as a migration unit, and the optical signal to be migrated and the target idle port are selected. The reward mechanism is as follows: if the utilization rate of the original congested port decreases by more than or equal to a first set threshold after migration, a points reward will be given. If the utilization rate of the target idle port after migration is less than or equal to the second set threshold, additional points will be awarded. If the migration time is less than the third set threshold, additional points will be awarded. If the utilization rate of the original congested port drops below the fourth set threshold after migration, an integral penalty will be applied.
6. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 5, characterized in that: The reward mechanism is as follows: if the utilization rate of the original congested port decreases by ≥30% after migration, 10 points will be awarded; if the utilization rate of the idle port is ≤50% after migration, an additional 5 points will be awarded; if the migration time is <1ms, an additional 3 points will be awarded; if the utilization rate of the original congested port decreases by <10% after migration, 3 points will be deducted.
7. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 1, characterized in that: The MRR-MZI switch array includes a first multimode interferometer, an MRR-MZI control array, and a second multimode interferometer connected in sequence according to the input and output order. The MRR-MZI control array includes multiple sets of microring resonators, each set of microring resonators including two microring resonators with the same wavelength; The first multimode interferometer is a beam splitter interferometer, and the second multimode interferometer is a beam combiner interferometer.
8. The multi-wavelength channel dynamic load balancing system based on the PPO algorithm according to claim 7, characterized in that: The electro-optic tuner achieves electro-optic switching by embedding a pin diode in a micro-ring resonator; The thermo-optical tuner achieves thermo-optical tuning by embedding a silicon resistive microheater.
9. A multi-wavelength channel dynamic load balancing method based on the PPO algorithm, characterized in that, Includes the following steps: Step 1: Build a multi-wavelength channel dynamic load balancing system based on the PPO algorithm as described in any one of claims 1-8; Step 2: Multiple optical signals of different wavelengths are input into the MRR-MZI switch array through multiple input ports of the MRR-MZI switch array for channel-specific phase modulation. Step 3: The traffic monitoring module collects the total optical power currently carried by each input port of the MRR-MZI switch array and the number of optical signal frames waiting to be transmitted in the port buffer queue in real time, and combines it with the port hardware characteristic parameters input by the topology association map module to calculate the bandwidth utilization of each input port of the MRR-MZI switch array. Step 4: The congestion status determination module determines whether the corresponding input port is congested based on the bandwidth utilization of each input port of the MRR-MZI switch array. If an input port is not congested, its non-congestion status command is output to the traffic monitoring module for continuous monitoring. If an input port is congested, its congestion status command is output to the PPO algorithm decision module. Step 5: When the PPO algorithm decision module receives the congestion status instruction, it adjusts the policy network parameters and generates the optimal migration decision by combining the correlation graph and the real-time congestion status through the near-end policy update mechanism. Step 6: The decision instruction issuing module parses the optimal migration decision instruction into a hardware control signal that can be recognized by the photonic execution layer, and performs thermo-optical tuning and electro-optical switching on the resonant state of the MRR-MZI switch array to realize wavelength migration control of the MRR-MZI switch array.
10. The multi-wavelength channel dynamic load balancing method based on the PPO algorithm according to claim 9, characterized in that: In step 4, the following conditions are used to determine whether the corresponding input port is congested: If the bandwidth utilization rate of an input port is less than or equal to 80% and the queue length is less than or equal to 50 frames, then the input port is determined to be not congested; if the bandwidth utilization rate of an input port is greater than 80% and / or the queue length is greater than 50 frames, then the input port is determined to be congested.