A Multi-Objective Optimization Routing Method for Loss and Crosstalk in Optical Switching Networks Based on EPM-D3QN Algorithm

Abstract

This invention discloses a multi-objective optimization routing method for optical switching networks based on the EPM-D3QN algorithm to address loss and crosstalk. The method first constructs a cellular optical switching network model and, combining port connection constraints and optical signal transmission characteristics, establishes a routing optimization model based on deep reinforcement learning. The core of the model, based on the D3QN algorithm, integrates an expert path initialization mechanism and a feasible action masking mechanism, effectively improving training efficiency and path search quality. Simultaneously, by designing a multi-objective reward function that comprehensively considers path loss, crosstalk, OSNR, and hop count, the model is guided to learn routing strategies with better overall transmission performance. Using the trained routing model, the final output is an optical routing path that satisfies the topological constraints of the optical switching network and exhibits low loss and low crosstalk characteristics, thereby significantly improving the transmission performance and resource utilization of the optical switching network.

Description

Technical Field

[0001] This invention relates to the field of optical switching technology, specifically to a multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm. Background Technology

[0002] With the widespread adoption of next-generation networks and the rapid development of data traffic services in the information society, data traffic in data centers and high-performance computing systems continues to grow, placing higher demands on the bandwidth, latency, energy consumption, and transmission reliability of on-chip and inter-chip interconnects. Compared to traditional electrical interconnect technologies, silicon-based optical interconnect technology has advantages such as high bandwidth, low power consumption, and strong resistance to electromagnetic interference, and has gradually become an important means of achieving short-distance, high-speed interconnects. As a key component in optical interconnect systems, optical switches enable flexible switching and path reconstruction of optical signals, and can be further cascaded to form optical networks with different topologies for large-scale data transmission tasks.

[0003] Among various optical switching network topologies, cellular optical switching networks belong to the multidirectional transmission type of optical networks. Compared to forward transmission networks, they have higher routing freedom and stronger reconfigurability. Compared to other multidirectional transmission networks such as triangular and quadrilateral networks, cellular optical switching networks have certain advantages in connectivity, path selection flexibility, and area utilization, thus possessing high research value in multidirectional controllable optical switching networks. Consistent with existing research on cellular optical switching networks, this type of network is typically constructed by cascading basic optical switching units, allowing optical signals to be flexibly forwarded in multiple directions, thereby forming different end-to-end routing paths.

[0004] However, in optical switching networks, optical signals are constrained during transmission by a combination of factors, including link insertion loss, waveguide crosstalk, device cascade attenuation, and crosstalk noise. As network size increases and routing complexity grows, focusing solely on a single loss metric for the selected path may result in lower transmission attenuation, but high crosstalk at several nodes or links along the path can still degrade signal quality, severely impacting communication reliability. Therefore, achieving a routing path with superior overall performance in cellular optical switching networks by simultaneously considering both transmission loss and crosstalk is a key challenge in current optical switching routing research.

[0005] With the development of deep reinforcement learning technology, algorithms such as DQN, DDPG, and PPO have been gradually applied to solving problems in combinatorial optimization, resource scheduling, and intelligent routing. Deep reinforcement learning can learn complex decision-making strategies through continuous interaction between the agent and the environment without explicitly exhaustively enumerating all paths, providing a new research approach for optical network routing problems. However, existing research on intelligent routing for optical switching networks is still relatively limited. Existing methods mostly focus on single loss optimization and do not adequately consider crosstalk factors, resulting in paths that are not necessarily the best in terms of overall performance. At the same time, facing the multi-branch, multi-state, and strongly coupled path decision-making scenarios in cellular optical switching networks, existing methods still have room for improvement in terms of path evaluation mechanisms, state representation methods, and multi-objective collaborative optimization capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-objective optimization routing method for optical switching networks based on the EPM-D3QN algorithm to address loss and crosstalk. This method is designed for cellular optical switching networks composed of cascaded basic optical switching units. Addressing the problems of existing routing methods, such as limited port connections, numerous invalid action explorations, difficulty in considering crosstalk effects with single loss optimization, and insufficient comprehensive transmission quality evaluation of the path, this invention constructs an improved Dueling Double DQN (D3QN) routing optimization model that integrates expert path initialization, a feasible action masking mechanism, and OSNR constraints. This model describes the end-to-end routing problem in optical switching networks as a Markov decision process. By jointly modeling cumulative path loss, cumulative crosstalk, output OSNR, port connection constraints, and historical access states, it achieves multi-objective collaborative optimization of the routing path in the optical switching network, thereby obtaining optical routing paths with low loss, low crosstalk, and superior transmission quality.

[0007] To achieve the above objectives, this invention provides a multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm, comprising the following steps:

[0008] S1: Construct a cellular optical switch network simulation array. Based on the basic optical switch units in the cellular optical switch network, establish an optical switch network model consisting of an optical switch unit set, an optical waveguide link set, a link transmission loss parameter set, a link crosstalk parameter set, and a port connection constraint set.

[0009] S2: Based on the optical switch network model established in step S1, the optical switch network routing problem is described as a Markov decision process, and a routing optimization model based on the EPM-D3QN algorithm is constructed; the EPM-D3QN algorithm is an improved D3QN algorithm that integrates expert path initialization, feasible action masking mechanism and OSNR constraint multi-objective reward function.

[0010] S3: Receive the request to establish an optical route, train the EPM-D3QN routing optimization model constructed in step S2, and obtain a loss and crosstalk multi-objective optimization routing model for cellular optical switch networks.

[0011] S4: Using the routing model trained in step S3, after inputting the starting point and ending point of the optical network, output the optical routing path that satisfies the optimal balance between overall loss and crosstalk.

[0012] In step S1, the cellular optical switch network is composed of multiple cascaded basic optical switch units. Each basic optical switch unit has three ports. After an optical signal is input from any port, it is output from the remaining feasible ports according to the port connection relationship. Each basic optical switch unit is connected to adjacent optical switch units through optical waveguide links. The outer optical switch units serve as boundary input / output nodes, and the inner optical switch units serve as intermediate forwarding nodes. A simple directed graph with weighting is used. To represent a cellular optical network simulation array, where, It is a collection of basic optical switching units in the network; Indicates from unit arrive A collection of optical waveguide links; Indicates from unit arrive Optical waveguide link The set of transmission losses. Indicates optical waveguide link The set of crosstalk parameters.

[0013] In step S2, an optical switch network routing optimization model is constructed based on the EPM-D3QN algorithm, which specifically includes a state space, an action space, a feasible action mask module, an expert path initialization module, a D3QN network module, and an OSNR-constrained multi-objective reward function module.

[0014] The state space is used to characterize the current routing state of the agent in the optical switch network, including at least the current optical switch unit location, the target optical switch unit location, the current incident port, the current candidate output port, the set of visited optical switch units, the current cumulative path loss, the current cumulative path crosstalk, and the current output OSNR estimate.

[0015] The action space represents the next-hop routing actions that the agent can choose at the current optical switch unit. Because the port connections of the optical switch units are constrained by physical structure, the agent cannot transmit optical signals in all directions. Therefore, this invention introduces a possible action masking mechanism to generate a set of possible actions in the current state based on the current optical signal input port, the current output port of the optical switch unit, the connection relationships of adjacent optical switch units, network boundary conditions, and the set of historical access nodes. It also blocks invalid actions such as no-port connection, out-of-bounds access, loops, and repeated access.

[0016] The action selection after the action mask satisfies:

[0017]

[0018] Where (s) represents the current state, and (a) represents the candidate action in the current state. This represents the set of possible actions that satisfy port connectivity constraints and topology constraints in the current state. This represents the action value function.

[0019] The expert path initialization module is used to improve the quality of experience samples in the early stages of training. Specifically, based on the optical switch network topology, link loss parameters, crosstalk parameters, and OSNR constraints, several candidate expert paths that satisfy port connection constraints are pre-generated, and the state transition samples corresponding to these expert paths are added to the experience replay pool. This allows the agent to use high-quality routing samples for policy learning in the early stages of training, reducing the proportion of invalid actions caused by completely random exploration and improving the model's convergence speed and training stability.

[0020] In step S3, the loss and crosstalk multi-objective optimization routing model for cellular optical switching networks is trained according to the EPM-D3QN algorithm, specifically as follows:

[0021] S3.1: Initialize the cellular optical switch network environment, determine the start and end points of the optical switch unit, and initialize the agent state, experience replay pool, online Q network parameters, target Q network parameters, maximum number of training rounds, and maximum number of action steps per round;

[0022] S3.2: Generate expert path samples that satisfy port connection constraints based on the expert path initialization module, and add the expert path samples to the experience replay pool;

[0023] S3.3: In the current state, generate a set of possible actions corresponding to the current state according to the possible action mask module, input the state information into the online Q network, and obtain the Q value corresponding to each candidate action in the set of possible actions;

[0024] S3.4: Select an action from the set of available actions according to the preset action selection strategy. After the agent executes the action, it enters the next state and obtains an immediate reward according to the multi-objective reward function constrained by OSNR.

[0025] The OSNR-constrained multi-objective reward function is used to simultaneously evaluate path loss, path crosstalk, output OSNR, and routing constraint satisfaction. The immediate reward obtained by the agent after performing an action consists of the target arrival reward, OSNR margin reward, link loss penalty, link crosstalk penalty, hop count penalty, and constraint penalty, and its expression is:

[0026]

[0027] in, This represents the immediate reward at step t; This represents the reward received upon reaching the target optical switch unit; This indicates the OSNR margin of the current path; This indicates the link loss or port switching loss introduced by the current action; This represents the crosstalk cost corresponding to the current action; Indicates the number of hops or the cost of detours; This indicates the constraint penalties corresponding to illegal actions, loops, repeated accesses, or unreachable actions. , , , and These represent the weighting coefficients for each reward or penalty item.

[0028] S3.5: Store the experience samples formed during the current interaction process into the experience replay pool, and randomly sample a small batch of experience samples from the experience replay pool; calculate the target Q value based on the sampled small batch of experience samples, and update the online Q network parameters using the error between the predicted Q value output by the online Q network and the target Q value.

[0029] The online Q-network adopts a Dueling structure, decomposing the state-action value into a state-value function and an action advantage function, the expression of which is:

[0030]

[0031] in, Represents the state value function. Represents the action advantage function. and These represent the network parameters for the state value branch and the action advantage branch, respectively.

[0032] The target Q value is calculated using the Double DQN method, specifically:

[0033]

[0034] in, This represents the target Q value corresponding to the i-th sample. Indicates an immediate reward. Indicates the discount factor. Indicates an online Q network, Indicates the target Q-network, Indicates the parameters of the online Q network. This represents the target Q-network parameters. The online Q-network is used to select the optimal action in the next state, while the target Q-network is used to evaluate the value of that action, thereby reducing the impact of Q-value overestimation on routing decisions in the traditional DQN algorithm.

[0035] In step S3.5, the loss function of the online Q-network is:

[0036]

[0037] Where N represents the number of small-batch empirical samples. This represents the target Q value corresponding to the i-th sample. Indicates the state of the online Q network. Next action The predicted Q value.

[0038] S3.6: Determine if the current round has ended. If the agent reaches the target light switch unit or reaches the maximum number of action steps per round, the current round ends. If it has not ended, return to step S3.3 to continue execution. Determine if the current number of training rounds has reached the maximum number of training rounds. If it has, the model training is complete. Otherwise, reinitialize the environment and agent state, and return to step S3.3 to continue training.

[0039] In step S4, the trained EPM-D3QN routing model is used for end-to-end routing reasoning in the optical switch network. After inputting the starting and ending optical switch units, the model outputs the optimal next-hop action based on the current state and the available action mask. The environment updates the current optical switch unit, incident port, cumulative loss, cumulative crosstalk, and OSNR state according to the action, and continues to make the next decision until the target optical switch unit is reached or the termination condition is met, thereby obtaining a complete optical routing path.

[0040] The comprehensive evaluation value of the optical routing path is expressed as:

[0041]

[0042] in, This represents the overall evaluation value of path P; This indicates the loss parameters corresponding to the link or port transition; This indicates the crosstalk parameters corresponding to link or port switching. This represents the OSNR margin of path P. Indicates the number of hops or the cost of detours; , , and These represent the weight coefficients for the corresponding evaluation items. The smaller the overall evaluation value, the better the overall performance of the corresponding optical routing path in terms of loss, crosstalk, OSNR, and path complexity. Attached Figure Description

[0043] Figure 1 is a flowchart of the multi-objective optimization routing method for optical switching network loss and crosstalk based on the EPM-D3QN algorithm of the present invention.

[0044] Figure 2 shows the 1×2 optical switch unit structure designed in this paper.

[0045] Figure 3 shows the invention provided by the present invention. Cellular optical switch network topology diagram;

[0046] Figure 4 is a schematic diagram of the process of training the optical switching routing model according to the EPM-D3QN algorithm of the present invention. Specific implementation methods

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] This embodiment provides a multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm. Referring to Figure 1, the routing method includes:

[0049] S1: Set up a A cellular optical network simulation array is constructed, and the optical switch network model is predefined based on the topological characteristics of the cellular optical switch network, the port connection relationship of the basic optical switch unit, and the boundary input-output constraints.

[0050] Figure 2 shows the structure of a 1×2 optical switch unit, which has three ports. Optical signals can be input from any one port and output from the other two ports depending on the switch state. Because the switching paths between different ports are different, the transmission loss and crosstalk parameters of the optical signal vary depending on the switching state of each port. Therefore, when establishing an optical switch network model, it is necessary to consider not only the topological connections between optical switch units but also the physical impairment parameters corresponding to the different port switching states within the basic optical switch unit. The cellular optical network shown in Figure 3 is composed of multiple basic optical switching units cascaded through optical waveguide links. Each node represents a basic optical switching unit, and the edges between nodes represent optical waveguide links between adjacent optical switching units. Outer optical switching units serve as boundary input / output nodes, while inner optical switching units serve as relay forwarding nodes. Except for the outer optical switching units, which have fewer adjacent connections due to their location at the network boundary, the remaining inner optical switching units are typically connected to three adjacent optical switching units, thus forming a cellular optical switching network with multi-directional controllable forwarding capabilities. A simple weighted directed graph can be used to illustrate this. To represent a cellular optical network simulation array, where, It is a collection of basic optical switching units in the network; Indicates from unit arrive A collection of optical waveguide links; Indicates from unit arrive Optical waveguide link The set of transmission losses. Indicates optical waveguide link The set of crosstalk parameters. For the cellular optical network, the optical signal is input through the outer basic optical switch unit, routed in the optical network, and finally output through the outer optical switch unit. The sequence of optical switch units passed through in this process constitutes an optical routing path. The overall cost of the optical routing path is determined by the cumulative path loss and the cumulative path crosstalk.

[0051] In step S1, the optical routing path It can be represented as ,in .

[0052] S2: Based on the cellular optical switch network model established in step S1, construct an optical switch network routing optimization model based on the EPM-D3QN algorithm, describe the optical switch network routing problem as a Markov decision process, and establish the state space, action space, experience replay pool, and reward function.

[0053] In this embodiment, the agent represents the optical signal performing path search in a cellular optical switch network. Starting from the originating optical switch unit, the agent selects the next-hop route action at each current optical switch unit until it reaches the destination optical switch unit or reaches the maximum number of steps.

[0054] The state space represents the current routing state of the agent, including the current optical switch unit location, the target optical switch unit location, the current optical signal input port, the current cumulative path loss, the current cumulative path crosstalk, the current path output OSNR estimate, and historical access node information. The state can be represented as:

[0055]

[0056] in, d represents the current optical switch unit; d represents the target optical switch unit; Indicates the current optical signal input port; This represents the current cumulative path loss; This indicates the current cumulative path crosstalk; This represents the estimated OSNR value at the current path output. This indicates information about historical access nodes.

[0057] The action space represents the next-hop routing actions that an agent can choose from at the current optical switch unit. Due to the port connection constraints of the basic optical switch unit, the agent can only select actions from the set of candidate actions that satisfy both the topology constraints and the port connection constraints.

[0058] The reward function is used to evaluate the routing quality after the agent performs an action. It comprehensively considers path loss, path crosstalk, OSNR performance, and destination arrival status. Positive rewards are given when the agent selects a path with low loss, low crosstalk, and a high OSNR margin; negative penalties are given when high loss, high crosstalk, loops, or unreachability occur, thereby guiding the agent to learn routing strategies with better overall transmission performance.

[0059] S3: Train the optical switch network routing optimization model constructed in step S2 based on the EPM-D3QN algorithm.

[0060] In this embodiment, as shown in Figure 4, training the optical switching routing model according to the EPM-D3QN algorithm specifically includes the following steps:

[0061] S3.1: Initialize the cellular optical switch network environment, read the optical switch unit set, optical waveguide link set, port connection relationship, link loss parameters and link crosstalk parameters, and set the training round number T, maximum action steps per round K, experience replay pool capacity, batch sampling size, and discount factor. The learning rate and the target network update cycle.

[0062] S3.2: Initialize the online Q-network and the target Q-network. Both the online Q-network and the target Q-network adopt a Dueling network structure, decomposing the action value function into a state value function and an action advantage function. The online Q-network is used for action selection and parameter updates, while the target Q-network is used to calculate the target Q-value and improve training stability. The action value function under the Dueling network structure can be expressed as:

[0063]

[0064] in, This represents the state value function, used to evaluate the value of the current state itself. This represents the action advantage function, used to evaluate the choice of action in the current state. Advantages compared to other actions; and These represent the network parameters for the state value branch and the action advantage branch, respectively. This represents the set of valid actions in the current state.

[0065] S3.3: Generate expert experience samples through an expert path initialization mechanism. Specifically, a preset path search algorithm is used to search for candidate paths that satisfy port connection constraints in the cellular optical switch network, and the candidate paths are filtered based on cumulative path loss, cumulative crosstalk, and OSNR margin. The filtered expert paths are decomposed into multiple state transition samples and stored in the experience replay pool. At the beginning of each training round, the starting optical switch unit and the ending optical switch unit are initialized to obtain the initial state. Let the current action step t=0;

[0066] S3.4: In the current state Based on the current optical switch unit, incident port, connection relationship between adjacent optical switch units, network boundary conditions, and historical access node information, a feasible action mask corresponding to the current state is generated. And obtain a set of effective actions. ;

[0067] S3.5: Set the current state The online Q-network is input to obtain the Q-values ​​corresponding to each candidate action, and invalid actions are masked according to the available action mask. For invalid actions, their corresponding Q-values ​​are not included in the action selection; for valid actions, a mask is used. Greedy strategy selects the current action In the early stages of training, actions are randomly selected from the set of effective actions with a certain probability to maintain the exploration capability; in the later stages of training, the selection gradually tends to focus on the effective actions with the largest current Q value to improve the quality of path decision-making.

[0068] The action selection process can be represented as:

[0069]

[0070] S3.6: The agent performs an action Then, the system moves from the current optical switching unit to the next optical switching unit, and updates the current incident port, cumulative path loss, cumulative path crosstalk, OSNR estimate, and historical access node set to obtain the next state. .

[0071] S3.7: Calculate the immediate reward corresponding to the current action based on the OSNR-perceived multi-objective reward function. The reward function comprehensively considers the target arrival reward, OSNR margin reward, link loss penalty, crosstalk penalty, path hop count penalty, and constraint penalty, and its expression is:

[0072]

[0073] in, This represents the immediate reward at step t; This represents the reward received upon reaching the target optical switch unit; This indicates the OSNR margin of the current path; This indicates the link loss or port switching loss introduced by the current action; This represents the crosstalk cost corresponding to the current action; Indicates the number of hops or the cost of detours; This indicates the constraint penalties corresponding to illegal actions, loops, repeated accesses, or unreachable actions. , , , and These represent the weighting coefficients for each reward or penalty item.

[0074] S3.8: The experience samples formed during the current interaction process ( Stored in the experience replay pool. Here, "done" represents a flag indicating whether the current round has ended. "done" is true when the agent reaches the target light switch unit or the current action step count reaches the maximum action step count K; otherwise, "done" is false.

[0075] S3.9: Randomly sample a small batch of experience samples from the experience replay pool and calculate the target Q-value using the Double DQN method. During the target Q-value calculation, the online Q-network selects the optimal action from the set of valid actions for the next state, and the target Q-network evaluates the value of this action, expressed as:

[0076]

[0077] in, This represents the target Q value corresponding to the i-th sample. Indicates an immediate reward. Indicates the discount factor. Indicates an online Q network, Indicates the target Q-network, Indicates the parameters of the online Q network. This represents the target Q-network parameters. By using online Q-network selection and target Q-network estimation, the impact of Q-value overestimation on routing decisions in traditional DQN can be reduced.

[0078] S3.10: Update the online Q-network parameters based on the error between the predicted Q-value and the target Q-value output by the online Q-network. The loss function is:

[0079]

[0080] Where N represents the number of small-batch empirical samples. This represents the target Q value corresponding to the i-th sample. Indicates the state of the online Q network. Next action The predicted Q-value is obtained. The parameters of the online Q-network are updated by minimizing the loss function.

[0081] S3.11: Determine if the target Q-network update conditions are met. If met, synchronize the online Q-network parameters to the target Q-network; otherwise, continue the current training process. Target network synchronization can be represented as:

[0082]

[0083] S3.12: Determine if the current round has ended. If the agent has reached the target light switch unit or the current action step has reached the maximum action step K, then the current round ends; if the current round has not ended, let t = t + 1, and return to step S3.4 to continue execution.

[0084] S3.13: Determine if the current number of training epochs has reached the maximum number of training epochs T. If the maximum number of training epochs has been reached, the model training is complete; otherwise, reinitialize the environment state and return to step S3.3 to continue training.

[0085] S4: Using the EPM-D3QN optical switch network routing model trained in step S3, output an optical routing path that meets the multi-objective optimization requirements of loss and crosstalk.

[0086] In this embodiment, the starting and ending points of the optical route are input into the trained EPM-D3QN routing model. The model autonomously outputs the next-hop routing decision based on the current state information and continuously updates the current state until the target node is reached. Ultimately, an optical route path that satisfies the topological constraints of the cellular optical switch network and has better overall loss and crosstalk performance is obtained.

Claims

1. The purpose of this invention is to provide a multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm, characterized in that... include: S1: Set up a A cellular optical switch network simulation array is constructed, and the network is predefined based on the topological characteristics of the cellular optical network simulation array. S2: Based on the optical switch network model established in step S1, the optical switch network routing problem is described as a Markov decision process. An optical switch network routing optimization model based on the EPM-D3QN algorithm is constructed, and the state space, action space, experience replay pool, and reward function are established. S3: The EPM-D3QN algorithm is used to train the optical switch network routing optimization model. During the training process, an expert path initialization mechanism and a feasible action mask mechanism are introduced, and the D2QN network structure is used for action value learning to obtain the trained optical switch network routing model. S4: Using the routing model trained in step S3, after inputting the starting point and ending point of the optical network, output the optical routing path that satisfies the optimal balance between overall loss and crosstalk.

2. The multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 1, characterized in that, In step S1, the cellular optical switch network is composed of multiple cascaded basic optical switch units. Each basic optical switch unit has three ports. After an optical signal is input from any port, it is output from the remaining feasible ports according to the port connection relationship. Each basic optical switch unit is connected to adjacent optical switch units through optical waveguide links. The outer optical switch units serve as boundary input / output nodes, and the inner optical switch units serve as intermediate forwarding nodes. A simple directed graph with weights is used. To represent a cellular optical network simulation array, where, It is a collection of basic optical switching units in the network; Indicates from unit arrive A collection of optical waveguide links; Indicates from unit arrive Optical waveguide link The set of transmission losses. Indicates optical waveguide link The set of crosstalk parameters.

3. The multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 1, characterized in that, In step S2, the state space includes the current optical switch unit position, the target optical switch unit position, the current optical signal input port, the cumulative path loss, the cumulative path crosstalk, the estimated OSNR value at the path output end, and the historical access node information; the action space represents the set of next-hop routing actions that satisfy the port connection constraints at the current optical switch unit.

4. The multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 1, characterized in that, In step S3, the optical switching routing model is trained according to the EPM-D3QN algorithm, including: S3.1: Initialize the cellular optical switch network environment, determine the start and end points of the optical switch units, and initialize the agent state, experience replay pool, online Q network parameters, target Q network parameters, maximum number of training rounds, and maximum number of action steps per round; S3.2: Generate expert path samples that satisfy port connection constraints based on the expert path initialization module, and add the expert path samples to the experience replay pool; S3.3: Receive the request to establish an optical route, determine the optical signal input and output ends, and train the EPM-D3QN routing optimization model constructed in step S2 to obtain a loss and crosstalk multi-objective optimization routing model for cellular optical switch networks. S3.4: Select an action from the set of available actions according to the preset action selection strategy. After the agent executes the action, it enters the next state and obtains an immediate reward according to the multi-objective reward function constrained by OSNR. S3.5: Store the experience samples generated during the current interaction process into the experience replay pool, and randomly sample a small batch of experience samples from the experience replay pool; calculate the target Q value based on the sampled small batch of experience samples, and update the online Q network parameters using the error between the predicted Q value output by the online Q network and the target Q value; when the preset target network update conditions are met, synchronize the online Q network parameters to the target Q network. S3.6: Determine if the current round has ended. If the agent reaches the target light switch unit or reaches the maximum number of action steps per round, the current round ends. If it has not ended, return to step S3.3 to continue execution. Determine if the current number of training rounds has reached the maximum number of training rounds. If it has, the model training is complete. Otherwise, reinitialize the environment and agent state, and return to step S3.3 to continue training.

5. A multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 4, characterized in that, In step S3.4, the formula for calculating the OSNR multi-objective optimization reward function is as follows:

6. The multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 4, characterized in that, In step S3.5, the online Q-network adopts a Dueling structure, decomposing the state-action value into a state value function and an action advantage function, the expression of which is: in, Represents the state value function. Represents the action advantage function. and These represent the network parameters for the state value branch and the action advantage branch, respectively.

7. A multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 4, characterized in that, In step S3.5, the target Q value is calculated using the Double DQN method, specifically as follows:

8. A multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 4, characterized in that, In step S3.5, the parameters of the online Q-network are updated based on the error between the predicted Q-value and the target Q-value output by the online Q-network. The loss function is:

9. A multi-objective optimization routing method for loss and crosstalk in optical switching networks based on the EPM-D3QN algorithm according to claim 4, characterized in that, In step S4, the comprehensive evaluation value of the optical routing path is expressed as: in, This represents the overall evaluation value of path P; This indicates the loss parameters corresponding to the link or port transition; This indicates the crosstalk parameters corresponding to link or port switching. This represents the OSNR margin of path P. Indicates the number of hops or the cost of detours; , , and These represent the weight coefficients of the corresponding evaluation items.

Images