Satellite optical network load balancing routing and wavelength allocation method

Abstract

The invention discloses a satellite optical network load balancing routing and wavelength allocation method, and relates to the field of satellite optical communication. The method comprises the following steps: establishing a dynamic simulation environment of a satellite optical network; defining a node as an agent, for an arriving service request, starting from a source node agent, taking a current node on a path as an active agent, and selecting a next hop node through deep reinforcement learning until a destination node is reached, thereby forming a candidate path and a to-be-allocated wavelength set; and calculating an allocation metric value for each wavelength to be allocated, and selecting the wavelength with the maximum allocation metric value as the optimal wavelength to be allocated to the current service. According to the method, the use condition of the wavelength in the satellite optical network is sensed through the intelligent agent, the path bypassing the blocking area is selected to avoid cascade blocking, the problem of local congestion on the satellite caused by unbalanced distribution of ground services in geography and time can be effectively solved, the service blocking rate is remarkably reduced, and the network load balancing degree is achieved.

Description

Technical Field

[0001] This invention belongs to the field of satellite optical communication, and in particular to a method for load balancing routing and wavelength allocation in satellite optical networks. Background Technology

[0002] With the rapid development of global digitalization, the demand for seamless coverage, high capacity, and low-latency broadband communication is increasing daily. Satellite internet, especially optical networks based on low Earth orbit satellite constellations, has become a key component in building an integrated space-air-ground information network. Satellite optical networks use lasers as the transmission medium and, combined with wavelength division multiplexing (WDM) technology, can significantly improve the transmission capacity of inter-satellite links. However, The high-speed movement of satellites causes continuous dynamic changes in network topology and frequent connection and disconnection of inter-satellite links. Simultaneously, global user service demands exhibit significant geographical unevenness and temporal fluctuations, easily leading to localized congestion and load imbalances within the network. Routing and wavelength allocation are core issues in optical networks, aiming to select paths and allocate wavelengths for service requests while satisfying wavelength consistency constraints. In the highly dynamic and unevenly loaded environment of satellite optical networks, designing efficient and reliable load balancing is crucial. The algorithm faces enormous challenges.

[0003] Satellite optical network routing algorithms can be categorized into centralized and distributed routing based on the topological distribution characteristics of the decision-making entities. Centralized routing uses a central controller to collect topology, link, and traffic information from the entire network to uniformly calculate and distribute the optimal path. Its advantages lie in achieving global optimization and efficient load balancing. However, it also suffers from poor scalability, high signaling overhead, and difficulty adapting to rapid network topology changes. In contrast, distributed routing decentralizes decision-making power to each satellite node, with each node making autonomous decisions based on exchanged local information. It boasts advantages such as fast response speed and low computational overhead, but due to the lack of a global perspective, it is prone to getting trapped in local optima.

[0004] In recent years, reinforcement learning techniques have been introduced to solve optimization problems in satellite optical networks. However, in satellite optical network scenarios, single-agent reinforcement learning methods face the curse of dimensionality in state and action spaces, while traditional distributed algorithms suffer from non-stationarity challenges due to local observations. Therefore, there is an urgent need for an intelligent reinforcement learning algorithm that can balance global optimization with distributed execution efficiency and adapt to the highly dynamic characteristics of satellite networks. Solution. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing technologies and provide a load balancing routing and wavelength allocation method for satellite optical networks. Based on multi-agent reinforcement learning technology, this method can effectively adapt to dynamic topology changes, intelligently balance network load, thereby reducing service blocking rates and improving overall network performance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for load balancing routing and wavelength allocation in a satellite optical network includes the following steps:

[0008] Step 1: Establish a dynamic simulation environment for the satellite optical network using a low Earth orbit satellite optical network scenario model. Abstract the satellite optical network into a topological undirected graph and simulate the source-destination pair requests of global business traffic on the satellite by mapping the unbalanced load of the ground network. Record the occupancy status of each wavelength on each link.

[0009] Step 2: Define each node in the undirected graph as an independent decision-making agent. For an arriving service request, starting from the source node agent, the current node on the path is designated as the active agent. The active agent selects the next-hop node based on the local observation state and a preset multi-objective reward function through deep reinforcement learning, extending the path hop by hop until the destination node is reached, forming candidate paths and a set of wavelengths to be assigned that meet wavelength consistency requirements. ;

[0010] Step 3, assigning wavelengths to candidate paths Calculate the allocation metric for each wavelength to be assigned. Calculate the wavelength respectively The average proportion of idle wavelength resources on two consecutive hop links is calculated and then averaged to obtain the total wavelength to be allocated for the entire path. The average proportion of idle wavelength resources after allocation; select the one with the largest allocation metric. wavelength Assigning the optimal wavelength to the current service, if the set of wavelengths to be assigned If different wavelengths yield the same allocation metric value, then the wavelength with the smallest index number is selected to ensure a balanced distribution of traffic load in the network.

[0011] Furthermore, the specific method of step 1 is as follows:

[0012] Step 1.1: Abstract the satellite optical network into an undirected topological graph. ,in, Representing satellite nodes in the network, distributed across... (Systems Tool Kit) STK On different orbital planes, each orbit has A model of the Iridium constellation with 100 satellites. This represents an inter-satellite link, where there are reverse seams between satellites. It is assumed that any inter-satellite link supports... Each wavelength is used to construct a wavelength resource set. ;

[0013] Step 1.2: Establish a delay model: This represents the time delay from the satellite node to the next-hop satellite node at time t, expressed as the propagation delay in the link;

[0014] Step 1.3: Establish a source-destination request model for uneven global business traffic. The satellite nodes corresponding to the source and destination users of optical routing requests are selected based on the traffic share of the five continents on the ground.

[0015] Step 1.4: Use The model simulates the process of request arrival and duration, where the service arrival rate follows a Poisson distribution and the duration follows an exponential distribution. After specifying the service arrival time and service duration, the traffic load is jointly determined by the arrival rate and the duration.

[0016]

[0017] In the formula, Let C represent a source-destination pair from source node o to destination node d, C represent the set of all source-destination pairs, p represent a path from a source-destination pair, and P represent the set of paths among all source-destination pairs. W represents the set of wavelengths on each link, and W represents the set of available wavelengths on each path.

[0018] This model will be used in optical networks without wavelength converters. The problem is expressed as a path wavelength vector.

[0019] A path will only be allocated if there is a wavelength request on the path and the wavelength consistency constraint on the path is met. This assumes that all links support wavelength matching. There are wavelengths, q∈{0∼15}.

[0020]

[0021] If path Upper allocation wavelength The variable takes Otherwise take .

[0022] Calculate the total number of available wavelengths in the link:

[0023]

[0024] The larger the value, the more idle the link is, and the more optical traffic it can carry.

[0025] To formally describe the load balancing routing and wavelength allocation problem, the optimization objective is defined as minimizing the total optical path cost. A deep reinforcement learning method is used to approximate the solution to this optimization objective:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] In the link superior, This represents a set of links on the path. This represents the set of available links along a path. For optical networks, traffic is actually measured in terms of the number of optical paths; that is, traffic on a link corresponds to the number of optical paths traversing that link, and traffic along a path corresponds to the number of optical paths using that path. This indicates the traffic along the link. The mean number of available wavelengths for the path. This represents the input flow of the path.

[0032] The first constraint is that the latency on each path must be below the maximum latency threshold to ensure service quality. The second constraint is the demand constraint for the source-destination pair. The third constraint is the wavelength consistency constraint on the path, where each wavelength on a link can only be used once. The fourth constraint is the constraint on the maximum number of wavelengths that can be used on a link.

[0033] Furthermore, in step 2, the state space of the decision-making agent contains at least the following information:

[0034] The source node of the current business and destination node Normalized identifiers;

[0035] Available wavelengths along the path ;

[0036] The shortest hop count from the current node's neighbor nodes to the destination node. Number of wavelengths available for a single hop Number of wavelengths available for two consecutive hops The propagation delay from the current node to its neighboring nodes. .

[0037] Furthermore, assume that the set of neighboring nodes of a satellite node is Where n is the number of neighboring nodes of the satellite node, and the satellite node The state at time t is ,in, These represent the source node and destination node of the service request, respectively. Represents a node Links between neighboring nodes The time delay, Representative path Due to wavelength consistency limitations, for any wavelength, it can only be used to establish an optical path if and only if it is idle on all links of the path. Represents a node neighboring nodes The shortest number of hops to the destination node; Indicates in Time Node Next-hop neighbor node The number of remaining available wavelengths in the link. Indicates in Time Node From the first hop neighbor node To the second-hop neighbor node The number of remaining available wavelengths in two consecutive hop links, where the next hop neighbor node Not the destination node Only then is the calculation from the current node performed. via the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links, and the next hop neighbor node. That is, the destination node When there is no subsequent hop constraint, the wavelength consistency constraint is automatically satisfied. 0 indicates a link wavelength It has already been occupied. Indicates link wavelength Unoccupied; wavelengths that satisfy the two-hop wavelength consistency are generated through routing selection, forming a set of wavelengths to be allocated. .

[0038] Furthermore, in step 2, the multi-objective reward function is in the form of a weighted sum, representing the immediate reward value obtained from the environment for each action of the decision-making agent, calculated as follows:

[0039]

[0040]

[0041]

[0042] in, Indicates the proportion of wavelength continuity. Indicates load cost, This represents the shortest number of hops to the destination node; These represent the weights of wavelength continuity, load cost, and the shortest hop count to the destination node in the reward, respectively. ,and ; This is a traffic correction factor, calculated based on the traffic intensity of the next-hop satellite coverage area. As traffic intensity increases, The value increases, which in turn increases the load cost penalty value.

[0043] To achieve load balancing by directing satellite optical network traffic to non-hotspot areas, links with higher traffic should have higher costs. Therefore, an exponential adjustment factor is used, defining a link cost adjustment factor δ to regulate path costs and make it easier for services to pass through non-hotspot areas. Based on the user traffic demand distribution model, for satellite... The traffic intensity of its coverage area is defined as the sum of the traffic intensity of users within the coverage area.

[0044] Will lead to satellite The link cost correction factor δ is defined as:

[0045]

[0046] In the formula: To correct the constant, when the satellite When users communicate within the coverage area ,satellite When there is no user communication within the coverage area . The satellite's flux intensity represents the satellite's The sum of user traffic intensity within the coverage area. When satellite User traffic intensity within the coverage area is At that time, access to the satellite of Cost Make adjustments to reduce the impact on non-hotspot areas. Cost; when user traffic intensity within satellite coverage area is greater than At that time, access to the satellite of Costs increase with the intensity of user traffic.

[0047] Furthermore, in step 3, the metric is assigned. The calculation method is as follows:

[0048]

[0049] in, Indicates a path. This represents the number of link indices. It is a link The set of adjacent links, It is a set The number of elements, It is a link The set of adjacent links, It is a set The number of elements.

[0050] The beneficial effects of this invention are as follows:

[0051] 1. This invention selects a path by calculating the number of consecutive available wavelengths in a satellite optical network and comprehensively considering wavelength continuity, load cost, and the shortest number of hops to the destination node.

[0052] 2. This invention selects the wavelength with the largest allocation metric value as the optimal wavelength and allocates it to the current service, which can balance the satellite optical network traffic.

[0053] 3. This method is applicable to different dynamic topologies of satellite optical networks, and can reduce service blocking rate and improve network load balancing. Attached Figure Description

[0054] Figure 1 This is a diagram of the Low Earth Orbit Satellite Optical Network (RWA). Detailed Implementation

[0055] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0056] A method for load balancing routing and wavelength allocation in a satellite optical network includes the following steps:

[0057] S1, Dynamic Network Environment Modeling: Establishing a dynamic simulation environment for the satellite optical network, including an undirected graph of the topology, a latency model, source-destination pairs of unbalanced global service traffic requests, and the real-time wavelength occupancy status of all links; specifically:

[0058] S1.1, abstracting the satellite optical network into a topological undirected graph. ,in, Representing satellite nodes in the network, distributed across... (Systems Tool Kit) STK On different orbital planes, each orbit has A model of the Iridium constellation with 100 satellites. This represents an inter-satellite link, where there are reverse seams between satellites. It is assumed that any inter-satellite link supports... Each wavelength is used to construct a wavelength resource set. ;

[0059] S1.2, Establish a time delay model: The delay from the satellite node to the next-hop satellite node at time t is represented by the propagation delay in the link. Nodes in the satellite optical network do not have a queuing mechanism; routing requests are discarded immediately after being rejected, so queuing delay is ignored.

[0060] S1.3, Establish a source-destination traffic request model for uneven global business traffic, and select the satellite node corresponding to the source-destination user of the optical routing request based on the traffic proportion of the five continents on the ground;

[0061] S1.4, using The model simulates the process of request arrival and duration, where the service arrival rate follows a Poisson distribution and the duration follows an exponential distribution. After specifying the service arrival time and service duration, the traffic load is jointly determined by the arrival rate and the duration.

[0062] S2, Distributed Routing: Each satellite node in the network is defined as an independent decision-making agent; for the incoming service request, starting from the source node agent, the current node on the path is used as an active agent to select the next hop node through deep reinforcement learning based on its local observations and a preset multi-objective reward function, and the path is extended hop by hop until the destination node is reached, forming a candidate path;

[0063] In this context, the local observation is a vector, and we assume that the set of neighboring nodes of the virtual satellite node is... Where n is the number of neighboring nodes of a satellite node. A satellite optical network model is defined to construct satellite nodes. The state at time t is ,in, These represent the source and destination nodes of the arriving service request, respectively. Represents a node Links between adjacent nodes The algorithm design takes into account both transmission delay and propagation delay. Representative path The state of the wavelength on the path is such that, due to wavelength consistency constraints, for any wavelength, it can only be used to establish an optical path if and only if it is idle on all links of the path. Represents a node neighboring nodes The shortest number of hops to reach the destination. 0 indicates that the link wavelength It has already been occupied. Indicates the wavelength on this link It is not occupied. , indicating in Time Node Next-hop neighbor node The number of remaining available wavelengths for the link; the higher the value, the more available wavelengths for the next-hop link. Indicates in Time Node From the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links; this variable directly measures the number of wavelengths available in a node. The maximum amount of continuous wavelengths reserved for subsequent hops; the higher the value, the more wavelengths are available for consecutive two-hop links. Among these, the next hop neighbor node... Not the destination node Only then is the calculation from the current node performed. via the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links, and the next hop neighbor node. That is, the destination node When this condition is met, it indicates that there are no subsequent hop constraints, and the wavelength consistency constraint is automatically satisfied. The above routing selection generates a wavelength set that satisfies the two-hop wavelength consistency constraint. It can be used to measure the future availability of wavelength resources for a link.

[0064] A low total path cost indicates that there are more idle resources on the path, fewer hotspot areas passed through, and lower latency. Choosing a path with a low total cost is beneficial for reducing congestion rate.

[0065] The multi-objective reward function is an instantaneous reward, representing the immediate reward value obtained from the environment for each action of the agent. The acquisition of this reward should be related to the algorithm's optimization objective. Let the normalized reward function be:

[0066]

[0067] in:

[0068]

[0069] For wavelength continuity ratio:

[0070]

[0071] The reward function should be related to the wavelength continuity ratio and load cost. and the shortest number of hops to the destination node related; , These represent the weights of wavelength continuity, load cost, and the shortest hop count from the destination node in the reward, respectively. The above-mentioned rewards are set for satellites that perform actions, which is to encourage intelligent agents to choose paths with balanced traffic load and idle wavelengths, thereby increasing the success rate of subsequent service requests.

[0072] S3, Wavelength Allocation: Wavelengths to be allocated for candidate paths Calculate the allocation metric for each wavelength to be assigned. Calculate the wavelength respectively The average proportion of idle wavelength resources on two consecutive hop links is calculated and then averaged to obtain the total wavelength to be allocated for the entire path. The average proportion of idle wavelength resources after allocation. The higher the value, the higher the success rate of subsequent services. Select the metric with the highest allocation value. wavelength Assigning the optimal wavelength to the current service, if the set of wavelengths to be assigned If different wavelengths yield the same allocation metric value, the wavelength with the smallest index number is selected to ensure a balanced distribution of traffic load across the network. The allocation metric value... The specific calculation method is as follows:

[0073]

[0074] in, For path, It is its link index number. It is the set of candidate wavelengths after routing selection. , It is a link The set of adjacent links, It is a set The number of elements, It is a link The set of adjacent links, It is a set The number of elements.

[0075] Nodes on the selected path and the set of wavelengths that satisfy two-hop wavelength continuity. Based on this, it is necessary to continue allocating wavelengths for the path, prioritizing the selection of wavelengths. A higher value indicates a higher wavelength. The proportion of idle wavelengths on adjacent links of two consecutive hops allocated is the key factor. A higher value indicates that the adjacent links of two consecutive hops are less occupied, which increases the success rate of subsequent wavelength allocation.

[0076] Here is a more specific example:

[0077] A method for load balancing routing and wavelength allocation in a satellite optical network includes the following steps:

[0078] S1, Determine the satellite optical network model: using methods such as... Figure 1 The low-orbit satellite optical network model shown is in In satellite networks, a dynamic simulation environment for the satellite optical network is established. This environment includes an undirected graph of the topology, a latency model, source-destination pairs of unbalanced global service traffic requests, and the real-time wavelength occupancy status of all links. The specific steps are as follows:

[0079] S1.1, abstracting the satellite optical network into a topological undirected graph. ,in, Representing satellite nodes in the network, distributed across... (Systems Tool Kit) STK On different orbital planes, each orbit has A model of the Iridium constellation with 100 satellites. This represents an inter-satellite link, where there are reverse seams between satellites. It is assumed that any inter-satellite link supports... Each wavelength is used to construct a wavelength resource set. ;

[0080] S1.2 Due to the high-speed motion of satellites, the physical topology changes frequently, making traditional static routing methods difficult to adapt to this dynamic nature. The virtual topology strategy discretizes the dynamically changing satellite network topology. Specifically, a complete system cycle... Divided into multiple time periods, such as Inter-satellite link switching only occurs at specific points in time. This occurs. Within each time period, the topology of the satellite network is assumed to remain constant.

[0081] S1.3, Establish a time delay model: This represents the time delay from the satellite node to the next-hop satellite node at time t, expressed as the propagation delay in the link. Since the nodes in the satellite optical network do not have a queuing mechanism, the routing request is immediately discarded after being rejected, so the queuing delay is ignored.

[0082] S1.4, Establish a source-destination traffic request model for uneven global business traffic, and select the destination user for optical routing requests based on the traffic share of satellite networks on the five continents.

[0083] S1.5, using The model simulates the arrival and duration of requests, where the service arrival rate follows a Poisson distribution and the duration follows an exponential distribution. After specifying the service arrival time and duration, the traffic load is jointly determined by the arrival rate and duration. This leads to the establishment of a business time model.

[0084] S2, Distributed Routing: Each satellite node in the network is defined as an independent decision-making agent; for arriving service requests, starting from the source node agent, the current node on the path is taken as the active agent, and based on its local observations and a pre-set multi-objective reward function, it is selected through reinforcement learning. The network selects the next-hop node and extends the path hop by hop until the destination node is reached, forming candidate paths. The specific method is as follows:

[0085] S2.1, Establish the state space: The current state space perceived by the agent includes normalized source / destination node IDs and link information. Normalized latency, wavelength status on the current path, link Available wavelengths and two-hop wavelength continuity index W i (Measure from) through arrive One of my neighbors At that time, the link and (Number of publicly available wavelengths), neighbors Normalized hop count to the destination node. Assuming a virtual satellite node. The set of adjacent nodes is , where n is the number of satellite nodes The number of adjacent nodes. Setting up a satellite optical network model to construct satellite nodes. At any moment The status is as follows:

[0086]

[0087] in, These represent the source and destination nodes that received the service request, respectively. Represents a node Links between adjacent nodes The algorithm design only considers the propagation delay. Representative path The state of the wavelength on the path is such that, due to wavelength consistency constraints, for any wavelength, it can only be used to establish an optical path if and only if it is idle on all links of the path. Represents a node neighboring nodes The shortest number of hops to reach the destination.

[0088]

[0089] This indicates the link wavelength It has already been occupied. Indicates the wavelength on this link It is not occupied.

[0090]

[0091] Indicates in Time Node Next-hop neighbor node The number of remaining available wavelengths for the link; the higher the value, the more available wavelengths for the next-hop link.

[0092]

[0093] Indicates in Time Node From the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links; this variable directly measures the number of wavelengths available in a node. The maximum amount of continuous wavelengths reserved for subsequent hops; the higher the value, the more wavelengths are available for consecutive two-hop links. Among these, the next hop neighbor node... Not the destination node Only then is the calculation from the current node performed. via the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links, and the next hop neighbor node. That is, the destination node When this condition is met, it indicates that there are no subsequent hop constraints, and the wavelength consistency constraint is automatically satisfied. The above routing selection generates a wavelength set that satisfies the two-hop wavelength consistency constraint. It can be used to measure the future availability of wavelength resources for a link.

[0094] S2.2, Establishing the Action Space: This mainly considers route selection by choosing the next hop node from the adjacent nodes of the current node; therefore, an action space is set up. ,in .when The current node indicates that the next hop will be selected; otherwise, it will not be selected (a satellite can have a maximum of four neighboring nodes). When selecting a next hop node from a neighboring node, if there is no available wavelength on the adjacent link that is the same as the one on the current path, routing will stop and the service request will be blocked.

[0095] The multi-objective reward function is an instantaneous reward, representing the immediate reward value obtained from the environment for each action of the agent. The acquisition of this reward should be related to the algorithm's optimization objective. Let the normalized reward function be:

[0096]

[0097] in:

[0098]

[0099]

[0100] represent Time Link Wavelength continuity ratio:

[0101]

[0102] The reward function should be proportional to the wavelength continuity. Load cost and the shortest number of hops to the destination node related; , These represent the weights of wavelength continuity, load cost, and the shortest hop count from the destination node in the reward, respectively. The above-mentioned reward is set for satellites that perform actions, which is to encourage intelligent agents to choose paths with balanced traffic load and idle wavelengths, thereby increasing the success rate of subsequent service requests.

[0103] S3, Wavelength Allocation: Wavelengths to be allocated for candidate paths Calculate the allocation metric for each wavelength to be assigned. Calculate the wavelength respectively The average proportion of idle wavelength resources on two consecutive hop links is calculated and then averaged to obtain the total wavelength to be allocated for the entire path. The average proportion of idle wavelength resources after allocation. The higher the value, the higher the success rate of subsequent services; select the one with the highest allocation metric. wavelength Assigning the optimal wavelength to the current service, if the set of wavelengths to be assigned If different wavelengths yield the same allocation metric value, then the wavelength with the smallest index number is selected to ensure a balanced distribution of traffic load in the network.

[0104] Among them, the allocation of metrics The specific calculation is as follows:

[0105]

[0106] in, For path, It is its link index number. It is the set of candidate wavelengths after routing selection. , It is a link The set of adjacent links, It is a set The number of elements, It is a link The set of adjacent links, It is a set The number of elements.

[0107] Nodes on the selected path and the set of wavelengths that satisfy two-hop wavelength continuity. Based on this, it is necessary to continue allocating wavelengths for the path, prioritizing the selection of wavelengths. A higher value indicates a higher wavelength. The proportion of idle wavelengths on adjacent links of two consecutive hops allocated is the key factor. A higher value indicates that the adjacent links of two consecutive hops are less occupied, which increases the success rate of subsequent wavelength allocation.

[0108] The specific method of step S3 is as follows:

[0109] For each wavelength in the candidate wavelength set Calculate its allocation metric. The specific method is as follows: after path selection, a continuously available set of candidate wavelengths is generated. Calculate the assigned wavelength The proportion of average idle wavelengths along this path:

[0110] After the routing phase is completed, a complete path is obtained. and candidate wavelength set ;

[0111] : Traverse the continuous links on the path ;

[0112] Find the union of their two-hop neighbor links. and the number of elements in the set;

[0113] : Calculation in In the process of allocating candidate wavelength sets After each wavelength, the allocation metric for the entire path is obtained. ;

[0114] Preferred selection A higher wavelength value indicates more idle resources in two consecutive hops along the path, resulting in a higher success rate for allocating wavelengths to subsequent services.

[0115] Intuitively measured: if wavelength is used The extent to which it "occupies" surrounding link resources.

[0116] In this method, the following is used Multi-agent deep reinforcement learning algorithms employ centralized training and distributed execution. This architecture addresses the multi-agent cooperative routing problem. During the execution and sampling phases, each agent first calculates its individual action value based on local observation history through an independent agent network. In each After the business process is completed, these interaction experiences are stored in the experience replay pool. During the training phase, when the pool has sufficient data, a batch of data is sampled and input into the core hybrid network. This network uses a supernetwork to receive the global state s and generate non-negative weights, applying them in a non-linear and monotonic manner to the local states of all agents. Aggregated into a global joint action value This monotonicity constraint This ensures that maximizing global benefits is equivalent to maximizing individual benefits. Finally, the system calculates... The target network (which periodically replicates parameters from the main network to maintain stability) outputs the target network. Temporal difference between values ​​( Errors are detected and all agents are updated simultaneously end-to-end using the backpropagation algorithm. The parameters of the network and hybrid network drive the agent to learn better cooperative strategies.

[0117] In summary, this invention utilizes an intelligent agent to perceive wavelength usage within the satellite optical network, selecting paths that bypass congestion areas to avoid cascading congestion. It employs distributed hop-by-hop routing based on local observation and multi-objective rewards, then selects the optimal wavelength from the publicly available wavelengths along the path to maximize the future availability of surrounding link resources. This invention effectively addresses the problem of localized congestion on satellites caused by uneven geographical and temporal distribution of terrestrial services, significantly reducing service congestion rates and improving the load balancing of the satellite optical network.

Claims

1. A method for load balancing routing and wavelength allocation in a satellite optical network, characterized in that, Includes the following steps: Step 1: Establish a dynamic simulation environment for the satellite optical network using a low Earth orbit satellite optical network scenario model. Abstract the satellite optical network into a topological undirected graph and simulate the source-destination pair requests of global business traffic on the satellite by mapping the unbalanced load of the ground network. Record the occupancy status of each wavelength on each link. Step 2: Define each node in the undirected graph as an independent decision-making agent. For an arriving service request, starting from the source node agent, the current node on the path is designated as the active agent. The active agent selects the next-hop node based on the local observation state and a preset multi-objective reward function through deep reinforcement learning, extending the path hop by hop until the destination node is reached, forming candidate paths and a set of wavelengths to be assigned that meet wavelength consistency requirements. ; Step 3, assigning wavelengths to candidate paths Calculate the allocation metric for each wavelength to be assigned. Calculate the wavelength respectively The average proportion of idle wavelength resources on two consecutive hop links is calculated and then averaged to obtain the total wavelength to be allocated for the entire path. The average proportion of idle wavelength resources after allocation; select the one with the largest allocation metric. wavelength Assigning the optimal wavelength to the current service, if the set of wavelengths to be assigned If different wavelengths yield the same allocation metric value, then the wavelength with the smallest index number is selected to ensure a balanced distribution of traffic load in the network.

2. The satellite optical network load balancing routing and wavelength allocation method according to claim 1, characterized in that, The specific method for step 1 is as follows: Step 1.1: Abstract the satellite optical network into an undirected topological graph. ,in, Representing satellite nodes in the network, distributed across... (Systems Tool Kit) STK On different orbital planes, each orbit has A model of the Iridium constellation with 100 satellites. This represents an inter-satellite link, where there are reverse seams between satellites. It is assumed that any inter-satellite link supports... Each wavelength is used to construct a wavelength resource set. ; Step 1.2: Establish a delay model: This represents the time delay from the satellite node to the next-hop satellite node at time t, expressed as the propagation delay in the link; Step 1.3: Establish a source-destination request model for uneven global business traffic. The satellite nodes corresponding to the source and destination users of optical routing requests are selected based on the traffic share of the five continents on the ground. Step 1.4: Use The model simulates the process of request arrival and duration, where the service arrival rate follows a Poisson distribution and the duration follows an exponential distribution. After specifying the service arrival time and service duration, the traffic load is jointly determined by the arrival rate and the duration.

3. The satellite optical network load balancing routing and wavelength allocation method according to claim 2, characterized in that, In step 2, the state space of the decision-making agent must contain at least the following information: The source node of the current business and destination node Normalized identifiers; Available wavelengths along the path ; The shortest hop count from the current node's neighbor nodes to the destination node. Number of wavelengths available for a single hop Number of wavelengths available for two consecutive hops The propagation delay from the current node to its neighboring nodes. .

4. The satellite optical network load balancing routing and wavelength allocation method according to claim 3, characterized in that, Assume the set of neighbor nodes of a satellite node is Where n is the number of neighboring nodes of the satellite node, and the satellite node The state at time t is ,in, These represent the source node and destination node of the service request, respectively. Represents a node Links between neighboring nodes The time delay, Representative path Due to wavelength consistency limitations, for any wavelength, it can only be used to establish an optical path if and only if it is idle on all links of the path. Represents a node neighboring nodes The shortest number of hops to the destination node; Indicates in Time Node Next-hop neighbor node The number of remaining available wavelengths in the link. Indicates in Time Node From the first hop neighbor node To the second-hop neighbor node The number of available wavelengths for consecutive two-hop links; where, next hop neighbor node Not the destination node Only then is the calculation from the current node performed. via the first jump neighbor To the second neighbor The number of remaining available wavelengths in two consecutive hop links, and the next hop neighbor node. That is, the destination node When there is no subsequent hop constraint, the wavelength consistency constraint is automatically satisfied. 0 indicates a link wavelength It has already been occupied. Indicates link wavelength Unoccupied; wavelengths that satisfy the two-hop wavelength consistency are generated through routing selection, forming a set of wavelengths to be allocated. .

5. A satellite optical network load balancing routing and wavelength allocation method according to claim 4, characterized in that, In step 2, the multi-objective reward function is a weighted sum, representing the immediate reward value obtained from the environment for each action of the decision-making agent. The calculation method is as follows: in, Indicates the proportion of wavelength continuity. Indicates load cost, This represents the shortest number of hops to the destination node; These represent the weights of wavelength continuity, load cost, and the shortest hop count to the destination node in the reward, respectively. ,and ; This is a traffic correction factor, calculated based on the traffic intensity of the next-hop satellite coverage area. As traffic intensity increases, The value increases, which in turn increases the load cost penalty value; Will lead to satellite The link cost correction factor δ is defined as: In the formula: To correct the constant, when the satellite When users communicate within the coverage area ,satellite When there is no user communication within the coverage area ; The satellite's flux intensity represents the satellite's The sum of user traffic intensity within the coverage area; when satellite User traffic intensity within the coverage area is At that time, access to the satellite of Cost Make adjustments to reduce the impact on non-hotspot areas. Cost; when user traffic intensity within satellite coverage area is greater than At that time, access to the satellite of Costs increase with the intensity of user traffic.

6. A satellite optical network load balancing routing and wavelength allocation method according to claim 5, characterized in that, In step 3, assign the metric. The calculation method is as follows: in, Indicates a path. This represents the number of link indices. It is a link The set of adjacent links, It is a set The number of elements, It is a link The set of adjacent links, It is a set The number of elements.

Images