A method for flexible routing of LEO mega-constellation based on graph reinforcement learning and related equipment

Abstract

The application provides a LEO mega-constellation flexible routing method based on graph reinforcement learning and related equipment, and relates to the technical field of satellite communication. The method comprises the following steps: dividing the constellation topology into multiple control domains, managing the multiple control domains by using master-slave controllers based on the SDN architecture; the master controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source-destination node pair into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller constructs a virtual traffic topology according to the total intra-domain traffic request and the fault link, selects the key link in the domain, extracts the link feature matrix of the virtual traffic topology by using the GNN, updates the weight of the key link by using the reinforcement learning algorithm, and then generates the intra-domain routing path based on the new link weight under the constraints of the path delay and the link bandwidth. The application can meet the high availability and low delay requirements of LEO mega-constellation data transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite communication, in particular to a LEO mega constellation elastic routing method based on graph reinforcement learning and related equipment. BACKGROUND

[0002] With the development of communication technology and the significant reduction of satellite manufacturing and launching costs, LEO mega constellation forms a large-scale network system by densely deploying small satellites to provide low-latency, high-bandwidth and wide-coverage network access services. As a key infrastructure for future global communication and next-generation Internet, routing plays a key role in satellite networks, not only determining the connectivity of data transmission, but also directly affecting the overall performance and operational efficiency of the network.

[0003] However, the unique network characteristics of LEO mega constellation pose many challenges to the design of routing algorithms. First, LEO satellites are exposed to open and harsh space environments, facing natural threats such as space debris and solar storms. At the same time, small satellites have a short lifespan and have various security vulnerabilities and backdoors, making them potential entry points for network attacks. Finally, geographical conditions such as uneven population distribution and economic development differences, combined with the dynamic characteristics of orbital motion, affect global traffic patterns, resulting in uneven network load, i.e. long-term overload of link bandwidth in hotspot areas, while other link bandwidth is idle and wasted. All of the above factors can lead to network failure of LEO mega constellation.

[0004] Combining the centralized control architecture of software-defined network (SDN) and intelligent routing methods based on deep reinforcement learning (DRL), which show good control ability for traffic transmission, are being widely researched to optimize traffic distribution. However, existing intelligent routing methods applied to LEO mega constellation face the problems of low robustness and difficulty in expansion. On the one hand, DRL-based routing usually needs to control all elements (links or data flows) of the target in the network, and the dense link elements in LEO mega constellation result in a sharp increase in state and action space, making it difficult for the model to effectively train and converge. In addition, these routing algorithms usually use traditional neural network architectures such as fully connected networks, which are prone to overfitting the training topology. Once the network failure causes the topology structure to change, the routing performance declines severely. SUMMARY

[0005] To meet the high availability and low latency requirements of LEO mega constellation data transmission, the present application proposes a LEO mega constellation elastic routing method based on graph reinforcement learning and related equipment, aiming to cope with frequent network failures and large network topologies.

[0006] In a first aspect, the present application provides a LEO mega constellation elastic routing method based on graph reinforcement learning, comprising:

[0007] Step 1: divide the constellation topology into multiple control domains, and manage the multiple control domains based on a master-slave controller using an SDN architecture; wherein all control domains are of the same shape and contain the same number of LEO satellites within the domain;

[0008] Step 2: the master controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source-destination node pair into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path;

[0009] Step 3: the slave controller constructs a virtual traffic topology based on the total intra-domain traffic request and the fault link, and selects a critical link within the domain based on the virtual traffic topology;

[0010] Step 4: the slave controller extracts a link feature matrix of the virtual traffic topology using GNN, and updates the weight of the critical link based on the link feature matrix using a reinforcement learning algorithm, and then generates an intra-domain routing path based on the new link weight under the constraints of path latency and link bandwidth.

[0011] Further, in step 2, the master controller calculates the global shortest path from the source node to the destination node using the A* algorithm; wherein the heuristic function selects the Manhattan distance.

[0012] Further, the heuristic function is:

[0013] h(n)=δ avg ×(|x n -x m |+|y n -y m |)

[0014] wherein δ avg represents the average latency of the link, and |x n -x m |+|y n -y m | is the Manhattan distance moving along the grid lines from the source node n to the destination node m.

[0015] Further, in step 3, the slave controller constructs a virtual traffic topology based on the intra-domain traffic request and the fault link, specifically including:

[0016] Collect real-time traffic requests within the domain, combine the real-time traffic requests with the intra-domain traffic requests from the master controller to obtain the total intra-domain traffic request; treat the fault link as a link completely exhausted by burst traffic; thereby generating a traffic demand matrix within the domain.

[0017] Further, in step 3, the critical link is selected within the domain based on the virtual traffic topology by a Monte Carlo algorithm, specifically including:

[0018] According to the traffic demand matrix in the domain, the local shortest path between the domain boundary node pairs and the load of each link under the local shortest path are calculated, and the sampling probability of each link is calculated; wherein the sampling probability of the link is proportional to the load of the link;

[0019] According to the sampling probability of the link, a fault scenario s containing λ links is repeatedly sampled from the domain, it is checked whether the fault scenario s already exists in the preset fault scenario set, if not, the maximum link utilization MLU corresponding to the total domain traffic request under the fault scenario s is calculated, and the fault scenario s is added to the preset fault scenario set, and the corresponding MLU is added to the preset MLU index set; if yes, the sampling is re-sampled;

[0020] It is checked whether the number of fault scenarios in the fault scenario set meets the requirements, if yes, the λ links with the highest MLU are selected as the key links, if not, the previous step is returned.

[0021] Further, in step 4, the controller extracts the link feature matrix of the virtual traffic topology by using GNN, specifically including:

[0022] The network state of the virtual traffic topology is initialized as a feature vector, and the message representation after iteration l times is defined :

[0023]

[0024] Wherein, denotes the adjacent node set of node v, denotes the feature vector of adjacent node u after iteration l-1 times, denotes the feature vector of the link composed of nodes u and v after iteration l-1 times, W (l) and B (l) denote the weight matrix, denotes the feature vector of node v after iteration l times, and σ denotes a nonlinear activation function ReLU, t denotes a message passing function;

[0025] After completing the total iteration L times, the feature vectors of all nodes are summarized into a whole graph level representation h G , which is represented as

[0026] Further, in step 4, the weight of the key link is updated by using a reinforcement learning algorithm, specifically including:

[0027] Define the state space: represent the network state as s t=[v,H,edge_index], wherein V represents node information, H represents link information, and edge_index is edge index information used to indicate a network connection relationship;

[0028] define an action set: represent the action corresponding to the time step t as a t ={a1,a2,…,a λ}, wherein a i is the virtual weight adjustment ratio of the i-th key link;

[0029] define a reward function: after executing the action a t , calculate the maximum link utilization MLU corresponding to the total intra-domain traffic request, and then calculate the reward r t =1 / MLU.

[0030] In a second aspect, the present application provides a LEO mega constellation elastic routing device based on graph reinforcement learning, comprising:

[0031] a constellation topology division module, configured to divide the constellation topology into a plurality of control domains; wherein all control domains have the same shape and contain the same number of LEO satellites in the domain;

[0032] a routing control module, configured to manage the plurality of control domains based on the SDN architecture using master-slave controllers; wherein the master controller calculates the global shortest path from the source node to the destination node, decomposes the cross-domain traffic request between the source-destination node pair into a plurality of intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller constructs a virtual traffic topology according to the total intra-domain traffic request and the fault link, selects key links in the domain based on the virtual traffic topology, extracts a link feature matrix of the virtual traffic topology using GNN, and updates the weights of the key links using a reinforcement learning algorithm based on the link feature matrix, and then generates an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0033] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the method of the first aspect.

[0034] In a fourth aspect, the present application provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the method of the first aspect.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] (1) The application can effectively deal with the problem of unstable routing caused by link failure or high load in LEO satellite network by constructing a routing method based on graph reinforcement learning, and ensure the high availability of data transmission in dynamic network environment.

[0037] (2) The application combines the master-slave controller design of the SDN architecture, the master controller is responsible for the global shortest path cross-domain planning, and the slave controller combines the virtual traffic topology and reinforcement learning to realize the control of the key link in the domain, dynamically adjusts the path, and improves the flexibility and accuracy of the routing decision.

[0038] (3) The application adopts Monte Carlo algorithm to identify high-load links, and combines reinforcement learning algorithm to control the weight of key links, which can minimize the influence of link failure on the overall network performance and enhance the fault tolerance of the network.

[0039] (4) The application can adapt to different sizes of network topology through the generalization ability of GNN, and still maintain high routing performance under the condition of topology structure change, overcoming the defect that the traditional intelligent routing method based on DRL is easily affected by network failure. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A general framework diagram of a graph reinforcement learning-based LEO mega constellation elastic routing method is provided for the embodiments of the application.

[0041] Figure 2 A basic mechanism diagram of A* algorithm in grid constellation topology is provided for the embodiments of the application.

[0042] Figure 3 A key link selection flowchart based on Monte Carlo simulation is provided for the embodiments of the application.

[0043] Figure 4 An intra-domain routing calculation flowchart based on graph reinforcement learning is provided for the embodiments of the application.

[0044] Figure 5 A DRL routing method based on GNN is provided for the embodiments of the application.

[0045] Figure 6 A structure diagram of a graph reinforcement learning-based LEO mega constellation elastic routing device is provided for the embodiments of the application.

[0046] Figure 7 A structure block diagram of an electronic device is provided for the embodiments of the application. DETAILED DESCRIPTION

[0047] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0048] As shown in Figure 1 , the embodiment of the present application provides a LEO mega-constellation flexible routing method based on graph reinforcement learning, comprising the following steps:

[0049] S101: The constellation topology is divided into multiple rectangular control domains according to a grid, and a master-slave controller is used to manage the multiple control domains based on an SDN architecture; wherein each control domain contains 3*3 LEO satellites;

[0050] S102: The master controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source-destination node pair into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path;

[0051] Specifically, since the LEO satellite network topology presents a highly regular grid shape, the A* algorithm is used to quickly determine the domain boundary nodes and inter-domain links passed. The heuristic function is selected as the Manhattan distance. The Manhattan distance, i.e. the straight-line distance between grid points, is suitable for estimating the shortest path problem in a grid layout. In combination with the demand for low latency in the present application, this physical distance is further converted into an estimate of the latency. First, the average latency δ avg of the link is obtained based on historical data or actual measurement, and then the heuristic function h(n) can be defined as h(n) = δ avg ×(|x n -x m |+|y n -y m |), where |x n -x m |+|y n -y m | calculates the Manhattan distance moving along the grid line from the source node n to the destination node m, as shown in Figure 2 .

[0052] S103: The slave controller constructs a virtual traffic topology according to the total intra-domain traffic request and the fault link, and selects the key link in the domain based on the virtual traffic topology;

[0053] Specifically, real-time traffic requests within the domain are collected, and these requests are merged with those from the main controller to obtain a total domain traffic request. Faulty links are treated as links completely exhausted by burst traffic, thereby generating a domain traffic demand matrix TM. Since various sudden network failures can lead to changes in the physical topology, this embodiment of the invention treats physical topology link failures as links completely exhausted by burst traffic and constructs a virtual traffic topology by combining this with the total domain traffic request. This transforms complex combinations of link failures into diverse traffic demands, thereby eliminating the uncertainty of topology changes caused by network failures and achieving the purpose of shielding against network failures.

[0054] S104: Extract the link feature matrix of the virtual traffic topology from the controller using GNN, and update the weights of key links using reinforcement learning algorithm based on the link feature matrix, thereby generating intra-domain routing paths based on the new link weights under the constraints of path delay and link bandwidth.

[0055] The resilient routing method provided in this invention mainly includes an inter-domain routing step (i.e., generating the global shortest path) handled by the master controller and an intra-domain routing step (i.e., generating the local shortest path within the domain) handled by the slave controller. Within each control domain, the SDN controller acquires routing requests from source and destination node pairs and constructs a virtual traffic topology based on failed links to mask network failures. Based on this virtual traffic topology, the application plane adjusts the weights of some critical links using graph reinforcement learning algorithms. Under constraints of path latency and link bandwidth, the routing path is calculated according to the new link weights, optimizing routing decisions and improving the path reliability of the satellite routing method under link failure conditions. This keeps end-to-end latency within an acceptable range, meeting the high availability and low latency requirements for LEO mega-constellation data transmission.

[0056] Based on the above embodiments, this embodiment of the invention employs the Monte Carlo algorithm to select critical links within the domain based on the virtual traffic topology. Specifically, the Monte Carlo algorithm is used to perform a finite N-order selection on combinations of fault scenarios. mc This simulation is used to identify critical links in the network. The probability of a link being selected is proportional to its initial load under the shortest path (SP) routing scheme, when N... mc When the value is 1, the critical link selection algorithm degenerates into selecting the top λ links with the highest load in the network. For example... Figure 3 As shown, the load on each link is first calculated based on the SP route and the current traffic demand TM, and the link sampling probability P is obtained. sample. When the number of simulations is less than N (N represents the number of LEO satellite nodes in the domain), a failure scenario s containing λ links is repeatedly sampled, and it is checked whether this scenario already exists in the set S. If s is a new scenario, the maximum link utilization (MLU) corresponding to all traffic demand under this scenario is calculated, and the scenario s and the MLU indicator i s are added to the sets S and I, respectively. Finally, the link set with the highest MLU indicator is selected from the stored failure scenarios as the critical link

[0057] In the above embodiment, the master controller decomposes the cross-domain traffic request into multiple intra-domain traffic requests, and the intra-domain routing process based on graph reinforcement learning (GNN+DR) of the corresponding slave controller ensures reliable transmission of the flow in its domain. On the basis of the above embodiment, in combination with Figure 4 and Figure 5 , the present embodiment further elaborates on the intra-domain routing process, as shown in Figure 4 , which mainly includes four stages.

[0058] In stage 1, the slave controller of each domain identifies the traffic request and the failed link, so as to combine the total intra-domain traffic request and the failure link regarded as exhausted by burst traffic, and finally obtain a brand-new traffic demand matrix TM. In stage 2, the critical link is selected as the control element in an online manner, and the GNN-based DRL agent interacts with the network environment to generate the weight of the critical link in the virtual traffic topology. The GNN is designed to process the network topology structure and generalize to different scale topology structures, so that the routing algorithm has scalability without additional repeated training. In stages 3 and 4, thanks to the division mode of the rectangular control domain, the network topology in each domain also presents a highly regularized grid structure. First, the minimum hop region of a given source and destination node is defined according to the Manhattan distance. In this region, multiple paths have the same minimum hop number, and the delay is also close to each other. Then, the A* algorithm is used to calculate the K minimum hop paths that meet the delay constraint in the minimum hop region according to the link weight of the virtual traffic topology. Finally, the routing convergence occurs in the virtual traffic topology, and the actual traffic forwarding relies on the physical topology.

[0059] As shown in Figure 5 , the detailed design of the state space, action space and reward function of the DRL agent is as follows:

[0060] 1) State space: the network state is represented as s t = [V, H, edge_index], including node information V, link information H, and edge index information edge_index indicating the connection relationship of the network.

[0061] Specifically, the node information v is a matrix with a dimension of 2N x L, containing network traffic demand information. V = [v1, v2, …, vN] (1) N ] T where N is the number of network nodes, and L is the length of the feature vector of each node. As shown in equation (1), v i is composed of the traffic vectors flowing into and out of the node, and is extended to a length L by padding with 0.

[0062] The link information H is a matrix with a dimension of E x 3, where E is the number of network links, and the 3-dimensional vector (x1, x2, x3) respectively represents the link delay, the failure link flag, and the critical link flag.

[0063] The edge index information edge_index is a matrix with a dimension of 2 x E, where the two elements in each column represent the existence of a link composed of the two nodes in the network, as shown in equation (2), where u i and v i represent the starting and ending nodes of the i-th edge, respectively.

[0064]

[0065] 2) Action set: Based on the above observation state, the DRL agent outputs the virtual weight adjustment ratio of the critical link and forms the virtual weight for routing calculation. Where, for time step t, the corresponding action is represented as a t = {a1, a2, …, a λ}, where a i is the virtual weight adjustment ratio of the i-th critical link, and λ is the number of critical links.

[0066] 3) Reward function: After changing the virtual weight of λ critical links for state s t , routing convergence is performed, the path of traffic demand is calculated, and the actual utilization of each link is obtained. The reward r t is defined as 1 / MLU to minimize the maximum link utilization, and the network performance is improved through DRL training. The smaller the MLU, the more favorable reward signal the agent can obtain.

[0067] In the GNN-based DRL agent, GCN is used as the feature extractor of the policy network and value network in the PPO algorithm, including three stages of initialization, message passing, and readout. In the initialization stage, the input s t is converted into a feature vector X (0) = φ(t), then message passing is achieved through the mechanism of aggregation and update, and the node features are updated using edge feature embedding until the nodes are stable.

[0068] where the message represents is:

[0069]

[0070] where, is the set of neighboring nodes of node v, is the feature vector of neighboring node u after iteration l-1 times, is the feature vector of the link composed of nodes u and v after iteration l-1 times, W (l) and B (l) is the weight matrix, is the feature vector of node v after iteration l times, and σ represents a nonlinear activation function ReLU, M t is a message passing function, which is used to perform nonlinear transformation or activation operation (such as ReLU, Sigmoid, etc.) on the features of nodes and links to realize feature aggregation.

[0071] In the readout phase, the information of all nodes in the graph is summarized into a graph-level representation, which is usually represented as the weighted sum of the node features output by the last layer L.

[0072] In the training process of the PPO control algorithm based on GCN, the GNN-based action network θ A and the value network θ C are first initialized. Then, the environment s t is reset and the action a t is performed, and the reward r t is obtained. This process is repeated to collect the s t a t r t sequence. Then, the reward accumulation is calculated based on the collected data. Finally, the network parameters are iteratively optimized using the gradient update method of PPO.

[0073] The elastic routing method provided by the embodiments of the present application generates actions from the controller according to the network state, calculates the reward function, and repeats the process to complete the training of the agent. Finally, the corresponding routing table is updated according to the new link weight, so as to maximize the path reliability under the delay constraint.

[0074] Corresponding to the above method, as shown in Figure 6 the embodiments of the present application also provide a LEO mega constellation elastic routing device based on graph reinforcement learning, which comprises a constellation topology division module and a routing control module.

[0075] The constellation topology division module is configured to divide the constellation topology into a plurality of control domains; all the control domains have the same shape and contain the same number of LEO satellites in the domains. The routing control module is configured to manage the plurality of control domains based on an SDN architecture using master-slave controllers; the master controller calculates a global shortest path from a source node to a destination node, decomposes a cross-domain traffic request between the source-destination node pair into a plurality of intra-domain traffic requests according to domain boundary nodes and inter-domain links passed by the global shortest path, the slave controller constructs a virtual traffic topology according to total intra-domain traffic requests and fault links, selects critical links in the domain based on the virtual traffic topology, extracts a link feature matrix of the virtual traffic topology by using GNN, and updates weights of the critical links based on the link feature matrix by using a reinforcement learning algorithm, and then generates an intra-domain routing path based on the new link weights under the constraints of path latency and link bandwidth.

[0076] It should be noted that the elastic routing device provided by the embodiment of the present application is used to implement the above-mentioned elastic routing method, and the functions thereof can be referred to the above-mentioned method embodiments, which will not be described here again.

[0077] Figure 7 An example of a schematic diagram of a physical structure of an electronic device is shown in Figure 7 As shown in the figure, the electronic device can include a processor 701, a communications interface 702, a memory 703, and a communications bus 704, wherein the processor 701, the communications interface 702, and the memory 703 complete mutual communication through the communications bus 704. The processor 701 can invoke a logical instruction in the memory 703 to execute an elastic routing method, which includes: dividing a constellation topology into a plurality of control domains, and managing the plurality of control domains based on an SDN architecture using master-slave controllers; all the control domains have the same shape and contain the same number of LEO satellites in the domains; the master controller calculates a global shortest path from a source node to a destination node, decomposes a cross-domain traffic request between the source-destination node pair into a plurality of intra-domain traffic requests according to domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller constructs a virtual traffic topology according to total intra-domain traffic requests and fault links, and selects critical links in the domain based on the virtual traffic topology; the slave controller extracts a link feature matrix of the virtual traffic topology by using GNN, and updates weights of the critical links based on the link feature matrix by using a reinforcement learning algorithm, and then generates an intra-domain routing path based on the new link weights under the constraints of path latency and link bandwidth.

[0078] In addition, the logic instructions in the memory 703 described above are implemented in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc.

[0079] The embodiments of the present application also provide a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can perform the elastic routing method provided by the method embodiments.

[0080] The embodiments of the present application also provide a non-transitory computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method provided by the method embodiments is implemented.

[0081] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary universal hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions essentially or the parts that make contributions to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A resilient routing method for LEO mega-constellations based on graph reinforcement learning, characterized in that, include: Step 1: Divide the constellation topology into multiple control domains, and use a master-slave controller based on the SDN architecture to manage multiple control domains; wherein, all control domains have the same shape and contain the same number of LEO satellites; Step 2: The main controller calculates the global shortest path from the source node to the destination node. Based on the domain boundary nodes and inter-domain links traversed by the global shortest path, it decomposes the cross-domain traffic request between the source and destination nodes into multiple intra-domain traffic requests. Step 3: The controller constructs a virtual traffic topology based on the total intra-domain traffic requests and faulty links, and selects critical links within the domain based on the virtual traffic topology; Step 4: Extract the link feature matrix of the virtual traffic topology from the controller using GNN, and based on the link feature matrix, update the weights of key links using reinforcement learning algorithm, and then generate intra-domain routing paths based on the new link weights under the constraints of path delay and link bandwidth.

2. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 1, characterized in that, In step 2, the main controller uses the A* algorithm to calculate the global shortest path from the source node to the destination node; the heuristic function chosen is the Manhattan distance.

3. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 2, characterized in that, The heuristic function is: h( n ) = δ avg ×(|x n -x m |+|y n -y m |) Where, δ avg |x represents the average latency of the link. n -x m |+|y n -y m | represents the Manhattan distance traveled from the source node n to the destination node m along the grid lines.

4. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 1, characterized in that, Step 3, whereby the controller constructs a virtual traffic topology based on intra-domain traffic requests and faulty links, specifically includes: Collect real-time traffic requests within the domain, merge the real-time traffic requests with the domain traffic requests from the main controller to obtain the total domain traffic requests; treat failed links as links completely exhausted by burst traffic; thereby generating a domain traffic demand matrix.

5. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 4, characterized in that, In step 3, the critical link within the domain is selected based on the virtual traffic topology using the Monte Carlo algorithm, specifically including: The local shortest path between pairs of nodes at the domain boundary is calculated based on the traffic demand matrix within the domain, as well as the load of each link under the local shortest path, and the sampling probability of each link is calculated; wherein, the sampling probability of a link is proportional to the load of the link. Based on the sampling probability of the links, a fault scenario s containing λ links is repeatedly sampled from within the domain. It is checked whether fault scenario s already exists in the preset fault scenario set. If not, the maximum link utilization (MLU) corresponding to the total intra-domain traffic requests under fault scenario s is calculated, and fault scenario s is added to the preset fault scenario set, and the corresponding MLU is added to the preset MLU metric set. If so, resampling is performed. Check if the number of fault scenarios in the fault scenario set meets the requirements. If yes, select the λ links with the highest MLU as the critical links. If not, return to the previous step.

6. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 1, characterized in that, Step 4, specifically the extraction of the link feature matrix of the virtual traffic topology from the controller using a GNN, includes: The network state of the virtual traffic topology is initialized as a feature vector, and the message representation after l iterations is defined. for: in, This represents the set of adjacent nodes of node v. Let represent the feature vector of the adjacent node u after iteration l-1. W represents the feature vector of the link composed of nodes u and v after l-1 iterations. (l) and B (l) Represents the weight matrix. Let M represent the feature vector of node v after l iterations, σ represent the nonlinear activation function ReLU, and M represent the feature vector of node v after l iterations. t This represents a message passing function; After a total of L iterations, the feature vectors of all nodes are aggregated into a full-graph level representation h. G , represented as 7. The LEO giant constellation resilient routing method based on graph reinforcement learning according to claim 1, characterized in that, In step 4, the weights of the critical links are updated using a reinforcement learning algorithm, specifically including: Define the state space: Represent the network state as s t = [V,H,edge_index], where V represents node information, H represents link information, and edge_index is the edge index information used to indicate network connection relationships; Define the action set: represent the action corresponding to time step t as a. t ={a1,a2,…,a λ }, where a i It is the virtual weight adjustment ratio of the i-th critical link; Define the reward function: Execute action a t Then, the maximum link utilization (MLU) corresponding to the total intra-domain traffic requests is calculated, and the reward r is then calculated. t It is 1 / MLU.

8. A LEO giant constellation elastic routing device based on graph reinforcement learning, characterized in that, include: The constellation topology partitioning module is used to divide the constellation topology into multiple control domains; all control domains have the same shape and contain the same number of LEO satellites. The routing control module is used to manage multiple control domains based on an SDN architecture using master-slave controllers. The master controller calculates the global shortest path from the source node to the destination node. Based on the domain boundary nodes and inter-domain links traversed by the global shortest path, it decomposes cross-domain traffic requests between the source and destination nodes into multiple intra-domain traffic requests. The slave controller constructs a virtual traffic topology based on the total intra-domain traffic requests and faulty links. Based on this virtual traffic topology, it selects critical links within the domain and extracts the link feature matrix of the virtual traffic topology using a GNN. Based on this link feature matrix, it updates the weights of the critical links using a reinforcement learning algorithm, thereby generating intra-domain routing paths based on the new link weights under the constraints of path latency and link bandwidth.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images