LEO giant constellation elastic routing method based on graph reinforcement learning and related equipment

Abstract

The invention provides an LEO giant constellation elastic routing method based on graph reinforcement learning and related equipment, and relates to the technical field of satellite communication. The method comprises the following steps: dividing a constellation topology into a plurality of control domains, and managing the plurality of control domains by adopting a master-slave controller based on an SDN architecture; the master controller calculates a global shortest path from a source node to a destination node, and decomposes a cross-domain traffic request between the source and destination node pairs into a plurality of intra-domain traffic requests according to domain boundary nodes and inter-domain links passed by the global shortest path; and the slave controller constructs a virtual traffic topology according to the total intra-domain traffic request and the fault link, selects a key link in the domain, extracts a link feature matrix of the virtual traffic topology by using GNN, updates the weight of the key link by using a reinforcement learning algorithm, and determines the total intra-domain traffic request. And generating an intra-domain routing path based on the new link weight under the constraint of path delay and link bandwidth. According to the invention, the requirements of high availability and low delay of LEO giant constellation data transmission can be met.

Description

Technical Field

[0001] The present invention relates to the field of satellite communication technology, and in particular to a LEO giant constellation elastic routing method based on graph reinforcement learning and related equipment. Background Art

[0002] With the development of communication technology and the significant reduction in satellite manufacturing and launch costs, LEO giant constellations form a large-scale network system by densely deploying small satellites to provide low-latency, high-bandwidth and wide-coverage network access services. As a key infrastructure for future global communications and the next generation of the Internet, routing plays a key role in satellite networks, which not only determines the connectivity of data transmission, but also directly affects the overall performance and operating efficiency of the network.

[0003] However, the unique network characteristics of LEO giant constellations pose many challenges to the design of routing algorithms. First, LEO satellites are exposed to an open and harsh space environment and face natural threats such as space debris and solar storms. At the same time, small satellites have a short lifespan and have multiple security vulnerabilities and backdoors, making them potential entry points for cyber attacks. Finally, geographical conditions such as uneven population distribution and differences in economic development and the dynamic characteristics of orbital motion affect global traffic patterns, resulting in unbalanced network loads, that is, link bandwidth in hot spots is overloaded for a long time, while other link bandwidths are idle and wasted. All of the above factors may lead to network failures in LEO giant constellations.

[0004] The combination of the centralized control architecture of software-defined networking (SDN) and the intelligent routing method based on deep reinforcement learning (DRL) shows good control capabilities over traffic transmission and is being widely studied to optimize traffic distribution. However, the existing intelligent routing methods applied to LEO giant constellations face the problems of low robustness and difficulty in scalability. On the one hand, DRL-based routing usually requires controlling all units of the target element (link or data flow) in the network, while the dense link elements in the LEO giant constellation cause the state and action space to increase dramatically, making it difficult to effectively train and converge the model. In addition, these routing algorithms usually use traditional neural network architectures such as fully connected, which are prone to overfitting the training topology. Once a network failure causes a change in the topology, the routing performance will degrade severely. Summary of the invention

[0005] In order to meet the high availability and low latency requirements of LEO giant constellation data transmission, the present invention proposes a LEO giant constellation elastic routing method and related equipment based on graph reinforcement learning, aiming to cope with frequent network failures and huge network topology.

[0006] In a first aspect, the present invention provides a LEO giant constellation elastic routing method based on graph reinforcement learning, comprising:

[0007] Step 1: Divide the constellation topology into multiple control domains, and use a master-slave controller based on the SDN architecture to manage the multiple control domains; where all control domains have the same shape and contain the same number of LEO satellites;

[0008] Step 2: The main controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path;

[0009] Step 3: The slave controller constructs a virtual traffic topology according to the total intra-domain traffic requests and the faulty links, and selects key links in the domain based on the virtual traffic topology;

[0010] Step 4: Use GNN to extract the link feature matrix of the virtual traffic topology from the controller, and based on the link feature matrix, use a reinforcement learning algorithm to update the weights of key links, and then generate an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0011] Furthermore, in step 2, the main controller uses the A* algorithm to calculate the global shortest path from the source node to the destination node; wherein the heuristic function selects the Manhattan distance.

[0012] Furthermore, the heuristic function is:

[0013] h(n)=δ avg ×(|x n -x m |+|y n -y m |)

[0014] Among them, δ avg represents the average delay of the link, |x n -x m |+|y n -y m | is the Manhattan distance from the source node n to the destination node m along the grid line.

[0015] Furthermore, in step 3, the slave controller constructs a virtual traffic topology according to the intra-domain traffic request and the faulty link, specifically including:

[0016] Collect the real-time traffic requests within the domain, merge the real-time traffic requests with the domain traffic requests from the main controller to obtain the total domain traffic requests; regard the failed link as a link completely exhausted by the burst traffic; thus generate the traffic demand matrix within the domain.

[0017] Further, in step 3, a key link is selected in the domain based on the virtual traffic topology by using a Monte Carlo algorithm, specifically including:

[0018] According to the traffic demand matrix in the domain, the local shortest path between the node pairs at the domain boundary and the load of each link under the local shortest path are calculated, and the sampling probability of each link is calculated; wherein the sampling probability of the link is proportional to the load of the link;

[0019] According to the sampling probability of the link, a fault scenario s containing λ links is repeatedly sampled from the domain, and the fault scenario s is checked to see if it already exists in the preset fault scenario set. If not, the maximum link utilization MLU corresponding to the total intra-domain traffic request under the fault scenario s is calculated, and the fault scenario s is added to the preset fault scenario set, and the corresponding MLU is added to the preset MLU indicator set; if yes, re-sample;

[0020] Check whether the number of fault scenarios in the fault scenario set meets the requirements. If so, select the λ links with the highest MLU as the key links. If not, return to the previous step.

[0021] Further, in step 4, the slave controller uses GNN to extract the link feature matrix of the virtual traffic topology, specifically including:

[0022] Initialize the network state of the virtual traffic topology as a feature vector and define the message representation after l iterations. for:

[0023]

[0024] in, represents the set of adjacent nodes of node v, represents the feature vector of the adjacent node u after iteration l-1 times, represents the feature vector of the link composed of nodes u and v after iteration l-1 times, W (l) and B (l) represents the weight matrix, represents the feature vector of node v after iteration l, σ represents the nonlinear activation function ReLU, M t Represents a message passing function;

[0025] After completing L total iterations, the feature vectors of all nodes are aggregated into a full-graph representation h G , expressed as

[0026] Furthermore, in step 4, a reinforcement learning algorithm is used to update the weights of key links, specifically including:

[0027] Define state space: The network state is represented as s t=[v,H,edge_index], where V represents node information, H represents link information, and edge_index is edge index information used to indicate network connection relationship;

[0028] Define the action set: The action corresponding to time step t is represented as a t ={a1,a2,…,a λ}, where a i is the virtual weight adjustment ratio of the i-th critical link;

[0029] Define the reward function: perform action a t After that, the maximum link utilization MLU corresponding to the total intra-domain traffic request is calculated, and then the reward r is calculated. t It is 1 / MLU.

[0030] In a second aspect, the present invention provides a LEO giant constellation elastic routing device based on graph reinforcement learning, comprising:

[0031] A constellation topology partitioning module, used for partitioning the constellation topology into a plurality of control domains; wherein all control domains have the same shape and contain the same number of LEO satellites;

[0032] A routing control module is used to manage multiple control domains using a master-slave controller based on an SDN architecture; wherein the master controller calculates the global shortest path from a source node to a destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller builds a virtual traffic topology according to the total intra-domain traffic requests and faulty links, selects key links in the domain based on the virtual traffic topology, and uses GNN to extract the link feature matrix of the virtual traffic topology, and based on the link feature matrix, uses a reinforcement learning algorithm to update the weights of the key links, and then generates an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0033] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0034] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) By constructing a routing method based on graph reinforcement learning, the present invention can effectively deal with the routing instability problem caused by link failure or high load in the LEO satellite network, and ensure the high availability of data transmission in a dynamic network environment.

[0037] (2) The present invention combines the master-slave controller design of the SDN architecture. The master controller is responsible for the cross-domain planning of the global shortest path, and the slave controller combines virtual traffic topology and reinforcement learning to control the key links in the domain, dynamically adjust the path, and improve the flexibility and accuracy of routing decisions.

[0038] (3) The present invention adopts the Monte Carlo algorithm to identify high-load links and combines it with the reinforcement learning algorithm to control the weights of key links, which can minimize the impact of link failure on the overall network performance and enhance the network's fault resistance.

[0039] (4) The present invention adapts to network topologies of different scales through the generalization ability of GNN, and can still maintain efficient routing performance when the topology structure changes, thus overcoming the defect that the traditional DRL-based intelligent routing method is easily affected by network failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 An overall framework diagram of a LEO giant constellation elastic routing method based on graph reinforcement learning provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the basic mechanism of the A* algorithm provided by an embodiment of the present invention in a grid constellation topology;

[0042] Figure 3 A schematic diagram of a critical link selection process based on Monte Carlo simulation provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a domain routing calculation process based on graph reinforcement learning provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a GNN-based DRL routing method provided in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the structure of a LEO giant constellation elastic routing device based on graph reinforcement learning provided by an embodiment of the present invention;

[0046] Figure 7 A structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a LEO giant constellation elastic routing method based on graph reinforcement learning, comprising the following steps:

[0049] S101: Divide the constellation topology into multiple rectangular control domains according to the grid, and use a master-slave controller based on the SDN architecture to manage the multiple control domains; each control domain contains 3*3 LEO satellites;

[0050] S102: The main controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path;

[0051] Specifically, given that the LEO satellite network topology presents a highly regular grid shape, this embodiment uses the A* algorithm to quickly determine the domain boundary nodes and inter-domain links passed through. The heuristic function selects the Manhattan distance. The Manhattan distance, that is, the straight-line distance between grid points, is suitable for estimating the shortest path problem in a grid layout. In combination with the present invention's requirement for low latency, this physical distance is further converted into an estimate of latency. First, the average latency δ of the link is obtained based on historical data or actual measurements. avg , then the heuristic function h(n) can be defined as h(n) = δ avg ×(|x n -x m |+|y n -y m |), here |x n -x m |+|y n -y m |The calculation is the Manhattan distance from the source node n to the destination node m along the grid line, such as Figure 2 shown.

[0052] S103: The slave controller constructs a virtual traffic topology according to the total traffic requests in the domain and the faulty links, and selects a key link in the domain based on the virtual traffic topology;

[0053] Specifically, real-time traffic requests within the domain are collected, and the real-time traffic requests are merged with the domain traffic requests from the main controller to obtain the total domain traffic requests; the faulty link is regarded as a link completely exhausted by burst traffic; thereby generating the domain traffic demand matrix TM. Since various sudden network failures will bring about changes in the physical topology, the embodiment of the present invention regards the link failure of the physical topology as the burst traffic completely exhausting the link, and constructs a virtual traffic topology in combination with the total domain traffic requests, that is, converting complex link failure combinations into diversified traffic demands, thereby eliminating the uncertainty of topology changes caused by network failures and achieving the purpose of shielding network failures.

[0054] S104: extracting the link feature matrix of the virtual traffic topology from the controller using GNN, and based on the link feature matrix, using a reinforcement learning algorithm to update the weights of key links, and then generating an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0055] The elastic routing method provided by the embodiment of the present invention mainly includes the inter-domain routing step (i.e., generating the global shortest path) that the master controller is responsible for and the intra-domain routing step (i.e., generating the local shortest path within the domain) that the slave controller is responsible for. In each control domain, the SDN controller obtains the routing request of the source and destination node pairs, and builds a virtual traffic topology in combination with the faulty link to shield the network failure; based on the virtual traffic topology, the application plane adjusts the weights of some key links through the graph reinforcement learning algorithm, and calculates the routing path according to the new link weight under the constraints of path delay and link bandwidth, optimizes the routing decision, improves the path reliability of the satellite routing method under link failure, and keeps the end-to-end delay within an acceptable range, meeting the high availability and low latency requirements of LEO giant constellation data transmission.

[0056] On the basis of the above embodiments, the embodiment of the present invention adopts the Monte Carlo algorithm to select key links in the domain based on the virtual traffic topology. Specifically, the Monte Carlo algorithm is used to perform a limited N combination of fault scenarios. mc The probability of a link being selected is proportional to the initial load of the link under the shortest path (SP) routing scheme. mc When is 1, the key link selection algorithm degenerates to selecting the first λ links with the largest load in the network. Figure 3 As shown in Figure 2, firstly, the load of each link is calculated based on the SP routing and the current traffic demand TM, and the link sampling probability P is obtained. sampleWhen the number of simulations is less than N (N represents the number of LEO satellite nodes in the domain), a fault scenario s containing λ links is repeatedly sampled and checked whether this scenario already exists in the set S. If s is a new scenario, the maximum link utilization (MLU) corresponding to all traffic demands in the scenario is calculated, and the scenario s and the MLU index i are combined. s Add them to the sets S and I respectively. Finally, select the link set with the highest MLU index from the stored fault scenarios as the key link

[0057] In the above embodiment, it is mentioned that the master controller decomposes the cross-domain traffic request into multiple intra-domain traffic requests, and the corresponding slave controller uses the intra-domain routing process based on graph reinforcement learning (GNN+DR) to ensure that the flow is reliably transmitted within its domain. Figure 4 and Figure 5 As shown, the embodiment of the present invention further describes the intra-domain routing process, as shown in FIG. Figure 4 As shown, it mainly includes 4 stages.

[0058] In phase 1, the slave controller of each domain identifies traffic requests and failed links, thereby combining the total intra-domain traffic requests and the failed links that are considered to be exhausted by burst traffic, and finally a new traffic demand matrix TM is obtained. In phase 2, key links are selected as control elements in an online manner, and the weights of key links in the virtual traffic topology are generated through the interaction of the GNN-based DRL agent with the network environment. Among them, GNN is designed to handle network topology and generalize to topologies of different scales, making the routing algorithm scalable without additional repeated training. In phases 3 and 4, thanks to the division of the rectangular control domain, the network topology in each domain also presents a highly regularized grid structure. First, the minimum hop area for a given source and destination node is defined according to the Manhattan distance. In this area, multiple paths have the same minimum number of hops and the delays are close to each other. Then, the A* algorithm is used to calculate K minimum hop paths that meet the delay constraints in the minimum hop area according to the link weights of the virtual traffic topology. Finally, routing convergence occurs in the virtual traffic topology, while the actual traffic forwarding depends on the physical topology.

[0059] Among them, Figure 5 As shown in the figure, the detailed design of the state space, action space and reward function of the DRL agent is as follows:

[0060] 1) State space: The network state is represented as s t =[V,H,edge_index], including node information V, link information H, and edge index information edge_index indicating the network connection relationship.

[0061] Specifically, the node information v is a matrix with a dimension of 2N×L, which contains network traffic demand information. V=[v1,v2,…,v N ] T , where N is the number of network nodes and L is the length of the feature vector of each node. As shown in formula (1), v i It consists of the flow vectors into and out of the node, extended to length L by zero padding.

[0062] The link information H is a matrix of dimension E×3, where E is the number of network links and the 3D vectors (x1, x2, x3) are link delay, fault link flag, and critical link flag, respectively.

[0063] The edge index information edge_index is a matrix with a dimension of 2×E. The two elements in each column represent the existence of a link consisting of these two nodes in the network, as shown in formula (2), where u i and v i They represent the starting and ending nodes of the i-th edge respectively.

[0064]

[0065] 2) Action set: Based on the above observation state, the DRL agent outputs the virtual weight adjustment ratio of the key link and forms the virtual weight for routing calculation. For time step t, the corresponding action is represented as a t ={a1,a2,…,a λ}, where a i is the virtual weight adjustment ratio of the i-th critical link, and λ is the number of critical links.

[0066] 3) Reward function: In state s t After changing the virtual weights of λ key links, perform routing convergence, calculate the paths of traffic demands, and obtain the actual utilization of each link. t Defined as 1 / MLU to minimize the maximum link utilization and improve network performance through DRL training. The smaller the MLU, the more favorable the reward signal the agent can obtain.

[0067] In the GNN-based DRL agent, GCN is used as the feature extractor of the policy network and value network in the PPO algorithm, which includes three stages: initialization, message passing, and readout. In the initialization stage, the input s t Convert to feature vector X (0) =φ(t), and then the message passing is realized through the aggregation and update mechanism, and the node features are updated by embedding the edge features until each node is stable.

[0068] Among them, after iteration l, the message indicates for:

[0069]

[0070] in, represents the set of adjacent nodes of node v, represents the feature vector of the adjacent node u after iteration l-1 times, represents the feature vector of the link composed of nodes u and v after iteration l-1 times, W (l) and B (l) represents the weight matrix, represents the feature vector of node v after iteration l, σ represents the nonlinear activation function ReLU, M t Represents the message passing function, which is used to perform nonlinear transformation or activation operations (such as ReLU, Sigmoid, etc.) on the features of nodes and links to achieve feature aggregation.

[0071] In the readout phase, the information of all nodes in the graph is summarized into a full-graph level representation, usually using the weighted sum of the node features output by the last layer L, expressed as

[0072] In the training process of the GCN-based PPO control algorithm, the GNN-based action network θ is first initialized. A and value network θ C Next, reset the environment t And perform action a t , get reward r t Repeat the process to collect s t a t r t Then, based on the collected data, the reward accumulation is calculated Finally, the network parameters are iteratively optimized using the gradient update method of PPO.

[0073] The elastic routing method provided by the embodiment of the present invention generates actions according to the network status from the controller, calculates the reward function, and repeats the process to complete the training of the intelligent agent. Finally, the corresponding routing table is updated according to the new link weight to maximize the path reliability under the delay constraint.

[0074] Corresponding to the above method, such as Figure 6 As shown, an embodiment of the present invention further provides a LEO giant constellation elastic routing device based on graph reinforcement learning, including: a constellation topology division module and a routing control module.

[0075] Among them, the constellation topology division module is used to divide the constellation topology into multiple control domains; wherein all control domains have the same shape and contain the same number of LEO satellites. The routing control module is used to manage multiple control domains based on the SDN architecture using a master-slave controller; wherein the master controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller builds a virtual traffic topology based on the total intra-domain traffic requests and faulty links, and selects key links in the domain based on the virtual traffic topology, and uses GNN to extract the link feature matrix of the virtual traffic topology, and based on the link feature matrix, uses a reinforcement learning algorithm to update the weights of key links, and then generates an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0076] It should be noted that the elastic routing device provided in the embodiment of the present invention is to implement the elastic routing method described above. The specific functions thereof can be referred to the method embodiment described above, and will not be described in detail here.

[0077] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor (processor) 701, a communication interface (Communications Interface) 702, a memory (memory) 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704. The processor 701 can call the logic instructions in the memory 703 to execute the elastic routing method, which includes: dividing the constellation topology into multiple control domains, and using a master-slave controller to manage the multiple control domains based on the SDN architecture; wherein all control domains have the same shape and contain the same number of LEO satellites; the master controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller builds a virtual traffic topology according to the total intra-domain traffic requests and faulty links, and selects key links in the domain based on the virtual traffic topology; the slave controller uses GNN to extract the link feature matrix of the virtual traffic topology, and based on the link feature matrix, uses a reinforcement learning algorithm to update the weights of the key links, and then generates an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

[0078] In addition, when the logic instructions in the above-mentioned memory 703 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0079] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the elastic routing method provided by the above-mentioned method embodiments.

[0080] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the methods provided by the above-mentioned method embodiments are implemented.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A LEO giant constellation elastic routing method based on graph reinforcement learning, characterized in that: include: Step 1: Divide the constellation topology into multiple control domains, and use a master-slave controller based on the SDN architecture to manage the multiple control domains; where all control domains have the same shape and contain the same number of LEO satellites; Step 2: The main controller calculates the global shortest path from the source node to the destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; Step 3: The slave controller constructs a virtual traffic topology according to the total intra-domain traffic requests and the faulty links, and selects key links in the domain based on the virtual traffic topology; Step 4: Use GNN to extract the link feature matrix of the virtual traffic topology from the controller, and based on the link feature matrix, use a reinforcement learning algorithm to update the weights of key links, and then generate an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

2. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 1, characterized in that: In step 2, the main controller uses the A* algorithm to calculate the global shortest path from the source node to the destination node; among them, the heuristic function selects the Manhattan distance.

3. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 2, characterized in that: The heuristic function is: h( n ) = δ avg ×(|x n -x m |+|y n -y m |) Among them, δ avg represents the average delay of the link, |x n -x m |+|y n -y m | is the Manhattan distance from the source node n to the destination node m along the grid line.

4. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 1, characterized in that: In step 3, the slave controller builds a virtual traffic topology according to the intra-domain traffic request and the faulty link, specifically including: Collect the real-time traffic requests within the domain, merge the real-time traffic requests with the domain traffic requests from the main controller to obtain the total domain traffic requests; regard the failed link as a link completely exhausted by the burst traffic; thus generate the traffic demand matrix within the domain.

5. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 4, characterized in that: In step 3, a key link is selected in the domain based on the virtual traffic topology by using a Monte Carlo algorithm, specifically including: According to the traffic demand matrix in the domain, the local shortest path between the node pairs at the domain boundary and the load of each link under the local shortest path are calculated, and the sampling probability of each link is calculated; wherein the sampling probability of the link is proportional to the load of the link; According to the sampling probability of the link, a fault scenario s containing λ links is repeatedly sampled from the domain, and the fault scenario s is checked to see if it already exists in the preset fault scenario set. If not, the maximum link utilization MLU corresponding to the total intra-domain traffic request under the fault scenario s is calculated, and the fault scenario s is added to the preset fault scenario set, and the corresponding MLU is added to the preset MLU indicator set; if yes, re-sample; Check whether the number of fault scenarios in the fault scenario set meets the requirements. If so, select the λ links with the highest MLU as the key links. If not, return to the previous step.

6. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 1, characterized in that: In step 4, the slave controller uses GNN to extract the link feature matrix of the virtual traffic topology, specifically including: Initialize the network state of the virtual traffic topology as a feature vector and define the message representation after l iterations. for: in, represents the set of adjacent nodes of node v, represents the feature vector of the adjacent node u after iteration l-1 times, represents the feature vector of the link composed of nodes u and v after iteration l-1 times, W (l) and B (l) represents the weight matrix, represents the feature vector of node v after iteration l, σ represents the nonlinear activation function ReLU, M t Represents a message passing function; After completing L total iterations, the feature vectors of all nodes are aggregated into a full-graph representation h G , expressed as 7. The LEO giant constellation elastic routing method based on graph reinforcement learning according to claim 1, characterized in that: In step 4, the weights of key links are updated using a reinforcement learning algorithm, including: Define state space: represent the network state as s t =[V, H, edge_index], where V represents node information, H represents link information, and edge_index is edge index information used to indicate network connection relationship; Define the action set: The action corresponding to time step t is represented as a t ={a1, a2, ..., a λ }, where a i is the virtual weight adjustment ratio of the i-th critical link; Define the reward function: perform action a t After that, the maximum link utilization MLU corresponding to the total intra-domain traffic request is calculated, and then the reward r is calculated. t It is 1 / MLU.

8. A LEO giant constellation elastic routing device based on graph reinforcement learning, characterized in that: include: A constellation topology partitioning module, used for partitioning the constellation topology into a plurality of control domains; wherein all control domains have the same shape and contain the same number of LEO satellites; A routing control module is used to manage multiple control domains using a master-slave controller based on an SDN architecture; wherein the master controller calculates the global shortest path from a source node to a destination node, and decomposes the cross-domain traffic request between the source and destination node pairs into multiple intra-domain traffic requests according to the domain boundary nodes and inter-domain links passed by the global shortest path; the slave controller builds a virtual traffic topology according to the total intra-domain traffic requests and faulty links, selects key links in the domain based on the virtual traffic topology, and uses GNN to extract the link feature matrix of the virtual traffic topology, and based on the link feature matrix, uses a reinforcement learning algorithm to update the weights of the key links, and then generates an intra-domain routing path based on the new link weights under the constraints of path delay and link bandwidth.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images