Distributed intelligent routing method for heaven and earth fusion network

Abstract

The invention provides a space-ground convergence network distributed intelligent routing method. The space-ground convergence network distributed intelligent routing method comprises the following steps: constructing a space-ground convergence network system comprising a ground network and a satellite network; training the graph neural network according to the topological structure of the heaven and earth fusion network system; connecting the trained graph neural network with a depth Q network, and training an intelligent agent of the depth Q network through the heaven and earth fusion network system; deploying the trained intelligent agent to each satellite node in the space-ground fusion network system, and making an autonomous routing decision by the intelligent agent on the satellite node according to the output corresponding to the graph neural network and the network state information; wherein the output corresponding to the graph neural network is a result of classifying the topology according to the local topology information corresponding to the satellite node. Therefore, congestion and loop conditions can be better avoided, the load balancing performance is improved, and the flexibility of the algorithm is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a distributed intelligent routing method for a space-ground fusion network. Background Art

[0002] With the rapid development of mobile communication technology, terrestrial networks can already provide customized, high-bandwidth, and low-latency services. However, due to geographical constraints, the vision of ubiquitous communication in the future 6G era cannot be realized using only terrestrial networks. Complementary to terrestrial networks, satellite networks can cover oceans, mountains, forests, deserts, and other areas where networks cannot be provided by ground cellular base stations, and play an important role in the next generation of wireless communication systems.

[0003] Therefore, with the continuous development of network and communication technologies, the integration of satellite networks and ground networks to build a global seamless, ubiquitous space-ground integrated network has become a future network development trend. In the future, the space-ground integrated network will include low-orbit satellite networks, high-orbit satellite networks, ground networks and other networks, ultimately realizing the vision of interconnecting everything and ubiquitous communication.

[0004] In general, the space-ground integrated network has the characteristics of large-scale, complex, highly dynamic, and highly heterogeneous. However, the use of traditional routing methods in the space-ground integrated network will cause problems such as poor scalability and high routing overhead. How to adapt to the above problems puts forward more stringent requirements on the routing algorithm of the space-ground integrated network. Summary of the invention

[0005] The purpose of the present invention is to provide a distributed intelligent routing method for a space-ground fusion network, which can solve the problems of large scale, high dynamics, and routing calculation overhead of the existing space-ground fusion network.

[0006] In order to achieve the above object, the present invention provides a distributed intelligent routing method for a space-ground fusion network, comprising the steps of:

[0007] Construct a space-ground integrated network system including ground network and satellite network;

[0008] Training the graph neural network according to the topological structure of the space-ground fusion network system;

[0009] Connecting the trained graph neural network to the deep Q network, and training the intelligent agent of the deep Q network through the ground-ground fusion network system;

[0010] The trained intelligent agent is deployed on each satellite node in the space-ground integrated network system, and the intelligent agent on the satellite node makes autonomous routing decisions based on the output corresponding to the graph neural network and the network status information; wherein the output corresponding to the graph neural network is the result of classifying the topology based on the local topology information corresponding to the satellite node.

[0011] Optionally, the structure of the space-ground integrated network system including the ground network and the satellite network includes:

[0012] Design and construct a mesh topology of low-orbit satellite constellations based on intersatellite links in the longitude and latitude directions;

[0013] Setting up high-orbit satellites in geosynchronous orbit;

[0014] A ground network connected to a satellite network is built, constraints are set and visibility between ground stations and satellites of the ground network is calculated to realize the construction of a space-ground integrated network system; wherein the ground network includes the ground stations and router nodes, each network node in the space-ground integrated network system has a corresponding logical number and database, and when a link switch occurs at the network node, updated topology information is sent to adjacent nodes to update the database of the adjacent nodes.

[0015] Optionally, the training of the graph neural network according to the topological structure of the space-ground fusion network system includes:

[0016] The local topological information in the database is used as training samples and input into the graph neural network for training; wherein the graph neural network is used to extract topological features in the local topological information to classify the topological graph, and the local topological information includes a feature matrix of nodes and a matrix representing the graph structure.

[0017] Optionally, the agent of the deep Q network is composed of a target network, a decision network and an experience replay buffer, the target network is used to stabilize the training process of the agent, the decision network is used to make action decisions based on the environment state, and the experience replay buffer is used to store experience tuples in the process of interaction between the agent and the environment, the experience tuples including state information, actions based on the state information, rewards obtained by performing the actions, and next state information obtained by inputting the actions into the environment.

[0018] Optionally, the agent on the satellite node makes an autonomous routing decision according to the output corresponding to the graph neural network and the network status information, including:

[0019] The output of the graph neural network, the adjacent node numbers, the adjacent node congestion status, the source node numbers and the destination node numbers are normalized by the agent of the current node, and then input into the decision network to obtain several Q values ​​of the output corresponding to the decision network; wherein the Q value represents the routing action of the current node to select the next node;

[0020] Sorting the Q values ​​output by the decision network, and selecting the routing action corresponding to the maximum Q value as the target action;

[0021] If the next hop node pointed to by the target action route is congested or a non-adjacent node, the routing action corresponding to the next order Q value is selected from the plurality of Q values ​​in sequence as the target action until the next hop node pointed to by the target action route is not congested and is an adjacent node.

[0022] Optionally, the agent on the satellite node makes an autonomous routing decision according to the output corresponding to the graph neural network and the network status information, further comprising:

[0023] If all the next-hop nodes pointed to by the routing actions corresponding to the Q values ​​are congested or non-adjacent nodes, packet loss processing is performed on the data of the current node and the previous-hop node is notified.

[0024] Optionally, the agent on the satellite node makes an autonomous routing decision according to the output corresponding to the graph neural network and the network status information, further comprising:

[0025] If a routing loop occurs after traversing all the Q values, packet loss processing is performed on the data of the current node, and the previous hop node is notified.

[0026] Optionally, after the agent selects the next hop node, the process includes:

[0027] For time-sensitive services, the agent selects the correct path by weighting the load balancing index and the positive reward to calculate the corresponding reward function;

[0028] For time-sensitive services, when the agent selects an incorrect path, the corresponding reward function is weighted and calculated according to the delay index;

[0029] For time-sensitive services, when the agent selects the next hop node and a routing loop or congestion occurs, the corresponding reward function is calculated using the packet loss rate indicator;

[0030] For price-sensitive businesses, the agent calculates the corresponding reward function based on the weighted cost index when selecting the wrong path.

[0031] Optionally, the agent calculates actions to generate actions according to an ∈-greedy strategy during training.

[0032] Optionally, the network status information includes adjacent node information, congestion information, service demand identifiers in data packets, previous hop nodes, source nodes, and destination node numbers collected in the space-ground integrated network system.

[0033] The distributed intelligent routing method of the space-ground fusion network described in the present invention can better adapt to the problems of large scale, high dynamics, and large routing calculation overhead of the space-ground fusion network, and has lower storage resources than the global routing algorithm; the present invention is dynamically designed for different business needs, and can better meet the requirements of multi-scenario and multi-business of the space-ground fusion network, better avoid congestion and loops, and improve load balancing performance. In addition, the present invention also introduces a graph neural network, which can effectively improve the flexibility of the algorithm compared to the virtual topology algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of the steps of the distributed intelligent routing method of the space-ground fusion network provided by an embodiment of the present invention;

[0035] Figure 2 A flow chart of the steps of the intelligent agent making autonomous routing decisions in the distributed intelligent routing method for the space-ground fusion network provided by an embodiment of the present invention;

[0036] Figure 3 A topological diagram of the space-ground fusion network system constructed by the space-ground fusion network distributed intelligent routing method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0038] It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification refer to the embodiment described, which may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Furthermore, when describing specific features, structures or characteristics in conjunction with an embodiment, whether or not there is an explicit description, it has been shown that it is within the knowledge of those skilled in the art to combine such features, structures or characteristics into other embodiments.

[0039] In addition, certain words are used in the specification and subsequent claims to refer to specific components or parts. Those with ordinary knowledge in the relevant field should understand that manufacturers can use different nouns or terms to refer to the same component or part. This specification and subsequent claims do not use differences in names as a way to distinguish components or parts, but use differences in the functions of components or parts as the criteria for distinction. "Including" and "including" mentioned throughout the specification and subsequent claims are open-ended terms and should be interpreted as "including but not limited to". In addition, the word "connected" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0040] Figure 1 A distributed intelligent routing method for a space-ground fusion network provided by an embodiment of the present invention is shown, comprising the following steps:

[0041] S101: constructing a space-ground integrated network system including a ground network and a satellite network; that is, in step S101, a space-ground integrated network system including a ground network and a satellite network is constructed, in which the satellites on the satellite network are connected through inter-satellite links, and the satellite network and the ground network are connected through satellite-to-ground links.

[0042] In an optional implementation, step S101 includes:

[0043] A mesh topology structure of a low-orbit satellite constellation is constructed according to the inter-satellite link design in the direction of longitude and latitude; a high-orbit satellite is set in a geosynchronous orbit; a ground network connected to the satellite network is built, constraints are set and the visibility of the ground station and the satellite of the ground network is calculated to realize the construction of a space-ground integrated network system; wherein the ground network includes the ground station and router nodes, each network node in the space-ground integrated network system has a corresponding logical number and database, and when a link switching occurs at the network node, the updated topology information is sent to the adjacent node to update the database of the adjacent node.

[0044] In specific implementation, this embodiment first designs and constructs a mesh topology structure of a low-orbit satellite constellation according to the inter-satellite links in the direction of longitude and latitude. Each satellite has 4 inter-satellite links, including 2 inter-orbit links and 2 intra-orbit links, which respectively connect 2 satellites in the same orbital plane and 2 satellites in adjacent orbital planes; secondly, a high-orbit satellite is set, which is located in the geosynchronous orbit; finally, a ground network is built, constraints are set, and the visibility of the ground station and the satellite is calculated; building a ground network and calculating the visibility of the ground station and the satellite requires comprehensive consideration of multiple factors, and setting reasonable constraints; through scientific planning and calculation, the performance and reliability of the ground network can be improved, and efficient satellite communication can be achieved. Among them, the ground network is mainly composed of ground stations and router nodes. Satellites can use different beams to connect to multiple ground stations at the same time. Each node in the network has its own logical number. When a link switch occurs at a node, the node will send updated topology information to the adjacent node. The adjacent node will update its own database based on the received topology information, and the data contains node business demand identifiers.

[0045] See also Figure 3 The space-ground fusion network system constructed in this embodiment mainly includes two parts: satellite network and ground network. Among them, the inter-satellite links in the low-orbit satellite network topology include two types of links: intra-orbit links and inter-orbit links. The intra-orbit link is a link between two adjacent satellites in the same orbital plane, which is a stable link; the inter-orbit link is a link established by two satellites in two adjacent orbital planes. When the two satellites reach the polar regions, the inter-satellite link will be disconnected due to the excessive relative angular velocity. When the satellites in two adjacent orbits run in opposite directions, a "reverse seam" will be formed. It is very difficult to establish and maintain a cross-seam link. Therefore, this embodiment assumes that there is no inter-satellite link between the satellites on both sides of the "reverse seam". There is no connection between some ground stations and routers in the ground network.

[0046] Specifically, the network topology of each space-ground fusion network system can be represented as an undirected graph G = {(V, E) | G = G sat ∪G ter ∪E stl}. Among them, G sat =(V sat ,E sat ) represents the satellite network, V sat = {v sat1 ,v sat2 ,...,v satn} represents each satellite node in the network, E sat ={(v sat1 ,v sat2 ),...,(v sati ,v satj )} represents the intersatellite link between satellite nodes; G ter =(Vter ,E ter ) represents the ground network, V ter = {v ter1 ,v ter2 ,...,v tern}It consists of a ground station and a router. ter ={(v ter1 ,v ter2 ),...,(v teri ,v terj )} represents the link between each node; E stl ={(v sat1 ,v ter2 ),...,(v sati ,v terj )} represents the satellite-to-ground link.

[0047] S102: According to the topological structure of the earth-ground fusion network system, the graph neural network is trained; the graph neural network is a neural network specially used to process graph structure data. The core of the graph neural network is the information transmission mechanism; each node updates its own state or feature representation by exchanging and sharing information with its adjacent nodes; this information transmission process is usually iterative, and after multiple iterations, the node can obtain information about other nodes in the graph, thereby better representing the structure and characteristics of the entire graph. The graph neural network to be trained in this embodiment can obtain the local topological information corresponding to the node from the node in the earth-ground fusion network system, and then classify the topological graph based on the topological features therein; specifically, step S102 includes: taking the local topological information in the database as a training sample, and inputting it into the graph neural network for training; wherein the graph neural network is used to extract the topological features in the local topological information to classify the topological graph, and the local topological information includes the feature matrix of the node and the matrix representing the graph structure.

[0048] The feature matrix of this embodiment is X, and the matrix representing the graph structure is A. Then the size of the feature matrix is ​​N×F, where N is the number of nodes in the local topology, and F is the dimension of each node feature. The features of the node mainly include the node number, node type, and node degree. The degree of the node refers to the number of edges directly connected to the node.

[0049] Optionally, the propagation formula of the graph neural network of this embodiment is:

[0050] Among them, s represents a set containing k nodes, represents the vector of the set at the tth level, is the parameter of the network, N(s) is the domain of s, and σ is the activation function.

[0051] S103: Connect the trained graph neural network with the deep Q network, and train the intelligent agent of the deep Q network through the ground-ground fusion network system; connect the graph neural network with the deep Q network so that the output of the graph neural network is used as the input of the deep Q network, and use the graph neural network to process the graph structure data of the ground-ground fusion network system to extract the feature representation of nodes and edges; these features can be fused with the input features of the deep Q network to provide the intelligent agent with richer environmental information. Then, the deep Q network can select the optimal action based on these input information, that is, select the optimal next hop node to achieve better routing decision performance.

[0052] The agent of the deep Q network consists of a target network, a decision network, and an experience replay buffer. The target network is used to stabilize the training process of the agent, the decision network is used to make action decisions based on the environment state, and the experience replay buffer is used to store the experience tuples in the process of the agent interacting with the environment. The experience tuples include state information, actions based on the state information, rewards obtained by executing the actions, and next state information obtained by inputting the actions into the environment. The target network of this embodiment is Q(s t ,a t ;w - ), the decision network is Q(s t ,a t ; w); that is, the target network and the decision network are two neural networks with different parameters but the same structure. The experience playback buffer stores the experience obtained after each action. An experience is composed of (s t ,a t ,r t ,s t+1 ) is used to represent the t Represented as state information, which is the basis for the agent's decision, a t For state s t The action under which is obtained according to the output of the decision network, r t is in state s t Next, perform action a t The reward obtained is calculated by the reward function, s t+1 To make action a t The new state obtained by inputting into the environment.

[0053] During the training process of the agent, the agent first selects an action through the decision network and interacts with the environment to obtain the next state and reward; then, the next state is evaluated using the target network to obtain the target Q value; finally, the parameters of the decision network are updated by minimizing the error between the current Q value and the target Q value.

[0054] Optionally, the agent generates actions according to the ∈-greedy strategy calculation action during training. This allows the deep Q network to explore all routing options as much as possible at the beginning of training, where the ∈-greedy strategy formula is:

[0055] Among them, p is a random number generated each time an action is selected, and its value is between 0 and 1.

[0056] The ∈-greedy strategy balances exploration (randomly selecting actions with a certain probability and trying different possibilities) and exploitation (selecting the action currently considered optimal to obtain the maximum reward). As training proceeds, the value of ∈ can be gradually reduced, making the agent more and more inclined to choose the optimal action, thereby improving the convergence speed and performance of the algorithm.

[0057] S104: Deploy the trained agent to each satellite node in the space-ground fusion network system, and the agent on the satellite node makes an autonomous routing decision based on the output of the graph neural network and the network status information; wherein the output of the graph neural network is the result of classifying the topology according to the local topology information corresponding to the satellite node. That is, the agent on each satellite node makes an autonomous routing decision based on the output of the graph neural network and the network status information obtained by it. The agent of the deep Q network is an entity that can interact with the environment and learn how to make decisions to maximize the cumulative reward. The agent of this embodiment makes an autonomous routing decision based on two data inputs, one of which is the output of the graph neural network connected to the deep Q network, and the other is the network status information collected by the current satellite node; preferably, the network status information includes the adjacent node information, congestion information, business demand identification in the data packet, previous hop node, source node and destination node number collected in the space-ground fusion network system. The autonomous routing decision of the agent of this embodiment can select the next action based on the input data as the environment state, that is, select the next hop node for data transmission.

[0058] See also Figure 2 In an optional implementation, step S104 specifically includes:

[0059] S201: After normalizing the output of the graph neural network and the adjacent node numbers, adjacent node congestion status, source node and destination node numbers through the intelligent agent of the current node, the output is input into the decision network to obtain several Q values ​​corresponding to the output of the decision network; wherein the Q value represents the routing action of the current node to select the next node; the adjacent node numbers, adjacent node congestion status, source node and destination node numbers are collected from the real network of the space-ground fusion network system.

[0060] S202: Sort the Q values ​​output by the decision network, and select the routing action corresponding to the maximum Q value as the target action. Multiple Q values ​​indicate that the agent has decided on multiple routing actions, which specifically refer to next-hop nodes, i.e., different Q values ​​indicate different next-hop nodes pointed to by the routing.

[0061] Specifically, after step S202, step S203 is also included. Step S203: determine whether the next hop node corresponding to the target action is congested or a non-adjacent node. If the next hop node is congested, or the next hop node is not an adjacent node of the current node, proceed to step S204.

[0062] S204: If the next hop node pointed to by the target action route is congested or a non-adjacent node, the routing action corresponding to the next order Q value is selected from the several Q values ​​in sequence as the target action, until the next hop node pointed to by the target action route is not congested and is an adjacent node. In this embodiment, multiple Q values ​​output by the decision network are sorted, specifically, they are sorted from large to small by value size, and then the largest Q value is extracted from them, and the routing action corresponding to the extracted Q value is used as the target action to enter step S203 for judgment to determine whether the corresponding next hop node is optional. If the next hop node is congested or a non-adjacent node, the current maximum Q value is abandoned, and the maximum Q value of the next order is selected for repeated judgment until it is determined that the next hop node corresponding to a certain Q value does not have congestion or a non-adjacent node. At this time, the next hop node corresponding to the Q value can be determined as the target node for data forwarding.

[0063] In the above example, the next hop nodes corresponding to all Q values ​​decided by the agent may not meet the requirements or a routing loop may occur. In this regard, step S104 of this embodiment further includes:

[0064] S205: If all the next-hop nodes pointed to by the routing actions corresponding to the Q values ​​are congested or non-adjacent nodes, the data of the current node is discarded and the previous-hop node is notified. The current node is recorded as the satellite node deployed by the agent making the current decision, that is, when all the routing pointing results decided by the agent are congested or non-adjacent nodes, the data of the current node is discarded, and the data processed by the packet discard is the data that needs to be forwarded; at the same time, the previous-hop node of the current node is no longer forwarded to the current node, but other satellite nodes are selected for forwarding.

[0065] S206: If a routing loop occurs after traversing all the Q values, the data of the current node is discarded and the previous hop node is notified. A routing loop refers to a phenomenon in which a data packet is continuously forwarded between certain routing nodes in the network and cannot reach the correct destination; for example, if the next hop node decided by the current node coincides with its previous hop node, the most typical routing loop phenomenon occurs here; the processing measures for this phenomenon are the same as those of the above step S205, and the specific measures can be referred to the above step S205, which will not be repeated here.

[0066] Optionally, in the training of the agent, when calculating the path, the cost of the path is designed as the weighted sum of delay and cost, and the weight is dynamically adjusted for time-sensitive services and price-sensitive services. s To the destination node v t The cost is defined as: For node v i With node v j The propagation delay between the source node and the destination node is L, e L is the price of the path, and κ is the weight of the price.

[0067] When calculating the reward function after selecting the next hop node, for time-sensitive services, the agent selects the correct path by weighting the load balancing index and the positive reward; when selecting the wrong path, it is weighted according to indicators such as latency; when there is a routing loop or congestion, it is calculated by indicators such as packet loss rate. For price-sensitive services, the calculation method when the agent selects the wrong path is adjusted, and it is specifically weighted according to indicators such as cost.

[0068] Optionally, the calculation formula of the reward function of this embodiment is:

[0069]

[0070] Where B represents the load balancing index, P represents the packet loss rate, T represents the difference between the propagation delay from the current satellite node to the destination node and the transmission delay from the next hop satellite node to the destination node, C represents the difference between the transmission delay from the current satellite node to the destination node and the cost from the next hop satellite node to the destination node; β, θ, λ, η are different weights; l∈L loop Indicates that a routing loop occurs on the current path, l∈L right Indicates that the calculated correct path is selected, l∈L loss Indicates that packet loss occurred due to congestion; b∈S t Indicates that the service is time-sensitive, b∈S p Indicates that the business is price sensitive.

[0071] This embodiment achieves multi-objective optimization of the network by designing different reward functions and path calculation methods according to different business needs; by adding indicators such as load balancing and packet loss rate to the reward mechanism of the deep Q network, the network load performance is effectively improved and the packet loss rate is reduced; it can also meet different business needs of data, dynamically adapt to changes in network topology, and perform well in multi-objective optimization.

[0072] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0073] The method according to the present invention can be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or implemented in a combination of the two. The executable code or part thereof for the method according to the present invention can be stored on a computer program product. The example of a computer program product includes a memory device, an optical storage device, an integrated circuit, a server, online software, etc. Preferably, the computer program product includes a non-temporary program code component stored on a computer-readable medium so as to perform the method according to the present invention when the program product is executed on a computer.

[0074] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A distributed intelligent routing method for a space-ground fusion network, characterized in that: Includes steps: Construct a space-ground integrated network system including ground network and satellite network; Training the graph neural network according to the topological structure of the space-ground fusion network system; Connecting the trained graph neural network to the deep Q network, and training the intelligent agent of the deep Q network through the ground-ground fusion network system; The trained intelligent agent is deployed on each satellite node in the space-ground integrated network system, and the intelligent agent on the satellite node makes autonomous routing decisions based on the output corresponding to the graph neural network and the network status information; wherein the output corresponding to the graph neural network is the result of classifying the topology based on the local topology information corresponding to the satellite node.

2. The distributed intelligent routing method for the space-ground fusion network according to claim 1 is characterized in that: The space-ground integrated network system comprising a ground network and a satellite network includes: Design and construct a mesh topology of low-orbit satellite constellations based on intersatellite links in the longitude and latitude directions; Setting up high-orbit satellites in geosynchronous orbit; A ground network connected to a satellite network is built, constraints are set and visibility between ground stations and satellites of the ground network is calculated to realize the construction of a space-ground integrated network system; wherein the ground network includes the ground stations and router nodes, each network node in the space-ground integrated network system has a corresponding logical number and database, and when a link switch occurs at the network node, updated topology information is sent to adjacent nodes to update the database of the adjacent nodes.

3. The distributed intelligent routing method for the space-ground fusion network according to claim 1 is characterized in that: The training of the graph neural network according to the topological structure of the space-ground fusion network system includes: The local topological information in the database is used as training samples and input into the graph neural network for training; wherein the graph neural network is used to extract topological features in the local topological information to classify the topological graph, and the local topological information includes a feature matrix of nodes and a matrix representing the graph structure.

4. The distributed intelligent routing method for the space-ground fusion network according to claim 1, characterized in that: The agent of the deep Q network consists of a target network, a decision network and an experience replay buffer. The target network is used to stabilize the training process of the agent, the decision network is used to make action decisions based on the environment state, and the experience replay buffer is used to store experience tuples in the process of interaction between the agent and the environment. The experience tuples include state information, actions based on the state information, rewards obtained by executing the actions, and next state information obtained by inputting the actions into the environment.

5. The distributed intelligent routing method for the space-ground fusion network according to claim 4 is characterized in that: The agent on the satellite node makes an autonomous routing decision based on the output of the graph neural network and the network status information, including: The output of the graph neural network, the adjacent node numbers, the adjacent node congestion status, the source node numbers and the destination node numbers are normalized by the agent of the current node, and then input into the decision network to obtain several Q values ​​of the output corresponding to the decision network; wherein the Q value represents the routing action of the current node to select the next node; Sorting the Q values ​​output by the decision network, and selecting the routing action corresponding to the maximum Q value as the target action; If the next hop node pointed to by the target action route is congested or a non-adjacent node, the routing action corresponding to the next order Q value is selected from the plurality of Q values ​​in sequence as the target action until the next hop node pointed to by the target action route is not congested and is an adjacent node.

6. The distributed intelligent routing method for the space-ground fusion network according to claim 5 is characterized in that: The agent on the satellite node makes an autonomous routing decision based on the output of the graph neural network and the network status information, and further includes: If all the next-hop nodes pointed to by the routing actions corresponding to the Q values ​​are congested or non-adjacent nodes, packet loss processing is performed on the data of the current node and the previous-hop node is notified.

7. The distributed intelligent routing method for the space-ground fusion network according to claim 5 is characterized in that: The agent on the satellite node makes an autonomous routing decision based on the output of the graph neural network and the network status information, and further includes: If a routing loop occurs after traversing all the Q values, packet loss processing is performed on the data of the current node, and the previous hop node is notified.

8. The distributed intelligent routing method for the space-ground fusion network according to claim 1, characterized in that: After the agent selects the next hop node, it includes: For time-sensitive services, the agent selects the correct path by weighting the load balancing index and the positive reward to calculate the corresponding reward function; For time-sensitive services, when the agent selects an incorrect path, the corresponding reward function is weighted and calculated according to the delay index; For time-sensitive services, when the agent selects the next hop node and a routing loop or congestion occurs, the corresponding reward function is calculated using the packet loss rate indicator; For price-sensitive businesses, the agent calculates the corresponding reward function based on the weighted cost index when selecting the wrong path.

9. The distributed intelligent routing method for the space-ground fusion network according to claim 1, characterized in that: The agent generates actions by calculating actions according to the ∈-greedy strategy during training.

10. The distributed intelligent routing method for the space-ground fusion network according to claim 1, characterized in that: The network status information includes adjacent node information, congestion information, service demand identifiers in data packets, previous hop nodes, source nodes, and destination node numbers collected in the space-ground integrated network system.

Images