Low earth orbit satellite network hybrid traffic routing method based on reinforcement learning

Abstract

The disclosure aims to provide a low-orbit satellite network hybrid service routing method based on reinforcement learning, comprising: priority division of hybrid services, obtaining the service type of any service; and using token bucket control and queue degradation mechanism to process any service, realizing differentiated scheduling; constructing a distributed reinforcement learning routing model with a satellite node as an agent, constructing a reward function according to the service type of the service; dynamically selecting a forwarding path combining node state and network topology, completing adaptive routing decision. The method of the disclosure can guarantee the priority forwarding of delay-sensitive services while providing basic service opportunities for best-effort services, effectively improving the overall resource utilization efficiency and service fairness; at the same time, the collaborative balance of delay guarantee and throughput optimization in a dynamic environment is realized.

Description

Technical Field

[0001] This disclosure relates to the field of low-Earth orbit (LEO) satellite network technology, and in particular to a hybrid service routing method for LEO satellite networks based on reinforcement learning. Background Technology

[0002] Low Earth Orbit (LEO) satellite networks are a crucial component of integrated air-space-ground-sea networks, providing long-range communication and global coverage, and possess significant application value. However, LEO satellite networks are characterized by complex and variable environments, highly dynamic topologies, and limited onboard resources. These characteristics pose severe challenges to ensuring the quality of service (QoS) of latency-sensitive services. As the scale of satellite networks and service demands continue to grow, ensuring end-to-end latency during scheduling and routing becomes increasingly difficult.

[0003] Existing research generally adopts a best-effort traffic transmission mechanism without prioritization, using the same routing strategy for different services. This results in traditional routing algorithms being insufficiently adaptable to the dynamic environment of low-Earth orbit satellites, unable to quickly respond to topology changes caused by satellite motion, and prone to routing path failure. At the same time, most existing studies treat different service types equally, adopting a "one-size-fits-all" routing strategy, without distinguishing the transmission requirements of latency-sensitive and best-effort services. This leads to a large number of latency-sensitive data packets failing due to transmission timeouts, which not only affects the quality of service but also wastes valuable resources such as onboard bandwidth.

[0004] Currently, some research focuses on routing design based on the spatiotemporal distribution characteristics of service traffic, such as multi-path search load balancing routing algorithms based on regional traffic segmentation. These algorithms dynamically calculate transition probabilities using link weight factors, turning factors, and node visibility parameters to guide regional traffic segmentation and ensure data transmission prioritizes links with lower congestion risk. However, this method treats different types of services equally, which may lead to latency-sensitive services choosing longer paths in satellite network environments, making it difficult to guarantee their quality of service (QoS) requirements. Furthermore, some research combines Software-Defined Networking (SDN) with traditional satellite networks. Under the SDN architecture, a priority assignment function that comprehensively considers service arrival rate and waiting time has been designed, proposing a joint inter-satellite routing algorithm based on service awareness and scheduling. This method employs different routing strategies for services with high and low QoS requirements. However, these methods still have certain limitations. On the one hand, they fail to fully consider the real-time load status of satellite nodes during path decision-making. On the other hand, while the in-orbit priority forwarding strategy reduces computational complexity, it may lead to suboptimal routing paths, thus affecting overall network performance.

[0005] In summary, how to efficiently forward data and optimize communication paths in a dynamically changing low-Earth orbit satellite network environment to improve the success rate of data packet delivery and meet the end-to-end latency requirements for different types of business needs remains a challenge that urgently needs to be addressed. Summary of the Invention

[0006] The purpose of this disclosure is to provide a reinforcement learning-based method for routing hybrid services in low-Earth orbit satellite networks, which addresses at least one technical problem in the prior art.

[0007] The technical solution disclosed herein is:

[0008] A hybrid service routing method for low-Earth orbit satellite networks based on reinforcement learning includes:

[0009] The system prioritizes mixed services to determine the service type for any given service; and employs token bucket control and queue degradation mechanisms to process any of these services, achieving differentiated scheduling.

[0010] A distributed reinforcement learning routing model is constructed using satellite nodes as intelligent agents, and a reward function is constructed based on the service type of the service.

[0011] By combining node status and network topology, forwarding paths are dynamically selected to complete adaptive routing decisions.

[0012] The process of prioritizing mixed services to determine the service type of any given service, and then using token bucket control and queue degradation mechanisms to process any of those services, includes:

[0013] Retrieve any mixed service data packet;

[0014] The services in the hybrid service data packets are divided into latency-sensitive service data packets and best-effort service data packets;

[0015] The token bucket control mechanism is used to enable high-priority queues to process latency-sensitive packets and low-priority queues to process best-effort packets.

[0016] A preset traffic limit is set, and data packets exceeding the preset traffic limit in the high-priority queue and low-priority queue are downgraded and treated as downgraded data packets.

[0017] If any data packet fails to acquire a token at the current node and enters the degraded queue for forwarding, it will still be divided into the corresponding queue according to the data packet type when the data packet arrives at the next satellite node.

[0018] Serve the low-priority queue when there are no packets in the high-priority queue;

[0019] When there are no data packets in the low-priority queue, the degraded queue will use the remaining bandwidth for transmission to avoid wasting resources.

[0020] Packets entering the degradation queue are forwarded according to the first-come, first-served standard.

[0021] The method of constructing a distributed reinforcement learning routing model using satellite nodes as agents, and constructing a reward function based on the service type of the service, includes:

[0022] Based on the Q-learning reinforcement learning strategy, any satellite is treated as an independent intelligent agent, and routing decisions are made in a distributed manner to build a reinforcement learning routing model and obtain the Q value of the current node forwarding to the destination node via neighboring nodes.

[0023] When any data packet arrives at the satellite node, scheduling is performed according to the priority queue corresponding to the service, and a candidate neighbor that meets the conditions is selected; the priority queue includes: delay-sensitive data packets processed by the high-priority queue and best-effort data packets processed by the low-priority queue;

[0024] Based on the candidate neighbor set, determine the next-hop node, complete the routing action, and update the environmental status information;

[0025] A reward function is constructed to dynamically update the Q value based on the environmental state information, so as to reflect the effect of the routing decision.

[0026] The step of determining the next-hop node based on the candidate neighbor set includes:

[0027] With probability Randomly select a neighbor as the next hop;

[0028] With probability Choose the neighbor with the highest Q value as the next hop.

[0029] The construction of the reward function includes:

[0030] From satellite node State transition to the next hop satellite node The reward function for the state is expressed as:

[0031] ;

[0032] in, and These represent the positive feedback reward for successfully forwarding a data packet to the destination node and the negative feedback penalty for failing to forward a data packet, respectively. , as well as These represent the weight values ​​for latency-sensitive data packets and best-effort data packets, respectively. Indicates the single-hop delay satisfaction; Represents the distance factor; This indicates the queue load of the satellite node.

[0033] The single-hop delay satisfaction The steps to obtain it include:

[0034] ;

[0035] in, This refers to the single-hop link latency cost; Budget for single-hop delay; It is a constant.

[0036] The distance factor The steps to obtain it include:

[0037] ;

[0038] in, For neighboring nodes Minimum number of hops remaining to reach the destination node; Indicates the maximum possible number of hops.

[0039] The queue load of the satellite node The steps to obtain it include:

[0040] ;

[0041] in, Indicates in From satellite nodes The space occupied by data packets in the buffer queue; This represents the queue capacity.

[0042] The beneficial effects of this disclosure include at least the following:

[0043] The method described in this disclosure addresses the problem of starvation for low-priority services caused by traditional strict priority queue scheduling under high load scenarios. It employs a queue scheduling strategy based on a token bucket control mechanism, independently setting token bucket bandwidth limits for each queue level and downgrading packets exceeding the limits. These packets are then placed in a degraded queue and scheduled according to a first-come, first-served principle. This ensures priority forwarding for latency-sensitive services while providing basic service opportunities for best-effort services, effectively improving overall resource utilization efficiency and service fairness. Furthermore, addressing the difficulty of existing satellite network routing algorithms in accommodating diverse service needs, a low-Earth orbit satellite network routing strategy based on reinforcement learning treats satellites as independent intelligent agents. Distributed routing decisions are made through Q-learning, and the reward function incorporates remaining deadline awareness. The per-hop budget for latency-sensitive services is dynamically calculated to ensure on-time arrival; for best-effort services, load balancing is emphasized, achieving a coordinated balance between latency assurance and throughput optimization in a dynamic environment. Attached Figure Description

[0044] Figure 1 This is the queue scheduling model in the reinforcement learning-based hybrid service routing method for low-Earth orbit satellite networks described in this disclosure;

[0045] Figure 2 This is a flowchart illustrating the packet degradation process in the reinforcement learning-based hybrid service routing method for low-Earth orbit satellite networks described in this disclosure.

[0046] Figure 3 This is a diagram of the Q-routing architecture in the reinforcement learning-based hybrid service routing method for low-Earth orbit satellite networks described in this disclosure. Detailed Implementation

[0047] The technical solution of this disclosure will be further described below with reference to the accompanying drawings.

[0048] This disclosure aims to address the problems of insufficient adaptability of traditional routing algorithms to dynamic environments and the lack of differentiated processing mechanisms for mixed services in current low-Earth orbit satellite network routing. It proposes a low-Earth orbit satellite network mixed service routing method based on reinforcement learning. Its goal is to plan a transmission path from the source node to the destination node for each data packet, maximizing the number of successfully transmitted data packets, under the premise of meeting the delay requirements of different types of data packets and ensuring that the satellite nodes in the selected path are in a non-congested state.

[0049] First, to address the service quality differences between latency-sensitive and best-effort services, this disclosure employs a dual-token bucket mechanism combined with strict priority scheduling. By setting bandwidth caps and degradation processing for high and low priorities, it prevents high-priority services from excessively preempting link resources, thus avoiding long-term starvation for low-priority services. This effectively alleviates the starvation problem for low-priority services while ensuring low latency for latency-sensitive services, balancing the fairness of resource allocation and overall transmission efficiency in mixed service scenarios.

[0050] Secondly, addressing the issue of insufficient adaptability of existing routing algorithms to dynamic environments, this disclosure deeply integrates service segmentation, differentiated reward functions, and reinforcement learning models. This not only ensures low-latency arrival for latency-sensitive services but also improves the throughput of best-effort services, thereby enhancing overall network resource utilization. By achieving load balancing and efficient utilization of onboard resources, it provides reliable support for hybrid services in low-Earth orbit satellite networks within an integrated air-space-ground-sea scenario. Specific Implementation Example 1:

[0052] This disclosure provides an embodiment:

[0053] The reinforcement learning-based hybrid service routing method for low-Earth orbit (LEO) satellite networks disclosed herein, based on the characteristics of highly dynamic LEO satellite network topology, limited on-board resources, and differentiated service requirements of hybrid services, improves the priority queue scheduling mechanism and combines it with a reinforcement learning-based routing strategy introduced into network routing to achieve efficient and differentiated transmission of latency-sensitive and best-effort services.

[0054] The steps of the method described in this disclosure include: prioritizing mixed services and obtaining the service type of any service; using token bucket control and queue degradation mechanism to process any of the services to achieve differentiated scheduling; constructing a distributed reinforcement learning routing model with satellite nodes as agents and constructing a reward function based on the service type of the service; and dynamically selecting forwarding paths by combining node status and network topology to complete adaptive routing decisions.

[0055] Specifically, in resource-constrained environments, reasonable priority management is an effective means of ensuring service quality for specific users. This embodiment focuses on a mixed routing scenario of latency-sensitive and best-effort services, with latency-sensitive services having higher priority.

[0056] In this embodiment, to better meet the strict upper limit of latency for latency-sensitive data packets while ensuring low packet loss rate and high throughput for best-effort data packets, a token bucket control mechanism is introduced based on strict priority queue scheduling. High-priority queues are used to process latency-sensitive data packets, while low-priority queues are used to process best-effort data packets. Data packets exceeding the preset traffic limit in both high-priority and low-priority queues are handled using a degradation strategy. Data packets entering the degradation queue are forwarded according to a first-come, first-served standard. The queue scheduling model is as follows: Figure 1 As shown.

[0057] This embodiment ensures low latency for latency-sensitive services through strict priority scheduling and introduces a dual-token bucket mechanism to set bandwidth limits for various services. Excess traffic automatically enters a degradation queue for processing on a first-come, first-served basis. This deep coupling of queue scheduling and token bucket control deeply absorbs bursty traffic using idle resources and effectively prevents high-priority services from occupying links for extended periods, fundamentally alleviating the starvation problem for low-priority services. Furthermore, it precisely categorizes services, introducing remaining deadline awareness for latency-sensitive services and prioritizing stable paths that meet latency budgets. For best-effort services, it focuses on optimizing resource utilization. Compared to traditional routing algorithms that do not distinguish between service types, this improves the satellite network's capacity to handle mixed services.

[0058] The diagram illustrates the packet degradation process for high-priority and low-priority queues. Figure 2 As shown. If a data packet successfully acquires a token, it will not enter the degraded queue and will be scheduled in its original priority queue order. If a data packet fails to acquire a token at the current node and enters the degraded queue for forwarding, it will still be assigned to the appropriate queue based on its type when it reaches the next satellite node. Different priority queues are scheduled according to strict priority; the scheduler will only serve a low-priority queue if there are no data packets in the high-priority queue. When there are no data packets in the low-priority queue, the degraded queue will use the remaining bandwidth for transmission to avoid resource waste.

[0059] In this embodiment, a model-free, Q-learning-based reinforcement learning strategy is introduced into network routing. Each satellite acts as an independent agent, making routing decisions in a distributed manner. After each forwarding action, the agent updates its Q-table based on environmental feedback. Q-routing is an application of the Q-learning algorithm to the routing problem, such as... Figure 3The proposed Q-routing architecture is shown. Each satellite node maintains a Q-table, which records the Q-values ​​of neighboring satellite nodes connected to the current satellite. The agent interacts with the external environment, receives feedback information, independently learns forwarding strategies, and forwards data packets to the next node.

[0060] During the data packet transmission and forwarding process, a reward is given for each data packet sent. The reward function in this embodiment addresses the two types of service requirements mentioned above, in order to better evaluate and dynamically adjust the forwarding strategy. On the one hand, it is necessary to provide bounded low-latency communication services for latency-sensitive services to solve the problem of timely transmission; on the other hand, it is necessary to meet the basic transmission requirements of best-effort services, thereby providing appropriate routing paths for different types of services.

[0061] Preferably, during the transmission of time-sensitive data packets with deadlines, the satellite node determines the remaining deadline of the currently transmitted data packet based on the current timeout. and remaining jumps To dynamically calculate the single-hop delay budget :

[0062] ;

[0063] When the remaining deadline of the data packet is greater than zero, this embodiment decomposes the end-to-end delay constraint of the data packet into local constraints for each hop, and uses single-hop delay satisfaction. To quantify the sufficiency of data packets during transmission. Single-hop link latency cost:

[0064] ;in, Distance factor, used to measure neighboring nodes Distance to the destination node For neighboring nodes The minimum number of remaining hops to the destination node, i.e., the logical distance.

[0065] ;

[0066] In addition, since satellite queue storage space is limited, if the data packet traffic is too large, it can easily cause node congestion, resulting in subsequent data packets being unable to enter the queue buffer and being dropped. Therefore, it is necessary to monitor the queue load of satellite nodes. Indicates in From satellite nodes The space occupied by data packets in the buffer queue, This represents the queue capacity.

[0067] ;

[0068] In summary, from the satellite node State transition to the next hop satellite node The reward function for the state can be calculated as:

[0069]

[0070] in, and These represent the positive feedback reward for successfully forwarding a data packet to the destination node and the negative feedback penalty for failing to forward the data packet. , , This indicates the weight values ​​corresponding to different service types. For data transmission of best-effort services, the main consideration is network load balancing, and single-hop latency is ignored in the calculation of the evaluation factor.

[0071] The operation flow of the hybrid service routing method for low-Earth orbit satellite networks described in this embodiment is as follows:

[0072] S1: Initialization Phase

[0073] Each satellite node independently initializes its own Q-table before the algorithm runs; each entry in the Q-table... Indicates starting from the current node via neighbor nodes Forward to the destination node The Q value.

[0074] S2: Packet arrival and neighbor filtering:

[0075] When data packet Reaching satellite node Scheduling is performed based on the queue scheduling model. The current data packet is recorded. destination node Initialize the set of visited nodes along the path. and the current node Add to collection This is done to prevent routing loops. Traverse the current node. All neighboring nodes Filter neighbors that meet the following conditions:

[0076] Neighbor nodes Not in the visited collection middle;

[0077] Neighbor node status identifier This indicates that the node is in a non-congested state.

[0078] For each eligible neighbor, read the corresponding information from the Q table. Values, and record the maximum value among them. The corresponding neighboring nodes.

[0079] S3: Next hop node selection:

[0080] Based on the selected set of candidate neighbors, the nodes are based on Greedy strategy to determine the next hop node Specifically, in terms of probability Randomly selecting a neighbor as the next hop, i.e., the exploration behavior, is based on probability. Choosing the neighbor with the highest Q-value as the next hop is the exploitation behavior. This strategy strikes a balance between exploring unknown paths and utilizing known optimal paths, adapting to dynamic changes in satellite network topology and avoiding getting trapped in local optima.

[0081] S4: Packet forwarding and status update:

[0082] Current node data packet Send to the selected next-hop node This completes a routing operation. Subsequently, the node updates its environmental state information to ensure that the algorithm can track changes in the network environment and maintain the temporal consistency of its decisions.

[0083] S5: Q value update:

[0084] After packet forwarding, the node updates its Q-value based on the immediate reward obtained from environmental feedback. The reward function integrates the service quality requirements of latency-sensitive and best-effort services, dynamically adjusting the Q-value to reflect the effectiveness of the routing decision.

[0085] S6: Iterative Loop and Policy Convergence:

[0086] Repeat the above steps until the preset termination condition is met. Each satellite node runs the above process independently and in parallel, continuously optimizing the Q-table through continuous interaction with the environment, and outputting the final optimized Q-table.

[0087] In summary, the method provided in this embodiment configures independent token buckets and sets bandwidth limits for high-priority and low-priority queues respectively. Data packets exceeding a preset traffic threshold are downgraded, entering a degraded queue and scheduled according to a first-come, first-served principle. This approach balances service guarantees for both latency-sensitive and best-effort services under high load conditions, alleviating the starvation problem of long-term service interruption for low-priority services and improving node resource utilization and service fairness. Simultaneously, a routing decision framework is built based on distributed Q-learning, using differentiated reward functions to optimize hybrid service paths. For latency-sensitive services, a single-hop delay satisfaction factor is introduced, dynamically allocating the delay budget for each hop based on the remaining deadline and number of hops to ensure end-to-end latency constraints are met. For best-effort services, the focus is on network load balancing, achieving differentiated transmission of services in a dynamic environment, improving overall network throughput while ensuring on-time arrival rates.

[0088] It needs to be clarified that:

[0089] In practical applications, the Q-learning described in this embodiment can be extended to a multi-agent deep reinforcement learning architecture, such as MADDPG. Each satellite node still acts as a distributed execution unit, and a centralized Critic network is introduced during the training phase to perform joint policy optimization using global state information. The replaced technical solution can better handle the cooperation and competition relationships between nodes and improve decision consistency in highly dynamic scenarios. Although the centralized training distributed execution architecture has advantages in theoretical performance, it has higher requirements for communication, computing, and storage resources, making it more difficult to deploy in real-world scenarios where low-Earth orbit satellite network resources are limited and the topology is highly dynamic. Specific Implementation Example 2:

[0091] This disclosure also provides an embodiment:

[0092] An electronic device includes: a storage medium and a processing unit; wherein the storage medium is used to store a computer program, and the processing unit exchanges data with the storage medium for executing the computer program through the processing unit when selecting a low-Earth orbit satellite routing strategy, performing the steps of the method as described in Specific Embodiment 1.

[0093] A computer-readable storage medium storing a computer program; when the computer program is run, it performs the steps of the method as described in Specific Embodiment 1.

[0094] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0095] The above disclosure only discloses a few specific implementation scenarios. However, this disclosure is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this disclosure.

Claims

1. A hybrid service routing method for low-Earth orbit satellite networks based on reinforcement learning, characterized in that, include: Prioritize mixed services and obtain the service type for any given service; The token bucket control and queue degradation mechanism are used to process any of the aforementioned services to achieve differentiated scheduling. A distributed reinforcement learning routing model is constructed using satellite nodes as intelligent agents, and a reward function is constructed based on the service type of the service. By combining node status and network topology, forwarding paths are dynamically selected to complete adaptive routing decisions.

2. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 1, characterized in that, The process involves prioritizing mixed services to determine the service type for any given service. And employing token bucket control and queue degradation mechanisms to process any of the aforementioned services, including: Retrieve any mixed service data packet; The services in the hybrid service data packets are divided into latency-sensitive service data packets and best-effort service data packets; The token bucket control mechanism is used to enable high-priority queues to process latency-sensitive packets and low-priority queues to process best-effort packets. A preset traffic limit is set, and data packets exceeding the preset traffic limit in the high-priority queue and low-priority queue are downgraded and treated as downgraded data packets.

3. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 2, characterized in that: If any data packet fails to acquire a token at the current node and enters the degraded queue for forwarding, it will still be divided into the corresponding queue according to the data packet type when the data packet arrives at the next satellite node. Serve the low-priority queue when there are no packets in the high-priority queue; When there are no data packets in the low-priority queue, the degraded queue will use the remaining bandwidth for transmission to avoid wasting resources.

4. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 2, characterized in that: Packets entering the degradation queue are forwarded according to the first-come, first-served standard.

5. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 1, characterized in that, The method of constructing a distributed reinforcement learning routing model using satellite nodes as agents, and constructing a reward function based on the service type of the service, includes: Based on the Q-learning reinforcement learning strategy, any satellite is treated as an independent intelligent agent, and routing decisions are made in a distributed manner to build a reinforcement learning routing model and obtain the Q value of the current node forwarding to the destination node via neighboring nodes. When any data packet arrives at the satellite node, scheduling is performed according to the priority queue corresponding to the service, and a candidate neighbor that meets the conditions is selected; the priority queue includes: delay-sensitive data packets processed by the high-priority queue and best-effort data packets processed by the low-priority queue; Based on the candidate neighbor set, determine the next-hop node, complete the routing action, and update the environmental status information; A reward function is constructed to dynamically update the Q value based on the environmental state information, so as to reflect the effect of the routing decision.

6. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 5, characterized in that, The step of determining the next-hop node based on the candidate neighbor set includes: With probability Randomly select a neighbor as the next hop; With probability Choose the neighbor with the highest Q value as the next hop.

7. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 5, characterized in that, The construction of the reward function includes: From satellite node State transition to the next hop satellite node The reward function for the state is expressed as: ； in, and These represent the positive feedback reward for successfully forwarding a data packet to the destination node and the negative feedback penalty for failing to forward a data packet, respectively. , as well as These represent the weight values ​​for latency-sensitive data packets and best-effort data packets, respectively. Indicates the single-hop delay satisfaction; Represents the distance factor; This indicates the queue load of the satellite node.

8. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 7, characterized in that, The single-hop delay satisfaction The steps to obtain it include: ； in, This refers to the single-hop link latency cost. Budget for single-hop delay; It is a constant.

9. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 7, characterized in that, The distance factor The steps to obtain it include: ； in, For satellite nodes Minimum number of hops remaining to reach the destination node; This indicates the maximum number of hops.

10. The low-Earth orbit satellite network hybrid service routing method based on reinforcement learning according to claim 7, characterized in that, The queue load of the satellite node The steps to obtain it include: ； in, Indicates in From satellite nodes The space occupied by data packets in the buffer queue; This represents the queue capacity.

Images