A quality of service aware routing method and system for large-scale low earth orbit satellites

Abstract

The application relates to a service quality perception routing method and system for large-scale low-orbit satellites, which comprises the following steps: acquiring global state information of a low-orbit satellite network; the global state information comprises satellite node positions, star link topologies, link time delays and network traffic density distributions; based on the global state information, the satellite network is divided into multiple areas, and each area is abstracted as a management node; satellite-level service quality indexes are aggregated as area-level service quality indexes, and the routing between areas is solved through the area-level service quality indexes, Markov decision and deep reinforcement learning methods; and based on the satellite adjacency relationship in each management node, the routing in each area is determined through a local probability forwarding method of satellites. Through hierarchical design and centralized control, the application effectively balances the calculation efficiency, QoS performance and scalability of the routing algorithm.

Description

Technical Field

[0001] This invention belongs to the field of satellite communication network technology, specifically relating to a service quality-aware routing method and apparatus for large-scale low-Earth orbit satellites. Background Technology

[0002] In recent years, with continuous breakthroughs in satellite communication technology, LEO satellite networks, due to their advantages such as global coverage, rapid deployment, and cross-regional interconnection, are gradually becoming a core pillar of the 6G global network system, demonstrating broad application prospects in large-scale business demands such as real-time voice communication, cloud gaming, and the Internet of Things. LEO constellation projects, represented by Starlink and OneWeb, are being rapidly deployed. However, with the increasing scale of satellite networks, the design of related routing schemes also faces severe challenges.

[0003] First, due to the high speed of satellite operation, the inter-satellite link topology changes rapidly over time, making it difficult for traditional static routing algorithms to adapt. Second, routing decisions on the LEO constellation network need to comprehensively consider various QoS constraints, the most representative indicators being latency and energy consumption. On the one hand, various onboard services have strict real-time requirements, so latency is currently the primary optimization indicator for many routing strategies; on the other hand, satellite operation is limited by its solar panel systems. Lower energy consumption means longer operational lifespan, so energy consumption optimization is crucial for ensuring the continuous operation of onboard missions. Therefore, achieving a relative balance among various QoS indicators is a major challenge in routing algorithm design. Furthermore, onboard computing resources are extremely limited, which imposes strict constraints on the computational complexity and deployment overhead of routing algorithms.

[0004] Currently, most satellite routing schemes are still based on improvements to traditional terrestrial routing algorithms. These algorithms, due to their simplicity and low computational overhead, were widely used in early satellite systems. However, they lack adaptability and struggle to effectively cope with rapid changes in satellite links. With the development of artificial intelligence, intelligent algorithms have been gradually introduced into satellite networks. However, these algorithms have also revealed problems such as high complexity, slow convergence, high computational consumption, and poor scalability in such large-scale networks. This severely limits the application and long-term operation of such methods in real-world large-scale LEO satellite networks. Furthermore, the introduction of SDN technology into satellite networks has improved management efficiency and flexibility through the separation of the control plane and data plane. However, combining the centralized control advantages of SDN with efficient intelligent routing algorithms remains a significant challenge. Therefore, how to achieve highly scalable, lightweight, and efficient routing while simultaneously optimizing QoS indicators such as latency and energy consumption in large-scale, highly dynamic LEO satellite networks is a problem that urgently needs to be solved. Summary of the Invention

[0005] To address the problems in the background art, a first aspect of the present invention provides a service quality-aware routing method for large-scale low-Earth orbit (LEO) satellites, comprising: acquiring global state information of a LEO satellite network; the global state information including satellite node locations, star link topology, link delays, and network traffic density distribution; based on the global state information, dividing the satellite network into multiple regions, and abstracting each region as a management node; aggregating satellite-level service quality indicators into region-level service quality indicators, and solving the routing between regions using the region-level service quality indicators, Markov decision and deep reinforcement learning methods; and determining the routing within each region based on the satellite adjacency relationships within each management node using a local probabilistic forwarding method based on satellite orientation.

[0006] In some embodiments of the present invention, the step of aggregating satellite-level service quality indicators into regional-level service quality indicators and solving the routing between regions using the regional-level service quality indicators, Markov decision-making, and deep reinforcement learning methods includes: constructing a multi-objective optimization function for inter-regional routing based on the regional-level service quality indicators; transforming the multi-objective optimization function into a Markov decision process; training a deep reinforcement learning model based on a greedy strategy of experience replay and dynamic decay; and solving the routing between regions using the trained deep reinforcement learning model.

[0007] Furthermore, the training of the deep reinforcement learning model includes: determining the iteration of the forwarding strategy through a dynamic decay-greedy strategy based on the initial state of the satellite currently forwarding the message, until the number of tuples in the experience replay buffer reaches a threshold; sampling the samples in the experience replay buffer based on weight priority, and updating the training network through gradient descent until the training network converges.

[0008] In some embodiments of the present invention, determining the route within each region based on the satellite adjacency relationship within each management node and using a local probability forwarding method based on satellite orientation includes: determining the link weight and weight adjustment factor of the inter-satellite link; determining the forwarding orientation of the current satellite's data packets based on whether the current satellite and the destination satellite are in the same region; calculating the forwarding probability of the current satellite in the horizontal and vertical links based on the link weight and the adjustment factor; and determining the route of the current satellite's data packets within the region based on the forwarding probability of the current satellite in the horizontal and vertical links.

[0009] Furthermore, the calculation of the forwarding probability of the current satellite in the horizontal and vertical links based on the link weight and adjustment factor includes: calculating the link weight using the adjustment factor, the data packet queue length in the buffer of the neighboring satellite in the corresponding direction of the current satellite, the distance between the current satellite and the neighboring satellite in the corresponding direction, and the bandwidth between the current satellite and the neighboring satellite in the corresponding direction; the corresponding direction includes horizontal and vertical; and calculating the forwarding probability of the current satellite in the corresponding direction based on the proportion of the link weight of each corresponding direction to the link weight of all corresponding directions.

[0010] In some embodiments of the present invention, dividing the satellite network into multiple regions based on the global state information includes: dividing the network service time into multiple consecutive time slices based on a virtual topology; describing the topology of satellite nodes and inter-satellite links in each time slice using an undirected graph; dividing the topology graph to obtain multiple non-overlapping regions, and recording the set of satellite nodes contained in each region and their adjacency relationships.

[0011] A second aspect of the present invention provides a service quality-aware routing system for large-scale low-Earth orbit (LEO) satellites, comprising: an acquisition module for acquiring global state information of a LEO satellite network; the global state information including satellite node locations, star link topology, link delays, and network traffic density distribution; a partitioning module for dividing the satellite network into multiple regions based on the global state information, and abstracting each region as a management node; a solution module for aggregating satellite-level service quality indicators into region-level service quality indicators, and solving for routes between regions using the region-level service quality indicators, Markov decision-making, and deep reinforcement learning methods; and a determination module for determining routes within each region based on satellite adjacency relationships within each management node, using a local probabilistic forwarding method based on satellite orientation.

[0012] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the quality of service-aware routing method for large-scale low-Earth orbit satellites provided in the first aspect of the present invention.

[0013] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the quality of service-aware routing method for large-scale low-Earth orbit satellites provided in the first aspect of the present invention.

[0014] The beneficial effects of this invention are:

[0015] 1. Reduced computational complexity: By dividing the global satellite network into multiple management areas through the SDN controller and abstracting them into super nodes, the complex routing problem is decomposed into two sub-problems: inter-area and intra-area, thereby significantly reducing the complexity of routing calculations;

[0016] 2. Improve inter-region routing efficiency: Based on the deep reinforcement learning algorithm (D3QN) and priority experience replay, offline training and online inference are performed under the centralized management of the SDN controller to quickly find the optimal inter-region skeleton path under the dynamic network topology, thereby improving training speed, convergence efficiency and overall network QoS performance;

[0017] 3. Achieve highly adaptable intra-regional routing: Design a lightweight local probabilistic forwarding algorithm that relies solely on local state and local information provided by the SDN controller for decision-making, avoiding the acquisition of the entire network state. This enables low-overhead, highly adaptable, fine-grained path selection and, in conjunction with inter-regional routing, ensures fast and efficient packet forwarding.

[0018] 4. Enhance network manageability: Based on the SDN architecture, the control layer and data layer are decoupled, and complex routing decisions are offloaded to the controller, reducing the computing burden on onboard equipment, improving network management flexibility and scalability, and further improving routing efficiency and network performance through global optimization. Attached Figure Description

[0019] Figure 1 This is a basic flowchart of a quality of service-aware routing method for large-scale low-Earth orbit satellites in some embodiments of the present invention.

[0020] Figure 2 This is a schematic diagram of a LEO satellite constellation scenario in some embodiments of the present invention;

[0021] Figure 3 This is a schematic diagram of low-Earth orbit satellite network topology partitioning in some embodiments of the present invention;

[0022] Figure 4 This is a schematic diagram of a global flow density distribution in some embodiments of the present invention;

[0023] Figure 5 This is a schematic diagram of a deep reinforcement learning algorithm based on priority experience replay in some embodiments of the present invention;

[0024] Figure 6 This is a schematic diagram of a lightweight local probabilistic forwarding algorithm in some embodiments of the present invention;

[0025] Figure 7 This is a schematic diagram of the structure of a quality of service-aware routing system for large-scale low-Earth orbit satellites in some embodiments of the present invention.

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0027] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0028] Example 1

[0029] refer to Figure 1 and Figure 2 In a first aspect, the present invention provides a service quality-aware routing method for large-scale low-Earth orbit (LEO) satellites, comprising: S100. acquiring global state information of a LEO satellite network; the global state information including satellite node locations, star link topology, link delays, and network traffic density distribution; S200. based on the global state information, dividing the satellite network into multiple regions, and abstracting each region as a management node; S300. aggregating satellite-level service quality indicators into regional-level service quality indicators, and solving the routing between regions using the regional-level service quality indicators, Markov decision and deep reinforcement learning methods; S400. based on the satellite adjacency relationships within each management node, determining the routing within each region using a local probabilistic forwarding method based on satellite orientation.

[0030] refer to Figure 2 In step S100 of some embodiments of the present invention, dividing the satellite network into multiple regions based on the global state information includes:

[0031] S101. Based on virtual topology, divide network service time into multiple consecutive time slices;

[0032] Specifically, Figure 1 The diagram illustrates the LEO satellite constellation. In this network, each satellite establishes stable two-way inter-satellite links with its four neighboring satellites, including two intra-orbit inter-satellite links and two inter-orbit inter-satellite links. To address the issue of frequent topology changes caused by high-speed satellite motion, this invention employs a virtual topology method to manage service time. Divided into several time slices Satellite topology only While the time point changes, it remains constant within the time slice.

[0033] S102. Within each time slice, describe the topology of satellite nodes and inter-satellite links using an undirected graph; S103. Divide the topology into multiple non-overlapping regions, and record the set of satellite nodes contained in each region and their adjacency relationships.

[0034] Specifically, within each time slice, all satellites and inter-satellite links together form a "+" shaped grid. Mathematically, this is represented by an undirected graph. Describe it. Among them... Represents a collection of satellites. This indicates the total number of satellites. Indicates the first One satellite; Represents the set of adjacent satellites. Indicates satellite The list of adjacent satellites.

[0035] In step S200 of some embodiments of the present invention, the satellite network is divided into multiple regions based on the global state information, and each region is abstracted as a management node;

[0036] Specifically, such as Figure 3 As shown, this satellite network further divides itself into multiple regions using the concept of topology partitioning for more efficient management and optimization. Each region serves as a management unit, denoted as . ,in Represents a set of regions. To represent the total number of regions, then use Represents the set of adjacent regions. Indicates the region The system generates a list of adjacent regions. These divided regions are then abstracted into supernodes, and various QoS indicators between individual satellites are converted into indicators between regions. Through this process, the system first calculates the skeleton path at the supernode level, and then uses this as the basis to calculate the forwarding path within the region, thereby reducing computational complexity.

[0037] In step S300 of some embodiments of the present invention, the aggregation of satellite-level service quality indicators into regional-level service quality indicators, and the solution of routes between regions using the regional-level service quality indicators, Markov decision-making, and deep reinforcement learning methods, includes:

[0038] S301. Construct a multi-objective optimization function for inter-regional routing based on regional-level service quality indicators;

[0039] Specifically, after topology partitioning, routing between these supernodes needs to consider QoS metrics such as latency and energy consumption to optimize satellite network routing performance. Therefore, this invention redefines the QoS metrics between regions.

[0040] Latency is the primary indicator for measuring the real-time performance of satellite networks, especially when carrying high-frequency interactive and real-time-sensitive services such as satellite telephony and live streaming. The magnitude of latency directly impacts service quality and user experience. After topology partitioning, the total end-to-end latency between supernodes... It is given by the following formula:

[0041] ,

[0042] in To represent the propagation delay between supernodes, This indicates the transmission latency between supernodes. This represents the queuing delay between supernodes. Of course, satellites perform operations such as unpacking data packets, protocol parsing, and routing table lookups during relay, resulting in some processing delay. However, this processing delay is typically only on the order of microseconds, far less than other delays. Therefore, processing delay is not considered in this invention.

[0043] The propagation delay between supernodes cannot be simply measured and calculated by the distance between the geometric centers of two regions. Therefore, this invention proposes the concept of a flow center, using the flow center... The distance between them indicates the supernode The distance between them. It is given by the following formula;

[0044] ,

[0045] in Let the position vector of a point in the region be . For the region Middle position The flow density, the global distribution of flow density is as follows Figure 3 As shown. Then the supernode Distance between for:

[0046] ,

[0047] Record No. Each time slot Data packets from the supernode To the next jump The propagation delay is Therefore, the propagation delay of the data packet can be calculated as follows:

[0048] ,

[0049] in It is the speed of light.

[0050] To address the transmission latency between supernodes, the inter-satellite links of low-Earth orbit satellites need to be considered as free-space optical communication (FSO) links. Based on this, the first... Each time slot Upper area arrive achievable link transmission rate It can be expressed as:

[0051] ,

[0052] in This represents the average link bandwidth between supernodes. supernode and The distance between them, and It is a parameter related to the average optical signal-to-noise ratio. This is the attenuation parameter, and the specific calculation method is as follows:

[0053] ,

[0054] ,

[0055] in, This represents the average optical signal-to-noise ratio. Given by the ratio of system average to peak power. Parameter It can be calculated as ,in It depends on visibility. ,wavelength and particle size distribution coefficient Therefore, the first The size in each time slot is Data packets from the supernode to its neighboring supernode The resulting transmission delay for:

[0056] ,

[0057] To address queuing latency between supernodes, this invention employs an M / M / C queuing model to model queuing behavior within supernodes. This model divides the supernode-level queuing system into four directional sub-queuing systems. Each sub-queuing system abstracts the boundary satellites in that direction as parallel servers, thus significantly reducing the complexity of queuing latency estimation for searching the skeleton path. When data packets originate from the region... Forward to region At that time, the average service rate on the corresponding subqueuing system can be estimated. as follows:

[0058]

[0059] in For the region arrive The boundary, For satellite The number of servers running in parallel. For satellite service rate The number of servers on this sub-queue system is calculated as follows:

[0060]

[0061] Similarly, the arrival rate of this sub-queue system is defined as... Then it can be estimated in the following way:

[0062]

[0063] in For data packets to reach the satellite The arrival rate. Therefore, when the... Region in each time slot correspond System utilization of the sub-queue system In this case, the queuing delay can be calculated as follows:

[0064]

[0065] in, The probability that a data packet needs to wait in this queuing system can be calculated using Erlang's C formula.

[0066] In low-Earth orbit satellite networks, operations such as packet forwarding and maintaining communication links consume significant amounts of satellite energy. This high energy consumption directly impacts satellite lifespan, consequently affecting the overall network's operational efficiency and sustainability. Therefore, selecting an energy-efficient routing path is a key consideration in this invention. Supernode In the Each time slot To the next jump Energy consumed in forwarding data packets It can be represented as:

[0067]

[0068] in Indicates the size of the data packet. For transmission rate, This indicates a region. Average antenna transmit power on the FSO link.

[0069] Based on the QoS metric model defined above, the optimization problem can be expressed as:

[0070] ;

[0071]

[0072] ;

[0073] ;

[0074] ;

[0075] .

[0076] The optimization objective is to minimize the latency and energy consumption generated during communication. and These are weighting coefficients used to uniformly quantify QoS metrics and appropriately balance the importance of each metric. Constraint 1 requires that the total latency generated by data packets during communication cannot exceed a preset tolerance threshold. Constraint 2 ensures that the energy consumption of a single hop between regions cannot exceed the maximum energy that the region can provide. Constraint 3 ensures that the load on each supernode cannot exceed its processing capacity. Constraint 4 ensures that the supernode... Only one adjacent region is selected as the next hop for data forwarding, where Defined as a term used to refer to Have you chosen? As a binary variable for the next hop.

[0077] It is understandable that, based on the established model, the hierarchical QoS-aware routing method proposed in this invention designs algorithms for both inter-regional and intra-regional routing problems.

[0078] S302. Transform the multi-objective optimization function into a Markov decision process;

[0079] Specifically, in the inter-region routing phase, the algorithm models the problem as consisting of quintuples. The MDP is described. This represents the state space, which is the set of states of the current region within the global network. Define the region. The state in the m-th time slot is:

[0080] ,

[0081] The Boolean value Done serves as a marker indicating whether the current region is the destination region. If it is the target area, then It is true if true, otherwise it is false. Represents the action space, when a data packet enters the region. At that time, the agent will select one of the four adjacent regions as its action. After being given the current state and selecting the appropriate action, the agent will receive the corresponding reward function. . This is the state transition probability matrix, representing the agent's transition from state to state. Take action Transition to the next state The probability of. This represents the reward discount factor. When... The closer the value is to 1, the more the agent values ​​long-term gains; conversely, the closer the value is to 1, the more the agent values ​​short-term gains.

[0082] S303. Train a deep reinforcement learning model based on a greedy strategy of experience replay and dynamic decay; solve the routing between regions using the trained deep reinforcement learning model.

[0083] Furthermore, the training of the deep reinforcement learning model includes: determining the iteration of the forwarding strategy through a dynamic decay-greedy strategy based on the initial state of the satellite currently forwarding the message, until the number of tuples in the experience replay buffer reaches a threshold; sampling the samples in the experience replay buffer based on weight priority, and updating the training network through gradient descent until the training network converges.

[0084] Specifically, refer to Figure 5 The training steps include:

[0085] (1) The SDN controller will obtain the global network status information and initialize the experience replay buffer. Training network and target network .in and These represent the parameters of the two networks, respectively.

[0086] It should be noted that, unlike the traditional Q network, and Instead of directly outputting the Q value, the calculation is divided into two branches. These two branches will calculate the state value separately. and Advantage Value Finally, these are summed to obtain the Q value. Specifically, the Q value is calculated as follows:

[0087] ,

[0088] Where s represents the state, a represents the action, and a' represents the predicted action.

[0089] This method of calculating the Q value avoids the problem of non-uniqueness in V and A value modeling, thus improving the stability of training.

[0090] (2) Obtain the initial state of the satellite currently forwarding the data packet. And according to the following dynamic decay - Greedy strategy makes decisions:

[0091] ,

[0092] in, It is a random parameter. The exploration rate will be updated as follows:

[0093] ,

[0094] in express The attenuation coefficient, This is the current training cycle.

[0095] (3) Performing actions Earn reward points and the next state The reward function is given by the following formula:

[0096] ,

[0097] in, and Ensure that the routing strategy chosen by the agent meets optimization requirements. These are respectively determined by latency. and energy consumption This is obtained through normalization. Furthermore, to avoid ping-pong routing issues, a reward value is set. As shown below:

[0098] ,

[0099] in, This is a relatively large reward constant when the data packet arrives at its destination area, while This is the penalty constant for each step before the data packet reaches its destination area. This is a weighting factor used to balance the above factors.

[0100] (4) Empirical tuples deposit middle.

[0101] (5) Repeat steps (2), (3), and (4) until... The number of tuples is greater than a predefined threshold.

[0102] (6) Calculation The importance of each sample is sampled by weight, and empirical tuples are sampled according to the weight priority. ,in This refers to the size of the batch.

[0103] (7) Minimize the loss function using gradient descent. Update it, and update it every certain number of training steps. .

[0104] (8) Repeat steps (5), (6), and (7) until the training cycle reaches its maximum value. Use the output at this point as the skeleton path obtained by inter-region routing.

[0105] It is understandable that after obtaining the skeleton path, it is necessary to further determine how the data packets are forwarded within the area. This invention stipulates that data packets will only be forwarded laterally or vertically in the direction of forwarding, and when the satellite containing the data packet and the destination location are in the same horizontal direction, they will only be forwarded laterally, and when they are in the same vertical direction, they will only be forwarded vertically.

[0106] In step S400 of some embodiments of the present invention, determining the route in each area based on the satellite adjacency relationship within each management node and using the local probabilistic forwarding method of satellite orientation includes: S401. Determining the link weight and weight adjustment factor of the inter-satellite link; S402. Determining the data packet forwarding orientation of the current satellite based on whether the current satellite and the destination satellite are in the same area;

[0107] S403. Based on the link weight and the adjustment factor, calculate the forwarding probability of the current satellite in the horizontal and vertical links;

[0108] Furthermore, the calculation of the forwarding probability of the current satellite in the horizontal and vertical links based on the link weight and adjustment factor includes: calculating the link weight using the adjustment factor, the data packet queue length in the buffer of the neighboring satellite in the corresponding direction of the current satellite, the distance between the current satellite and the neighboring satellite in the corresponding direction, and the bandwidth between the current satellite and the neighboring satellite in the corresponding direction; the corresponding direction includes horizontal and vertical; and calculating the forwarding probability of the current satellite in the corresponding direction based on the proportion of the link weight of each corresponding direction to the link weight of all corresponding directions.

[0109] Specifically, assign weights to the current satellite's horizontal links respectively. With vertical link weight As shown below:

[0110] ,

[0111] in Information tuples directly observed by the SDN controller. This indicates the length of the data packet queue in the buffer of the neighboring satellite in the corresponding direction. Indicates the distance to neighboring satellites in the corresponding direction. ( ) indicates the corresponding direction (horizontal) h Vertical v The link bandwidth between neighboring satellites. This is an adjustment factor used to calculate the weights. Therefore, the lateral forwarding probability of the current satellite node can be calculated. and vertical forwarding probability As shown below:

[0112] .

[0113] S404. Based on the forwarding probabilities of the current satellite in the horizontal and vertical links, determine the routing of the current satellite's data packets within the area.

[0114] Specifically, refer to Figure 6 The specific steps are as follows:

[0115] (1) Initialize link weights Regulatory factors .

[0116] (2) Determine the current satellite forwarding direction. When the target satellite is not in the same area, forward the satellite towards the center of the next hop area; otherwise, forward the satellite towards the target satellite.

[0117] (3) The SDN controller observes the environmental tuples of the horizontal and vertical links in the current satellite forwarding direction.

[0118] (4) Calculate link weights and .

[0119] (5) Calculate the forwarding probability.

[0120] (6) The data packet is routed to the next-hop satellite according to the forwarding probability and this satellite is used as the current satellite.

[0121] (7) Repeat steps (2), (3), (4), (5), and (6) until the data packet reaches the destination satellite.

[0122] By combining the aforementioned inter-regional and intra-regional routing algorithms, this invention enables efficient routing with low overhead, low latency, and low energy consumption in large-scale LEO satellite networks. Furthermore, this implementation method offers excellent flexibility and scalability to meet the needs of practical applications, making it widely applicable to future low-Earth orbit satellite communication systems.

[0123] Example 2

[0124] refer to Figure 7 In a second aspect, the present invention provides a service quality-aware routing system 1 for large-scale low-Earth orbit satellites, comprising: an acquisition module 11 for acquiring global state information of a low-Earth orbit satellite network; the global state information including satellite node locations, star link topology, link delays, and network traffic density distribution; a partitioning module 12 for dividing the satellite network into multiple regions based on the global state information, and abstracting each region as a management node; a solving module 13 for aggregating satellite-level service quality indicators into region-level service quality indicators, and solving for routes between regions using the region-level service quality indicators, Markov decision and deep reinforcement learning methods; and a determination module 14 for determining routes within each region based on satellite adjacency relationships within each management node, using a local probabilistic forwarding method based on satellite orientation.

[0125] Furthermore, the solution module 13 includes: a construction unit for constructing a multi-objective optimization function for inter-regional routing based on regional service quality indicators; a transformation unit for transforming the multi-objective optimization function into a Markov decision process; a training unit for training a deep reinforcement learning model based on an experience replay and a dynamically decaying greedy strategy; and a solution unit for solving the inter-regional routing using the trained deep reinforcement learning model.

[0126] Example 3

[0127] refer to Figure 8 A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the quality of service-aware routing method for large-scale low-Earth orbit satellites according to the first aspect of the present invention.

[0128] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0129] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 8 Each box shown can represent a device or multiple devices as needed.

[0130] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0131] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to:

[0132] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A quality-of-service (QoS)-aware routing method for large-scale low-Earth orbit (LEO) satellites, characterized in that, include: Obtain global status information of low-Earth orbit satellite networks; The global state information includes satellite node locations, star link topology, link latency, and network traffic density distribution. Based on the global state information, the satellite network is divided into multiple regions, and each region is abstracted as a management node; The step of dividing the satellite network into multiple regions based on the global state information includes: dividing the network service time into multiple consecutive time slices based on virtual topology; describing the topology of satellite nodes and inter-satellite links in each time slice using an undirected graph; dividing the topology graph to obtain multiple non-overlapping regions, and recording the set of satellite nodes contained in each region and their adjacency relationships. Satellite-level service quality indicators are aggregated into regional-level service quality indicators. Routes between regions are then solved using these regional-level service quality indicators, Markov decision processes, and deep reinforcement learning methods. Specifically, a multi-objective optimization function for inter-regional routing is constructed based on the regional-level service quality indicators. This multi-objective optimization function is then transformed into a Markov decision process. A deep reinforcement learning model is trained based on a greedy strategy of experience replay and dynamic decay. Finally, the routes between regions are solved using the trained deep reinforcement learning model. Based on the satellite adjacency relationships within each management node, routes within each area are determined using a local probabilistic forwarding method based on satellite orientation.

2. The quality of service-aware routing method for large-scale low-Earth orbit satellites according to claim 1, characterized in that, The training deep reinforcement learning model includes: Based on the initial state of the satellite currently forwarding the message, the iteration of the forwarding strategy is determined through a dynamic decay-greedy strategy until the number of tuples in the experience replay buffer reaches a threshold. Samples in the experience replay buffer are sampled based on weight priority, and the training network is updated by gradient descent until the training network converges.

3. The quality of service-aware routing method for large-scale low-Earth orbit satellites according to claim 1, characterized in that, The method of determining routes within each area based on satellite adjacency relationships within each management node and using a local probabilistic forwarding method according to satellite orientation includes: Determine the link weights and weight adjustment factors for inter-satellite links; Determine the forwarding direction of the current satellite's data packets based on whether the current satellite and the target satellite are in the same area; Based on the link weight and the adjustment factor, calculate the forwarding probability of the current satellite in the horizontal and vertical links; Based on the forwarding probabilities of the current satellite in the horizontal and vertical links, the routing of the current satellite's data packets within the region is determined.

4. The quality of service-aware routing method for large-scale low-Earth orbit satellites according to claim 3, characterized in that, The calculation of the current satellite's forwarding probability in the horizontal and vertical links based on link weight and adjustment factors includes: The link weight is calculated using the adjustment factor, the length of the data packet queue in the buffer of the neighboring satellite in the corresponding direction of the current satellite, the distance between the current satellite and the neighboring satellite in the corresponding direction, and the bandwidth between the current satellite and the neighboring satellite in the corresponding direction. The corresponding directions include horizontal and vertical directions; Based on the proportion of the link weight in each corresponding direction to the link weight in all corresponding directions, the forwarding probability of the current satellite in the corresponding direction is calculated.

5. A quality of service-aware routing system for large-scale low-Earth orbit satellites, characterized in that, include: The acquisition module is used to acquire global status information of the low-Earth orbit satellite network; The global state information includes satellite node locations, star link topology, link latency, and network traffic density distribution. The partitioning module is used to divide the satellite network into multiple regions based on the global state information, and to abstract each region as a management node; The step of dividing the satellite network into multiple regions based on the global state information includes: dividing the network service time into multiple consecutive time slices based on virtual topology; describing the topology of satellite nodes and inter-satellite links in each time slice using an undirected graph; dividing the topology graph to obtain multiple non-overlapping regions, and recording the set of satellite nodes contained in each region and their adjacency relationships. The solution module is used to aggregate satellite-level service quality indicators into regional-level service quality indicators, and solve the routing between regions using the regional-level service quality indicators, Markov decision-making, and deep reinforcement learning methods: Based on the regional-level service quality indicators, a multi-objective optimization function for inter-regional routing is constructed; the multi-objective optimization function is transformed into a Markov decision process; a deep reinforcement learning model is trained based on a greedy strategy of experience replay and dynamic decay; and the routing between regions is solved using the trained deep reinforcement learning model. The determination module is used to determine the route within each area based on the satellite adjacency relationship within each management node, using a local probabilistic forwarding method based on satellite orientation.

6. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the quality of service-aware routing method for large-scale low-Earth orbit satellites as described in any one of claims 1 to 4.

7. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the quality of service-aware routing method for large-scale low-Earth orbit satellites as described in any one of claims 1 to 4.

Images