Satellite communication routing method and system, storage medium and satellite

Abstract

The invention provides a satellite communication routing method and system, a storage medium and a satellite. The method comprises the following steps: acquiring a satellite set of a congestion area broadcasted by a network control center; updating satellite network topology information; and after a user data packet is acquired, calculating a next-hop satellite of the user data packet by adopting a routing algorithm model based on deep reinforcement learning. According to the satellite communication routing method and system, the storage medium and the satellite, the local routing decision is made based on deep reinforcement learning, distributed routing of a satellite network is achieved, and signaling overhead and link time delay are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of wireless communications, and in particular to a satellite communication routing method, system, storage medium and satellite. Background Art

[0002] The integration of ground communication systems and satellite communication systems is an important technical feature of the next-generation 6G communication system. Among them, the satellite communication system is mainly composed of low-orbit satellites, medium-orbit satellites and high-orbit satellites, and the satellites are connected through inter-satellite links. Compared with the ground communication system, the satellite communication system has the advantages of wide coverage, large transmission capacity, and no geographical restrictions. It is an important means to achieve seamless global coverage of 6G.

[0003] In satellite communication systems, routing is an important technical means to achieve end-to-end transmission of user data. However, the high-speed movement of satellites and the frequent on-off of inter-satellite links will lead to highly dynamic changes in network topology, which brings severe technical challenges to the design of routing algorithms. In traditional centralized routing, the status changes of satellite nodes and inter-satellite links need to be sent to the network control center in a timely manner to ensure the effectiveness of routing decisions. On the one hand, the status change update brings large signaling overhead and consumes link bandwidth resources; on the other hand, adjacent satellites are usually far apart, resulting in large link propagation delays. In addition, the current low-orbit satellite system is evolving towards a large-scale giant constellation, and the substantial increase in the number of satellites in the network has further exacerbated the impact of the above problems. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a satellite communication routing method, system, storage medium and satellite, which make local routing decisions based on deep reinforcement learning, realize distributed routing of satellite networks, and effectively reduce signaling overhead and link delay.

[0005] In a first aspect, the present invention provides a satellite communication routing method, which is applied to a satellite, and the method comprises the following steps: obtaining a set of satellites in a congested area broadcast by a network control center; updating satellite network topology information; and after obtaining a user data packet, using a routing algorithm model based on deep reinforcement learning to calculate the next-hop satellite of the user data packet.

[0006] In an implementation of the first aspect, the satellite in the congested area refers to a satellite whose user data queue length to be forwarded exceeds a preset value, and the satellite in the congested area uses control signaling to send the congestion status to the network control center so that the network control center generates and broadcasts a set of satellites in the congested area.

[0007] In an implementation manner of the first aspect, updating the satellite network topology information includes the following steps:

[0008] Get satellite ephemeris information;

[0009] Obtaining intersatellite link establishment criteria;

[0010] The satellite network topology information is updated based on the satellite ephemeris information and the inter-satellite link establishment criteria.

[0011] In an implementation of the first aspect, the routing algorithm model based on deep reinforcement learning includes a state, an action, and a reward; the state S is composed of the node states of all adjacent satellites having intersatellite links with the satellite and the link states of the corresponding intersatellite links; the action Among them, a n ∈{0,1}, 1≤n≤N max , N max Indicates the maximum number of adjacent satellites supported by a single satellite, a n =1 means that the nth adjacent satellite of the satellite is the next hop satellite, a n =0 means that the nth adjacent satellite of the satellite is not the next-hop satellite; the reward is used to evaluate the merits of selecting action A in state S;

[0012] When all the adjacent satellites are not the destination satellites of the user data packet, the satellite obtains an optimal action that meets a predetermined criterion according to the state S; the adjacent satellite corresponding to the non-zero element in the optimal action is selected as the next-hop satellite;

[0013] When a certain adjacent satellite is the destination satellite, the user data packet is directly forwarded to the certain connected satellite.

[0014] In an implementation of the first aspect, the node status includes the available queue length of the satellite, the available computing resources of the satellite, the available energy of the satellite, and the shortest path delay from the satellite to the destination satellite; the link status includes the available communication bandwidth and shortest path delay of the inter-satellite link.

[0015] In an implementation of the first aspect, the predetermined criterion is to minimize the maximum statistical long-term cumulative discounted reward or minimize the loss function; the maximum statistical long-term cumulative discounted reward refers to a statistical function including current rewards and future expected rewards; the minimum loss function refers to a loss function including the reward;

[0016] The Reward where w j Represents the weighting coefficient, 0≤w j ≤1, Indicates satellite s to the nth adjacent satellite The shortest path delay between The nth adjacent satellite To the destination satellite d The shortest path delay, Indicates the nth adjacent satellite The available queue length, Represents satellite s and its nth adjacent satellite The available communication bandwidth of the intersatellite link between Indicates the nth adjacent satellite Available computing resources, Indicates the nth adjacent satellite of available energy.

[0017] In an implementation of the first aspect, Calculated using any of the following methods:

[0018] 1) Obtaining the nth adjacent satellite based on the satellite network topology information and the shortest path algorithm To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation

[0019] 2) removing the satellite set in the congested area and the inter-satellite link set connected to the satellite set implementing the congested area from the satellite network topology information to obtain updated network topology information; obtaining the nth adjacent satellite based on the updated network topology information and the shortest path algorithm; To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation

[0020] In a second aspect, the present invention provides a satellite communication routing system, applied to a satellite, the system comprising an acquisition module, an update module and a routing module;

[0021] The acquisition module is used to acquire a set of satellites in a congested area broadcast by a network control center;

[0022] The updating module is used to update the satellite network topology information;

[0023] The routing module is used to calculate the next-hop satellite of the user data packet by using a routing algorithm model based on deep reinforcement learning after obtaining the user data packet.

[0024] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which implements the above-mentioned satellite communication routing method when executed by a processor.

[0025] In a fourth aspect, the present invention provides a satellite, comprising: a processor and a memory;

[0026] The memory is used to store computer programs;

[0027] The processor is used to execute the computer program stored in the memory so as to enable the satellite to execute the above-mentioned satellite communication routing method.

[0028] As described above, the satellite communication routing method, system, storage medium and satellite of the present invention have the following beneficial effects:

[0029] (1) The satellite makes local routing decisions based on deep reinforcement learning based on the acquired satellite information of the congested area and the status information of adjacent satellites and intersatellite links, thus realizing distributed routing of the satellite network.

[0030] (2) Compared with centralized routing, it effectively reduces signaling overhead and inter-satellite long-distance transmission delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shown is a flow chart of a satellite communication routing method in one embodiment of the present invention;

[0032] Figure 2 Shown is a schematic diagram of a scenario of a satellite communication system in an embodiment of the present invention;

[0033] Figure 3 Shown is a schematic diagram of the structure of a satellite communication routing system in one embodiment of the present invention;

[0034] Figure 4 Shown is a schematic structural diagram of a satellite in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0037] The satellite communication system research scenario of the present invention is as follows Figure 1 As shown in Figure 1, users access the communication network through satellites, satellites are connected to each other through intersatellite links, and user data reaches the destination via the satellite communication network. The satellite that the user accesses is called the source satellite (denoted as s0), and the satellite covering the destination area is called the destination satellite (denoted as s d ). An edge computing server is deployed on the satellite to execute the satellite communication routing method of the present invention. In addition, the satellite is also equipped with a memory with a certain storage capacity for storing the user data queue to be forwarded and the satellite network topology information.

[0038] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0039] like Figure 2 As shown, in one embodiment, the satellite communication routing method of the present invention is applied to a satellite, and the method includes steps S1 to S3.

[0040] Step S1: Obtain a set of satellites in a congested area broadcast by a network control center.

[0041] Specifically, when the length of the user data queue to be forwarded in the satellite memory exceeds a preset value, the satellite is determined to be in a congested area, and the satellite uses control signaling to send the congestion status to the network control center so that the network control center generates and broadcasts the satellite set in the congested area. The network control center can be deployed at a ground station, or on a high-orbit satellite, a medium-orbit satellite, or a low-orbit satellite with strong computing and storage capabilities. The broadcasting method of the network control center can be a periodic broadcasting method, or it can be an event-triggered broadcasting method when a satellite in a congested area changes.

[0042] In one embodiment, the control signaling includes a satellite number, a congestion indication, and a timestamp. In another embodiment, the control signaling includes a satellite number, a queue occupancy rate, and a timestamp.

[0043] Step S2: Update satellite network topology information.

[0044] Specifically, the satellite first obtains satellite ephemeris information and intersatellite link establishment criteria, and then updates the satellite network topology information according to the satellite ephemeris information and the intersatellite link establishment criteria. The satellite ephemeris information is used to obtain the positions of different satellites in the satellite network, and the intersatellite link establishment criteria are used to indicate the connectivity status of the intersatellite links between different satellites under the current network topology. Therefore, the satellite network topology information can be represented as a graph G = (V, E). V represents all satellite nodes, and E represents the connection relationship (or edge) between satellite nodes. When there is an intersatellite link between two satellites, the edge value is 1, otherwise it is 0.

[0045] Step S3: After obtaining the user data packet, a routing algorithm model based on deep reinforcement learning is used to calculate the next hop satellite of the user data packet.

[0046] Specifically, the routing algorithm model based on deep reinforcement learning (DRL) of the present invention is deployed on each satellite. The satellite selects the next hop satellite by running the model. The input of the routing algorithm model based on deep reinforcement learning is state S, and the output is action A and reward R. Taking satellite s as an example, the state S, action A and reward R are defined as follows:

[0047] State S consists of the node states of all neighboring satellites that have intersatellite links with satellite s, and the link states of the corresponding intersatellite links. Among them, all neighboring satellites of satellite s0 send their node states to satellite s0 through intersatellite links. The nth neighboring satellite is denoted as 1≤n≤N max , N max Indicates the maximum number of neighboring satellites supported by a single satellite. In one embodiment, the satellite node status includes the following elements: the available queue length of the satellite (denoted as ), the available computing resources of the satellite (denoted as ), the available energy of the satellite (denoted as ), the shortest path delay from the satellite to the destination satellite (denoted as ). Take the nth adjacent satellite For example, its node state can be expressed as When the number of adjacent satellites that actually establish intersatellite links with the satellite is less than N max When the node status element that is not supported is set to 0, the satellite node status element may be set to 0. In another embodiment, the satellite node status includes the values ​​of the above elements after normalization. In one embodiment, the link status includes the following elements: the available communication bandwidth of the link (denoted as ), the shortest path delay (denoted as ). Similarly, with satellite s and the nth adjacent satellite As an example, the intersatellite link between When the number of adjacent satellites that actually establish intersatellite links with the satellite is less than N max When the link state element that is not supported is set to 0, the link state element that is not supported may be set to 0. In another embodiment, the link state includes the values ​​of the above elements after normalization.

[0048] Action A is represented by Among them, a n ∈{0,1}, 1≤n≤N max ,and a n =1 indicates the nth neighboring satellite of satellite s is the next hop satellite; otherwise, a n = 0 means that the nth adjacent satellite of satellite s is not the next-hop satellite. When the number of adjacent satellites that actually establish intersatellite links with the satellite is less than N max When N is used, the unsupported action elements are set to 0. For example, if N max =4, but when the satellite moves to the Earth's polar region, the satellite only supports the establishment of two inter-satellite links. At this time, action A can be expressed as A=(a1,a2,0,0).

[0049] The reward R is used to evaluate the quality of the routing algorithm model based on deep reinforcement learning in selecting action A in state S. In one embodiment, the reward R is expressed as:

[0050]

[0051] Among them, w j (1≤j≤5) is the weighting coefficient, satisfying 0≤w j ≤1, Indicates satellite s to the nth adjacent satellite The shortest path delay between The nth adjacent satellite To the destination satellite d The shortest path delay, Indicates the nth adjacent satellite The available queue length, Represents satellite s and its nth adjacent satellite The available communication bandwidth of the intersatellite link between Indicates the nth adjacent satellite Available computing resources, Indicates the nth adjacent satellite of available energy.

[0052] In one embodiment, the nth adjacent satellite is calculated To the destination satellited The shortest path delay When the nth adjacent satellite is obtained based on the satellite network topology information and the shortest path algorithm (such as Dijkstra algorithm) To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation

[0053] In another embodiment, the nth adjacent satellite is calculated To the destination satellite d The shortest path delay When the nth adjacent satellite is obtained, the satellite set in the congested area and the inter-satellite link set connected to the satellite set in the congested area are removed from the satellite network topology information to obtain updated network topology information; and the nth adjacent satellite is obtained based on the updated network topology information and the shortest path algorithm. To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation

[0054] It should be noted that It can be calculated by an adjacent satellite and sent to satellite s0 via an intersatellite link, or it can be directly calculated by satellite s0 based on the current network topology information.

[0055] Calculating the next-hop satellite of the user data packet using a routing algorithm model based on deep reinforcement learning includes the following steps:

[0056] 31) When all the adjacent satellites are not the destination satellites of the user data packet, the satellite obtains the optimal action that meets the predetermined criteria according to the state S; the adjacent satellite corresponding to the non-zero element in the optimal action is selected as the next-hop satellite. At the same time, the next-hop satellite continues to calculate the next-hop satellite of the user data packet according to the routing algorithm model based on deep reinforcement learning until the user data packet reaches the destination satellite. Among them, the predetermined criteria are minimizing and maximizing the statistical long-term cumulative discounted reward or minimizing the loss function; the maximizing statistical long-term cumulative discounted reward refers to a statistical function that includes the current reward and the expected future reward; the minimum loss function refers to a loss function that includes the reward, such as an entropy function.

[0057] 32) When a certain adjacent satellite is the destination satellite, directly forward the user data packet to the certain connected satellite.

[0058] The satellite communication routing method of the present invention is further described below through specific embodiments.

[0059] In this embodiment, the routing algorithm model based on deep reinforcement learning is deployed on each satellite, and the satellite selects the next hop satellite by running the model. Taking satellite s as an example, the state S, action A and reward R are defined as follows:

[0060] State S consists of the node states of all neighboring satellites that have intersatellite links with satellite s, and the link states of the corresponding intersatellite links. The nth neighboring satellite is denoted as 1≤n≤4, the maximum number of adjacent satellites supported by a single satellite is 4. The satellite node state includes the following elements: the available queue length of the satellite (denoted as ), the available computing resources of the satellite (denoted as ), the available energy of the satellite (denoted as ), the shortest path delay from the satellite to the destination satellite (denoted as ). Take the nth adjacent satellite For example, its node state can be expressed as The link status includes the following elements: the available communication bandwidth of the link (denoted as ), the shortest path delay (denoted as ). Similarly, with satellite s and the nth adjacent satellite As an example, the intersatellite link between

[0061] Action A is represented by A = (a1, a2, a3, a4). n ∈{0,1}, 1≤n≤4, and a n =1 indicates the nth neighboring satellite of satellite s is the next hop satellite; otherwise, a n =0 means that the nth adjacent satellite of satellite s is not the next-hop satellite.

[0062] The reward R is used to evaluate the quality of the action A selected by the routing algorithm model based on deep reinforcement learning in state S.

[0063] Step 1: Obtain the satellite set of the congested area broadcast by the network control center.

[0064] When the length of the user data queue to be forwarded in the satellite memory exceeds a preset value, the satellite is determined to be in a congested area, and the satellite sends the congestion status to the network control center in the form of control signaling. The control signaling content includes the satellite number, congestion indication, and timestamp.

[0065] The network control center broadcasts the satellite set in the congested area to the satellites in the network. The network control center can be deployed at the ground station. The broadcast method adopts event triggering, that is, the broadcast is triggered when the satellite in the congested area changes.

[0066] Step 2: Update satellite network topology information.

[0067] Step 3: After obtaining the user data packet, a routing algorithm model based on deep reinforcement learning is used to calculate the next-hop satellite of the user data packet.

[0068] Satellite s0 runs a routing algorithm model based on deep reinforcement learning, that is, according to the current state S, it obtains the optimal action A that meets the predetermined criteria (such as maximizing the statistical long-term cumulative discounted reward, minimizing the loss function, etc.), where the adjacent satellites corresponding to the non-zero elements in A are selected as the next-hop forwarding satellite s1. d When it is an adjacent satellite of satellite s0, satellite s0 directly forwards the user data packet to the destination satellite s d , the algorithm ends.

[0069] When the user data packet arrives at satellite s1, satellite s1 calculates the next hop of the user data packet according to the routing algorithm model and forwards it to satellite s2 until it reaches the destination satellite s d .

[0070] The protection scope of the satellite communication routing method described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0071] An embodiment of the present invention further provides a satellite communication routing system, which can implement the satellite communication routing method described in the present invention. However, the implementation device of the satellite communication routing system described in the present invention includes but is not limited to the structure of the satellite communication routing system listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0072] like Figure 3 As shown, in one embodiment, the satellite communication routing system of the present invention is applied to a satellite, and includes an acquisition module 31 , an update module 32 and a routing module 33 .

[0073] The acquisition module 31 is used to acquire the satellite set in the congested area broadcast by the network control center.

[0074] The updating module 32 is used to update the satellite network topology information.

[0075] The routing module 33 is connected to the update module 32 and the routing module 33, and is used to calculate the next hop satellite of the user data packet by using a routing algorithm model based on deep reinforcement learning after obtaining the user data packet.

[0076] The structures and principles of the acquisition module 31, the update module 32 and the routing module 33 correspond one to one with the steps in the above-mentioned satellite communication routing method, so they will not be described in detail here.

[0077] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0078] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0079] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0080] The embodiment of the present invention also provides a computer-readable storage medium. A person skilled in the art can understand that all or part of the steps in the satellite communication routing method of the above embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0081] An embodiment of the present invention further provides a satellite, wherein the satellite includes a processor and a memory.

[0082] The memory is used to store computer programs.

[0083] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0084] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the satellite executes the above-mentioned satellite communication routing method.

[0085] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0086] like Figure 4As shown, the satellite of the present invention is in the form of a general-purpose computing device. The components of the satellite may include, but are not limited to: one or more processors or processing units 41, a memory 42, and a bus 43 connecting different system components (including the memory 42 and the processing unit 41).

[0087] Bus 43 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0088] Satellites typically include a variety of computer system readable media. These media can be any available media that can be accessed by the satellite, including volatile and non-volatile media, removable and non-removable media.

[0089] Memory 42 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 421 and / or cache memory 422. The satellite may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 423 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 43 via one or more data medium interfaces. Memory 42 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0090] A program / utility 424 having a set (at least one) of program modules 4241 may be stored, for example, in the memory 42, such program modules 4241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 4241 generally perform the functions and / or methods of the embodiments described herein.

[0091] The satellite may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the satellite, and / or any device that enables the satellite to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communications may occur via input / output (I / O) interfaces 44. Furthermore, the satellite may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks such as the Internet) via network adapters 45. Figure 4 As shown, network adapter 45 communicates with other modules of the satellite via bus 43. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the satellite, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0092] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A satellite communication routing method, applied to a satellite, characterized in that: The method comprises the following steps: Obtain the satellite set of the congested area broadcast by the network control center; Update satellite network topology information; After the user data packet is obtained, a routing algorithm model based on deep reinforcement learning is used to calculate the next hop satellite of the user data packet.

2. The satellite communication routing method according to claim 1, characterized in that: The satellites in the congested area refer to the satellites whose user data queue length to be forwarded exceeds a preset value. The satellites in the congested area use control signaling to send the congestion status to the network control center so that the network control center generates and broadcasts the satellite set in the congested area.

3. The satellite communication routing method according to claim 1, characterized in that: Updating satellite network topology information includes the following steps: Get satellite ephemeris information; Obtaining intersatellite link establishment criteria; The satellite network topology information is updated based on the satellite ephemeris information and the inter-satellite link establishment criteria.

4. The satellite communication routing method according to claim 1, characterized in that: The routing algorithm model based on deep reinforcement learning includes states, actions and rewards; the state S is composed of the node states of all adjacent satellites with which the satellite has intersatellite links and the link states of the corresponding intersatellite links; the action Among them, a n ∈{0,1}, 1≤n≤N max , N max Indicates the maximum number of adjacent satellites supported by a single satellite, a n =1 means that the nth adjacent satellite of the satellite is the next hop satellite, a n =0 means that the nth adjacent satellite of the satellite is not the next-hop satellite; the reward is used to evaluate the merits of selecting action A in state S; When all the adjacent satellites are not the destination satellites of the user data packet, the satellite obtains an optimal action that meets a predetermined criterion according to the state S; the adjacent satellite corresponding to the non-zero element in the optimal action is selected as the next-hop satellite; When a certain adjacent satellite is the destination satellite, the user data packet is directly forwarded to the certain connected satellite.

5. The satellite communication routing method according to claim 4, characterized in that: The node status includes the available queue length of the satellite, the available computing resources of the satellite, the available energy of the satellite, and the shortest path delay from the satellite to the destination satellite; the link status includes the available communication bandwidth and shortest path delay of the inter-satellite link.

6. The satellite communication routing method according to claim 4, characterized in that: The predetermined criteria are maximizing the statistical long-term cumulative discounted reward or minimizing the loss function; the maximizing statistical long-term cumulative discounted reward refers to a statistical function including the current reward and the expected future reward; the minimizing loss function refers to a loss function including the reward; The Reward where w j Represents the weighting coefficient, 0≤w j ≤1, Indicates satellite s to the nth adjacent satellite The shortest path delay between The nth adjacent satellite To the destination satellite d The shortest path delay, Indicates the nth adjacent satellite The available queue length, Represents satellite s and its nth adjacent satellite The available communication bandwidth of the intersatellite link between Indicates the nth adjacent satellite Available computing resources, Indicates the nth adjacent satellite of available energy.

7. The satellite communication routing method according to claim 6, characterized in that: Calculated using any of the following methods: 1) Obtaining the nth adjacent satellite based on the satellite network topology information and the shortest path algorithm To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation 2) removing the satellite set in the congested area and the inter-satellite link set connected to the satellite set in the congested area from the satellite network topology information, and obtaining updated network topology information; Acquire the nth adjacent satellite based on the updated network topology information and the shortest path algorithm To the destination satellite d The sum of the lengths of all intersatellite links included in the shortest path is divided by the speed of light as the result of calculation 8. A satellite communication routing system, applied to a satellite, characterized in that: The system includes an acquisition module, an update module and a routing module; The acquisition module is used to acquire a set of satellites in a congested area broadcast by a network control center; The updating module is used to update the satellite network topology information; The routing module is used to calculate the next-hop satellite of the user data packet by using a routing algorithm model based on deep reinforcement learning after obtaining the user data packet.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the satellite communication routing method according to any one of claims 1 to 7 is implemented.

10. A satellite, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so as to enable the satellite to perform the satellite communication routing method according to any one of claims 1 to 7.

Images