Satellite Internet of Things energy sensing routing method based on MPNN and related device

Abstract

The invention discloses an MPNN-based satellite Internet of Things energy sensing routing method and a related device. The method comprises the following steps: initializing satellite Internet of Things parameter information; establishing a satellite node-inter-satellite link relation graph model; obtaining the current state of each satellite node and the current state of an inter-satellite link in the satellite node-inter-satellite link relation graph model; and inputting the current state of each satellite node and the current state of the inter-satellite link into the trained routing decision framework based on the deep Q network, and performing satellite Internet of Things energy sensing routing according to an output result of the trained routing decision framework based on the deep Q network. According to the method and the related device, information of the whole network can be obtained with low expenditure so as to expand the sensing domain, the satellite network is optimized as a whole system, the service quality is guaranteed, meanwhile, energy use between satellites is balanced, and the service life of a satellite battery is prolonged.

Description

Technical Field

[0001] The present invention belongs to the technical field of satellite Internet of Things, and relates to a satellite Internet of Things energy perception routing method based on MPNN and related devices. Background Art

[0002] Satellite Internet of Things combines satellite networks with the Internet of Things to form a ubiquitous Internet of Things system based on a space-ground integrated information network architecture. As an extension of the terrestrial Internet of Things at the global three-dimensional space level, the satellite Internet of Things is not only a comprehensive application service platform integrating multiple information technologies, but also can provide seamless connection services around the world, especially in remote areas, oceans and air that are difficult to cover by terrestrial networks. Satellite Internet of Things has shown important strategic and economic significance.

[0003] As the core of the satellite IoT communication protocol, routing bears the heavy responsibility of data transmission and directly determines the transmission performance of the satellite IoT. A flexible and effective routing mechanism can ensure the normal operation of the satellite IoT and make timely adjustments when the network status changes, thereby ensuring the network's quality of service (QoS). Therefore, it is of great significance to study the routing method of the satellite IoT.

[0004] The network performance of routing not only depends on the communication protocol itself, but is also closely related to the satellite's energy storage system, in which the battery pack plays a key role as a core component. Due to the limited size of the satellite and the limited battery capacity, the energy management of the battery pack has become an extremely important issue. Excessive discharge will significantly shorten the service life of the battery, which in turn affects the long-term stable operation of the satellite. At the same time, the earth's occlusion effect and the uneven distribution of business loads lead to significant differences in the charging and discharging cycles of different satellites, and the remaining power of each node is also different, which brings additional complexity to the design of the routing algorithm.

[0005] Therefore, when designing routing methods, it is necessary not only to consider power management, but also to take into account the quality of service. The conflict between power factors and service quality requirements makes it a key challenge in the current satellite IoT routing design to reasonably balance inter-satellite energy consumption, avoid excessive discharge of satellite nodes, and extend their lifespan while ensuring network performance.

[0006] The energy saving problem of satellite networks has been widely discussed by many researchers. In the paper Satellite QoS routing algorithm based on energy aware and load balancing, HAO L et al. proposed a routing strategy based on energy awareness and load balancing to meet the different communication needs of users based on the limited energy resources on the satellite, but the algorithm did not make full use of satellite network information. In the paper Maximum lifetime routing with guaranteed throughput in leo satellite network, Y. Wang et al. proposed an algorithm to maximize the lifetime of low-orbit satellite networks. The algorithm performed well in extending the network life cycle and traffic balance, but did not consider the impact of discharge depth. In the paper Towards energy-efficient routing in satellite networks, Yang Y et al. proposed an energy-saving satellite routing method, aiming to reduce the DoD of each satellite node, but the overhead of obtaining real-time network traffic demand information is large. In the paper Adaptive routing with guaranteed delay bounds using safe reinforcement learning, GN Seetanadi et al. proposed an intelligent routing algorithm based on deep reinforcement learning, which extracted network features through neural networks and used reinforcement learning to make routing decisions. Compared with traditional methods and reinforcement learning-based methods, intelligent routing algorithms based on deep reinforcement learning can better learn network status and make more appropriate routing decisions based on it. However, most intelligent routing algorithms based on deep reinforcement learning find it difficult to accurately extract and fully utilize network topology information.

[0007] In summary, the existing technologies have certain shortcomings: First, these algorithms mainly focus on optimizing the energy consumption of a single satellite, reducing the energy consumption of a specific single satellite, and ignoring the overall energy balance of the satellite network. Second, due to the limitations of the perception domain, these algorithms are difficult to effectively perceive the state changes of the entire network, and the overhead of obtaining network information is large. They can often only make decisions within a local range and are prone to falling into the local optimal problem. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a satellite Internet of Things energy-aware routing method and related devices based on MPNN. The method and related devices can obtain network-wide information with low overhead to expand the perception domain, and optimize the satellite network as a whole system, while ensuring the quality of service, balancing inter-satellite energy usage and extending the satellite battery life.

[0009] To achieve the above object, the present invention discloses a satellite Internet of Things energy-aware routing method based on MPNN, comprising:

[0010] Initialize satellite IoT parameter information;

[0011] Establish a satellite node-intersatellite link relationship graph model;

[0012] Obtaining the current status of each satellite node and the current status of the intersatellite link in the satellite node-intersatellite link relationship graph model;

[0013] The current status of each satellite node and the current status of the intersatellite link are input into the trained routing decision framework based on the deep Q network, and the satellite Internet of Things energy-aware routing is performed according to the output results of the trained routing decision framework based on the deep Q network.

[0014] The further improvement of the satellite IoT energy-aware routing method based on MPNN described in the present invention is:

[0015] Furthermore, the process of initializing the satellite IoT parameter information is as follows:

[0016] 1a) Based on virtual nodes, the satellite network is represented as a graph G N =(S,L), where S = {s1,s2,…,s i ,…,s n} represents the satellite set, s i represents the i-th satellite, o(i) and k(i) represent satellite s respectively. i The orbit number and the satellite number on the orbit, L = {l i,n ,l i,m ,…,l i,j ,…,l j,n} represents the intersatellite link set, z i,j Indicates satellites i To Satellites j link, T represents the satellite network operation period, t path Indicates the time interval for adjacent satellites to exchange routing information. There are N satellite networks. Q Business, business q k =[s s ,s d ,Fk ], where s s Indicates business q k The source node of the request, s d Indicates business q k The destination node of the request, F k Indicates business q k Data transfer rate;

[0017] 1b) Establish satellite communication model;

[0018] 1c) Establish energy model;

[0019] 1d) Build a satellite battery life model.

[0020] Furthermore, the process of establishing the satellite node-intersatellite link relationship graph model is as follows:

[0021] 2a) Satellite node-intersatellite link relationship diagram is represented by G T =(V,E), where V and E represent the vertex set and edge set respectively;

[0022] 2b) Construct a vertex set V and add all satellites in set S and all links in set L as graph G T = vertices in (V,E), V = S∪L;

[0023] 2c) Construct an edge set L, connect the satellites corresponding to each link with the vertices corresponding to the satellites at both ends to form a graph G T = the edge in (V,E), E = {(s i ,l i,j )|s i ∈S,l i,j ∈L}∪{(l i,j ,s j )|s j ∈S,l i,j ∈L}.

[0024] Furthermore, the state of the i-th satellite node at time t W node represents the weight matrix, mapping node features to the state space, b node represents the bias vector and ReLU represents the activation function.

[0025] Furthermore, the state of the intersatellite link between the i-th satellite node and the j-th satellite node at time t is W link represents the weight matrix, mapping link features to the state space, b link represents the bias vector, the vertex l of the intersatellite link i,j The characteristic at time t is expressed as for Time Link i,j The total delay of the i-th satellite s i Characteristics at time t Among them, p q represents the collection rate of all satellite vertices on the sun-side panels, t i Indicates the remaining time that the satellite vertex is on the sun-facing side, are the node characteristics and link characteristics of each satellite at the current time step t, respectively.

[0026] Furthermore, the optimization objective reward function r of the routing decision framework based on the deep Q network during the training process is t It consists of satellite battery cycle life consumption, battery remaining power and business completion status.

[0027] Furthermore, the optimization objective reward function r t for:

[0028]

[0029] in, Indicates the total cycle life consumption of the battery, Indicates the minimum cycle life consumption life, Indicates the minimum battery charge. Indicates the maximum link utilization, represents the number of links in the entire network that exceed the threshold, F represents the reward for reaching the end point, and d p Indicates the path delay.

[0030] The present invention discloses a satellite Internet of Things energy sensing routing system based on MPNN, comprising:

[0031] Initialization module, used to initialize satellite IoT parameter information;

[0032] Establishing a module for establishing a satellite node-intersatellite link relationship graph model;

[0033] An acquisition module, used to acquire the current state of each satellite node and the current state of the intersatellite link in the satellite node-intersatellite link relationship graph model;

[0034] The decision module is used to input the current status of each satellite node and the current status of the inter-satellite link into the trained routing decision framework based on the deep Q network, and perform satellite Internet of Things energy-aware routing according to the output results of the trained routing decision framework based on the deep Q network.

[0035] The present invention discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the MPNN-based satellite Internet of Things energy-aware routing method are implemented.

[0036] The present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the MPNN-based satellite Internet of Things energy-aware routing method are implemented.

[0037] The present invention has the following beneficial effects:

[0038] The satellite Internet of Things energy-aware routing method and related devices based on MPNN described in the present invention use the learning ability of MPNN for non-Euclidean space data to interact and extract features of satellite node and intersatellite link status information, obtain the satellite link status information of the entire network through low interaction overhead, and then efficiently exchange information from adjacent satellites regularly and aggregate the status of the entire network, so that each node can comprehensively consider the status of neighboring nodes, capture the complex relationship and dependency between satellite nodes, and thus more accurately understand the status of satellite nodes and the entire network. In addition, it should be noted that the present invention combines the powerful learning and decision-making capabilities of DQN, uses the node status information and business information extracted by MPNN, and adopts the ε-greedy strategy to strike a balance between exploring new paths and using known optimal paths. By considering the satellite network as a whole, the overall state and local needs of the network are comprehensively considered to achieve efficient and balanced energy-aware routing. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 It is a flow chart for realizing the present invention;

[0041] Figure 2 A scene structure diagram used in the present invention;

[0042] Figure 3 It is a method framework diagram of the present invention;

[0043] Figure 4 It is a flow chart of the intersatellite link state interaction and feature extraction algorithm based on MPNN of the present invention;

[0044] Figure 5 A routing decision framework diagram based on a deep Q network is provided for the present invention;

[0045] Figure 6 The flowchart of the training routing decision deep Q network algorithm of the present invention;

[0046] Figure 7 This is a flow chart of the routing decision algorithm using a trained deep Q network of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0049] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0050] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0051] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0052] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0055] Embodiment 1

[0056] refer to Figure 1 , Figure 2 and Figure 3 The satellite Internet of Things energy-aware routing method based on MPNN of the present invention comprises the following steps:

[0057] 1) Initialize satellite IoT parameter information;

[0058] 2) Establish a satellite node-intersatellite link relationship graph model;

[0059] 3) Intersatellite link status interaction and feature extraction based on MPNN;

[0060] 4) Build a routing decision framework based on deep Q network;

[0061] 5) Training routing decision deep Q network;

[0062] 6) Use the trained deep Q network to make routing decisions.

[0063] The specific operations of step 1) are:

[0064] 1a) Based on virtual nodes, the satellite network can be represented as a graph G N =(S,L), where S = {s1,s2,…,s i ,…,s n} represents the satellite set, s i represents the i-th satellite, o(i) and k(i) represent satellite s respectively. i The orbit number and the satellite number on the orbit, L = {l i,n ,l i,m ,…,l i,j ,…,l j,n} represents the intersatellite link set, z i,j Indicates satellites i To Satellites j link, T represents the satellite network operation period, t path Indicates the time interval for adjacent satellites to exchange routing information. There are N satellite networks. Q Business, business q k =[s s ,s d ,F k ], where s s Indicates business q k The source node of the request, s d Indicates business q k The destination node of the request, F k Indicates business q k Data transfer rate;

[0065] 1b) Establish satellite communication model;

[0066] The specific process of step 1b) is:

[0067] 1b1) Establish a delay model: link l at time t i,j The total delay is

[0068] 1b2) Establish the residual bandwidth model at the current moment: Assume that the transmission rate of each intersatellite link is B, then the residual bandwidth of the intersatellite link at the current moment can be expressed as the total transmission rate of each intersatellite link minus the transmission rate of the service, that is:

[0069] 1b3) Establish the remaining bandwidth model at the current moment: The link utilization rate of the intersatellite link is the ratio of the link throughput to the total transmission rate of the link:

[0070] 1c) Establishing an energy model: The energy model includes satellite energy collection model, consumption model and current satellite power model;

[0071] 1c1) The satellite energy collection model is: in, represents the satellite s at time t i The charging capacity, p q represents the collection rate of all satellites on the solar panels, Δ(t) is the charging time, θ i (t) represents the relative position of sunlight to satellite s at time t i The angle of incidence of the solar panel;

[0072] 1c2) Satellite energy consumption model: service q k From the source satellite s Transmit to the destination satellite d , satellites i Energy consumed Among them, satellite s i The energy consumed by the transmission service and the satellite s i+1 The energy consumed by the receiving service can be expressed as: P tx Indicates the transmit power, P rx Indicates the received power. Another part of the energy is used to query the routing table, which is uniformly expressed as:

[0073] 1c3) Satellites i Remaining power in, Indicates satellites i Maximum power of

[0074] 1d) Establishing a satellite battery life model: The relationship between the depth of discharge (DoD) and cycle life of satellite batteries can usually be expressed as: in is the cycle life, σ and ε are constant coefficients related to battery characteristics, It is the DoD value.

[0075] 2) Establish a satellite node-intersatellite link relationship graph model;

[0076] The specific operations of step 2) are:

[0077] 2a) Satellite node-intersatellite link relationship diagram is represented by G T =(V,E), where V and E represent the vertex set and edge set respectively;

[0078] 2b) Construct a vertex set V and add all satellites in set S and all links in set L as graph GT = vertices in (V,E), that is, V = S∪L;

[0079] 2c) Construct an edge set L, connect the satellites corresponding to each link with the vertices corresponding to the satellites at both ends to form a graph G T = the edge in (V,E), that is, E = {(s i ,l i,j )|s i ∈S,l i,j ∈L}∪{(l i,j ,s j )|s j ∈S,l i,j ∈L}.

[0080] Step 3) Inter-satellite link status interaction and feature extraction based on MPNN;

[0081] refer to Figure 4 , the specific operation of step 3) is:

[0082] 3a) Obtain the node characteristics and link characteristics of each satellite at the current time step t Among them, satellite s i The characteristic at time t is expressed as Among them, p q represents the collection rate of all satellite vertices on the sun-side panels, t i Indicates the remaining time that the satellite vertex is on the sun-facing side. i,j The characteristic at time t is expressed as

[0083] 3b) Generate status information in, W node represents the weight matrix, mapping node features to the state space, b node Represents the bias vector, ReLU represents the activation function, which is used to introduce nonlinearity. W link represents the weight matrix, mapping link features to the state space, b link represents the bias vector;

[0084] 3c) Each satellite exchanges information with its four neighboring nodes to obtain vertex and link status information

[0085] 3d) Each satellite uses the message function Generate a satellite vertex message where, is the attention weight of the satellite vertex’s own state, indicating the importance of its own state in message aggregation. is the intersatellite link vertex li,j For satellite vertex s i The attention weight, M l is a message function that combines its own state and neighbor state to generate a message. is the satellite vertex s i The set of all connected intersatellite link vertices;

[0086] 3e) Each satellite uses the message function Generate a link message, where is the attention weight of the intersatellite link vertex’s own state, indicating the importance of its own state in message aggregation. is the satellite vertex s k For the intersatellite link vertex l i,j The attention weight, M s It is a message function that combines its own state and neighbor state to generate a message;

[0087] 3f) Each satellite uses the update function Update the vertex state, where U s is the update function for the intersatellite link vertex, which is set to a nonlinear transformation;

[0088] 3g) Each satellite uses the update function Update link status, where U l is the update function for satellite vertices, set to nonlinear transformation;

[0089] 3h) t = t + 1;

[0090] 3i) Output the final states of the satellite and the intersatellite link at time step t+1 respectively in, represents the state of satellite node s1 at time step t+1, represents the state of satellite node s2 at time step t+1, Represents satellite node s n The state at time step t+1. ISL 1,2 The state at time step t+1, ISL 1,3 The state at time step t+1, ISL i,j The state at time step t+1.

[0091] 4) Build a routing decision framework based on deep Q network;

[0092] refer to Figure 5 , the specific operation of step 4) is:

[0093] The routing decision framework of the deep Q network consists of a state space, an action space, and a reward function:

[0094] 4a) The state space is the set of environmental states of the deep reinforcement learning agent. The state space is defined as: the characteristics of the satellite network and the information of the service output based on the MPNN intersatellite link state interaction and feature extraction, expressed as in, Indicates satellite i A collection of adjacent satellites;

[0095] 4b) Action space selects an output interface for the service. The action space A is defined as: A = [E, W, N, S], which represents the four output directions of east, west, north, and south respectively;

[0096] 4c) After each routing decision is made, a reward value is given according to the designed reward function. The size of the reward value reflects the quality of this decision. t It consists of satellite battery cycle life consumption, battery remaining power and service completion status, namely:

[0097]

[0098] in, Indicates the total cycle life consumption of the battery, Indicates the minimum cycle life consumption life, Indicates the minimum battery charge. Indicates the maximum link utilization, represents the number of links in the entire network that exceed the threshold, F represents the reward for reaching the end point, and d p Indicates the path delay.

[0099] 5) Training routing decision deep Q network;

[0100] refer to Figure 5 and Figure 6 , the specific operation of step 5) is:

[0101] The specific operations are as follows:

[0102] 5a) Initialize Q network parameters: Initialize the online Q network Q online (s,a;θ) and the target Q network Q target (s, a; θ′), and set its initial parameters to random values ​​θ=θ′, where θ is the weight parameter of the online Q network and θ′ is the weight parameter of the target Q network, and the two remain consistent during initialization.

[0103] 5b) Initialize the experience replay pool: Initialize an empty data buffer D to store the experience sample data generated during the training process.

[0104] 5c) Set training hyperparameters: Set the maximum number of iterations L, which indicates the maximum number of cycles in the entire training process; set the time step T, which indicates the number of time steps run in each iteration process; set the batch size M, which indicates the number of samples taken from the data buffer for training; set the update interval C, which is used to control the frequency of updating the online Q network target Q network; set the discount factor γ, which is used to balance the weight of immediate rewards and future rewards; the learning rate α is used to update the learning rate of the Q network parameters.

[0105] 5d) Set the number of iterations epoch = 1;

[0106] 5e) If epoch ≤ L;

[0107] 5f) Determine the input features s of DQN t ;

[0108] 5g) Set time step t = 1;

[0109] 5h) If t ≤ T:

[0110] 5i) According to the current state s t Choose an action using the ∈-greedy strategy Among them, ∈ is the exploration rate, which is used to balance exploration and exploitation.

[0111] 5j) Execute the selected action a t , and obtain immediate reward r based on environmental feedback t And according to step 3) transfer to the next state s t+1 .

[0112] 5k) will experience sample (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool D.

[0113] 5l) Randomly extract samples of batch size M from the experience replay pool D to form a batch

[0114] 5m) For each sample (s i ,a i ,r i ,s i+1 ), calculate the target Q value y i , where y i =r i +γargmaxai Q target (s i ,a i );

[0115] 5n) For each sample (s) in the batch i ,a i ,r i ,s i+1 ), update the parameters θ of the online Q network by minimizing the loss function. Loss function is the mean square error (MSE): Among them, Q online (s i ,a i ; θ) is the online Q network for state s i and action a i Q value, y i is the target Q value.

[0116] 5o) Update parameters using gradient descent: Among them, α is the learning rate;

[0117] 5p) Check whether the current iteration number meets the condition epoch%C==0. If the condition is met, the target Q network Q target The parameters of the current online Q network Q online Parameters, that is, θ′=θ. If the condition is not met, the target network parameters remain unchanged;

[0118] 5q) After each time step cycle ends, the time step counter t is increased by 1, that is, t=t+1. If t≤T, jump to step 5h), otherwise, jump to step 5r);

[0119] 5r) epoch = epoch + 1, if epoch ≤ L, jump to step 5e), otherwise, terminate the algorithm and output the final satellite routing strategy and model parameter θ′.

[0120] Build a routing decision framework based on deep Q network, refer to Figure 7 , the specific operations are as follows:

[0121] 6a) Set the routing decision period T and initialize the time step t = 1;

[0122] 6b) When the current satellite node receives the service request, jump to step 6c), otherwise, jump to step 6g);

[0123] 6c) The satellite node checks the routing table. If the routing table contains the route for the requested service, it jumps to step 6f). Otherwise, it calculates the current state s t, jump to step 6d);

[0124] 6d) Utilization Calculate the next hop with the largest Q value for the service request;

[0125] 6e) The next hop of the service is a t Write to the satellite node routing table;

[0126] 6f) forwarding the service according to the next hop of the service in the routing table;

[0127] 6g) t = t + 1;

[0128] 6h) If t≤T, go to step 6b), otherwise, the routing work is completed.

[0129] Embodiment 2

[0130] The satellite Internet of Things energy sensing routing system based on MPNN of the present invention comprises:

[0131] Initialization module, used to initialize satellite IoT parameter information;

[0132] Establishing a module for establishing a satellite node-intersatellite link relationship graph model;

[0133] An acquisition module, used to acquire the current state of each satellite node and the current state of the intersatellite link in the satellite node-intersatellite link relationship graph model;

[0134] The decision module is used to input the current status of each satellite node and the current status of the inter-satellite link into the trained routing decision framework based on the deep Q network, and perform satellite Internet of Things energy-aware routing according to the output results of the trained routing decision framework based on the deep Q network.

[0135] In this embodiment, the process of initializing the satellite IoT parameter information is as follows:

[0136] 1a) Based on virtual nodes, the satellite network is represented as a graph G N =(S,L), where S = {s1,s2,…,s i ,…,s n} represents the satellite set, s i represents the i-th satellite, o(i) and k(i) represent satellite s respectively. i The orbit number and the satellite number on the orbit, L = {l i,n ,l i,m ,…,l i,j ,…,l j,n} represents the intersatellite link set, z i,j Indicates satellites i To Satellites jlink, T represents the satellite network operation period, t path Indicates the time interval for adjacent satellites to exchange routing information. There are N satellite networks. Q Business, business q k =[s s ,s d ,F k ], where s s Indicates business q k The source node of the request, s d Indicates business q k The destination node of the request, F k Indicates business q k Data transfer rate;

[0137] 1b) Establish satellite communication model;

[0138] 1c) Establish energy model;

[0139] 1d) Build a satellite battery life model.

[0140] In this embodiment, the process of establishing the satellite node-intersatellite link relationship graph model is as follows:

[0141] 2a) Satellite node-intersatellite link relationship diagram is represented by G T =(V,E), where V and E represent the vertex set and edge set respectively;

[0142] 2b) Construct a vertex set V and add all satellites in set S and all links in set L as graph G T = vertices in (V,E), V = S∪L;

[0143] 2c) Construct an edge set L, connect the satellites corresponding to each link with the vertices corresponding to the satellites at both ends to form a graph G T = the edge in (V,E), E = {(s i ,l i,j )|s i ∈S,l i,j ∈L}∪{(l i,j ,s j )|s j ∈S,l i,j ∈L}.

[0144] In this embodiment, the state of the i-th satellite node at time t W node represents the weight matrix, mapping node features to the state space, b node represents the bias vector and ReLU represents the activation function.

[0145] In this embodiment, the state of the inter-satellite link between the i-th satellite node and the j-th satellite node at time t W link represents the weight matrix, mapping link features to the state space, b link represents the bias vector, the vertex l of the intersatellite link i,j The characteristic at time t is expressed as for Time Link i,j The total delay of the i-th satellite s i Characteristics at time t Among them, p q represents the collection rate of all satellite vertices on the sun-side panels, t i Indicates the remaining time that the satellite vertex is on the sun-facing side, are the node characteristics and link characteristics of each satellite at the current time step t, respectively.

[0146] In this embodiment, the optimization objective reward function r of the routing decision framework based on the deep Q network during the training process is t It consists of satellite battery cycle life consumption, battery remaining power and business completion status.

[0147] In this embodiment, the optimization objective reward function r t for:

[0148]

[0149] in, Indicates the total cycle life consumption of the battery, Indicates the minimum cycle life consumption life, Indicates the minimum battery charge. Indicates the maximum link utilization, represents the number of links in the entire network that exceed the threshold, F represents the reward for reaching the end point, and d p Indicates the path delay.

[0150] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0151] Embodiment 3

[0152] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of implementing the energy-aware routing method of the satellite Internet of Things based on MPNN are implemented, for example, including: initializing satellite Internet of Things parameter information; establishing a satellite node-intersatellite link relationship graph model; obtaining the current state of each satellite node and the current state of the intersatellite link in the satellite node-intersatellite link relationship graph model; inputting the current state of each satellite node and the current state of the intersatellite link into a trained routing decision framework based on a deep Q network, and performing satellite Internet of Things energy-aware routing according to the output result of the trained routing decision framework based on a deep Q network. Wherein, the memory may include a memory, such as a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory, etc.; the processor, the network interface, and the memory are interconnected through an internal bus, and the internal bus may be an industrial standard architecture bus, a peripheral component interconnection standard bus, an extended industrial standard structure bus, etc., and the bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include memory and nonvolatile memory and provides instructions and data to the processor.

[0153] Embodiment 4

[0154] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of implementing the energy-aware routing method of the satellite Internet of Things based on MPNN are implemented, for example, including: initializing satellite Internet of Things parameter information; establishing a satellite node-intersatellite link relationship graph model; obtaining the current state of each satellite node and the current state of the intersatellite link in the satellite node-intersatellite link relationship graph model; inputting the current state of each satellite node and the current state of the intersatellite link into a trained routing decision framework based on a deep Q network, and performing satellite Internet of Things energy-aware routing according to the output result of the trained routing decision framework based on the deep Q network. Specifically, the computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include a read-only memory (ROM), a hard disk, a flash memory, an optical disk, a magnetic disk, etc.

[0155] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0156] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0157] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0159] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and disclosure of the invention. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0160] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0161] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A satellite Internet of Things energy-aware routing method based on MPNN, characterized in that: include: Initialize satellite IoT parameter information; Establish a satellite node-intersatellite link relationship graph model; Obtaining the current status of each satellite node and the current status of the intersatellite link in the satellite node-intersatellite link relationship graph model; The current status of each satellite node and the current status of the intersatellite link are input into the trained routing decision framework based on the deep Q network, and the satellite Internet of Things energy-aware routing is performed according to the output results of the trained routing decision framework based on the deep Q network.

2. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 1 is characterized in that: The process of initializing the satellite IoT parameter information is as follows: 1a) Based on virtual nodes, the satellite network is represented as a graph G N =(S,L), where S = {s1,s2,…,s i ,…,s n } represents the satellite set, s i represents the i-th satellite, o(i) and k(i) represent satellite s respectively. i The orbit number and the satellite number on the orbit, L = {l i,n ,l i,m ,…,l i,j ,…,l j,n } represents the intersatellite link set, z i,j Indicates satellites i To Satellites j link, T represents the satellite network operation period, t path Indicates the time interval for adjacent satellites to exchange routing information. There are N satellite networks. Q Business, business q k =[s s ,s d ,F k ], where s s Indicates business q k The source node of the request, s d Indicates business q k The destination node of the request, F k Indicates business q k Data transfer rate; 1b) Establish satellite communication model; 1c) Establish energy model; 1d) Build a satellite battery life model.

3. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 1 is characterized in that: The process of establishing the satellite node-intersatellite link relationship graph model is as follows: 2a) Satellite node-intersatellite link relationship diagram is represented by G T =(V,E), where V and E represent the vertex set and edge set respectively; 2b) Construct a vertex set V and add all satellites in set S and all links in set L as graph G T = vertices in (V,E), V = S∪L; 2c) Construct an edge set L, connect the satellites corresponding to each link with the vertices corresponding to the satellites at both ends to form a graph G T = the edge in (V,E), E = {(s i ,l i,j )|s i ∈S,l i,j ∈L}∪{(l i,j ,s j )|s j ∈S,l i,j ∈L}.

4. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 1 is characterized in that: The state of the i-th satellite node at time t W node represents the weight matrix, mapping node features to the state space, b node represents the bias vector and ReLU represents the activation function.

5. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 1 is characterized in that: The state of the intersatellite link between the i-th satellite node and the j-th satellite node at time t W link represents the weight matrix, mapping link features to the state space, b link represents the bias vector, the vertex l of the intersatellite link i,j The characteristic at time t is expressed as for Time Link i,j The total delay of the i-th satellite s i Characteristics at time t Among them, p q represents the collection rate of all satellite vertices on the sun-side panels, t i Indicates the remaining time that the satellite vertex is on the sun-facing side, are the node characteristics and link characteristics of each satellite at the current time step t, respectively.

6. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 1 is characterized in that: The optimization objective reward function r of the routing decision framework based on the deep Q network during training t It consists of satellite battery cycle life consumption, battery remaining power and business completion status.

7. The satellite Internet of Things energy-aware routing method based on MPNN according to claim 6 is characterized in that: The optimization objective reward function r t for: in, Indicates the total cycle life consumption of the battery, Indicates the minimum cycle life consumption life, Indicates the minimum battery charge. Indicates the maximum link utilization, represents the number of links in the entire network that exceed the threshold, F represents the reward for reaching the end point, and d p Indicates the path delay.

8. A satellite Internet of Things energy-aware routing system based on MPNN, characterized in that: include: Initialization module, used to initialize satellite IoT parameter information; Establishing a module for establishing a satellite node-intersatellite link relationship graph model; An acquisition module, used to acquire the current state of each satellite node and the current state of the intersatellite link in the satellite node-intersatellite link relationship graph model; The decision module is used to input the current status of each satellite node and the current status of the inter-satellite link into the trained routing decision framework based on the deep Q network, and perform satellite Internet of Things energy-aware routing according to the output results of the trained routing decision framework based on the deep Q network.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the MPNN-based satellite Internet of Things energy-aware routing method are implemented as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the MPNN-based satellite Internet of Things energy-aware routing method are implemented as described in any one of claims 1 to 7.

Images