A low earth orbit satellite offloading and scheduling method

Abstract

The present application relates to a kind of low-orbit satellite unloading and scheduling method, belong to wireless communication field.The method includes: S1: modeling system model;S2: modeling Internet of Things equipment computing task model;S3: modeling Internet of Things equipment computing task unloading mode;S4: modeling computing task queue model;S5: modeling observation satellite data transmission service model;S6: modeling observation satellite data transmission service queue model;S7: modeling task transmission and execution energy consumption model;S8: modeling data transmission energy consumption model;S9: modeling system utility model;S10: modeling Markov decision process;S11: based on DQN algorithm determines unloading and scheduling strategy.The present application is based on system utility optimization design low-orbit satellite network design unloading and scheduling strategy, can realize system performance optimization.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication and relates to a method for unloading and scheduling low-orbit satellites. Background Technology

[0002] With the rapid increase in demand for mobile internet services, resource-intensive applications such as cloud gaming, augmented reality, and image recognition are emerging. These resource-intensive tasks pose significant challenges to the computing and caching capabilities of terrestrial networks. Low-Earth orbit (LEO) satellite IoT offers advantages such as wide coverage, large system capacity, and resilience. Satellite communication can offload the computing tasks of terrestrial IoT devices to LEO satellites, alleviating the processing pressure on these devices. Simultaneously, Earth observation technologies utilize LEO satellite networks to dynamically monitor and analyze the Earth's natural environment, economic crops, and traffic conditions, providing macroscopic, accurate, comprehensive, and continuous information data. Therefore, designing offloading and scheduling strategies that comprehensively consider the computing needs of IoT devices and the data transmission requirements of observation satellites, as well as the characteristics of LEO satellite networks, has become an important research topic.

[0003] Existing literature has studied the issues of mission offloading and data transmission in satellite networks, but few studies have comprehensively considered the joint mission offloading and data transmission service scheduling issues under resource competition in low-Earth orbit satellite networks. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for offloading and scheduling low-Earth orbit (LEO) satellites, which comprehensively considers the needs of IoT device computing tasks and observation satellite data transmission services as well as the offloading and scheduling problems of LEO satellite networks, effectively maximizing the system's utility and thereby improving the system's performance.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for unloading and scheduling low-Earth orbit satellites, specifically including the following steps:

[0007] S1: Modeling system model;

[0008] S2: Modeling IoT device computing task models and computing task offloading modes;

[0009] S3: Modeling a computational task queue model;

[0010] S4: Modeling observation satellite data transmission service model and transmission service queue model;

[0011] S5: Modeling the energy consumption of task transmission and execution;

[0012] S6: Model the energy consumption of data transmission;

[0013] S7: Modeling the system utility function model;

[0014] S8: Transform the optimization problem of the system utility function into a Markov decision process;

[0015] S9: Determine the offloading and service scheduling strategy based on the DQN algorithm.

[0016] Furthermore, in step S1, the system model is as follows: the low-Earth orbit satellite network consists of N observation satellites, K relay satellites, M IoT devices, and one ground station. The observation satellites collect ground data, transmit it to the relay satellites, and then forward it to the ground station for caching. The IoT devices generate computing tasks, which need to be sent to the relay satellites or the ground station for task execution.

[0017] OS n Represents the nth observation satellite, RS k Represents the k-th relay satellite, RS k Its computing power is ID m Let m represent the m-th IoT device; each relay satellite has W sub-channels, and the available bandwidth of each sub-channel is B. Multiple IoT devices can use orthogonal frequency division multiple access (OFDM) technology to communicate with the relay satellite; the system time is divided into T time slots, and the length of each time slot is τ.

[0018] make This indicates the physical link identifier between the observation satellite and the relay satellite. This indicates the OS n With RS k A physical link exists in time slot t; otherwise... make This indicates the physical link identifier between the IoT device and the relay satellite. This indicates ID m With RS k A physical link exists in time slot t; otherwise... make This indicates that an identifier exists between the relay satellite and the ground station. RS k There must be a physical link with the ground station in time slot t, otherwise

[0019] Furthermore, in step S2, the modeled IoT device computing task is as follows: the computing task of each IoT device arrives randomly in each time slot, let θ m,t Indicates that time slot t arrives at ID m The computational task is to model θ. m,t For θ m,t = <D m,t ,F m,t >, where Dm,t Representing task θ m,t workload, F m,t Indicates the task θ to be processed m,t Central processing unit cycles required per bit of data;

[0020] IoT devices offload computing tasks to relay satellites, which can process tasks using either satellite offloading mode or ground station offloading mode. In satellite offloading mode, the relay satellite receives the task from the IoT device and executes it using its onboard computing resources. In ground station offloading mode, the relay satellite forwards the task to the ground station in a subsequent time slot, where the task is executed. Let λ m,t,k,t′ ∈{0,1} represents θ m,t The task transfer variable, if λ m,t,k,t′ =1, then it means θ m,t It is transmitted to the relay satellite RS in time slot t′. k Otherwise λ m,t,k,t′ =0; let Representing task θ m,t The satellite offloading mode selection variable, if Then it represents θ m,t The data is unloaded to RS in time slot t′. k and in RS k Execute here, otherwise make Representing task θ m,t The ground station unloading mode selection variable, if Then it represents θ m,t It is unloaded to relay satellite RS in time slot t′. k And unload to the ground station in time slot t″, otherwise

[0021] Furthermore, in step S3, the specific model for the computational task queue is as follows: Let Indicates time slot t, located at ID m The length of the task queue at that location is calculated. The queue update formula is modeled as follows:

[0022]

[0023] in, It is an ID m Maximum queue length;

[0024] make RS k The task queue length is calculated in time slot t, then RS k The queue update formula at the location is modeled as follows:

[0025]

[0026] in, It is RS k Calculate the maximum queue length for the task; y m,t ′, k,t ∈{0,1} represents RS k Task calculation identifier, if y m,t′,k,t =1, then it means RS k Execute task θ in time slot t m,t′ Otherwise y m,t′,k,t =0,D m,t′,k,t RS k θ needs to be calculated in time slot t. m,t′ The workload, and its update formula, can be modeled as follows:

[0027]

[0028] Among them, D m,t′,k,t In RS k The initial values ​​can be modeled as:

[0029] Furthermore, in step S4, the modeled observation satellite data transmission service model is specifically as follows: Each time slot observation satellite continuously observes the ground, collects data transmission services, and transmits them to the ground station via relay satellite in subsequent time slots; Let Indicates OS n The amount of data transmission service data collected in time slot t, let The relay satellite selection variable represents the number of satellites observed. This indicates the observation satellite OS n Data is transmitted to the relay satellite RS in time slot t. k ,otherwise make Ground station selection variables for relay satellites, if RS k OS in time slot t′ n The data is transmitted to the ground station, otherwise...

[0030] The specific model for the observation satellite data transmission service queue is as follows: Let Indicates OS n Given the service queue length in time slot t, then the OS n The queue update formula at the location is modeled as follows:

[0031]

[0032] in, For OS n Maximum queue length, To observe satellite OSn In time slot t and relay satellite RS k The link rate between them is modeled as follows:

[0033]

[0034] in, P represents the transmitting antenna gain of the observation satellite and the receiving antenna gain of the relay satellite, respectively. n For OS n The transmit power, k s T is the Boltzmann constant. s E represents the system thermal noise temperature. b N0 is the energy consumption required for a satellite to transmit one bit, and N0 is the power spectral density of the inter-satellite link noise. For OS n In time slot t and RS k The free space loss of the inter-link is modeled as follows:

[0035]

[0036] in, Indicates OS n In time slot t and relay satellite RS k The distance between them, f is the carrier frequency, and c represents the speed of light;

[0037] make RS k Given the data transmission service queue length in time slot t, then RS k The queue update formula at the location is modeled as follows:

[0038]

[0039] in, For RS k Maximum queue length for data transmission services RS k The link rate between time slot t and the ground station is modeled as follows:

[0040]

[0041] in, For RS k The transmission power, G g Representing RS respectively k The gain of the transmitting antenna and the gain of the receiving antenna at the ground station. RS k In time slot t, the rain attenuation coefficient of the ground station transmission link, RS kThe free space loss between time slot t and the ground station is modeled as follows:

[0042]

[0043] in, For RS k The distance between time slot t and the ground station.

[0044] Furthermore, in step S5, the specific model for task transmission and execution energy consumption is defined as follows: Representing task θ m,t The energy consumption for transmission and processing is modeled as follows:

[0045]

[0046] in, Indicates IoT device ID m Task θ m,t The energy consumption required for transmission to the relay satellite is modeled as follows:

[0047]

[0048] in, For ID m The transmission power, For time slot t′, the IoT device ID m With relay satellite RS k The link rate between them is modeled as follows:

[0049]

[0050] Among them, G m For ID m transmit antenna gain, For RS k The receiving antenna gain, σ 2 Indicates noise power. Indicates IoT device ID m In time slot t′ and RS k Rain attenuation coefficient of the transmission link between them Indicates the IoT device ID for time slot t′. m With RS k The free space loss of the inter-link is modeled as follows:

[0051]

[0052] in, For ID m In time slot t and RS k The distance between them, f is the carrier frequency, and c represents the speed of light;

[0053] Representing task θ m,t The energy consumption required for mission execution using the satellite offloading mode is modeled as follows:

[0054]

[0055] Where, ε k RS k The correlation coefficient, RS k Computational power;

[0056] In the expression Representing task θ m,t The energy consumption for transmission and execution required in the ground station offloading mode is modeled as follows:

[0057]

[0058] Where, ε g Represents the correlation coefficient of ground stations. This indicates the computing power of the ground station.

[0059] Furthermore, in step S6, the data transmission energy consumption model is specifically designed as follows: Let The energy consumption required for the observation satellite to transmit data to the relay satellite in time slot t is modeled as follows: make Indicates relay satellite RS k The energy consumption required to transmit data to the ground station in time slot t is modeled as follows:

[0060] Furthermore, in step S7, the model of the system utility function is as follows: U t The utility function representing time slot t is modeled as: U t =μ1E t +μ2Q t , of which E t The energy consumption of the system in time slot t is represented by the model as follows: Q t The system queue size for time slot t is modeled as follows:

[0061] Furthermore, in step S8, the system utility function optimization problem is transformed into a Markov decision process, which includes three parts: state space, action space, and reward. Specifically, the state space of time slot t is modeled. in, This represents the set of task queues for IoT devices in time slot t. This represents the set of satellite data transmission service queues for time slot t. This represents the set of relay satellite computing task queues in time slot t. This represents the set of relay satellite data transmission service queues in time slot t. This represents the set of rain attenuation coefficients for the transmission link in time slot t. This represents the set of free-space losses in the link between the observation satellite and the relay satellite in time slot t. This represents the set of free-space losses in the link between the relay satellite and the ground station in time slot t. This represents the set of free-space losses in the link between the IoT device and the relay satellite in time slot t;

[0062] Modeling the action space of time slot t Where, λ t ={λ m,t′,k,t The set of transmission strategies between all IoT devices and relay satellites in time slot t is represented by {1≤m≤M,1≤t′≤t,1≤k≤K}. This represents the set of satellite offloading strategies for time slot t. This represents the set of ground station unloading strategies for time slot t. This indicates the transmission strategy between the observation satellite and the relay satellite in time slot t. This indicates the data transmission strategy between the relay satellite and the ground station in time slot t;

[0063] The benefit of the modeling system in time slot t is:

[0064] Furthermore, in step S9, the task offloading and service scheduling strategy is determined based on the DQN algorithm, specifically including: setting the parameters required during DQN training, including the learning rate and discount rate; initializing the parameters θ of the main Q-network and the parameters θ′ of the target Q-network in the DQN model; and setting the current state... Input to the main Q network to obtain all actions corresponding to Select actions based on a greedy strategy. Get instant rewards State transition to Obtain transfer data And store them in the experience replay pool; randomly select transfer samples from the experience replay pool. Input to the neural network; output from the main Q-network and the target Q-network. and Where γ represents the discount factor, the backpropagation method is used to update the parameter θ by gradient, and θ′=θ is periodically set to complete the network parameter update; given the initial state of the system, the DQN algorithm is run to iteratively update the parameters of the main Q network and the target Q network until the algorithm converges, and the unloading and scheduling strategy is determined based on the trained Q network.

[0065] The beneficial effects of this invention are as follows: This invention comprehensively considers the resource constraints of low-orbit satellite networks, the computing tasks of Internet of Things devices, and the offloading and scheduling problems of observation satellite data transmission services. At the same time, it introduces the DQN algorithm in deep reinforcement learning to determine the optimal offloading and scheduling strategy, which effectively maximizes the utility of the system and improves the performance of the system.

[0066] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0067] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0068] Figure 1 This is a schematic diagram illustrating the offloading and scheduling scenario for low-Earth orbit satellite networks.

[0069] Figure 2 Methods for unloading and scheduling low-Earth orbit satellites;

[0070] Figure 3 This is a schematic diagram of the DQN algorithm. Detailed Implementation

[0071] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0072] Please see Figures 1-3 , Figure 1 This invention provides a method for offloading and scheduling low-Earth orbit (LEO) satellites, which is illustrated in the diagram of a low-Earth orbit (LEO) satellite network offloading and scheduling scenario. The method consists of multiple IoT devices, multiple observation satellites and relay satellites, and a ground station. It takes into account the different characteristics of computing tasks and data transmission services, as well as the impact of computing, caching, and communication resources on the LEO satellites. In this invention, the computing tasks of IoT devices can be processed using the computing resources of computing satellites or ground stations, and the data transmission services of observation satellites are forwarded to the ground station via relay satellites.

[0073] like Figure 2As shown, the method specifically includes the following steps:

[0074] S1: Modeling system model, specifically: In this model, the low-Earth orbit satellite network consists of N observation satellites, K relay satellites, M IoT devices, and one ground station. The observation satellites collect ground data, transmit it to the relay satellites, and then forward it to the ground station for caching. The IoT devices generate computing tasks, which need to be sent to the relay satellites or the ground station for task execution.

[0075] OS n Represents the nth observation satellite, RS k Represents the k-th relay satellite, RS k Its computing power is ID m Let m represent the m-th IoT device; each relay satellite has W sub-channels, and the available bandwidth of each sub-channel is B. Multiple IoT devices can use orthogonal frequency division multiple access (OFDM) technology to communicate with the relay satellite; the system time is divided into T time slots, and the length of each time slot is τ.

[0076] make This indicates the physical link identifier between the observation satellite and the relay satellite. This indicates the OS n With RS k A physical link exists in time slot t; otherwise... make This indicates the physical link identifier between the IoT device and the relay satellite. This indicates ID m With RS k A physical link exists in time slot t; otherwise... make This indicates that an identifier exists between the relay satellite and the ground station. RS k There must be a physical link with the ground station in time slot t, otherwise

[0077] S2: Modeling the computational task model for IoT devices, specifically: the computational task for each IoT device arrives randomly in each time slot, let θ m,t Indicates that time slot t arrives at ID m The computational task is to model θ. m,t For θ m,t = <D m,t ,F m,t >, where D m,t Representing task θ m,t workload, F m,t Indicates the task θ to be processed m,t Central processing unit cycles required per bit of data.

[0078] S3: Modeling the computing task offloading mode of IoT devices, specifically: IoT devices offload computing tasks to relay satellites, which can process tasks using either satellite offloading mode or ground station offloading mode; in satellite offloading mode, after receiving the task from the IoT device, the relay satellite uses its onboard computing resources to execute the task; in ground station offloading mode, the relay satellite forwards the task to the ground station in subsequent time slots, and the ground station completes the task execution.

[0079] Let λ m,t,k,t′ ∈{0,1} represents θ m,t The task transfer variable, if λ m,t,k,t′ =1, then it means θ m,t It is transmitted to the relay satellite RS in time slot t′. k Otherwise λ m,t,k,t′ =0; let Representing task θ m,t The satellite offloading mode selection variable, if Then it represents θ m,t The data is unloaded to RS in time slot t′. k and in RS k Execute here, otherwise make Representing task θ m,t The ground station unloading mode selection variable, if Then it represents θ m,t It is unloaded to relay satellite RS in time slot t′. k And unload to the ground station in time slot t″, otherwise

[0080] S4: Model the computational task queue model, specifically: Let Indicates time slot t, located at ID m The length of the task queue at that location is calculated. The queue update formula is modeled as follows:

[0081]

[0082] in, It is an ID m Maximum queue length;

[0083] make RS k The task queue length is calculated in time slot t, then RS k The queue update formula at the location is modeled as follows:

[0084]

[0085] in, It is RS kCalculate the maximum queue length for the task, y m,t′,k,t ∈{0,1} represents RS k Task calculation identifier, if y m,t′,k,t =1, then it means RS k Execute task θ in time slot t m,t′ Otherwise y m,t′,k,t =0,D m,t′,k,t RS k θ needs to be calculated in time slot t. m,t′ The workload, and its update formula, can be modeled as follows:

[0086]

[0087] Among them, D m,t′,k,t In RS k The initial values ​​can be modeled as:

[0088]

[0089] S5: Modeling the satellite data transmission service model, specifically: Observation satellites continuously observe the ground in each time slot, collect data transmission data, and transmit it to the ground station via relay satellite in subsequent time slots; Indicates OS n The amount of data transmission service data collected in time slot t, let The relay satellite selection variable represents the number of satellites observed. This indicates the observation satellite OS n Data is transmitted to the relay satellite RS in time slot t. k ,otherwise make Ground station selection variables for relay satellites, if RS k OS in time slot t′ n The data is transmitted to the ground station, otherwise...

[0090] S6: Modeling the satellite data transmission service queue model, specifically: [The following is a separate, unrelated sentence:] Let... Indicates OS n Given the service queue length in time slot t, then the OS n The queue update formula at the location is modeled as follows:

[0091]

[0092] in, For OS n Maximum queue length, To observe satellite OS n In time slot t and relay satellite RS k The link rate between them is modeled as follows:

[0093]

[0094] in, P represents the transmitting antenna gain of the observation satellite and the receiving antenna gain of the relay satellite, respectively. n For OS n The transmit power, k s T is the Boltzmann constant. s E represents the system thermal noise temperature. b N0 is the energy consumption required for a satellite to transmit one bit, and N0 is the power spectral density of the inter-satellite link noise. For OS n In time slot t and RS k The free space loss of the inter-link is modeled as follows:

[0095]

[0096] in, Indicates OS n In time slot t and relay satellite RS k The distance between them;

[0097] make RS k Given the data transmission service queue length in time slot t, then RS k The queue update formula at the location is modeled as follows:

[0098]

[0099] in, For RS k Maximum queue length for data transmission services RS k The link rate between time slot t and the ground station is modeled as follows:

[0100]

[0101] in, For RS k The transmission power, G g Representing RS respectively k The gain of the transmitting antenna and the gain of the receiving antenna at the ground station. RS k In time slot t, the rain attenuation coefficient of the ground station transmission link, RS k The free space loss between time slot t and the ground station is modeled as follows:

[0102]

[0103] in, For RS k The distance between time slot t and the ground station.

[0104] S7: Modeling the energy consumption of task transmission and execution, specifically: Definition Representing task θ m,t The energy consumption for transmission and processing is modeled as follows:

[0105]

[0106] in, Indicates IoT device ID m Task θ m,t The energy consumption required for transmission to the relay satellite is modeled as follows:

[0107]

[0108] in, For ID m The transmission power, For time slot t′, the IoT device ID m With relay satellite RS k The link rate between them is modeled as follows:

[0109]

[0110] Among them, G m For ID m transmit antenna gain, For RS k The receiving antenna gain, σ 2 Indicates noise power. Indicates IoT device ID m In time slot t′ and RS k Rain attenuation coefficient of the transmission link between them Indicates the IoT device ID for time slot t′. m With RS k The free space loss of the inter-link is modeled as follows:

[0111]

[0112] Where c represents the speed of light. For ID m In time slot t and RS k The distance between them, where f is the carrier frequency;

[0113] In the expression Representing task θ m,tThe energy consumption required for mission execution using the satellite offloading mode is modeled as follows:

[0114]

[0115] Where, ε k RS k The correlation coefficient, RS k Computational power;

[0116] In the expression Representing task θ m,t The energy consumption for transmission and execution required in the ground station offloading mode is modeled as follows:

[0117]

[0118] Where, ε g Represents the correlation coefficient of ground stations. This indicates the computing power of the ground station.

[0119] S8: Energy consumption for modeling data transmission, specifically: [Instructions to be added] The energy consumption required for the observation satellite to transmit data to the relay satellite in time slot t is modeled as follows:

[0120]

[0121] make Indicates relay satellite RS k The energy consumption required to transmit data to the ground station in time slot t is modeled as follows:

[0122]

[0123] S9: Modeling the system utility model, specifically:

[0124] Let U t The utility function representing time slot t is modeled as follows:

[0125] U t =μ1E t +μ2Q t ,

[0126] Among them, E t The energy consumption of the system in time slot t is represented by the model as follows:

[0127]

[0128] U t In the expression, Q t The system queue size for time slot t is modeled as follows:

[0129]

[0130] The model for the system utility function is as follows:

[0131]

[0132] S10: Modeling a Markov Decision Process, specifically: transforming the system utility function optimization problem into a Markov decision process, which includes three parts: state space, action space, and reward, as detailed below:

[0133] Modeling the state space of time slot t in, This represents the set of task queues for IoT devices in time slot t. This represents the set of satellite data transmission service queues for time slot t. This represents the set of relay satellite computing task queues in time slot t. This represents the set of relay satellite data transmission service queues in time slot t. This represents the set of rain attenuation coefficients for the transmission link in time slot t. This represents the set of free-space losses in the link between the observation satellite and the relay satellite in time slot t. This represents the set of free-space losses in the link between the relay satellite and the ground station in time slot t. This represents the set of free-space losses in the link between the IoT device and the relay satellite in time slot t;

[0134] Modeling the action space of time slot t Where, λ t ={λ m,t′,k,t The set of transmission strategies between all IoT devices and relay satellites in time slot t is represented by {1≤m≤M,1≤t′≤t,1≤k≤K}. This represents the set of satellite offloading strategies for time slot t. This represents the set of ground station unloading strategies for time slot t. This indicates the transmission strategy between the observation satellite and the relay satellite in time slot t. This indicates the data transmission strategy between the relay satellite and the ground station in time slot t;

[0135] The modeling system's gain in time slot t is R. t =-U t .

[0136] S11: Determine the unloading and scheduling strategy based on the DQN algorithm (DQN algorithm such as...) Figure 3 As shown), specifically: setting the parameters required during DQN training, including the learning rate and discount rate; initializing the parameters θ of the main Q-network and θ′ of the target Q-network in the DQN model; and setting the current state... Input to the main Q network to obtain all actions corresponding to Select actions based on a greedy strategy. Get instant rewards State transition to Obtain transfer data And store them in the experience replay pool; randomly select transfer samples from the experience replay pool. Input to the neural network; output from the main Q-network and the target Q-network. and Using the backpropagation method, the gradient updates the parameter θ, and periodically let θ′=θ to complete the network parameter update; given the initial state of the system, run the DQN algorithm to iteratively update the parameters of the main Q network and the target Q network until the algorithm converges, and determine the unloading and scheduling strategy based on the trained Q network.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for unloading and scheduling low-Earth orbit satellites, characterized in that, The method specifically comprises the following steps: S1: modeling a system model; S2: modeling an Internet of Things device computing task model and a computing task offloading mode; S3: modeling a computing task queue model; S4: modeling an observation satellite data transmission service model and a transmission service queue model; S5: modeling a task transmission and execution energy consumption model; S6: modeling a data transmission energy consumption model; S7: modeling a system utility function; S8: converting an optimization problem of the system utility function into a Markov decision process; S9: determining an offloading and service scheduling strategy based on a DQN algorithm; In step S1, the system model for modeling is: a low-orbit satellite network is composed of N one observation satellite, K one relay satellite, M one Internet of Things device, and one ground station, the observation satellite collects ground data, transmits to the relay satellite and is forwarded by the relay satellite to the ground station for caching; the Internet of Things device generates a computing task, which needs to be sent to the relay satellite or the ground station for task execution; make Indicates the first n One observation satellite, Indicates the first k One relay satellite, Its computing power is ,make Indicates the first m Each relay satellite has one IoT device; W There are 1 sub-channel, and the available bandwidth of each sub-channel is 1. B Multiple IoT devices communicate with relay satellites using orthogonal frequency division multiple access (OFDM) technology; the system time is divided into... T There are 1 time slot, and the length of each time slot is 1. ; Let denote the physical link identification between the observation satellite and the relay satellite, if , denote and there exists a physical link between the observation satellite and the relay satellite in time slot t , otherwise ; let denote the physical link identification between the IOT device and the relay satellite, if , denote and there exists a physical link between the IOT device and the relay satellite in time slot t , otherwise ; let denote the link existence identification between the relay satellite and the ground station, if , denote and the ground station exist a physical link in time slot t , otherwise ; In step S2, the modeling Internet of Things device calculates the task model: the random arrival of the computing task of each time slot Internet of Things device, let represent the time slot t arrival of the computing task, the modeling is , wherein represents the task amount of the task, represents the central processing unit cycle required for processing the task per bit of data; The Internet of Things device offloads the computing task to the relay satellite, which adopts a satellite offloading mode and a ground station offloading mode to process the task; in the satellite offloading mode, the relay satellite executes the task by using the on-satellite computing resource carried by the relay satellite after receiving the task of the Internet of Things device; in the ground station offloading mode, the relay satellite forwards the task to the ground station in a subsequent time slot, and the ground station completes the task execution; let denote the task transmission variable of , if , then denote that the task is transmitted to the relay satellite in the time slot , otherwise ; let denote the satellite offloading mode selection variable of the task , if , then denote that the task is offloaded to in the time slot and is executed at , otherwise ; let denote the ground station offloading mode selection variable of the task , if , then denote that the task is offloaded to the relay satellite in the time slot and is offloaded to the ground station in the time slot , otherwise ; In step S3, the modeling of the computing task queue model is specifically: let represent the time slot t , the length of the computing task queue at , and then the queue update formula of is modeled as: wherein is the maximum queue length; make express In the time slot t The length of the task queue is calculated, then The queue update formula at the location is modeled as follows: in, yes Calculate the maximum queue length for tasks; express Task calculation identifier, if , then it means In the time slot t Execute the task ,otherwise , express In the time slot t Calculation required The task load, and its update formula model is as follows: ， wherein, In The initial value of the is modeled as: ; In step S4, the modeled observation satellite data transmission service model is as follows: Each time slot observation satellite continuously observes the ground, collects data transmission data, and transmits it to the ground station via relay satellite in subsequent time slots; Let express In the time slot t The amount of data collected for data transmission services is so large that The relay satellite selection variable represents the number of satellites observed. This indicates that the satellite is under observation. In the time slot t Transmit data to relay satellite ,otherwise ;make Ground station selection variables for relay satellites, if ,express In the time slot Will The data is transmitted to the ground station, otherwise... ; The specific model for the observation satellite data transmission service queue is as follows: Let express In the time slot t The length of the business queue, then The queue update formula at the location is modeled as follows: wherein, is the maximum queue length, of the relay satellite, is the observation satellite is the link rate between the time slot t and the relay satellite, is modeled as: in, , These represent the transmitting antenna gain of the observation satellite and the receiving antenna gain of the relay satellite, respectively. for The transmission power, Boltzmann's constant, The system thermal noise temperature, The energy required for a satellite to transmit one bit. The noise power spectral density of the inter-satellite link. for In the time slot t and The free space loss of the inter-link is modeled as follows: wherein denotes in a time slot t between the relay satellite distance, carrier frequency, denotes the speed of light; make express In the time slot t The length of the data transmission service queue, then The queue update formula at the location is modeled as follows: wherein is the maximum queue length of the data transmission service at the time slot denotes in the time slot t the link rate between the satellite and the ground station is modeled as: wherein is the transmit power, , respectively denote the transmit antenna gain and the receive antenna gain of the ground station, denotes the rain attenuation coefficient of the transmission link between the satellite and the ground station at time slot t , denotes the free space loss between the satellite and the ground station at time slot , modeled as wherein is in a time slot t distance from the ground station; In step S5, the modeling of the task transmission and execution energy consumption model is defined as follows: represents the transmission and processing energy consumption of a task is modeled as: wherein, representing internet of things devices transmitting tasks The energy consumption required for a task to be transmitted to a relay satellite is modeled as: wherein, is the transmit power of the is the time slot Internet of Things device with a relay satellite between the Internet of Things device and the relay satellite is modeled as:​ wherein, is the transmit antenna gain, is the receive antenna gain, denotes the noise power, denotes an Internet of Things device transmits in a time slot with a rain fade coefficient of the transmission link between denotes a time slot an Internet of Things device with a free space loss of the link between​​ wherein is in a time slot t with a distance between is a carrier frequency denotes the speed of light; representative tasks The energy consumption of the tasks required to adopt the satellite offloading mode is modeled as: wherein represents the correlation coefficient of represents the computing power; in the expression representing a task The transmission and execution energy consumption required for the ground station offloading mode is modeled as: wherein, a correlation coefficient indicative of the ground station, a computing power indicative of the ground station; In step S6, the data transmission energy consumption model is specifically designed as follows: Let Indicates the observation satellite in the time slot t The energy consumption required to transmit data to a relay satellite is modeled as follows: ;make Indicates relay satellite In the time slot t The energy consumption required to transmit data to the ground station is modeled as follows: ; In step S7, the modeling system utility function model is: where denotes the utility function of time slot t , modeled as: where, denotes the energy consumption of the system in time slot t , modeled as: ; denotes the system queue size in time slot t , modeled as: .

2. The LEO satellite offloading and scheduling method of claim 1, wherein, In step S8, the system utility function optimization problem is converted into a Markov decision process, which includes a state space, an action space and a reward, specifically: modeling the state space of the time slot t wherein, represents the time slot t a set of IoT device task queues, represents the time slot t a set of observation satellite TT&C task queues, represents the time slot t a set of relay satellite computing task queues, represents the time slot t a set of relay satellite TT&C task queues, represents the time slot t a set of rain attenuation coefficients of the transmission link, represents the time slot t a set of free space losses of the link between the observation satellite and the relay satellite, represents the time slot t a set of free space losses of the link between the relay satellite and the ground station, represents the time slot t a set of free space losses of the link between the IoT device and the relay satellite;​ Modeling time slots t Action space of the time slots wherein, representing time slots t a set of transmission strategies of all internet of things devices with the relay satellite, representing time slots t a set of satellite offloading strategies, representing time slots t a set of ground station offloading strategies, representing time slots t a transmission strategy of the observation satellite with the relay satellite, representing time slots t a transmission strategy of the relay satellite with the ground station for data service; The modeling system in time slots t yields: .

3. The LEO satellite offloading and scheduling method of claim 2, wherein, In step S9, the task offloading and service scheduling strategy is determined based on the DQN algorithm, specifically including: setting parameters required in the DQN training process, including a learning rate and a discount rate; initializing parameters of a main Q network and parameters of a target Q network in the DQN model ; inputting a current state to the main Q network to obtain corresponding to all actions , selecting an action according to a greedy strategy, obtaining an instant reward , transferring a state to , obtaining transition data , and storing the transition data in an experience replay pool; randomly selecting transition samples from the experience replay pool and inputting the transition samples to a neural network; outputting and from the main Q network and the target Q network, wherein represents a discount factor, updating parameters by using a back propagation method, and periodically updating network parameters 1. A method for unloading and scheduling low-Earth orbit satellites, characterized in that, The method specifically comprises the following steps: to complete the network parameter updating; given an initial state of a system, running the DQN algorithm, iteratively updating parameters of the main Q network and the target Q network, until the algorithm converges, and determining the offloading and scheduling strategy based on the trained Q network.

Images