Service migration method considering inter-satellite link volatility in satellite edge calculation

Abstract

According to the service migration method considering inter-satellite link volatility in satellite edge calculation, when modeling is carried out on a service migration problem in satellite, the inter-satellite link volatility is considered, and the influence of migration process restart or path switching caused by dynamic change of network topology on service migration is relieved; a service migration problem is solved by using a flexible actor-critic (SAC) algorithm, and a maximum entropy model is adopted to enhance the strategy exploration capability, so that the high-dimensional solution space problem in satellite edge calculation can be effectively solved, and the purpose of rapid solution in a dynamic environment is achieved; according to the invention, when the service required by the user is migrated to different Low Earth Orbit (LEO) satellites along with time to process the request of the user, the continuity of the service of the user can be ensured, the distance between the user and the satellite where the service is located is prevented from being too long, the end-to-end time delay of the user is reduced, and the total cost of the service of the user is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer applications, and particularly relates to a service migration method considering the volatility of inter-satellite links in satellite edge computing. Background Art

[0002] Satellite networks have key advantages in improving global communication and data transmission due to their global coverage, high bandwidth, and high reliability. Especially in remote areas and places with limited communication infrastructure, satellite networks can provide ubiquitous and low-latency communication support for users. Among them, Low Earth Orbit (LEO) satellites have the advantages of low orbital altitude, short transmission delay, and small path loss compared with Medium Earth Orbit (MEO) satellites and Geosynchronous Earth Orbit (GEO) satellites. Therefore, LEO satellite networks are considered the most promising satellite communication networks.

[0003] With the help of Mobile Edge Computing (MEC) by sinking computing resources to locations close to users, Satellite Edge Computing (SEC) technology has emerged. This technology can effectively reduce the user-perceived latency by deploying MEC servers on LEO satellites to provide computing resources. Existing research on SEC technology focuses on network architecture, computing offloading, and resource allocation, etc., and there is less research on service migration-related work. However, in the SEC network, how to migrate the service instances required by users from the source satellite to the target satellite is crucial, which determines the distance between the user and the service instances they need, and has a profound impact on system performance.

[0004] In the SEC network, there is a high-speed relative motion between users and satellites. To maintain the continuity of user services, the communication between users and the satellites where their service instances are located may require multi-hop transmission, seriously affecting the user's end-to-end latency. In addition, affected by the high-speed movement of satellites, the SEC network topology exhibits significant time-varying characteristics. The inter-satellite links frequently experience disconnection and reconnection processes, resulting in high volatility of inter-satellite links and seriously affecting the communication quality of the network. Summary of the Invention

[0005] The present invention addresses the deficiencies in the above-mentioned background art and provides a service migration method considering the volatility of inter-satellite links in satellite edge computing. The volatility of inter-satellite links is considered in satellite edge computing to mitigate the impact of the restart of the migration process or path switching caused by the dynamic change of the network topology on service migration. The Soft Actor-Critic (SAC) algorithm is used to solve the service migration problem. The maximum entropy model is adopted to enhance the policy exploration ability, which can effectively solve the high-dimensional solution space problem in satellite edge computing and achieve the purpose of quickly solving in a dynamic environment.

[0006] A service migration method considering the volatility of inter-satellite links in satellite edge computing includes the following steps:

[0007] Step S1: Consider a satellite edge computing network architecture with LEO satellites as edges and users as terminals, and construct an inter-satellite link dynamic volatility coefficient according to the process of disconnection and reconnection of inter-satellite links within a period.

[0008] Step S2: Based on the inter-satellite link dynamic volatility coefficient, establish a service migration cost model. Then, based on the propagation delay, transmission delay, queuing delay, and computing delay of user requests, establish a service processing cost model. Finally, according to the service migration cost model and the service processing cost model, formalize the service migration problem considering the volatility of inter-satellite links.

[0009] Step S3: Use the Soft Actor-Critic algorithm to solve the service migration problem considering the volatility of inter-satellite links, establish a Soft Actor-Critic network model, train its neural network parameters, and use the trained Soft Actor-Critic network model to select a migration location for the service required by the user request.

[0010] The beneficial effects of the present invention are:

[0011] 1) When modeling the service migration problem in satellite edge computing, the volatility of inter-satellite links is considered to mitigate the impact of the restart of the migration process or path switching caused by the dynamic change of the network topology on service migration.

[0012] 2) The Soft Actor-Critic algorithm is used to solve the service migration problem. The maximum entropy model is adopted to enhance the policy exploration ability, which can effectively solve the high-dimensional solution space problem in satellite edge computing and achieve the purpose of quickly solving in a dynamic environment.

[0013] 3) When migrating the service required by the user to different LEO satellites over time to handle the user's request, the continuity of the user service can be guaranteed, the distance between the user and the satellite where its service is located can be avoided from being too far, the end-to-end delay of the user can be reduced, and the total cost of the user service can be lowered. Description of the Drawings

[0014] Figure 1 This is the edge computing architecture diagram of the embodiment of the present invention.

[0015] Figure 2 This is the specific flowchart of the embodiment of the present invention.

[0016] Figure 3 This is the schematic diagram of the service migration action selection model based on the flexible actor-critic algorithm in the embodiment of the present invention.

[0017] Figure 4 This is the schematic diagram of the service migration algorithm training process based on the flexible actor-critic algorithm in the embodiment of the present invention.

[0018] Figure 5 This is the experimental result graph of the average user delay in the embodiment of the present invention.

[0019] Figure 6 This is the experimental result graph of the average user service cost in the embodiment of the present invention. Detailed implementation manners

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings of the specification.

[0021] As Figure 1 shown, the edge computing architecture diagram of the embodiment of the present invention is specifically described as follows:

[0022] With the high-speed relative movement between the LEO satellite and the user, the service instance required by the user is migrated to different LEO satellites to process the user's request; when the service instance required by the user is migrated to the user's access satellite, the user request is directly processed on its access satellite; when the service instance required by the user is migrated to other satellites, the user request is transmitted from its access satellite to the satellite where its required service instance is located through the inter-satellite link for processing.

[0023] As Figure 2 shown, the specific process of the embodiment of the present invention includes:

[0024] Step S1: Consider a satellite edge computing network architecture with LEO satellites as the edge and users as the end, and construct the dynamic volatility coefficient of the inter-satellite link according to the process of disconnection and reconnection of the inter-satellite link within a period.

[0025] Step S1-1: Build the network architecture of satellite edge computing:

[0026] Consider a LEO satellite network with N satellites, each satellite is equipped with an edge server to serve as an edge computing node, represented by the set ; among them, the computing power of satellite n is C n , and the storage capacity is Zn ; Using a time-slot model, a finite period of time is divided into T time slots, and the time-slot set is denoted as Each time slot has a length of τ; Using the snapshot technology, the satellite network is topologically sampled at the beginning of each time slot, and the dynamic topology of the satellite network is described as a discrete-time graph sequence {G t (V, E(t))|t = 1, 2, …, T}; where V represents the set of all satellites in the satellite network; E(t) represents the set of inter-satellite links existing within time slot t; The adjacency matrix A(t) serves as a mathematical representation of the topology, and its element a i,j (t) = 1 if and only if (i, j) ∈ E(t), that is, satellite i and satellite j are connected in time slot t; Each satellite has limited computing power, and satellite n has a computing queue The size of which represents the load of satellite n in time slot t.

[0027] Considering there are U users on the ground, denoted as The service instance requested by user u in time slot t is To improve the conciseness of the writing and the consistency of terms, the service instance will be collectively referred to as service in the subsequent descriptions; where, u A (t) represents the input data volume per unit time; u C (t) represents the computing intensity required for the service; u Z (t) represents the storage resources required for the service; The satellite directly connected to user u in time slot t is called the access satellite of user u, denoted as The satellite serving user u in time slot t is called the service satellite of user u, denoted as Define the service migration decision of user u in time slot t as x u (t); where, is a binary variable; When it means that the service of processing user u is migrated to satellite n in time slot t, that is, satellite n is the service satellite of user u in time slot t.

[0028] Step S1-2: Construct the dynamic volatility coefficient of inter-satellite links:

[0029] The time-varying topology of the satellite network caused by the high-speed movement of satellites leads to frequent disconnections and reconnections of inter-satellite links; Use the satellite volatility coefficient to quantify the process of disconnection and reconnection of inter-satellite links. When switching time slots, the lower the volatility coefficient of a satellite, the smaller the probability of disconnection or reconnection of the links it connects, and the lower the service migration cost; For satellite its volatility coefficient F i is defined as:

[0030]

[0031] Among them, represents the number of satellites that satellite i can connect to within the period T; is an effective connection indication function, which is expressed as:

[0032]

[0033] Among them, is a link activation indication function, which is determined by the adjacency matrix A(t) = [a i,j (t)] N×N of the satellite network topology, and is expressed as:

[0034]

[0035] Based on the satellite volatility coefficient, the link dynamic volatility coefficient m i,j between satellite i and satellite j is defined as:

[0036]

[0037] Among them, F i and F j represent the volatility coefficients of satellite i and satellite j respectively.

[0038] Step S2: Based on the link dynamic volatility coefficient between satellites, establish a service migration cost model, then based on the propagation delay, transmission delay, queuing delay, and computing delay of the user request, establish a service processing cost model, and finally formalize the service migration problem considering the satellite link volatility according to the service migration cost model and the service processing cost model.

[0039] Step S2-1: Establish a service migration cost model:

[0040] Consider the impacts of two factors, namely service interruption delay and satellite link volatility, on the service migration cost; among them, the service interruption delay directly affects the user's end-to-end delay, while the satellite link volatility affects the reliability and resource consumption of the migration path.

[0041] In order to reduce the service interruption delay during the migration process, a service real-time migration based on the pre-transfer mechanism is adopted; before the migration, the data and application running status required for the service requested by the user will be migrated from the source satellite to the service satellite; however, even with the pre-transfer mechanism, whenever a migration occurs, the interruption delay is still non-zero, and the interruption delay increases with the increase of the migration distance; in addition to the interruption delay, the satellite link volatility also affects the service migration cost; the lower the link dynamic volatility coefficient, the smaller the probability of link disconnection or reconnection, and the lower the service migration cost; therefore, the cost required for the service to migrate between satellite n' and satellite n is calculated as follows:

[0042]

[0043] Among them, represents the set of satellites through which the service migrates from satellite n' to serving satellite n during time slot t; m i,j is the link dynamic volatility coefficient between satellite i and satellite j; represents the interruption delay of service migration between satellite i and satellite j; among them, d i,j is the distance between satellite i and satellite j; β i,j is the migration speed of the service between satellite i and satellite j.

[0044] The service migration decision of user u at time slot t-1 is The service migration decision at time slot t is Then the migration cost of the service required by user u at time slot t is expressed as:

[0045]

[0046] Step S2-2: Establish a service processing cost model:

[0047] Use the service processing delay as a measure of the service processing cost; the service processing delay consists of three parts: communication delay, queuing delay, and computing delay.

[0048] The service migration decision directly affects the end-to-end delay of the user; when the service migrates to the access satellite L u (t) to process the user request, that is, when L u (t) = S u (t), the communication delay only includes the space-ground communication delay; when the service migrates to other satellites to process the user request, that is, when L u (t) ≠ S u (t), the communication delay consists of two parts: space-ground communication delay and inter-satellite communication delay.

[0049] The space-ground communication delay consists of space-ground propagation delay and space-ground transmission delay; given the distance u between user u and its access satellite L and the speed of light c, the space-ground propagation delay and the space-ground transmission delay are respectively expressed as:

[0050]

[0051] Among them, represents the transmission rate through the link between user u and its access satellite L u (t); and respectively represent the bandwidth, communication power, and noise power of the link between user u and its access satellite L u (t).

[0052] The inter-satellite communication delay consists of the inter-satellite propagation delay and the inter-satellite transmission delay; the inter-satellite propagation delay and the inter-satellite transmission delay are respectively expressed as:

[0053]

[0054] wherein, represents the set of satellites passed by when forwarding the request of user u from access satellite L u (t) to serving satellite n; d i,j represents the distance between satellite i and satellite j; if i = j, then d i,j = 0; represents the transmission rate of the link between satellite i and satellite j; wherein, B i,j , P i,j and N i,j respectively represent the bandwidth, communication power, and noise power of the link between satellite i and satellite j.

[0055] Therefore, the communication delay of user u is expressed as:

[0056]

[0057] When the request of user u is processed on serving satellite n, it may be necessary to wait for calculation, thus generating a queuing delay; according to the computing power C n of serving satellite n and its computing queue the queuing delay is expressed as:

[0058]

[0059] The computing delay required for serving satellite n to process the request of user u is expressed as:

[0060]

[0061] wherein, u A (t) represents the amount of data input by user u per unit time; u C (t) represents the computing intensity required for the service requested by user u.

[0062] Therefore, the service processing cost of user u is expressed as:

[0063]

[0064] Step S2-3: Formalize the service migration problem considering the inter-satellite link volatility according to the service migration cost model and the service processing cost model:

[0065] Considering the migration cost of user services and the processing cost of user services These two metrics are integrated into the total cost of user services by using a linear weight

[0066]

[0067] where λ ∈ [0, 1] is the weight coefficient for balancing the service migration cost and the service processing cost; and are the service migration cost and service processing cost of user u at time slot t after being normalized by the Min-Max method, respectively.

[0068] Given a time range The goal of service migration is to minimize the average total cost of user services The service migration problem considering the inter-satellite link volatility is formulated as the following form:

[0069]

[0070]

[0071] where C1 represents the migration decision constraint; C2 represents that the service required by each user can only be migrated to one satellite at each time slot; C3 represents that the total storage capacity of all services on satellite n cannot exceed the storage capacity of satellite n.

[0072] Step S3: Use the flexible actor-critic algorithm to solve the service migration problem considering the inter-satellite link volatility, establish a flexible actor-critic network model, train its neural network parameters, and use the trained flexible actor-critic network model to select the migration location for the service required by user requests.

[0073] Step S3-1: Use the flexible actor-critic algorithm to solve the service migration problem considering the inter-satellite link volatility, reformulate the service migration problem through a Markov decision process, and represent it with a triple (S, A, R); where S represents the state space of the environment; A represents the action space of all users; R represents the reward obtained after the user is in state s ∈ S and executes action a ∈ A; the specific steps are as follows:

[0074] Step S3-1-1: State space S: At time slot t (t ≥ 1), the environmental state s(t) consists of five parts, which are defined as Among them, G t (V, E(t)) represents the satellite network topology graph in time slot t; N L (t) = {L1(t), …, L u (t), …, L U (r)} represents the access satellites of each user in time slot t; represents the computing queues of each satellite in time slot t; X u (t - 1) = {x1(t - 1), …, x u (t - 1), …, x U (t - 1)} represents the migration decisions of the services required by each user in time slot t - 1; M(t) represents the link dynamic volatility coefficient between every two connected satellites in time slot t.

[0075] Step S3 - 1 - 2: Action space A: At the beginning of time slot t (t ≥ 1), the central controller selects an appropriate action a(t) according to the state s(t) to make migration decisions for the services required by users; The action a(t) in time slot t = {x1(t), …, x u (t), …, x U (t)} represents the migration decisions of the services required by all users; Since there are N possible migration locations for the services required by users and a total of U users, the action dimension in time slot t is N U .

[0076] Step S3 - 1 - 3: Reward R: In time slot t, the central controller executes the action a(t) according to the state s(t) to obtain the direct reward r(t); Since reinforcement learning usually aims to maximize the reward function to obtain the optimal action, therefore, to minimize the total cost of user services, the negative opposite of the total cost of user services is used as part of the reward function; At the same time, a penalty factor based on the storage capacity limit of each LEO satellite is additionally introduced during the calculation of the reward function; Therefore, the final reward function can be rewritten as:

[0077]

[0078] Among them, r1 and r2 are the reward coefficient and the penalty coefficient respectively; Z n represents the storage capacity of satellite n.

[0079] Step S3 - 2: Establish a flexible actor - critic network model. The flexible actor - critic network model includes an actor network, two main critic networks, two target critic networks, and an experience buffer.

[0080] The actor network includes an input layer, an output layer, and two hidden layers, and its network parameters are defined as The input of the actor network is the environmental state s(t) observed at time slot t, and the output is the service migration action a(t) of all users.

[0081] Both of the two main critic networks consist of an input layer, an output layer, and two hidden layers, and their network parameters are θ1 and θ2 respectively; the inputs of both of the two main critic networks are the environmental state s(t) observed at time slot t and the output a(t) of the actor network, and the output is the corresponding soft Q value. and Take and the smaller value in [ ] as the true Q value to prevent overestimating the Q value;

[0082] Both of the two target critic networks consist of an input layer, an output layer, and two hidden layers, and their network parameters are and The inputs of the target critic network are the same as those of the main critic network, and the output is the corresponding soft Q value. and Take and the smaller value in [ ] as the true Q value to suppress the problem of overestimation; the target critic network provides a stable Q value estimate through delayed parameter update (soft update) to avoid training oscillation caused by the rapid change of the main critic network.

[0083] The experience buffer stores the historical experience of the interaction between the central controller and the environment for batch sampling during network training, and each piece of experience is saved in the form of a quadruple (s(t), a(t), r(t), s(t + 1)); where, s(t) represents the environmental state at time slot t; a(t) represents the service migration action of all users at time slot t; r(t) represents the direct reward obtained after service migration according to the service migration action a(t) of all users; s(t + 1) represents the environmental state observed after executing the action a(t) when the environmental state is s(t).

[0084] Step S3-3: Train the parameters in the flexible actor-critic network model, and the specific steps are as follows:

[0085] Step S3-3-1: Randomly initialize the actor network parameters the main critic network parameters θ1, θ2, the target critic network parameters and the initial value of the given entropy regularization coefficient α.

[0086] Step S3-3-2: Continuously interact with the environment using the actor network to obtain the quadruple (s(t), a(t), r(t), s(t + 1)), and store it in the experience replay pool.

[0087] Step S3-3-3: Repeat step S3-3-2 until enough quadruples are collected.

[0088] Step S3-3-4: Obtain samples by performing mini-batch sampling from the experience replay pool.

[0089] Step S3-3-5: Calculate the soft Q-value of the sample according to the following formula:

[0090]

[0091] where γ is the discount factor; α is the entropy regularization coefficient that controls the importance of the entropy value relative to the cumulative return; π(·|s(t)) represents the probability distribution of generating an action by the policy (actor network) in state s(t); represents the entropy of the policy π(·|s(t)) in state s(t); s0 = s means that when calculating the Q-value, the initial state is s; a0 = a means that when calculating the Q-value, the action taken at s0 = s is a.

[0092] Step S3-3-6: Update the parameters of each main critic network by performing gradient descent on the loss function L C (θ i ) of the main critic network. The loss function L C (θ i ) of the main critic network is calculated by the following formula:

[0093]

[0094] where represents the soft Q-value when the main critic network with parameter θ i executes action a(t) in state s(t); represents the target value obtained by adopting the parameters of the target critic network, and its value is calculated by the following formula:

[0095]

[0096] where is the probability of the actor network outputting action a(t+1) in state s(t+1); represents the soft Q-value predicted by the target critic network with parameter

[0097] in the next state s(t+1) and action a(t+1). Perform gradient ascent to update the parameters of the actor network The loss function of the actor network It is calculated by the following formula:

[0098]

[0099] where represents the action sampled in real time from the current actor network

[0100] Step S3-3-8: In addition to the parameters of the main critic network and the actor network, the entropy regularization coefficient α also needs to be updated; α is self-adjusted by gradient descent of the loss function L(α) and is calculated by the following formula:

[0101]

[0102] where represents the target entropy value

[0103] Step S3-3-9: To maintain the stability of learning, use the soft update method to update the parameters of the target critic network through θ i Update the parameters of the target critic network This process is expressed as:

[0104]

[0105] where τ ∈ (0, 1) is the update coefficient of the target critic network

[0106] Step S3-3-10: Repeat S3-3-2 to S3-3-8 until the reward value converges to complete the training

[0107] Step S3-4: Input the observed system state s(t), and output the actions a(t) of all users through the trained flexible actor-critic network model; according to the definition of the action space in Step S3-1, convert a(t) into the service migration decision corresponding to this action; the central controller migrates the services required by users to the corresponding locations according to this service migration decision

[0108] To prove the effectiveness of the present invention, preliminary experiments were conducted. The method proposed in the present invention was compared with the SAC, DDQN, GSP, and RAN methods. Among them, in SAC, the method is also based on the flexible actor-critic algorithm, but the inter-satellite link volatility is not considered in this method; in DDQN, the method is based on deep reinforcement learning, and a double Q-network is used for decision evaluation to reduce the overestimation problem of traditional Q-learning. The optimal migration action is sampled in the Q-network through the ε-greedy strategy, so as to optimize the service migration strategy between satellites and solve the high-dimensional dynamic migration decision problem. In GSP, the method is based on the greedy algorithm, and the migration decision of the service required by each user is updated through continuous iteration to determine the final migration decision; in RAN, the method randomly selects the migration location of the service required by each user. The comparison results are as Figure 5 and Figure 6 shown. It can be seen from the experimental results that the service migration method considering the inter-satellite link volatility proposed in the present invention has the best performance both in terms of the average user delay and the total average service cost of users. The method proposed in the present invention is more suitable for use in satellite edge computing.

[0109] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modification or change made by those of ordinary skill in the art according to the disclosure of the present invention shall be included in the protection scope recorded in the claims.

Claims

1. A service migration method considering the volatility of inter-satellite links in satellite edge computing, characterized in that: The method includes the following steps: Step S1: Consider a satellite-edge computing network architecture with a low-earth orbit (LEO) satellite as the edge and a user as the end, and construct an inter-satellite link dynamic volatility coefficient according to the disconnection and reconnection process of the inter-satellite link within a period. Step S2: Based on the inter-satellite link dynamic volatility coefficient, establish a service migration cost model. Then, based on the propagation delay, transmission delay, queuing delay, and computing delay of the user request, establish a service processing cost model with the goal of minimizing the delay. Finally, according to the service migration cost model and the service processing cost model, formalize the service migration problem considering the volatility of the inter-satellite link. Step S3: Use the flexible actor-critic algorithm to solve the service migration problem considering the volatility of the inter-satellite link, establish a flexible actor-critic network model, train its neural network parameters, and use the trained flexible actor-critic network model to select a migration location for the service required by the user request.

2. The service migration method considering the volatility of inter-satellite links in satellite edge computing according to claim 1, wherein: In step S1, the specific steps are as follows: Step S1-1: Build the network architecture of satellite-edge computing: Consider a LEO satellite network with N satellites, each of which is equipped with an edge server, making it an edge computing node, represented by the set ; among them, the computing power of satellite n is C n , and the storage capacity is Z n ; adopt the time-slot model, divide a finite period of time into T time slots, and the time-slot set is represented as Each time slot has a length of τ; adopt the snapshot technology to sample the topology of the satellite network at the beginning of each time slot, and describe the dynamic topology of the satellite network as a discrete-time graph sequence {G t (V, E(t))|t = 1, 2, …, T}; where, V represents the set of all satellites in the satellite network; E(t) represents the set of inter-satellite links existing within time slot t; the adjacency matrix A(t) is used as the mathematical representation of the topology, and its element a i,j (t) = 1 if and only if (i, j) ∈ E(t), that is, satellite i and satellite j are connected in time slot t; the computing power of each satellite is limited, and satellite n has a computing queue The size of represents the load of satellite n in time slot t. Consider that there are U users on the ground, denoted as The service instance requested by user u at time slot t is where, u A (t) represents the input data volume per unit time; u C (t) represents the computing intensity required by the service; u Z (t) represents the storage resources required by the service; The satellite directly connected to user u at time slot t is called the access satellite of user u, denoted as The satellite serving user u at time slot t is called the service satellite of user u, denoted as Define the service migration decision of user u at time slot t as x u (t); where, is a binary variable; when it means that the service of processing user u is migrated to satellite n at time slot t, that is, satellite n is the service satellite of user u at time slot t; Step S1-2: Construct the inter-satellite link dynamic volatility coefficient: The satellite network topology variation caused by the high-speed movement of satellites leads to frequent disconnection and reconnection of inter-satellite links; the process of disconnection and reconnection of inter-satellite links is quantified by the satellite volatility coefficient. When switching time slots, the lower the volatility coefficient of a satellite, the smaller the probability of disconnection or reconnection of the links it connects, and the lower the service migration cost; for a satellite whose volatility coefficient is F i is defined as: Among them, represents the number of satellites that satellite i can connect to within the period T; is an effective connection indication function, which is expressed as: Among them, is the link activation indication function, which is determined by the adjacency matrix A(t) = [a i,j (t)] N×N of the satellite network topology, and is expressed as: Based on the satellite volatility coefficient, the link dynamic volatility coefficient m between satellite i and satellite j i,j is defined as: Among them, F i and F j respectively represent the volatility coefficients of satellite i and satellite j.

3. The service migration method considering the volatility of inter-satellite links in satellite edge computing according to claim 1, wherein: In step S2, the specific steps are as follows: Step S2-1: Establish the service migration cost model: Consider the impacts of two factors, namely service interruption delay and inter-satellite link volatility, on the service migration cost. Among them, the service interruption delay directly affects the user's end-to-end delay, while the inter-satellite link volatility affects the reliability and resource consumption of the migration path. To reduce the service interruption delay during the migration process, service real-time migration based on a pre-transfer mechanism is adopted; before migration, the data and application running status required for the service requested by the user will be migrated from the source satellite to the service satellite; however, even with the pre-transfer mechanism, whenever a migration occurs, the interruption delay is still non-zero, and the interruption delay increases with the increase in the migration distance; in addition to the interruption delay, the inter-satellite link volatility also affects the service migration cost; the lower the link dynamic volatility coefficient, the smaller the probability of link disconnection or reconnection, and the lower the service migration cost; therefore, the cost required for the service to migrate between satellite n' and satellite n is calculated as follows: Among them, represents the set of satellites passed by the migrated service from satellite n' to serving satellite n during time slot t; m i,j is the link dynamic volatility coefficient between satellite i and satellite j; represents the interruption delay of service migration between satellite i and satellite j; among them, d i,j is the distance between satellite i and satellite j; β i,j is the migration speed of the service between satellite i and satellite j; The service migration decision of user u at time slot t-1 is The service migration decision at time slot t is Then the migration cost of the service required by user u at time slot t is expressed as: Step S2-2: Establish the service processing cost model: Use the service processing delay as a measure of the service processing cost. The service processing delay consists of three parts: communication delay, queuing delay, and computing delay. The service migration decision directly affects the user's end-to-end delay; when the service migrates to the access satellite L of user u u (t) processes the user request, that is, when L u (t) = S u (t), the communication delay only includes the space-ground communication delay; when the service migrates to other satellites to process the user request, that is, when L u (t) ≠ S u (t), the communication delay consists of two parts: the space-ground communication delay and the inter-satellite communication delay; The space-ground communication delay consists of the space-ground propagation delay and the space-ground transmission delay; given the distance between user u and the satellite L u (t) and the speed of light c, the space-ground propagation delay and the space-ground transmission delay are respectively expressed as: Among them, represents the transmission rate of the link between user u and its access satellite L u (t); and respectively represent the bandwidth, communication power, and noise power of the link between user u and its access satellite L u (t). The inter-satellite communication delay consists of the inter-satellite propagation delay and the inter-satellite transmission delay; the inter-satellite propagation delay and the inter-satellite transmission delay are respectively expressed as: Among them, represents the set of satellites through which the request of user u is relayed from access satellite L u (t) to serving satellite n in time slot t; d i,j represents the distance between satellite i and satellite j; if i = j, then d i,j = 0; represents the transmission rate of the link between satellite i and satellite j; among them, B i,j , P i,j and N i,j respectively represent the bandwidth, communication power and noise power of the link between satellite i and satellite j; Therefore, the communication delay of user u is expressed as: When the request of user u is processed on service satellite n, waiting for computation may occur, resulting in queuing delay; according to the computing power C of service satellite n n and its computing queue Queuing delay It is expressed as: The computing delay required for the service satellite n to process the request of user u Expressed as: Among them, u A (t) represents the amount of data input by user u per unit time; u C (t) represents the computing intensity required for the service requested by user u; Therefore, the service processing cost of user u is expressed as: Step S2-3: According to the service migration cost model and the service processing cost model, formalize the service migration problem considering the volatility of the inter-satellite link: Considering the migration cost of user services and the processing cost of user services These two indicators are integrated into the total cost of user services using linear weights where, λ∈[0,1] is the weight coefficient for balancing the service migration cost and the service processing cost; and are the service migration cost and the service processing cost of user u at time slot t after normalization by the Min-Max method, respectively; Given time range The goal of service migration is to minimize the total cost of average user services Formulate the service migration problem considering the inter-satellite link volatility as follows: Among them, C1 represents the migration decision constraint; C2 represents that the service required by each user can only be migrated to one satellite in each time slot; C3 represents that the total storage capacity of all services on satellite n cannot exceed the storage capacity of satellite n.

4. The service migration method considering the volatility of inter-satellite links in satellite edge computing according to claim 1, characterized in that: In step S3, the specific steps are as follows: Step S3-1: Use the flexible actor-critic algorithm to solve the service migration problem considering the volatility of the inter-satellite link, reformulate the service migration problem through a Markov decision process, and represent it with a triple (S, A, R). Among them, S represents the state space of the environment; A represents the action space of all users; R represents the reward obtained after the user is in state s ∈ S and executes action a ∈ A. Step S3-2: Establish a flexible actor-critic network model. The flexible actor-critic network model includes an actor network, two main critic networks, two target critic networks, and an experience buffer. The actor network includes an input layer, an output layer, and two hidden layers, and its network parameters are defined as The input of the actor network is the environmental state s(t) observed at time slot t, and the output is the service migration actions a(t) of all users; Both main critic networks consist of an input layer, an output layer, and two hidden layers, with their network parameters being θ1 and θ2 respectively; the inputs of both main critic networks are the environmental state s(t) observed at time slot t and the output a(t) of the actor network, and the output is the corresponding soft Q value. and Take and the smaller value in as the true Q value to prevent overestimation of the Q value. Both target critic networks consist of an input layer, an output layer, and two hidden layers, with their network parameters being and The input of the target critic network is the same as that of the main critic network, and the output is the corresponding soft Q-value. and Take and The smaller value in is used as the true Q-value to suppress the problem of overestimation; the target critic network provides a stable Q-value estimate through delayed parameter update (soft update) to avoid training oscillations caused by the rapid changes of the main critic network. The experience buffer stores the historical experiences of the interaction between the central controller and the environment for batch sampling during network training. Each experience is saved in the form of a quadruple (s(t), a(t), r(t), s(t+1)); where s(t) represents the environmental state at time slot t; a(t) represents the service migration actions of all users at time slot t; r(t) represents the immediate reward obtained after service migration according to the service migration actions a(t) of all users; s(t+1) represents the environmental state observed after executing action a(t) when the environmental state is s(t). Step S3-3: Train the parameters in the flexible actor-critic network model; Step S3-4: Input the observed system state s(t), and output the actions a(t) of all users through the trained flexible actor-critic network model; according to the definition of the action space in Step S3-1, convert a(t) into the service migration decision corresponding to this action; the central controller migrates the services required by users to the corresponding locations according to this service migration decision.

5. The service migration method considering the volatility of inter-satellite links in satellite edge computing according to claim 4, wherein: The specific steps of Step S3-1 are as follows: Step S3-1-1: State space S: At time slot t (t≥1), the environmental state s(t) consists of five parts, which are defined as where, G t (V, E(t)) represents the satellite network topology graph at time slot t; N L (t) = {L1(t), …, L u (t), …, L U (t)} represents the access satellites of each user at time slot t; represents the computing queues of each satellite at time slot t; X u (t - 1) = {x1(t - 1), …, x u (t - 1), …, x U (t - 1)} represents the migration decisions of the required services requested by each user at time slot t - 1; M(t) represents the link dynamic volatility coefficient between every two connected satellites at time slot t; Step S3-1-2: Action space A: At the beginning of time slot t (t≥1), the central controller selects an appropriate action a(t) according to the state s(t) to make a migration decision for the services required by the users; the action a(t) at time slot t = {x1(t), …, x u (t), …, x U (t)} represents the migration decisions for all the services required by the users; since there are N possible migration locations for the services required by the users and there are a total of U users, the action dimension at time slot t is N U ; Step S3-1-3: Reward R: At time slot t, the central controller executes action a(t) according to state s(t) to obtain the immediate reward r(t); since reinforcement learning usually aims to maximize the reward function to obtain the optimal action, therefore, in order to minimize the total user service cost, the negative value of the total user service cost is used as part of the reward function; at the same time, a penalty factor based on the storage capacity limit of each LEO satellite is additionally introduced during the calculation of the reward function; therefore, the final reward function can be rewritten as: where r1 and r2 are the reward coefficient and the penalty coefficient respectively; Z n represents the storage capacity of satellite n.

6. The service migration method considering the volatility of inter-satellite links in satellite edge computing according to claim 4, wherein: The specific steps of Step S3-3 are as follows: Step S3-3-1: Randomly initialize the parameters of the actor network The parameters θ1, θ2 of the main critic network and the parameters of the target critic network and the initial value of the given entropy regularization coefficient α; Step S3-3-2: Use the actor network to continuously interact with the environment to obtain the quadruple (s(t), a(t), r(t), s(t+1)), and store it in the experience replay pool; Step S3-3-3: Repeat Step S3-3-2 until enough quadruples are collected; Step S3-3-4: Perform mini batch sampling from the experience replay pool to obtain samples; Step S3-3-5: Calculate the soft Q value of the sample according to the following formula: Among them, γ is the discount factor; α is the entropy regularization coefficient that controls the importance of the entropy value relative to the cumulative return; π(·|s(t)) represents the probability distribution of the policy (actor network) generating an action in state s(t). represents the entropy of the policy π(·|s(t)) in state s(t); s0 = s means that when calculating the Q value, the initial state is s; a0 = a means that when calculating the Q value, the action taken at s0 = s is a. Step S3-3-6: By performing gradient descent on the loss function \(L\) of the main critic network C (\(\theta\) i ), update the parameters of each main critic network The loss function \(L\) of the main critic network C (\(\theta\) i ) is calculated by the following formula: Among them, represents the soft Q-value when the main critic network with parameter θ i executes action a(t) in state s(t); represents the target value obtained by adopting the parameters of the target critic network and its value is calculated by the following formula: wherein, is the probability that the actor network outputs the action a(t + 1) in the state s(t + 1); represents the soft Q value predicted by the target critic network with parameters at the next state s(t + 1) and the action a(t + 1); Step S3-3-7: Update the parameters of the actor network by performing gradient ascent on the loss function of the actor network to update the parameters of the actor network The loss function of the actor network is calculated by the following formula: Among them, represents the action sampled in real time from the current actor network; Step S3-3-8: In addition to the parameters of the main critic network and actor network, the entropy regularization coefficient α also needs to be updated; α self-adjusts through the gradient descent of the loss function L(α), and is calculated by the following formula: Among them, represents the target entropy value; Step S3-3-9: To maintain the stability of learning, the soft update method is used to update the target critic network parameters by θ i This process is expressed as:​ where τ∈(0,1) is the update coefficient of the target critic network. Step S3-3-10: Repeat S3-3-2 to S3-3-8 until the reward value converges to complete the training.

Images