Mixed edge cloud-oriented satellite-ground fusion network service function chain arrangement method

Abstract

The invention discloses a mixed edge cloud-oriented satellite-ground convergence network service function chain (SFC) arrangement method. The method comprises the following steps: firstly, establishing a mixed edge cloud-oriented satellite-ground network model and a service function chain model; secondly, designing a service cost model and a link reliability model for the scene; thirdly, establishing a service function chain arrangement optimization problem model; then, converting the problem model by using a Markov decision process, performing feature extraction on the current network and service state, and inputting the features into a strategy generation network to obtain a deployment strategy; carrying out link mapping by adopting a Dijkstra algorithm based on reliability; and finally, optimizing an arrangement strategy by using deep reinforcement learning to realize an optimization target. According to the invention, the arrangement of the service function chain is carried out under the assistance of the cloud computing center in the satellite-ground convergence network environment, and efficient resource allocation and management are realized by taking the maximization of the service acceptance rate and the minimization of the service cost as targets.

Description

Technical Field

[0001] The present invention belongs to the field of satellite-ground fusion network technology, and specifically relates to a satellite-ground fusion network service function chain orchestration method for a hybrid edge cloud. Background Art

[0002] The integration of satellite and ground networks can achieve seamless global coverage and broadband access, meet a wide range of business needs, and improve user experience. Faced with the large scale and high dynamics of satellite-to-ground networks, traditional network architectures and hardware are difficult to effectively manage resources and services. Network function virtualization virtualizes the functions of traditional physical network devices and transforms them into virtual network functions (VNFs) that can run on general-purpose hardware, thereby achieving flexible and efficient management and dynamic configuration of network resources. When the service requested by the user arrives at the network node, it is described as a service function chain (SFC) consisting of VNFs with a specific order, and VNFs are deployed on physical nodes according to efficient orchestration strategies to establish end-to-end data paths and complete the service.

[0003] As network services continue to increase, the demand for resources has also exploded. However, due to the limited satellite resources and server capacity constraints, if SFC is deployed only relying on satellite-to-ground edge servers, the success rate of service delivery may drop significantly under high traffic demand, directly affecting service quality and continuity. In addition, a large amount of data sharing and exchange is required when processing complex orchestration tasks, which places unprecedented high demands on the reliability of network transmission links. Due to the high-speed movement of satellites in orbit, their connection with ground nodes is constantly changing, which leads to intermittent connection. This intermittent connection phenomenon means that the communication link between the satellite and the ground may be interrupted from time to time, which is a major problem for services that require continuous and stable connections.

[0004] Therefore, it is of great significance to use the powerful computing and storage capabilities of cloud computing centers as a powerful supplement to edge nodes, conduct SFC orchestration of satellite-ground networks for hybrid edge clouds, and further consider the high reliability of network links. Summary of the invention

[0005] The purpose of the present invention is to provide a service function chain arrangement method that can improve the service acceptance rate and reduce the service cost in a satellite-to-ground network scenario facing a hybrid edge cloud, while taking into account service reliability. The technical solution to achieve the purpose of the present invention is: a satellite-to-ground network service function chain arrangement method for a hybrid edge cloud, comprising the following steps:

[0006] Step 1: Establish a satellite-ground network model and service function chain model for hybrid edge cloud;

[0007] Step 2: Establish service cost model and link reliability model;

[0008] Step 3: With the goal of improving service acceptance rate and reducing service cost, the SFC scheduling optimization problem model is established by jointly considering node resources, delay and reliability constraints;

[0009] Step 4: Use the Markov decision process to transform the problem model, extract features of the current satellite-ground network and service function chain status, input the extracted features into the policy generation network, and obtain the VNF deployment strategy;

[0010] Step 5: According to the VNF deployment strategy, the reliability-based Dijkstra algorithm is used to perform link mapping;

[0011] Step 6: Use deep reinforcement learning to optimize the orchestration strategy to achieve the optimization goal, and complete the service function chain orchestration according to the optimization strategy.

[0012] Furthermore, the satellite-ground network model and service function chain model for hybrid edge cloud described in step 1 are as follows:

[0013] For the satellite-to-ground network model, the connection between two physical nodes is predictable due to the periodic motion of satellite nodes. Consider a period P, which is divided into T time slots. During the duration of each time slot t∈T, it can be assumed that the network topology remains unchanged. First, the physical network is modeled as a weighted undirected graph G = (V, E). Where V is the set of network nodes, V = V E ∪V C , V C represents the cloud computing center, V E Represents the set of edge network nodes. E =V S ∪V G , V S represents the satellite node, V G Represents the ground edge network node. E represents the connection relationship between network nodes, E=E S ∪E G ∪E GS , E S represents the set of physical links between satellite nodes, E G represents the set of physical links between ground nodes, E GS Represents the set of physical links between ground nodes and satellite nodes.

[0014] For each SFC service request q, it is modeled as a weighted directed graph G q =(V q , E q ), where V q represents the set of VNFs, E qIndicates a virtual link. Also defined is the fth VNF in service request q, |v q | represents the total number of VNFs included in the service request q. express The computing resources requested, express and The bandwidth resources requested by the virtual link between them.

[0015] Furthermore, the establishment of the service cost model and the link reliability model in step 2 includes:

[0016] The present invention measures resource cost by calculating the average resource utilization of nodes. The total resource cost of each service is expressed as R q It is defined as follows:

[0017] R q =R N +R B (1)

[0018]

[0019] In formula (2), R N represents the computing resource cost consumed by deploying VNF to physical nodes, R run Represents the operating energy consumption. Binary variable x f,v,q =1 means that VNF ​​f of service q is embedded in physical node v, if not embedded then x f,v,q = 0. IR is a constant, indicating the additional cost required to start a service node, R on Indicates whether the service node is open. The fewer open service nodes, the less computing resources the nodes consume, and the lower the service deployment cost. In formula (3), R B Represents the bandwidth resource cost of mapping virtual links to physical links. Binary variable The virtual link between VNF i and VNF j representing service q is embedded into the physical link E n,m On, otherwise The fewer the number of physical links in the virtual link mapping, the lower the bandwidth resource overhead.

[0020] The present invention quantifies the reliability of the space link by constructing a reliability probability function. The greater the reliability probability, the higher the reliability of the space link. In this paper, the definition of the link reliability probability function is affected by two aspects. First, the farther the distance between the nodes, the lower the reliability probability of the space link. Second, the smaller the change in the distance between the nodes, the higher the reliability probability of the space link. The definition is as follows:

[0021]

[0022] In the formula, represents the distance between physical nodes n and m at time slot t, It represents the change of the distance between two nodes at time slot t relative to the previous time point t-1. Let α and β, respectively, represent the corresponding weights, where α+β=1 and 0≤α,β≤1.

[0023] Furthermore, the establishment of the SFC scheduling optimization problem model described in step 3 is specifically as follows:

[0024]

[0025] Constraints (6) and (7) indicate that the initial and final VNFs must be embedded on the source and target nodes; constraint (8) indicates that as a receiving SFC, each of its VNFs is restricted to be embedded on only one network node; constraint (9) indicates that the overall delay of each service must be within its deadline S q Constraint (10) indicates that network nodes must satisfy the flow conservation, that is, the incoming flow should be equal to the outgoing flow; Constraints (11) and (12) respectively indicate that the computing and bandwidth resources allocated to SFC cannot exceed the total resources owned by the network node; Constraint (13) indicates that each service needs to satisfy the link reliability constraint, where the constant P q Indicates the link reliability threshold of service q.

[0026] Step 4 above includes:

[0027] Step 4-1: Convert the service function chain arrangement problem into a Markov decision process. At each time step t = (1, ..., T), select a physical node to place the VNF in turn until all VNFs are embedded, that is, T represents the number of VNFs contained in the current SFC. The specific definition is as follows:

[0028] State: Define the state as Contains the characteristics of the entire physical network topology and the relevant characteristics of the service function chain to be deployed. Represents the current state of the physical network. Among them, A∈R |V|×|V| is the adjacency matrix of the network, X∈R |V|×M is the characteristic matrix of the physical node. The number of adjacent physical links, the reliability of adjacent physical links, and the remaining link bandwidth of the node As the feature of each physical node, it is normalized to [0,1]. is the state of the current service function chain, the amount of resources requested by the tth VNF The bandwidth b required by the tth virtual link t,t+1, the reliability requirements of the current service function chain and the number of VNFs remaining to be placed.

[0029] Action: At time step t, the decision agent selects a physical server node and instantiates the VNF, taking action a t Expressed as in Indicates that the available resources exceed c f,q Select node n from the candidate physical nodes; otherwise, If a t =0 means deployment failed.

[0030] Reward: The immediate reward in the SFC request q orchestration process and the reward of the final state have different reward functions. The immediate reward in the intermediate process is defined as:

[0031]

[0032] Among them, θ1 and θ2 are the resource consumption coefficient of the current VNF ​​and the reliability coefficient of the mapping link at the current time step, respectively. -α1 represents the penalty for violating the constraint, which is set to a large constant value to prevent the agent from making invalid decisions.

[0033] Similarly, the reward function in the final state should evaluate the entire SFC orchestration process, and the final states in three cases are considered and reward functions are designed for them.

[0034]

[0035] Among them, β0 and β1 represent the reward values ​​of SFC orchestration, and β0 is greater than β1, so as to reward the current orchestration and punish the routing paths that violate the reliability constraints. θ3 is introduced to promote the improvement of reliability when the reliability constraints are violated.

[0036] Step 4-2: At each time step t, use the graph attention network (GAT) to explore the node and link characteristics of the current physical network. The input to GAT is the feature vector set of the physical node, represented by Z = {z1, z2, …, z N}(z∈R F ), where N is the number of nodes and F is the number of physical node features. In the output layer of the physical network embedding, an N×F hidden Matrix Z t , where F hidden Represents the dimensionality of the hidden features.

[0037] Step 4-3: Consider the SFC deployment problem as a sequence data processing problem and use the LSTM-based sequence-to-sequence network model as a proxy for VNF node selection. The details are as follows:

[0038] The encoder takes an input sequence After the LSTM unit outputs the final hidden state h t ;

[0039] In order to accurately capture the complex relationship in the SFC request, the weighted average of the encoder hidden states is calculated as the context vector c t , used to represent the encoder hidden state h t For time step j, the decoder generates a hidden state d j ,(h1,...,h T ) is the encoder hidden state. First, calculate the i-th hidden state h in the encoder i and the jth hidden state d in the decoder j The attention weights between are as follows:

[0040]

[0041] The score() function is used to measure the encoder hidden state h when generating an action at time step j. i The importance and matching degree are calculated as follows:

[0042]

[0043] Among them, ; represents the connection of two vectors, and W a is a learnable weight matrix.

[0044] Based on the attention weights, the weighted average of the encoder hidden states is calculated as the jth context vector:

[0045]

[0046] Then, the decoder calculates the current state d according to the t , context vector c t , and the flattened feature vector Z output by GAT t , combined with the agent's current strategy, outputs the value of each node and uses the fully connected layer to determine the node selection probability distribution. Based on the probability distribution, the intelligent agent will perform action a t , that is, selecting a suitable physical node based on conditional probability.

[0047] Step 5: Use the reliability-based Dijkstra algorithm to find the connection a in the physical network t and a t-1 If there is a path that meets the conditions, the current VNF ​​is executed according to action a. tComplete the placement and link mapping; otherwise, the current SFC deployment fails and the previously occupied physical resources will be released. The edge weight calculation formula of the algorithm physical link is as follows:

[0048]

[0049] in, Indicates physical link E n,m The current remaining bandwidth resources, I(·) is an indicator function, when x≥0, I(x)=1, otherwise I(x)=0.

[0050] Step 6: Use the PPO algorithm to train the strategy generation network in parallel to speed up the training.

[0051] Compared with the prior art, the present invention has the following significant advantages: (1) In the satellite-ground fusion network environment, the cloud computing center is used to assist in the SFC orchestration, solving the problem of insufficient computing and communication resources of edge nodes. (2) The present invention solves the SFC orchestration problem in the satellite-ground network for hybrid edge cloud with the goal of maximizing service acceptance rate and minimizing service cost, and proposes a reliable SFC deployment method based on PPO to achieve the optimization goal by optimizing the VNF selection strategy, in which the GAT and Seq2Seq models extract the characteristics of the physical network and the ordered information of the SFC request, generate the SFC deployment strategy, and optimize the link selection based on the improved link mapping algorithm to improve the link reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a framework diagram of the service function chain orchestration system in the satellite-ground network scenario for the hybrid edge cloud in the present invention.

[0053] Figure 2 A diagram of a service function chain orchestration method based on deep reinforcement learning in the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below with reference to the accompanying drawings.

[0055] The present invention discloses a satellite-ground fusion network service function chain orchestration method for hybrid edge cloud, comprising the following steps:

[0056] Combination Figure 1 , the satellite-ground network model and service function chain model for hybrid edge cloud described in step 1 are as follows:

[0057] For the satellite-to-ground network model, the connection between two physical nodes is predictable due to the periodic motion of satellite nodes. Consider a period P, which is divided into T time slots. In the duration of each time slot t∈T, it can be assumed that the network topology remains unchanged. First, the physical network is modeled as a weighted undirected graph G=(V,E). Where V is the set of network nodes, V=V E ∪V C , V C represents the cloud computing center, V E Represents the set of edge network nodes. E =V S ∪V G , V S represents the satellite node, V G Represents the ground edge network node. E represents the connection relationship between network nodes, E=E S ∪E G ∪E GS , E S represents the set of physical links between satellite nodes, E G represents the set of physical links between ground nodes, E GS Represents the set of physical links between ground nodes and satellite nodes.

[0058] For each SFC service request q, it is modeled as a weighted directed graph G q =(V q ,E q ), where V q represents the set of VNFs, E q Indicates a virtual link. Also defined is the fth VNF in service request q, |v q | represents the total number of VNFs included in the service request q. express The computing resources requested, express and The bandwidth resources requested by the virtual link between them.

[0059] The establishment of the service cost model and link reliability model described in step 2 includes:

[0060] The present invention measures resource cost by calculating the average resource utilization of nodes. The total resource cost of each service is expressed as R q It is defined as follows:

[0061] Rq=R N +R B (20)

[0062]

[0063] In formula (21), R N represents the computing resource cost consumed by deploying VNF to physical nodes, R run Represents the operating energy consumption. Binary variable x f,v,q =1 means that VNF ​​f of service q is embedded in physical node v, if not embedded then x f,v,q = 0. IR is a constant, indicating the additional cost required to start a service node, R on Indicates whether the service node is open. The fewer open service nodes are, the less computing resources the nodes consume, and the lower the service deployment cost. In formula (22), R B Represents the bandwidth resource cost of mapping virtual links to physical links. Binary variable The virtual link between VNF i and VNF j representing service q is embedded into the physical link E n,m On, otherwise The fewer the number of physical links in the virtual link mapping, the lower the bandwidth resource overhead.

[0064] The present invention quantifies the reliability of the space link by constructing a reliability probability function. The greater the reliability probability, the higher the reliability of the space link. In this paper, the definition of the link reliability probability function is affected by two aspects. First, the farther the distance between the nodes, the lower the reliability probability of the space link. Second, the smaller the change in the distance between the nodes, the higher the reliability probability of the space link. The definition is as follows:

[0065]

[0066] In the formula, represents the distance between physical nodes n and m at time slot t, It represents the change of the distance between two nodes at time slot t relative to the previous time point t-1. Let α and β, respectively, represent the corresponding weights, where α+β=1 and 0≤α,β≤1.

[0067] The specific model of establishing the SFC scheduling optimization problem described in step 3 is as follows:

[0068]

[0069] Constraints (25) and (26) indicate that the initial and final VNFs are to be embedded on the source and target nodes; constraint (27) indicates that as a receiving SFC, each of its VNFs is restricted to be embedded on only one network node; constraint (28) indicates that the overall delay of each service must be within its deadline S qConstraint (29) indicates that the network nodes must satisfy the flow conservation, that is, the incoming flow should be equal to the outgoing flow; constraints (30) and (31) respectively indicate that the computing and bandwidth resources allocated to the SFC cannot exceed the total resources owned by the network node; constraint (32) indicates that each service needs to satisfy the link reliability constraint, where the constant P q Indicates the link reliability threshold of service q.

[0070] Combination Figure 2 , the above step 4 includes:

[0071] Step 4-1: Convert the service function chain arrangement problem into a Markov decision process. At each time step t = (1, ..., T), select a physical node to place the VNF in turn until all VNFs are embedded, that is, T represents the number of VNFs contained in the current SFC. The specific definition is as follows:

[0072] State: Define the state as Contains the characteristics of the entire physical network topology and the relevant characteristics of the service function chain to be deployed. Represents the current state of the physical network. Among them, A∈R |V|×|V| is the adjacency matrix of the network, X∈R |V|×M is the characteristic matrix of the physical node. The number of adjacent physical links, the reliability of adjacent physical links, and the remaining link bandwidth of the node As the feature of each physical node, it is normalized to [0,1]. is the state of the current service function chain, the amount of resources requested by the tth VNF The bandwidth b required by the tth virtual link t,t+1 , the reliability requirements of the current service function chain and the number of VNFs remaining to be placed.

[0073] Action: At time step t, the decision agent selects a physical server node and instantiates the VNF, taking action a t Represented as a t = in Indicates that the available resources exceed c f,q Select node n from the candidate physical nodes; otherwise, If a t =0 means deployment failed.

[0074] Reward: The immediate reward in the SFC request q orchestration process and the reward of the final state have different reward functions. The immediate reward in the intermediate process is defined as:

[0075]

[0076] Among them, θ1 and θ2 are the resource consumption coefficient of the current VNF ​​and the reliability coefficient of the mapping link at the current time step, respectively. -α1 represents the penalty for violating the constraint, which is set to a large constant value to prevent the agent from making invalid decisions.

[0077] Similarly, the reward function in the final state should evaluate the entire SFC orchestration process, and the final states in three cases are considered and reward functions are designed for them.

[0078]

[0079] Among them, β0 and β1 represent the reward values ​​of SFC orchestration, and β0 is greater than β1, so as to reward the current orchestration and punish the routing paths that violate the reliability constraints. θ3 is introduced to promote the improvement of reliability when the reliability constraints are violated.

[0080] Step 4-2: At each time step t, use the graph attention network (GAT) to explore the node and link characteristics of the current physical network. The input to GAT is the feature vector set of the physical node, represented by Z = {z1, z2, …, z N}(z∈R F ), where N is the number of nodes and F is the number of physical node features. In the output layer of the physical network embedding, an N×F hidden Matrix Z t , where F hidden Represents the dimensionality of the hidden features.

[0081] Step 4-3: Consider the SFC deployment problem as a sequence data processing problem and use the LSTM-based sequence-to-sequence network model as a proxy for VNF node selection. The details are as follows:

[0082] The encoder takes an input sequence After the LSTM unit outputs the final hidden state h t ;

[0083] In order to accurately capture the complex relationship in the SFC request, the weighted average of the encoder hidden states is calculated as the context vector c t , used to represent the encoder hidden state h t For time step j, the decoder generates a hidden state d j ,(h1,...,h T ) is the encoder hidden state. First, calculate the i-th hidden state hi in the encoder and the j-th hidden state d in the decoder j The attention weights between are as follows:

[0084]

[0085] The score() function is used to measure the encoder hidden state h when generating an action at time step j. i The importance and matching degree are calculated as follows:

[0086]

[0087] Among them, ; represents the connection of two vectors, and W a is a learnable weight matrix.

[0088] Based on the attention weights, the weighted average of the encoder hidden states is calculated as the jth context vector:

[0089]

[0090] Then, the decoder calculates the current state d according to the t , context vector c t , and the flattened feature vector Z output by GAT t , combined with the agent's current strategy, outputs the value of each node and uses the fully connected layer to determine the node selection probability distribution. Based on the probability distribution, the intelligent agent will perform action a t , that is, selecting a suitable physical node based on conditional probability.

[0091] Step 5: Combine Figure 2 , using the reliability-based Dijkstra algorithm to find the connection a in the physical network t and a t-1 If there is a path that meets the conditions, the current VNF ​​is executed according to action a. t Complete the placement and link mapping; otherwise, the current SFC deployment fails and the previously occupied physical resources will be released. The edge weight calculation formula of the algorithm physical link is as follows:

[0092]

[0093] in, Indicates physical link E n,m The current remaining bandwidth resources, I(·) is an indicator function, when x≥0, I(x)=1, otherwise I(x)=0.

[0094] Step 6: Combine Figure 2 , the PPO algorithm is used to train the strategy generation network in parallel to speed up the training. The specific process is as follows:

[0095] During the training process, the agent continuously gains experience information through interaction with the environment. With each interaction with the environment, the agent extracts the latest collected experience from the experience cache and uses it to update its strategy. Each time the parameters are updated, the Actor network and the Critic network are optimized with the strategy loss function and the state-value loss function as the objectives respectively. The loss function of the Actor network is defined as follows:

[0096]

[0097] in, is the ratio of the probability of the new strategy to the old strategy, π θ (a t |s t ) is a new strategy, is the old strategy; clip is the truncation function, the purpose of which is to control the range of changes between the new and old strategies to [1-ε, 1+ε]; It represents the expectation of multiple samples. is the advantage function, which is used to evaluate the advantage of taking action a relative to the average behavior in state s. The specific definition is as follows:

[0098]

[0099] δ t =r t +γV(s t+1 )-V(s t ) (41)

[0100] Among them, δ t represents the TD error at time t; r t represents the reward value after taking an action; γ represents the reward discount factor; λ represents the GAE hyperparameter used to control the weight of future rewards; T represents the total number of time steps.

[0101] Finally, we use gradient ascent to update the parameters of the Actor network and the Critic network, and continuously update the network parameters θ by n update After that, the parameter θ old Update to θ. The update method is as follows:

[0102]

[0103] δ t =r t +γV(s t+1 ,w t+1 )-V(s t ,w t ) (43)

[0104]

[0105] Among them, w t ,θ t Respectively represent the parameters of the current strategy network and value network; w t+1 ,θ t+1 Respectively represent the updated network parameters; α w , α θ is the network learning rate; Indicates the gradient of w; It means to find the gradient of θ. The pseudo code of the algorithm is defined as:

[0106]

[0107] The above content describes the implementation process and advantages of the present invention. Those skilled in the art should understand that the present invention may have various changes and improvements without departing from the principles of the present invention, and these changes and improvements fall within the scope of the present invention claimed for protection.

Claims

1. A satellite-ground fusion network service function chain orchestration method for hybrid edge cloud, characterized in that: The following steps are involved: Step 1: Establish a satellite-ground network model and service function chain model for hybrid edge cloud; Step 2: Establish a service cost model and a link reliability model; Step 3: With the goal of improving service acceptance rate and reducing service cost, the SFC scheduling optimization problem model is established by jointly considering node resources, delay and reliability constraints; Step 4: Use the Markov decision process to transform the problem model, extract features of the current satellite-ground network and service function chain status, input the extracted features into the policy generation network, and obtain the VNF deployment strategy; Step 5: According to the VNF deployment strategy, the reliability-based Dijkstra algorithm is used to perform link mapping; Step 6: Use deep reinforcement learning to optimize the orchestration strategy to achieve the optimization goal, and complete the service function chain orchestration according to the optimization strategy.

2. The satellite-ground fusion network service function chain arrangement method for hybrid edge cloud according to claim 1 is characterized in that The satellite-ground network model and service function chain model for hybrid edge cloud described in step 1 are as follows: For the satellite-to-ground network model, the connection between two physical nodes is predictable due to the periodic motion of satellite nodes. Consider a period P, which is divided into T time slots. In the duration of each time slot t∈T, it can be assumed that the network topology remains unchanged. First, the physical network is modeled as a weighted undirected graph G=(V,E). Where V is the set of network nodes, V=V E ∪V C , V C represents the cloud computing center, V E Represents the set of edge network nodes. E =V S ∪V G , V S represents the satellite node, V G Represents the ground edge network node. E represents the connection relationship between network nodes, E=E S ∪E G ∪E GS , E S represents the set of physical links between satellite nodes, E G represents the set of physical links between ground nodes, E GS Represents the set of physical links between ground nodes and satellite nodes. Model each SFC service request q as a weighted directed graph G q =(V q ,E q ), where V q represents the set of VNFs, E q Indicates a virtual link. Also defined is the fth VNF in service request q, |v q | represents the total number of VNFs included in the service request q. express The computing resources requested, express and The bandwidth resources requested by the virtual link between them.

3. The satellite-ground fusion network service function chain arrangement method for hybrid edge cloud according to claim 1 is characterized in that The establishment of the service cost model and link reliability model described in step 2 includes: The present invention measures resource cost by calculating the average resource utilization of nodes. The total resource cost of each service is expressed as R q It is defined as follows: R q =R N +R B (1) In formula (2), R N represents the computing resource cost consumed by deploying VNF to physical nodes, R run Represents the operating energy consumption. Binary variable x f,v,q =1 means that VNF ​​f of service q is embedded in physical node v, if not embedded then x f,v,q = 0. IR is a constant, indicating the additional cost required to start a service node. on Indicates whether the business node is open. The fewer open nodes, the less computing resources the nodes consume, and the lower the service deployment cost. In formula (3), R B Represents the bandwidth resource cost of mapping virtual links to physical links. Binary variable The virtual link between VNF i and VNF j representing service q is embedded into the physical link E n,m On, otherwise The fewer physical links a virtual link maps to, the lower the bandwidth resource overhead. The present invention quantifies the reliability of the space link by constructing a reliability probability function. The greater the reliability probability, the higher the reliability of the space link. In this paper, the definition of the link reliability probability function is affected by two aspects. First, the farther the distance between the nodes, the lower the reliability probability of the space link. Second, the smaller the change in the distance between the nodes, the higher the reliability probability of the space link. The definition is as follows: In the formula, represents the distance between physical nodes n and m at time slot t, It represents the change of the distance between two nodes at time slot t relative to the previous time point t-1. Let α and β, respectively, represent the corresponding weights, where α+β=1 and 0≤α,β≤1.

4. The satellite-ground fusion network service function chain arrangement method for hybrid edge cloud according to claim 1 is characterized in that The SFC scheduling optimization problem model described in step 3 is established as follows: Constraints (6) and (7) indicate that the initial and final VNFs must be embedded on the source and target nodes; constraint (8) indicates that as a receiving SFC, each of its VNFs is restricted to be embedded on only one network node; constraint (9) indicates that the overall delay of each service must be within its deadline S q Constraint (10) indicates that the network nodes must satisfy the flow conservation, that is, the incoming flow should be equal to the outgoing flow; Constraints (11) and (12) respectively indicate that the computing and bandwidth resources allocated to the SFC cannot exceed the total resources owned by the network nodes; Constraint (13) indicates that each service needs to satisfy the link reliability constraint, where the constant P q Indicates the link reliability threshold of service q.

5. The satellite-ground fusion network service function chain arrangement method for hybrid edge cloud according to claim 1 is characterized in that Step 4 specifically includes: Step 4-1: Convert the service function chain arrangement problem into a Markov decision process. At each time step t = (1, ..., T), select a physical node to place the VNF in turn until all VNFs are embedded, that is, T represents the number of VNFs contained in the current SFC. The specific definition is as follows: State: Define the state as Contains the characteristics of the entire physical network topology and the relevant characteristics of the service function chain to be deployed. Represents the current state of the physical network. Among them, A∈R |V|×|V| is the adjacency matrix of the network, X∈R |V|×M is the characteristic matrix of the physical node. The number of adjacent physical links, the reliability of adjacent physical links, and the remaining link bandwidth of the node As the feature of each physical node, it is normalized to 0,1. is the state of the current service function chain, the amount of resources requested by the tth VNF The bandwidth b required by the tth virtual link t,t+1 , the reliability requirements of the current service function chain and the number of VNFs remaining to be placed. Action: At time step t, the decision agent selects a physical server node and instantiates the VNF, taking action a t Expressed as in Indicates that the available resources exceed c f,q Select node n from the candidate physical nodes; otherwise, If a t =0 means deployment failed. Reward: The immediate reward in the SFC request q orchestration process and the reward of the final state have different reward functions. The immediate reward in the intermediate process is defined as: Among them, θ1 and θ2 are the resource consumption coefficient of the current VNF ​​and the reliability coefficient of the mapping link at the current time step, respectively. -α1 represents the penalty for violating the constraint, which is set to a large constant value to prevent the agent from making invalid decisions. Similarly, the reward function in the final state should evaluate the entire SFC orchestration process, and the final states in three cases are considered and reward functions are designed for them. Among them, β0 and β1 represent the reward values ​​of SFC orchestration, and β0 is greater than β1, so as to reward the current orchestration and punish the routing paths that violate the reliability constraints. θ3 is introduced to promote the improvement of reliability when the reliability constraints are violated. Step 4-2: Use the graph attention network (GAT) to explore the physical network node and link characteristics at the current time step t. The input to GAT is the feature vector set of the physical node, represented by Z = {z1, z2, …, z N }(z∈R F ), where N is the number of nodes and F is the number of physical node features. In the output layer of the physical network embedding, an N×F hidden Matrix Z t , where F hidden Represents the dimensionality of the hidden features. Step 4-3: Consider the SFC deployment problem as a sequence data processing problem and use the LSTM-based sequence-to-sequence network model as a proxy for VNF node selection. The details are as follows: First, the encoder takes an input sequence After the LSTM unit outputs the final hidden state h t ; Second, in order to accurately capture the complex relationship in the SFC request, the weighted average of the encoder hidden states is calculated as the context vector c t , used to represent the encoder hidden state h t For time step j, the decoder generates a hidden state d j ,(h1,...,h T ) is the encoder hidden state. First, calculate the i-th hidden state h in the encoder i and the jth hidden state d in the decoder j The attention weights between are as follows: The score() function is used to measure the encoder hidden state h when generating an action at time step j. i The importance and matching degree are calculated as follows: Among them, ; represents the connection of two vectors, and W a is a learnable weight matrix. Based on the attention weights, the weighted average of the encoder hidden states is calculated as the jth context vector: Then, the decoder calculates the current state d according to the t , context vector c t , and the flattened feature vector Z output by GAT t , combined with the agent's current strategy, outputs the value of each node and uses the fully connected layer to determine the node selection probability distribution. Based on the probability distribution, the intelligent agent will perform action a t , that is, selecting a suitable physical node based on conditional probability.

6. The satellite-ground fusion network service function chain arrangement method for hybrid edge cloud according to claim 1 is characterized in that According to the VNF deployment strategy described in step 5, the reliability-based Dijkstra algorithm is used for link mapping, as follows: Use reliability-based Dijkstra algorithm to find connections in physical networks. t and a t-1 If there is a path that meets the conditions, the current VNF ​​is executed according to action a. t Complete the placement and link mapping; otherwise, the current SFC deployment fails and the previously occupied physical resources will be released. The edge weight calculation formula of the algorithm physical link is as follows: in, Indicates physical link E n,m The current remaining bandwidth resources, I(·) is an indicator function, when x≥0, I(x)=1, otherwise I(x)=0.

7. The method for arranging satellite-ground fusion network service function chains for hybrid edge cloud according to claim 1 is characterized in that Step 6 uses the PPO algorithm to train the strategy generation network in parallel to speed up the training. The details are as follows: During the training process, the agent continuously gains experience information through interaction with the environment. With each interaction with the environment, the agent extracts the latest collected experience from the experience cache and uses it to update its strategy. Each time the parameters are updated, the Actor network and the Critic network are optimized with the strategy loss function and the state-value loss function as the objectives respectively. Specifically, the loss function of the Actor network is defined as follows: in, is the ratio of the probability of the new strategy to the old strategy, π θ (a t |s t ) is a new strategy, is the old strategy; clip is the truncation function, the purpose of which is to control the range of changes between the new and old strategies to [1-ε, 1+ε]; It represents the expectation of multiple samples. is the advantage function, which is used to evaluate the advantage of taking action a relative to the average behavior in state s. The specific definition is as follows: δ t =r t +γV(s t+1 )-V(s t ) (22) Among them, δ t represents the TD error at time t; r t represents the reward value after taking an action; γ represents the reward discount factor; λ represents the GAE hyperparameter used to control the weight of future rewards; T represents the total number of time steps. Finally, use gradient ascent to update the parameters of the Actor network and the Critic network, and continuously update the network parameters θ by n update After that, the parameter θ old Update to θ. The update method is as follows: δ t =r t +γV(s t+1 ,w t+1 )-V(s t ,w t ) (24) Among them, w t ,θ t Respectively represent the parameters of the current strategy network and value network; w t+1 ,θ t+1 Respectively represent the updated network parameters; α w , α θ is the network learning rate; It means to find the gradient of w; It means to find the gradient of θ. The pseudo code of the algorithm is defined as:

Images