A star-ground fusion network service function chain arrangement method for a hybrid edge cloud

Abstract

The application discloses a kind of star-ground fusion network service function chain (SFC) arrangement methods for hybrid edge cloud.The method is as follows: firstly, establish the star-ground network model and service function chain model for hybrid edge cloud;Second, design the service cost model, link reliability model for the scene;Then, establish service function chain arrangement optimization problem model;Then, using Markov decision process to transform problem model, the current network and service state are extracted, input into policy generation network, and deployment strategy is obtained;Reliable Dijkstra algorithm is used for link mapping;Finally, the arrangement strategy is optimized using deep reinforcement learning, and the optimization target is realized.The application uses cloud computing center to assist in the arrangement of service function chain under the star-ground fusion network environment, to maximize service acceptance rate and minimize service cost as the goal to realize efficient resource allocation and management.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of satellite-ground integrated network, and particularly relates to a service function chain orchestration method for a hybrid edge cloud-oriented satellite-ground integrated network. BACKGROUND

[0002] Satellite and ground network integration can achieve global seamless coverage and broadband access, meet extensive business needs, and improve user experience. In the face of large-scale and high-dynamic satellite-ground networks, traditional network architecture and hardware are difficult to effectively manage resources and services. Network function virtualization changes the functions of traditional physical network devices into virtual network functions (VNFs) that can run on general-purpose hardware, thereby realizing flexible and efficient management and dynamic configuration of network resources. The service requested by a user reaches a network node and is described as a service function chain (SFC) composed of VNFs with a specific order, and VNFs are deployed on physical nodes according to an efficient orchestration strategy to establish an end-to-end data path and complete the service.

[0003] With the continuous increase in network services, the demand for resources also increases explosively. However, due to the limited nature of satellite resources and the limitation of server capacity, if only relying on satellite-ground edge servers to deploy SFCs, the success rate of service delivery may decrease significantly under high traffic demand, directly affecting the quality and continuity of services. In addition, a large amount of data sharing and exchange is required when processing complex orchestration tasks, which puts unprecedented high requirements on the reliability of network transmission links. Due to the high-speed movement of satellites in orbit, the connection between them and the ground nodes will change constantly, which leads to intermittent connection. This intermittent connection phenomenon means that the communication link between the satellite and the ground may be interrupted from time to time, which is a major problem for services that require continuous and stable connection.

[0004] Therefore, it is of great significance to use the powerful computing and storage capabilities of cloud computing centers as a powerful supplement to edge nodes, to perform SFC orchestration for a hybrid edge cloud-oriented satellite-ground network, and to further consider the high reliability of network links. SUMMARY

[0005] The application aims to provide a service function chain orchestration method that can improve service acceptance rate and reduce service cost while considering service reliability in a hybrid edge cloud-oriented satellite-ground network scenario. The technical solution for achieving the purpose of the application is: a service function chain orchestration method for a hybrid edge cloud-oriented satellite-ground network, comprising the following steps:

[0006] Step 1: establishing a hybrid edge cloud-oriented satellite-ground network model and a service function chain model;

[0007] Step 2: establishing a service cost model and a link reliability model;

[0008] Step 3, in order to improve service acceptance rate and reduce service cost, considering node resources, delay and reliability constraints, an SFC orchestration optimization problem model is established;

[0009] Step 4, using Markov decision process to transform the problem model, the current satellite-ground network and service function chain state are extracted, and the extracted features are input into the policy generation network to obtain the VNF deployment strategy;

[0010] Step 5, according to the VNF deployment strategy, link mapping is performed using a reliability-based Dijkstra algorithm;

[0011] Step 6, using deep reinforcement learning to optimize the orchestration strategy to achieve the optimization goal, and completing the service function chain orchestration according to the optimized strategy.

[0012] Further, the satellite-ground network model and the service function chain model in step 1 are as follows:

[0013] For the satellite-ground network model, due to the periodic motion of satellite nodes, the connection between two physical nodes is predictable. Consider a period P, which is divided into T time slots. In the duration of each time slot t∈T, it is assumed that the network topology remains unchanged. First, the physical network is modeled as a weighted undirected graph G=(V,E). Where V is the set of network nodes, V=V E ∪V C , V C represents a cloud computing center, V E represents a set of edge network nodes. Where V E =V S ∪V G , V S represents a satellite node, V G represents a ground edge network node. E represents the connection relationship between nodes, E=E S ∪E G ∪E GS , E S represents a set of physical links between satellite nodes, E G represents a set of physical links between ground nodes, E GS represents a set of physical links between ground nodes and satellite nodes.

[0014] For each SFC service request q, it is modeled as a weighted directed graph G q =(V q ,E q ), where V q represents a set of VNFs, E qrepresents a virtual link. Meanwhile, define |v q | represents the total number of VNFs contained in service request q. Then represents the requested computing resources, represents and the requested bandwidth resources of the virtual link between

[0015] Further, the establishment of the service cost model and the link reliability model in step 2 includes:

[0016] The present application measures the resource cost by calculating the average resource utilization of the nodes. The total resource cost of each service is represented by R q , which is defined as follows:

[0017] R q = R N + R B (1)

[0018]

[0019] In equation (2), R N represents the computing resource cost consumed by the deployment of VNF to the physical node, R run represents the operating energy consumption. The binary variable x f,v,q = 1 indicates that the VNF f of service q is embedded on the physical node v, and x f,v,q = 0 if not. IR is a constant, representing the additional cost required to start the service node, and R on represents whether the service node is open. The fewer the open service nodes, the less computing resources the node consumes, and the lower the service deployment cost. In equation (3), R B represents the bandwidth resource cost of the virtual link mapped to the physical link. The binary variable represents that the virtual link between VNF i and VNF j of service q is embedded on the physical link E n,m , otherwise The fewer the number of physical links in the virtual link mapping, the lower the bandwidth resource overhead.

[0020] The present application quantifies the reliability of the spatial link by constructing a reliability probability function. The greater the reliability probability, the higher the reliability of the spatial link. In this paper, the definition of the link reliability probability function is influenced by two aspects. First, the farther the distance between nodes, the lower the spatial link reliability probability. Second, the smaller the change in the distance between nodes, the higher the spatial link reliability probability. The definition is as follows:

[0021]

[0022] wherein, denotes the distance between physical nodes n and m at time slot t, denotes the change of the distance between two nodes at time slot t relative to the previous time point t-1. Let a and b denote the corresponding weights, where a + b = 1 and 0 < a, b < 1.

[0023] Further, the step 3 of establishing the SFC orchestration optimization problem model is specifically as follows:

[0024]

[0025] Constraints (6) and (7) indicate that the initial and final VNFs are to be embedded on the source node and the target node; constraint (8) indicates that as a received SFC, each VNF thereof is limited to be embedded on only one network node; constraint (9) indicates that the overall delay of each service must be within its deadline S q ; constraint (10) indicates that the network node is to satisfy the flow conservation, i.e., the inflow flow should be equal to the outflow flow; constraints (11) and (12) respectively indicate that the computation and bandwidth resources allocated for the SFC cannot exceed the total amount of resources possessed by the network node; and constraint (13) indicates that each service needs to satisfy the link reliability constraint, wherein the constant P q denotes the link reliability threshold value of the service q.

[0026] The above step 4 includes:

[0027] Step 4-1: converting the service function chain orchestration problem into a Markov decision process. At each time step t = (1, …, T), a physical node is selected in turn to place the VNF, until all VNFs are embedded, i.e., T denotes the number of VNFs contained in the current SFC. The specific definition is as follows:

[0028] State: the state is defined as including the features of the entire physical network topology and the related features of the service function chain to be deployed. representing the current state of the physical network. Wherein, A e R |V|×|V| is the adjacency matrix of the network, X e R |V|×M is the feature matrix of the physical node. The remaining computing resources of the physical node, the number of adjacent physical links, the reliability of the adjacent physical links, and the remaining link bandwidth of the node are taken as the features of each physical node, and are normalized to [0, 1]. is the state of the current service function chain, and the resource amount required by the tthvirtual link is b t,t+1, the reliability requirements of the current service function chain and the number of VNFs remaining to be placed.

[0029] Action: At time step t, the decision agent selects a physical server node and instantiates the VNF, taking action a t Expressed as in Indicates that the available resources exceed c f,q Select node n from the candidate physical nodes; otherwise, If a t =0 means deployment failed.

[0030] Reward: The immediate reward during the SFC request q orchestration process and the reward at the final state have different reward functions. The immediate reward in the intermediate process is defined as:

[0031]

[0032] where θ1 and θ2 are the resource consumption coefficient of the current VNF ​​and the reliability coefficient of the mapping link at the current time step, respectively. −α1 represents the penalty for violating the constraint, which is set to a large constant value to prevent the agent from making invalid decisions.

[0033] Similarly, the reward function in the final state should evaluate the entire SFC orchestration process. The final states in three cases are considered and reward functions are designed for them.

[0034]

[0035] Where β0 and β1 represent the reward values ​​of the SFC orchestration, and β0 is greater than β1, thereby rewarding the current orchestration and penalizing routing paths that violate reliability constraints. θ3 is introduced to promote reliability improvement when reliability constraints are violated.

[0036] Step 4-2: At each time step t, use the graph attention network (GAT) to explore the node and link characteristics of the current physical network. The input to GAT is the feature vector set of the physical node, with Z = {z1,z2,…,z N}(z∈R F ), where N is the number of nodes and F is the number of physical node features. In the output layer of the physical network embedding, an N×F hidden Matrix Z t , where F hidden Represents the dimensionality of the latent features.

[0037] Step 4-3: Consider the SFC deployment problem as a sequence data processing problem and use the LSTM-based sequence-to-sequence network model as a proxy for VNF node selection. The details are as follows:

[0038] The encoder takes an input sequence The final hidden state h t is outputted through the LSTM unit

[0039] To precisely capture the complex relationships in SFC requests, a weighted average of the encoder hidden states is computed as the context vector c t to represent the relevance between the encoder hidden states h t and the output sequence. For time step j, the generated hidden state d j in the decoder is computed as follows: T (h1,...,h i ) are the encoder hidden states. First, the attention weight between the i-th hidden state h j in the encoder and the j-th hidden state d i in the decoder is computed as follows:

[0040]

[0041] where the score() function is used to measure the importance and matching degree of the encoder hidden state h i when generating an action at time step j, which is computed as follows:

[0042]

[0043] where represents the concatenation of two vectors, and W a are learnable weight matrices.

[0044] Based on the attention weight, a weighted average of the encoder hidden states is computed as the j-th context vector:

[0045]

[0046] Then, the decoder outputs the value of each node and determines the node selection probability distribution using a fully connected layer based on the current time step state d t , the context vector c t , and the flattened feature vector Z t output by the GAT, combined with the current policy of the agent. Based on the probability distribution, the intelligent agent will perform an action a t , i.e., select an appropriate physical node according to the conditional probability.

[0047] Step 5: Use the reliability-based Dijkstra algorithm to find the path connecting a t and a t-1 in the physical network. If there is a path that meets the conditions, the current VNF will be placed according to the action a tThe placement is completed and the link mapping is completed, otherwise, the current SFC deployment fails, and the previously occupied physical resources will be released.

[0048]

[0049] wherein, denotes the physical link E n,m The current residual bandwidth resource, I(·) is an indicator function, when x is greater than or equal to 0, I(x) = 1, otherwise I(x) = 0.

[0050] Step 6: the PPO algorithm is used to perform parallel training on the strategy generation network, so as to accelerate the training speed.

[0051] Compared with the prior art, the present application has the following advantages: (1) the SFC arrangement is assisted by a cloud computing center in a star-ground fusion network environment, so as to solve the problem of insufficient edge node computing and communication resources; (2) the present application solves the SFC arrangement problem in a star-ground network facing a hybrid edge cloud, with the target of maximizing service acceptance rate and minimizing service cost, and proposes a reliable SFC deployment method based on PPO, which realizes the optimization target by optimizing the VNF selection strategy, wherein the GAT and Seq2Seq models extract the features of the physical network and the ordered information of the SFC request, generate the SFC deployment strategy, and optimize the link selection by using an improved link mapping algorithm to improve the link reliability. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The figure is a service function chain arrangement system framework diagram in a star-ground network scenario facing a hybrid edge cloud in the present application.

[0053] Figure 2 The figure is a service function chain arrangement method based on deep reinforcement learning in the present application. DETAILED DESCRIPTION

[0054] The present application will be further described in detail below with reference to the accompanying drawings.

[0055] The present application discloses a star-ground fusion network service function chain arrangement method facing a hybrid edge cloud, which comprises the following steps:

[0056] In combination with Figure 1 The star-ground network model facing a hybrid edge cloud and the service function chain model in step 1 are as follows:

[0057] For the satellite-to-ground network model, the connection between two physical nodes is predictable due to the periodic motion of satellite nodes. Consider a period P, which is divided into T time slots. During the duration of each time slot t∈T, the network topology can be assumed to remain unchanged. First, the physical network is modeled as a weighted undirected graph G=(V,E). Where V is the set of network nodes, V=V E ∪V C , V C represents the cloud computing center, V E Represents the set of edge network nodes. E =V S ∪V G , V S represents the satellite node, V G Represents the ground edge network node. E represents the connection relationship between network nodes, E=E S ∪E G ∪E GS , E S represents the set of physical links between satellite nodes, E G represents the set of physical links between ground nodes, E GS Represents the set of physical links between ground nodes and satellite nodes.

[0058] For each SFC service request q, it is modeled as a weighted directed graph G q =(V q ,E q ), where V q represents the set of VNFs, E q Indicates a virtual link. Also defines is the fth VNF in service request q, |v q | represents the total number of VNFs included in the service request q. express The requested computing resources, express and The bandwidth resources requested by the virtual link between them.

[0059] The establishment of the service cost model and link reliability model described in step 2 includes:

[0060] The present invention measures resource cost by calculating the average resource utilization of nodes. The total resource cost of each service is expressed as R q It is defined as follows:

[0061] Rq=R N +R B (20)

[0062]

[0063] In formula (21), R N represents the computing resource cost consumed by VNF deployment to the physical node, R run represents the running energy consumption. The binary variable x f,v,q = 1 indicates that the VNF f of the service q is embedded on the physical node v, and x f,v,q = 0 indicates that the VNF f of the service q is not embedded on the physical node v. IR is a constant, representing the additional cost required to start the service node, R on represents whether the service node is open. The fewer the open service nodes, the less the computing resources consumed by the node, and the lower the service deployment cost. In formula (22), R B represents the bandwidth resource cost of the virtual link mapped to the physical link. The binary variable represents that the virtual link between the VNF i and the VNF j of the service q is embedded on the physical link E n,m , otherwise The fewer the number of physical links in the virtual link mapping, the lower the bandwidth resource overhead.

[0064] The present application quantifies the reliability of the spatial link by constructing a reliability probability function. The greater the reliability probability, the higher the reliability of the spatial link. In this paper, the definition of the link reliability probability function is influenced by two aspects. First, the farther the distance between nodes, the lower the spatial link reliability probability. Second, the smaller the change in the distance between nodes, the higher the spatial link reliability probability. The definition is as follows:

[0065]

[0066] In the formula, d represents the distance between the physical nodes n and m at time slot t, represents the change of the distance between the two nodes at time slot t relative to the previous time point t-1. Let α and β, respectively, represent the corresponding weights, where α+β=1 and 0≤α,β≤1.

[0067] The establishment of the SFC orchestration optimization problem model described in step 3 is as follows:

[0068]

[0069] Constraints (25) and (26) indicate that the initial and final VNFs are to be embedded on the source node and the target node; constraint (27) indicates that as a received SFC, each VNF thereof is limited to being embedded on only one network node; and constraint (28) indicates that the overall delay of each service must be within its deadline S qIn, constraint (29) indicates that the network node should satisfy the flow conservation, i.e., the flow in should be equal to the flow out; constraints (30) and (31) indicate that the computational and bandwidth resources allocated for SFC should not exceed the total amount of resources owned by the network node; constraint (32) indicates that each service needs to satisfy the link reliability constraint, where constant P q represents the link reliability threshold of service q.

[0070] In combination Figure 2 , the above step 4 includes:

[0071] Step 4-1: converting the service function chain orchestration problem into a Markov decision process. At each time step t = (1, …, T), a physical node is selected in turn to place VNF until all VNFs are embedded, i.e., T represents the number of VNFs contained in the current SFC. The specific definition is as follows:

[0072] State: define the state as including the features of the entire physical network topology and the related features of the service function chain to be deployed. representing the current state of the physical network. Wherein, A ∈ R |V|×|V| is the adjacency matrix of the network, X ∈ R |V|×M is the feature matrix of the physical node. The remaining computational resources of the physical node, the number of adjacent physical links, the reliability of adjacent physical links, and the remaining link bandwidth of the node are taken as the features of each physical node, and are normalized to [0, 1]. is the state of the current service function chain, which is composed of the resource amount required by the tth VNF, the bandwidth b t,t+1 required by the tth virtual link, the reliability requirement of the current service function chain, and the number of VNFs remaining to be placed.

[0073] Action: at time step t, the decision agent selects a physical server node and instantiates the VNF, and the action a t is represented as a t = wherein represents selecting node n from the candidate physical nodes whose available resources exceed c f,q ; otherwise, if a t = 0, it indicates that the deployment fails.

[0074] Reward: the immediate reward and the reward of the final state in the SFC request q orchestration process have different reward functions. The immediate reward in the middle process is defined as:

[0075]

[0076] where θ1 and θ2 are the resource consumption coefficient of the current VNF and the reliability coefficient of the mapping link at the current time step, respectively. -α1 represents the penalty for violating the constraint, which is set to a large constant value to prevent the agent from making invalid decisions.

[0077] Similarly, the reward function in the final state should evaluate the entire SFC orchestration process, considering the final state in three cases and designing a reward function for it.

[0078]

[0079] where β0 and β1 represent the reward value of SFC orchestration, and β0 is greater than β1, so as to reward the current orchestration and punish the routing path that violates the reliability constraint. θ3 is introduced to promote the improvement of reliability when the reliability constraint is violated.

[0080] Step 4-2: At each time step t, use the graph attention network (GAT) to explore the node and link features of the current physical network. The input of GAT is the feature vector set of physical nodes, which is denoted as Z = {z1, z2, …, zN} (z ∈ R N ), where N is the number of nodes, and F represents the number of physical node features. In the output layer of the physical network embedding, an N × F F matrix Z hidden is generated, where F t represents the dimension of the hidden features.

[0081] Step 4-3: Treat the SFC deployment problem as a sequence data processing problem, and use an LSTM-based sequence-to-sequence network model as the agent for VNF node selection. Specifically as follows:

[0082] The encoder takes an input sequence hidden ; The LSTM unit outputs the final hidden state h t ;

[0083] In order to accurately capture the complex relationships in the SFC request, the weighted average value of the encoder hidden state is calculated as the context vector c t , which is used to represent the correlation between the encoder hidden state h t and the output sequence. For time step j, the decoder generates hidden state d j , (h1,...,h T ) are the encoder hidden states. First, calculate the attention weight between the i-th hidden state hi in the encoder and the j-th hidden state d j in the decoder as follows:

[0084]

[0085] where the score() function measures the importance and matching degree of the encoder hidden state h i when generating action at time step j, which is calculated as follows:

[0086]

[0087] where denotes the concatenation of two vectors, and W a are learnable weight matrices.

[0088] Based on the attention weight, the weighted average of the encoder hidden state is calculated as the jth context vector:

[0089]

[0090] Then, the decoder outputs the value of each node and processes the node selection probability distribution using a fully connected layer based on the current time step state d t , the context vector c t , and the flattened feature vector Z t output by the GAT, combined with the current policy of the agent. Based on the probability distribution, the intelligent agent will perform action a t , that is, select an appropriate physical node according to the conditional probability.

[0091] Step 5: combined Figure 2 , use the reliability-based Dijkstra algorithm to find the path connecting a t and a t-1 in the physical network. If there is a path that meets the conditions, place the current VNF according to action a t and complete the link mapping; otherwise, the current SFC deployment fails, and the previously occupied physical resources will be released. The edge weight calculation formula of the algorithm physical link is as follows:

[0092]

[0093] where, denotes the current remaining bandwidth resource of physical link E n,m , and I(·) is the indicator function, I(x) = 1 when x ≥ 0, otherwise I(x) = 0.

[0094] Step 6: combined Figure 2 , use the PPO algorithm to train the policy generation network in parallel to speed up the training. The specific process is as follows:

[0095] During training, the agent continuously gains experience through interaction with the environment. With each interaction, the agent extracts the latest experience from its experience cache and uses it to update its policy. Each time a parameter update is performed, the actor network and critic network optimize using the policy loss function and state-value loss function, respectively. The actor network's loss function is defined as follows:

[0096]

[0097] in, is the ratio of the probability of the new strategy to the old strategy, π θ (a t |s t ) is a new strategy, is the old strategy; clip is the truncation function, the purpose of which is to control the range of changes between the new and old strategies to [1-ε,1+ε]; It represents the expectation of multiple samples. is the advantage function, which is used to evaluate the advantage of taking action a relative to the average behavior in state s. The specific definition is as follows:

[0098]

[0099] δ t =r t +γV(s t+1 )-V(s t ) (41)

[0100] Among them, δ t represents the TD error at time t; r t Represents the reward value after taking the action; γ represents the reward discount factor; λ represents the GAE hyperparameter used to control the weight of future rewards; T represents the total number of time steps.

[0101] Finally, the parameters of the Actor network and the Critic network are updated using gradient ascent, and the network parameters θ are continuously updated by n update After that, the parameter θ old Update to θ. The update method is as follows:

[0102]

[0103] δ t =r t +γV(s t+1 ,w t+1 )-V(s t ,w t ) (43)

[0104]

[0105] where w t , θ t are the parameters of the policy network and value network at the current time step, respectively; w t+1 , θ t+1 are the updated network parameters; α w , α θ are the network learning rates; denotes the gradient with respect to w denotes the gradient with respect to θ

[0106]

[0107] The above describes the implementation process and advantages of the present application. Those skilled in the art should understand that, without departing from the principles of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A method for Star-ground fusion network service function chain (SFC) orchestration for hybrid edge cloud, characterized in that, The method comprises the following steps: Step 1, establishing a hybrid edge cloud-oriented satellite-ground network model and a SFC model; Step 2, establishing a service cost model and a link reliability model; Step 3, establishing a SFC arrangement optimization problem model by jointly considering node resources, delay and reliability constraints to improve service acceptance rate and reduce service cost; the optimization problem is represented as: s.t. where z q = 1 indicates SFC q is successfully deployed, otherwise z q = 0; R q is the total cost for each SFC; Q indicates the set of SFCs; Constraints C1 and C2 ensure that initial and final virtual network functions (VNFs) are instantiated on source and target nodes, x f,v,q = 1 if VNF f of SFC q is instantiated on physical node v, otherwise x f,v,q = 0; constraint C3 ensures that each VNF has a unique mapped physical node; constraint C4 ensures that the overall delay D q of each SFC is within its deadline S q ; constraint C5 ensures that network nodes have to satisfy the flow conservation, denotes that the virtual link between VNFi and VNFj of service q is instantiated on physical link E n,m , otherwise constraints C6 and C7 ensure that the computation and bandwidth resources allocated for SFCs cannot exceed the total amount of resources owned by the network nodes, V is the set of network nodes and E is the set of links between network nodes; constraint C8 ensures that links satisfy a link reliability threshold P q , denotes the reliability probability function of a spatial link; Step 4, using a Markov decision process to transform the problem model, extracting features of the current satellite-ground network and SFC state, inputting the extracted features into a policy generation network to obtain a VNF deployment policy; Step 5, according to the VNF deployment policy, performing link mapping by using a reliability-based Dijkstra algorithm; Step 6, optimizing the arrangement policy by using deep reinforcement learning to achieve the optimization goal, and completing SFC arrangement according to the optimized policy.

2. The hybrid edge cloud oriented satellite-ground converged network (SFC) orchestration method of claim 1, wherein The hybrid edge cloud-oriented satellite-ground network model and the SFC model in step 1 are as follows: For the satellite-ground network model, define an arrangement period P, which is divided into T time slots; the physical network is modeled as G=(V,E); wherein V = V E ∪ V C , V C represents a cloud computing center, V E represents a set of edge ground nodes and satellite nodes; define each SFC request as G q = (V q , E q ), wherein V q represents a set of VNFs, E q represents a virtual link; define as the fth VNF in the service request q, |v q | represents the total number of VNFs contained in the service request q; represents the requested computing resources, represents and the requested bandwidth resources of the virtual link between and.

3. The hybrid edge cloud oriented satellite-ground converged network (SFC) orchestration method of claim 1, wherein The establishment of the service cost model and the link reliability model in step 2 comprises: The total cost of each service is defined as R q : R q = R N + R B R N denotes the cost of computing resources, R B denotes the bandwidth resource cost of the virtual link mapping to the physical link; Define a spatial link reliability probability function as: denotes the distance between physical nodes n and m at time slot t, denotes the change in distance between two nodes at time slot t relative to the previous time point t-1, α+β=1 and 0≤α,β≤1.

4. The hybrid edge cloud oriented satellite-ground converged network (SFC) orchestration method of claim 1, wherein Step 4 comprises the following steps: Step 4-1: convert the SFC arrangement problem into a Markov decision process; at each time step t=(1,…,T), a physical node is selected to place a VNF, until all VNFs are embedded, and T represents the number of VNFs contained in the current SFC; the specific definition is as follows: State: the current state of the physical network and the state of the current SFC consists of, defined as Action: At time step t, the decision agent selects a physical server node and instantiates the VNF, the action a t is represented as a t = where represents the selection of node n from the candidate physical nodes with available resources exceeding c f,q ; otherwise, if a t = 0 indicates a deployment failure; Reward: define an immediate reward function in the intermediate process as: Where θ1 and θ2 are the resource consumption coefficient of the current VNF and the reliability coefficient of the current time step mapping link; -α1 represents the penalty for violating constraints; Define the reward function of the final state as: Where β0 and β1 represent the reward value of the SFC arrangement, and β0>β1; θ3 is a reliability constraint coefficient; Step 4-2: Explore the physical network node and link features at the current time step t using the graph attention network (GAT); the input of GAT is the feature vector set Z = {z1, z2, …, zN} of the physical node (z e R N F ), where N is the number of nodes, and F represents the number of physical node features; in the output layer of the physical network embedding, an N x F hidden matrix Z t is generated, where F hidden represents the dimension of the hidden features;​ Step 4-3: treat the SFC deployment problem as a sequence data processing problem, and use an LSTM-based sequence-to-sequence network model as a proxy for VNF node selection; the specific definition is as follows: The input of the encoder is the current SFC state The final hidden state h is output through the LSTM unit t The context vector c t The decoder outputs the value of each node and determines the node selection probability distribution using a fully connected layer based on the context vector c t , and the output vector Z of the GAT t , combined with the current policy of the agent, and processes to determine the node selection probability distribution; based on the probability distribution, the intelligent agent will perform the action a t。 5. The hybrid edge cloud oriented satellite-ground converged network (SFC) orchestration method of claim 1, wherein Step 5: according to the VNF deployment policy, perform link mapping by using a reliability-based Dijkstra algorithm, as follows: find a path in the physical network using the reliability-based Dijkstra algorithm t and a t-1 ; if there is a path that satisfies the condition, place the current VNF according to action a t and complete the link mapping; otherwise, the current SFC deployment fails, and the previously occupied physical resources will be released; the edge weight of the physical link is defined as follows: wherein represents a physical link E n,m the current residual bandwidth resources, I(·) is an indicator function, I(x) = 1 when x ≥ 0, otherwise I(x) = 0.

6. The hybrid edge cloud oriented satellite-ground converged network (SFC) orchestration method of claim 1, wherein Step 6: update the parameters of the policy network by using the PPO algorithm for parallel training; the loss function of the policy network is defined as follows: where, is the ratio of the probabilities of the new and old policies, π θ (a t |s t ) is the new policy, is the old policy; clip is a clipping function that controls the range of the change of the new and old policies to be within [1-ε, 1+ε]; represents taking the expectation over multiple samples; is the advantage function, which is calculated as follows δ t = r t + γV(s t+1 ) - V(s t ) where δ t denotes the TD error at time t; r t denotes the reward value after taking the action; γ denotes the reward discount factor; λ denotes a GAE hyperparameter that controls the weight of future rewards; and T denotes the total number of time steps. The parameters of the policy network and the evaluation network are updated using a gradient ascent strategy, and the network parameters θ are updated continuously n times update After that, the parameters θ old are updated to θ; the updating method is as follows: delta t = r t + gamma V(s t+1 , w t+1 ) - V(s t , w t ) where w t , θ t represent the parameters of the policy network and the value network at the current time, respectively; w t+1 , θ t+1 represent the updated network parameters; a w , a θ are the network learning rates; represents the gradient of w with respect to the loss function; represents the gradient of θ with respect to the loss function; the above process is repeated using the new policy parameters until the number of iterations is reached.

Images