Service function chain embedding optimization method for heterogeneous reliable perception in space-air-ground network

Abstract

The invention discloses a heterogeneous reliable perception service function chain (SFC) embedding optimization method in a space-air-ground integrated network (SAGIN), and the method comprises the steps: building an embedding optimization model in consideration of the use of computing resources, a memory and bandwidth; and solving the embedding optimization model by adopting a hybrid algorithm, calculating an optimal scheme of SFC embedding and backup in the SAGIN by utilizing the autonomous decision-making capability of deep reinforcement learning through a reliability-aware SFC embedding algorithm, and performing backup by utilizing a corresponding actuator in a data center so as to improve the service reliability. When the network load exceeds a predefined threshold, SFC re-embedding is performed using a re-embedding algorithm based on a greedy policy to further improve service reliability and improve the acceptance of the SFC. According to the invention, uncertain user requests can be met, and the reliability of service quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of space-air-ground integrated networks, and particularly to an optimization method for embedding heterogeneous reliable sensing service function chains in a space-air-ground network. Background Art

[0002] The continuous development of social informatization has promoted the rapid evolution of 5G and 6G. Compared with the previous generation of communication networks, 6G is expected to support more complex and diverse application scenarios, and future communication networks will show multi-dimensional interconnection. Due to the lack of a coordination mechanism between the ground and air networks and limited coverage at the current stage, the 6G network needs to break through the terrain and environmental limitations and build a space-air-ground integrated network (SAGIN). SAGIN is an integrated network architecture that integrates ground networks, space-based networks, and satellite networks with different orbits, types, and performances in space. It provides an ubiquitous, intelligent, coordinated, and efficient information guarantee infrastructure for various network services within a large range of space.

[0003] To provide reliable services in the space-air-ground integrated network SAGIN, an appropriate embedding scheme of the service function chain (SFC) is required to ensure service quality. However, the complex topology, limited infrastructure resources, and heterogeneous hardware and software in SAGIN currently make it difficult for existing reliable SFC embedding schemes to provide high service quality. Although SFC backup is a feasible solution, the selection of backup locations poses a great challenge to the three-dimensional SAGIN topology and the uncertainty of user requirements therein. The existing problems are as follows: 1. The devices in SAGIN are heterogeneous and distributed in different dimensions, resulting in a complex topology. The connection conditions and network states of devices at different levels are also different from those of the ground topology, affecting service reliability and latency. 2. Some resources of the SAGIN infrastructure are limited. Embedding too many virtual network function (VNF) backups in the same infrastructure will cause network services to be unavailable, leading to a sharp decline in service quality. 3. The dynamic and uncertain nature of user requirements in the network makes it difficult for SAGIN to continuously provide reliable services.

[0004] Therefore, to solve the above problems, an optimization method for embedding heterogeneous reliable sensing service function chains in a space-air-ground network is needed, which can meet uncertain user requests, improve the reliability of service quality, and increase the acceptance rate of SFC. Summary of the Invention

[0005] In view of this, the object of the present invention is to overcome the defects in the prior art, and provide an optimization method for embedding service function chains with heterogeneous reliable perception in the space-air-ground network, which can meet uncertain user requests, improve the reliability of service quality, and increase the acceptance rate of SFCs.

[0006] The optimization method for embedding service function chains with heterogeneous reliable perception in the space-air-ground network of the present invention includes:

[0007] Considering the use of computing resources, memory, and bandwidth, an embedding optimization model is constructed;

[0008] The embedding optimization model is solved to make the embedding optimization model obtain the optimal solution, and the optimal solution is used as the best solution for SFC embedding.

[0009] Furthermore, the embedding optimization model includes an objective function and constraint conditions;

[0010] The objective function is:

[0011]

[0012] Among them, the set of SFC requests is represented as Each SFC request Is formalized as s = {f1, f2, f3,..., f n}, f i Represents the i-th VNF in the SFC; And Are binary variables, respectively indicating whether f i Is placed on the physical node u and whether the virtual link Is mapped to the physical link l u,v On; and the variable Represents the hop count of the link , and the hop count between adjacent devices Since the request passes through more than one device and physical link, all links that the request needs to pass through need to provide the bandwidth required by the current request, and the bandwidth consumption is η u Represents the consumption cost of unit computing resources; Represents the computing resources required by f i ; υ u Represents the consumption cost of unit memory; Represents the memory resources required by f i ; Represents the input cost required by unit bandwidth; Represents the virtual link Required bandwidth; Represents the physical device; Indicates a physical link;

[0013] The constraint conditions include backup constraint, reliability constraint, resource constraint, and latency constraint.

[0014] Furthermore, the backup constraint is:

[0015]

[0016] The reliability constraint is:

[0017]

[0018] Among them, R s and respectively represent the reliability of s and f i 's reliability; Represents the lowest tolerable reliability;

[0019] The resource constraint is:

[0020]

[0021] Among them, Represents the total CPU resources; Represents the total memory resources; Represents the bandwidth of the link;

[0022] The latency constraint is:

[0023]

[0024] Among them, Represents the processing latency of device u; Represents the transmission latency of the link; D s Represents the maximum acceptable latency; Represents the latency of the connection between ground devices; Represents the latency of the connection between airborne devices; Represents the latency of the connection between space satellites.

[0025] Furthermore, solving the embedded optimization model specifically includes:

[0026] Regarding the embedding positions of all backups of VNF as an action using DDQN, and setting rewards related to the investment to minimize the investment during the training of DDQN; in DDQN, using triples to represent the state space Action space And the reward function

[0027] State space The middle state tensor includes the lowest tolerable reliability of the SFC, the reliability of relevant hardware, and the resources provided by physical nodes; the actions taken by the agent of DDQN are to select the VNF backup deployment location and whether the current VNF backup is the primary instance. This operation is divided into two parts. One is the VNF placement action, and the other is the action of selecting the primary instance after backup; the reward function is set by prompting the corresponding feedback provided by the environment after the agent executes the placement action.

[0028] Input the software and hardware parameters of the SFC as the initial state tensor received by the agent in DDQN; the initial state tensor includes the lowest tolerable reliability, network topology parameters, SFC parameters, and the cost of unit resources.

[0029] For each VNF in each SFC request, perform the following operations:

[0030] Select action a according to the ε-greedy rule; randomly initialize action a; execute action a and obtain the reward; learn from the previous experience and the randomly generated actions and rewards to obtain the replayed experience, and store the replayed experience in the replay set of DDQN; if the reliability constraint of the embedding optimization model is less than the lowest tolerable reliability, reject the SFC request, otherwise pass the SFC request.

[0031] Furthermore, when an SFC request arrives, obtain the current state of the network. If the network load trigger condition is satisfied, perform the re-embedding of the SFC according to the following method:

[0032] Block subsequent SFCs; regard all devices that satisfy the network load trigger condition as a set; for each device in the set, perform the following operations: normalize the total CPU and memory resources of the device; regard the total CPU and memory resources of the device as the total resources of the device; select μ device backups according to the top-k algorithm, and select the device with the largest resources as the primary instance; update the device resource amount.

[0033] Among them, the network load trigger condition is that the maximum usage of memory or computing resources in the network exceeds the maximum load specified by the network system; when the lowest tolerable reliability of the SFC is , the minimum number of backups μ is len is the shortest length of the SFC, v min represents the minimum reliability among the physical devices where VNFs are deployed; φ min represents the minimum f i reliability in the SFC.

[0034] The beneficial effects of the present invention are as follows: An optimization method for embedding service function chains with heterogeneous reliable perception in an air-space-ground network disclosed by the present invention constructs an embedding optimization model and uses two sub-algorithms, namely the RASE algorithm and the GSR algorithm, to solve the embedding optimization model. Among them, the RASE algorithm will utilize the autonomous decision-making ability of deep reinforcement learning to calculate the best solutions for SFC embedding and backup in the SAGIN, and corresponding actuators will perform backups in the data center to improve the reliability of services. When the network load exceeds a predefined threshold, the GSR algorithm is used to perform SFC re-embedding to further improve service reliability and increase the acceptance rate of SFCs. Description of the Drawings

[0035] The present invention will be further described below in conjunction with the drawings and embodiments:

[0036] Figure 1 It is a schematic diagram of the SAGIN topology for SFC embedding in an air-space-ground integrated network;

[0037] Figure 2 It is a schematic diagram of the hybrid algorithm framework of the RASE algorithm and the GSR algorithm. Specific Embodiments

[0038] The following further describes the present invention in conjunction with the drawings of the specification, as shown in the figure:

[0039] This embodiment discloses an optimization method for embedding service function chains with heterogeneous reliable perception in an air-space-ground network, including:

[0040] Considering the use of computing resources, memory, and bandwidth, construct an embedding optimization model;

[0041] Solve the embedding optimization model to make the embedding optimization model obtain an optimal solution, and use the optimal solution as the best solution for SFC embedding.

[0042] In this embodiment, Figure 1 The topology of the SAGIN is depicted, where the devices include three layers: the satellite layer, the air and ground layers. Each layer has various heterogeneous devices. For example, the ground layer includes base stations, vehicles, and mobile phones. Drones may exist in the air, and the satellite layer involves satellites in various orbits. This topology can be represented by an undirected graph:

[0043] denoted as, where represents the set of physical devices, represents the links between these devices. Due to the construction objectives and characteristics of the SAGIN, physical links may exist between and within layers. Therefore, let Here represents the links between satellite devices in different orbits, represents the link between air devices (such as drones, etc.), while is the link between a ground device and an orbital satellite device.

[0044] The parameter includes various resource parameters and attribute parameters of the SAGIN topology. Among them, is the computing resource amount of a network node in SAGIN, is the memory size of the node, represents the processing delay of the node for a task, is the bandwidth size of link l u,v . is the transmission delay of l u,v . Compared with the transmission delay in SAGIN, the propagation delay and queuing delay of SFC can be ignored, so they are ignored in the subsequent calculations of the present invention. Finally, v u represents the reliability of network device u, that is, the probability that the device can operate stably without downtime. Therefore, 1 - v u can represent the probability that the device fails to provide services or terminates.

[0045] The SFC request is a series of request flows sent by the user, and this request flow passes through each required VNF in turn. The set of SFC requests containing k SFCs is represented as where each SFC can be formalized as s = {f1, f2, f3,..., f n}, where f i represents the i-th VNF in the SFC.

[0046] Let where and represent the computing resources and memory required in f i in the VNF respectively. represents the link bandwidth required between VNF f i and f i+1 . is the reliability of VNF f i , then represents the probability that the current VNF fails and stops providing services. Finally, D s represents the maximum delay that the current SFC can tolerate.

[0047] Deploying SFC requires consuming various resources, including computing resources, memory resources, and bandwidth. When Internet service providers consider deploying SFC on their physical devices, one of the key factors they generally focus on is the consumption cost required for embedding. Considering the use of computing resources, memory, and bandwidth, the problem of reliable SFC embedding and re-embedding (HRSER) is modeled, and an embedding optimization model is constructed.

[0048] The embedding optimization model includes an objective function and constraint conditions;

[0049] The objective function is:

[0050]

[0051] Among them, the set of SFC requests is represented as Each SFC request is formalized as s = {f1, f2, f3, …, f n}, f i represents the i-th VNF in the SFC; and are binary variables, representing whether f i is placed on the physical node u and whether the virtual link is mapped to the physical link l u,v respectively; and the variable represents the hop count of the link , and the hop count between adjacent devices Since the request passes through more than one device and physical link, all the links that the request passes through need to provide the bandwidth required by the current request, and the bandwidth consumption is η u represents the consumption cost of unit computing resources; represents the computing resources required by f i ; υ u represents the consumption cost of unit memory; represents the memory resources required by f i ; ζ (u,v) represents the input cost required for unit bandwidth; represents the virtual link required bandwidth; represents physical devices; represents physical links.

[0052] The constraint conditions include backup constraints, reliability constraints, resource constraints, and delay constraints;

[0053] 1. Each VNF deploys at least one backup on physical devices to ensure the availability of SFC, and the model adopts distributed backup, so that there can only be one backup of the same type of VNF on each physical device. Then there are the following backup constraints:

[0054]

[0055] 2. Assume that each SFC has a minimum tolerable reliability threshold, and the model aims to ensure that the overall reliability of the SFC exceeds this minimum threshold. It is known that the reliability of the SFC is determined by the reliability of each VNF that composes it and the device reliability. Then the reliability constraint is:

[0056]

[0057] Among them, R s and respectively represent the reliability of s and f i ; represents the lowest tolerable reliability;

[0058] In addition, the reliability of each VNF depends on whether at least one of all its backups is available, and the availability of the backup means that both software and hardware are available, that is, the probabilities of software reliability and hardware reliability need to be met.

[0059] Therefore, the reliability definitions of VNF and backup can be expressed by the following equations:

[0060]

[0061] The above two expressions respectively represent whether VNF f i is available and whether the backup on device u is available. The former is composed of the probabilities of all its backups, and the latter is composed of the reliability of the VNF itself and the reliability of the device.

[0062] 3. In SAGIN, the bandwidth, memory, and computing resources of devices are limited. Therefore, the resource constraints are:

[0063]

[0064]

[0065] Among them, represents the total CPU resources; represents the total memory resources; represents the bandwidth of the link;

[0066] 4. An important point in the quality of service of SFC is the service response delay. Therefore, the sum of the processing delay and the transmission delay of the SFC is less than the maximum acceptable delay D s; The delay constraint is as follows:

[0067]

[0068] Among them, represents the processing delay of device u; represents the transmission delay of the link; D s represents the maximum acceptable delay; The complex topology of SAGIN results in different physical connection delays for different layers. The maximum delay in the connection between space satellites is the highest, followed by that in the air, and the device delay on the ground is considered the lowest, and the minimum delay is the same. represents the delay in the connection between ground devices; represents the delay in the connection between air devices; represents the delay in the connection between space satellites.

[0069] In this embodiment, based on the heterogeneous reliable sensing SFC embedding and re-embedding problem in SAGIN, that is, the HRSER problem, the present invention formulates the HRSER problem as a non-linear integer programming, and uses the Reliable-aware SFC Embedding (RASE) algorithm and the Greedy SFC Re-embedding (GSR) algorithm based on the greedy strategy to solve the embedding optimization model.

[0070] Such as Figure 2 The hybrid algorithm framework shown. There is a monitor in the server, which is used to detect whether the network load exceeds the threshold and determine whether re-embedding is required when an SFC request enters. If the network load exceeds the threshold, the SFC that reaches the device bottleneck in the network will be re-embedded. Otherwise, it will be directly embedded through the RASE algorithm.

[0071] In the RASE algorithm, multiple backups can be arbitrarily selected as the primary instance of the service. However, considering the consumption generated by switching an instance to another backup after a failure, and due to the low delay between the device and the ground server and the large scale of server resources, device nodes such as ground servers have priority when being selected as the primary instance. Subsequently, in the GSR algorithm, the lower limit of the number of backups that meet the constraints is proved, so that the minimum number of backups can be selected for greedy strategy-based re-embedding.

[0072] The core ideas of the RASE algorithm and the GSR algorithm are to use Deep Reinforcement Learning (DRL) and greedy strategies to solve problems and obtain SFC embedding and re-embedding solutions in SAGIN. The two algorithms are relatively independent. Only the monitor detects the network state at the initial stage to decide which solution to use. The two algorithms use the same network state parameters and the same data, and are just two solutions selected during deployment.

[0073] Performing SFC embedding in SAGIN not only needs to consider the heterogeneous parameter changes between different physical devices, but also needs to consider the spatial distribution of these devices. Therefore, a method based on DRL and greedy strategies is used to effectively solve this problem by setting an effective reward function and designing a reasonable action space and state space.

[0074] Designing a good DRL agent can capture the features and complex network topologies concerned by the problem, and at the same time reduce the trial-and-error costs caused by frequent constraint violations. The Double Deep Q-learning (DDQN) is a solution for DRL to handle discrete action spaces, which learns from previous experiences and randomly generated actions and rewards through an experience replay buffer.

[0075] In the HRSER problem, DDQN can regard the embedding positions of all backups of VNF as an action, and at the same time minimize the cost during training by setting a reward related to the cost. In DDQN, first, a triple is used to represent the state space, action space, and reward function. Next, these three elements are described separately.

[0076] (1) State space. To correctly represent the system state, the state tensor should include the parameters of SFC and physical devices in SAGIN. Considering the reliability of SFC, the state tensor includes the lowest tolerable reliability of SFC the relevant hardware reliability, and the resources provided by physical nodes. Therefore, it is represented as follows:

[0077]

[0078] After calculation, the length of the state tensor This length will affect the computational complexity of DDQN, which will be analyzed later.

[0079] (2) Action Space. The actions taken by the agent of DDQN are to select the backup deployment location of the VNF and whether the current VNF backup is the primary instance. This operation needs to be divided into two parts, one is the VNF placement action, and the other is the action of selecting the primary instance after backup. The action space is defined as follows:

[0080]

[0081] Similarly, the length of the action tensor can be obtained.

[0082] (3) Reward Function. The reward obtained by the agent from the environment can encourage it to make more accurate decisions. Minimize the input consumption while complying with the constraints. The reward is the corresponding feedback provided by the environment after the agent executes the placement action. Therefore, the reward is expressed as the negative of the input, and the reward function is:

[0083]

[0084] Among them, α≥0, β≥0 and both are constants. The delay between two devices containing VNF backups is added to the reward function to ensure that the selected backup is closer to the ground. If a complete SFC contains different partial devices of the three-layer network, the agent can make decisions with the help of the delay balance term and select a backup scheme with lower delay. In other words, due to the deployment of multiple backups, if the SFC violates the delay constraint due to passing through a VNF instance with high delay, it will tend to select a backup with low delay, so as to ensure the minimization of end-to-end delay. Through training, the delay between the selected backups will be lower, and the agent will be more inclined to select operations with lower input and finally achieve convergence.

[0085] In Algorithm 1, the software and hardware parameters of the input SFC are used as the initial state tensor received by the agent in DDQN. Lines 2 - 16 of Algorithm 1 execute the training steps of DDQN, where θ represents the parameters of the neural network. The best action generated after the ε-greedy strategy needs to be screened through Lines 6 - 8, and the initially selected primary instance is random. Finally, if the obtained backup does not meet the constraint conditions, the corresponding SFC is rejected.

[0086] Algorithm 1. RASE Algorithm:

[0087] Input: The minimum tolerable reliability of the SFC

[0088] Network topology parameters

[0089] SFC parameters The cost v of unit resource u , η u , ζ l ;

[0090] Output: Embedding action

[0091]

[0092]

[0093] Return: a

[0094] To ensure the reliability of the SFC, a large number of backups are usually deployed into the network topology in the early stage. However, this approach will cause an overloaded network. When the network is overloaded and the device resource utilization rate reaches the threshold Ψ, the overall performance of the network and the SFC acceptance rate will be greatly affected, thus affecting the service experience and service quality, where Ψ is the maximum load specified by the network system. When the network load trigger condition is met, SFC re-embedding will be considered. The trigger condition is defined as:

[0095]

[0096] The load of a single device is a reflection of the load of the entire network. If the maximum usage of memory or computing resources in a network exceeds the threshold, it means that this network is about to enter a heavily loaded state. Because the load of this node is too large, a series of subsequent requests may be rejected or the service may become unavailable due to device downtime. Thus, the trigger condition expresses a trend and signal of network performance degradation. Therefore, when the trigger condition is met, subsequent SFCs will be blocked, and the GSR algorithm that triggers the greedy strategy for SFC re-embedding needs to be used.

[0097] When an SFC request arrives, the monitor first obtains the current state of the entire network. The monitor determines whether the trigger condition is met according to the state of the network. Since the monitor will block the flow at this time, it needs to solve the re-embedding problem as soon as possible.

[0098] Since the algorithm time complexity of the greedy algorithm is faster than that of the RASE algorithm for inference, the monitor should use the faster greedy algorithm instead of letting the trained agent perform re-embedding to minimize the re-embedding time and reduce the overall embedding consumption.

[0099] The greedy strategy GSR is shown in Algorithm 2. Since re-embedding is triggered when the network load reaches the bottleneck, the number of backups should be minimized after re-embedding to avoid the situation of being embedded in the same location due to using the agent trained by RASE. Let μ be the minimum number of backups, and the minimum number of backups μ is len is the shortest length of the SFC, v min represents the minimum reliability among the physical devices where VNFs are deployed; φ minIt represents the smallest f in the SFC i Reliability

[0100] The greedy strategy GSR needs to aggregate the total normalized CPU and memory resources of the same device. The monitor selects the first μ devices for backup and selects the device with the largest resources as the primary instance. The load status of the devices in SAGIN is first obtained by the monitor when the SFC request flow arrives. The resource utilization rate is calculated in Algorithm 2 and checked whether it exceeds the threshold. If the threshold is reached, the subsequent SFC is blocked and re-embedding is performed according to lines 5 - 13. The eligible devices are put into a set and then backup selection is carried out

[0101] Algorithm 2. GSR algorithm

[0102] Input: SFC s

[0103] Output: Embedding action

[0104]

[0105]

[0106] Return: a

[0107] The present invention deeply studies the problem of heterogeneous reliable SFC embedding and re-embedding in SAGIN. Existing technologies cannot be well applied to the HRSER problem because they do not consider the environmental characteristics of SAGIN. The complex network topology and device heterogeneity lead to more comprehensive issues to be considered when embedding SFC. The dynamics and scarcity of resources may require frequent adjustments after SFC embedding. To address these challenges, the present invention proposes a hybrid algorithm that combines DDQN and greedy strategy to improve the acceptance rate of SFC

[0108] In view of the heterogeneity of SAGIN, the present invention proposes a RASE algorithm based on DDQN. This algorithm enhances the reliability of SFC by considering backups. In addition, the RASE algorithm uses DDQN to select the primary instance of each VNF. To cope with the high network load in the later stage, a greedy GSR algorithm is further proposed for re-embedding, effectively improving the acceptance rate of SFC. In addition, the lower bound of the number of SFC backups for re-embedding is given. Through simulation experiments on real datasets, the RASE algorithm and GSR algorithm proposed by the present invention show robustness for different numbers of neurons and learning rates. Compared with existing excellent algorithms, the SFC acceptance rate of the hybrid algorithm adopted by the present invention is close to 100%, and the investment is lower, only half of that required by algorithms with a similar SFC acceptance rate

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for embedding and optimizing service function chains with heterogeneous reliable perception in air-ground-space networks, characterized in that: Consider the use of computing resources, memory, and bandwidth to build an embedding optimization model; The embedding optimization model is solved so that the embedding optimization model obtains an optimal solution, and the optimal solution is used as the best solution for SFC embedding.

2. The method for embedding and optimizing service function chains with heterogeneous reliable perception in air-ground-space networks according to claim 1 is characterized in that: The embedded optimization model includes an objective function and constraints; The objective function is: Among them, the set of SFC requests is expressed as Each SFC request Formally, it can be expressed as s = {f1,f2,f3,…,f n }, f i represents the i-th VNF in the SFC; and are binary variables, representing f i Whether to place it on the physical node u and the virtual link Whether to map to physical link l u,v On; while the variable Indicates link The number of hops between adjacent devices Since the request does not just go through one device and physical link, all links that the request goes through need to provide the bandwidth required by the current request. The bandwidth consumption is η u Indicates the consumption cost of unit computing resources; represents f i Required computing resources; u Indicates the unit memory consumption cost; represents f i Required memory resources; Indicates the investment cost required for unit bandwidth; Indicates a virtual link Required bandwidth; Represents a physical device; Indicates a physical link; The constraints include backup constraints, reliability constraints, resource constraints and delay constraints.

3. The method for embedding and optimizing service function chains with heterogeneous reliable perception in air-ground-space networks according to claim 2 is characterized in that: The backup constraints are: The reliability constraints are: Among them, R s and Represent the reliability of s and f respectively i Reliability; represents the minimum tolerable reliability; The resource constraints are: in, Indicates the total number of CPU resources; Indicates the total amount of memory resources; Indicates the bandwidth of the link; The delay constraints are: in, Indicates the processing delay of device u; Indicates the transmission delay of the link; D s Indicates the maximum acceptable delay; Indicates the delay in the connection between ground equipment; Indicates the latency of the connection between devices in the air; Represents the latency of connections between satellites in space.

4. The method for embedding and optimizing service function chains with heterogeneous reliable perception in air-ground-space networks according to claim 1, characterized in that: Solving the embedding optimization model specifically includes: DDQN is used to treat the embedding positions of all backups of VNF as an action, and the input-related rewards are set to minimize the input when training DDQN. In DDQN, triples are used to represent the state space. Action Space And the reward function State Space The state tensor in the middle includes the minimum tolerable reliability of SFC, the reliability of related hardware, and the resources provided by the physical node; the action taken by the DDQN agent is to select the VNF backup deployment location and whether the current VNF ​​backup is the primary instance. This operation is divided into two parts, one is the VNF placement action, and the other is the action of selecting the primary instance after the backup; the reward function is set by prompting the agent to provide corresponding feedback from the environment after performing the placement action; Input the hardware and software parameters of the SFC as the initial state tensor accepted by the agent in the DDQN; the initial state tensor includes the minimum tolerable reliability, network topology parameters, SFC parameters, and the cost per unit resource; For each VNF in each SFC request, perform the following operations: Select action a according to the ε-greedy rule; randomly initialize action a; execute action a and obtain reward; learn from previous experience and randomly generated actions and rewards to obtain replayed experience, and store the replayed experience in the replay set of DDQN; if the reliability constraint of the embedded optimization model is less than the minimum tolerable reliability, reject the SFC request, otherwise pass the SFC request.

5. The method for embedding and optimizing service function chains with heterogeneous reliable perception in air-ground-space networks according to claim 4 is characterized in that: When an SFC request arrives, the current state of the network is obtained. If the network load trigger condition is met, the SFC is re-embedded as follows: Block subsequent SFCs; treat all devices that meet the network load triggering conditions as a set; for each device in the set, perform the following operations: normalize the total number of CPU and memory resources of the device; use the total number of CPU and memory resources of the device as the total resources of the device; select μ devices for backup according to the top-k algorithm, and select the device with the largest resources as the primary instance; update the device resource amount; The network load trigger condition is that the maximum usage of memory or computing resources in the network exceeds the maximum load specified by the network system; when the minimum tolerable reliability of SFC is When len is the shortest length of SFC, ν min represents the minimum reliability of the physical device where the VNF is deployed; φ min It means the smallest f in SFC i reliability.

Images