High-reliability service function chaining deployment method based on double deep Q network

Abstract

The application discloses a high-reliability service function chain deployment method based on a double deep Q network, and comprises the following steps: step 1, establishing a satellite network model and an SFC model; step 2, establishing an optimization problem model; step 3, modeling the SFC deployment problem as a Markov decision process (MDP), designing a state space, including the topology structure of the satellite network and the available computing resource capacity of nodes; designing an action space, including the selected primary node and backup node for a virtual network function (VNF); and a reward function comprehensively considering the end-to-end delay of a primary path and the SFC reliability, then using a high-reliability SFC deployment algorithm based on DDQN, and sequentially selecting a primary satellite node and a backup satellite node for each VNF through a two-stage deployment strategy, i.e., a primary path stage and a backup path stage, to maximize the goal of coping with the challenge of deploying a high-reliability service function chain in a satellite network with limited resources and dynamic topology changes.

Description

Technical Field

[0001] To address the challenges of deploying highly reliable service function chains (SFCs) in resource-constrained and dynamically changing satellite networks, this invention models the SFC deployment problem as a Markov decision process (MDP) and proposes a highly reliable service function chain deployment algorithm HRSFCD (High Reliability Service Function Chains Deployment) based on a dual deep Q network (DDQN). Background Technology

[0002] With the rapid development of the internet, satellites are undertaking computing tasks in addition to traffic forwarding. Due to the high launch and operation costs of satellites, it has become a trend to carry multiple computing tasks on a single satellite. Meanwhile, high-reliability, low-latency communication, as one of the application scenarios of 5G, places extremely high demands on the performance of satellite networks. However, the dynamic nature of satellite network topology and node failures can easily lead to service interruptions. In traditional network architectures based on dedicated hardware, providing backups for satellite nodes is a feasible way to improve service reliability and ensure normal service operation. However, more backups lead to greater resource consumption.

[0003] Network Functions Virtualization (SFC) technology decouples network functions from dedicated hardware, enabling satellites to undertake multiple computing tasks and significantly improving resource utilization. Therefore, satellite-based network services not only face the risk of hardware failure but also the potential problems caused by software failures. Thus, the key to solving the problem lies in how to comprehensively consider hardware and software reliability in resource-constrained and dynamically topological satellite networks, designing solutions to improve SFC reliability, and proposing deployment strategies to ensure high SFC reliability.

[0004] When deploying SFCs in satellite networks, the probability of multiple failure events occurring simultaneously is extremely low. This invention focuses on single-point failure scenarios, where at most one hardware or software failure event occurs. By optimizing deployment strategies, an effective balance is achieved between end-to-end latency and SFC reliability. Summary of the Invention

[0005] To address the technical problems mentioned in the background section, this invention employs a highly reliable service function chain (SFC) deployment method based on a dual-depth Q-network. This method includes the following steps: Step 1, constructing a satellite network model, including satellite nodes, node computing resource capacity, node hardware failure probability, physical links, and the shortest propagation delay between nodes. Step 2, constructing a service function chain (SFC) model, including SFC requests, virtual network nodes (VNFs), the maximum tolerable end-to-end delay, VNF computing resource requirements, and VNF software failure probability. Step 3, constructing an optimization problem model based on the satellite network model and the service function chain model, including the main path end-to-end delay model, the SFC reliability model, and constraints.

[0006] Furthermore, the specific implementation process of constructing the satellite network model in step 1 above is as follows;

[0007] A satellite network consists of several satellite nodes and links between them, represented by a connected graph G(V,E). Each satellite node possesses certain computing resources and has a certain probability of failure. Where V = {v1, v2, ..., v...} k ,...v |V|} represents all satellite nodes v k The set of |V|, where |V| represents the number of satellite nodes in the network. This represents the set of available computing resource capacities for all satellite nodes in time slot t, where Indicates v k Available computing resource capacity in time slot t. Ph = {Ph1, Ph2, ..., Ph} |V|} represents the hardware failure probability of all satellite nodes. E represents the set of physical links between satellite nodes. This represents the matrix representing the shortest propagation delay between satellite nodes in time slot t.

[0008] The specific implementation process of building the service function chain model in step 2 above is as follows;

[0009] Define SR = {SR1, SR2, ..., SR} |Sr|} represents the set of SFCs requests, where |Sr| represents the number of SFCs. (Using tuples) <VNF i D i Cd i Ps i > Indicates a request for SR i ∈SR. Indicates the formation of a request SR i The set of VNFs, where f i,j Let D represent the j-th VNF. i Indicates SR iMaximum tolerable end-to-end delay. This represents the computational resource requirements of each VNF. This represents the probability of software failure for each VNF.

[0010] The specific implementation process of constructing the optimization problem model in step 3 above is as follows;

[0011] Satellite network applications such as real-time communication are sensitive to latency and reliability. High latency will lead to a decline in service quality, and low reliability will increase the risk of service interruption. Therefore, end-to-end latency and reliability are used as indicators for optimization problems.

[0012] First, establish the end-to-end latency model for the main path.

[0013] The main path is the path consisting of the main satellite nodes of all VNFs and the links between them. Satellite nodes are selected sequentially for each VNF according to its sequence number. The first VNF ​​deployed is called the starting VNF. This indicates that there are one or more endpoint VNFs. Based on this, the SFC request SR is... i Main path end-to-end delay Dp i Defined as the maximum value of the shortest delay between the starting VNF and the ending VNFs. The formula is as follows:

[0014]

[0015] Where k1 and k2 represent satellite nodes, f i,j Let represent the j-th VNF in the i-th SFC request, and C represent the set of endpoint VNFs. It is a binary variable. This indicates that the VNF software Deployed on the main hardware node superior.

[0016] Secondly, establish an SFC reliability model.

[0017] Put SR i The reliability is defined as SR i The probability of no failure, using Indicated. Because it is SR i All VNF software selects one primary hardware node and one backup hardware node respectively, so SR i reliability Reliability of the main path and the reliability of backup paths related.

[0018] (a) Calculation;

[0019] Reliability of the main path This refers to the probability that all VNF software and the host hardware node of all VNF software are functioning correctly. (SR) i The probability that all VNF software works properly is expressed in Rs. i The calculation formula is as follows:

[0020]

[0021] Where Ps i,j Indicates VNF software f i,j The probability of software failure, then 1-Ps i,j f i,j Software reliability. SR i The probability that all the main hardware nodes hosting all VNF software are functioning correctly is expressed by Rh. i The calculation formula is as follows:

[0022]

[0023] Ph k Represents node v k The probability of hardware failure, then 1-Ph k Indicates v k Hardware reliability. This indicates that when different VNFs are deployed to the same node v k At that time, v k The hardware reliability only participates in one computation. Therefore, the main path reliability... The calculation method is as follows:

[0024]

[0025] (b) Calculation;

[0026] VNF software f i,j Deployed on satellite node v k The above is referred to as a VNF instance. If v k It is f i,j The primary node, when f i,j Software malfunction or v k Hardware failure caused instance If it fails to function properly, then the primary node v should take over. k The processed data stream will be forwarded to f i,j Processing is performed on the backup node. At this time, it is handled by f. i,j The backup nodes, the master nodes of other VNF software, and the links between the nodes constitute the SR. i A backup path, which will be used because of the instance Backup path used in case of failure This indicates that, since a backup node was selected for each VNF software, under the single point of failure scenario mentioned in this article, SR... i There is 1 main path and |VNF i | Backup path. SR i Backup path reliability Refers to SR i The sum of the reliability of all backup paths.

[0027] For backup path There are two reasons why it works. First, examples. VNF software f i,j A failure occurred, but the primary hardware node v k Normal. In this case, the path The hardware nodes involved in the operation can be divided into two categories: software and hardware. i,j The backup hardware node and the primary hardware node where other VNF software resides are considered. All hardware nodes are traversed, and if a node belongs to at least one of the two categories mentioned above, its hardware reliability is included in the calculation. Therefore, the backup path... The probability that all the hardware on it works properly is expressed as in Used to determine whether it belongs to the main hardware node where other VNF software resides. It is a binary variable. f i,j It is deployed to the backup node in time slot t. Above, in this formula, is used to determine whether it is software f. i,j Backup hardware nodes. SR i All VNF software works normally, and the probability is expressed as in equation (2). Therefore, in this case, the backup path The reliability is expressed as:

[0028]

[0029] in Indicates the backup path The probability that all the hardware on the device is working properly, Rs i Indicates the backup path The probability that all VNF software works normally, Ps i,j ·(1-Ph k ) indicates VNF software f i,j A failure occurred, but the primary hardware node v k Normal probability.

[0030] Second, examples VNF software fi,j Normal, but the main hardware node v k Fault. In this case, the path The hardware nodes involved in the operation can be divided into two categories: master nodes and virtual nodes. k The backup hardware node for the VNF software, and the primary hardware node where other VNF software resides. Because v k The error occurred, so it iterated through all elements except v. k All hardware nodes other than those listed above, if they belong to at least one of the two node categories mentioned above, will have their hardware reliability included in the calculation. Therefore, the backup path... The probability that all the hardware on it works properly is expressed as in Used to determine whether it belongs to the main hardware node where other VNF software resides. The value used to determine whether something belongs to the master node is v. k Backup hardware nodes for VNF software. SR i All VNF software works normally, and the probability is expressed as in equation (2). Therefore, the backup path in this case is... Reliability The calculation formula is as follows:

[0031]

[0032] in Indicates the backup path The probability that all the hardware on the device is working properly, Rs i Indicates the backup path The probability that all VNF software works normally, (1-Ps) i,j Ph k Indicates VNF software f i,j Normal, but the main hardware node v k The probability of failure.

[0033] Therefore, backup path reliability Represented as:

[0034]

[0035] in, Indicates VNF software f i,j A failure occurred, but the primary hardware node v k The reliability of the backup path under normal circumstances Indicates VNF software f i,j Normal, but the main hardware node v k The reliability of the backup path in the event of a failure.

[0036] Furthermore, SR i Reliability of all backup paths Represented as:

[0037]

[0038] Finally, SR i reliability The calculation formula is:

[0039]

[0040] in Indicates SR i The reliability of the main path, Indicates SR i The reliability of all backup paths.

[0041] Then, the deployment of the service function chain is subject to some limitations.

[0042] (a) The computing resources of satellite nodes cannot be over-allocated by VNFs.

[0043]

[0044] (b) Each VNF may select only one primary satellite node and one backup satellite node.

[0045]

[0046] (c) For any f i,j ∈VNF i f is not allowed i,j The primary node and backup node are the same.

[0047]

[0048] (d) The end-to-end delay of the SFC must not exceed its delay constraint.

[0049]

[0050] Finally, an optimization problem model is established.

[0051] To optimize the end-to-end latency and reliability of the main path in SFC, the optimization objective is defined as:

[0052] obj:ω·Dp i +(1-ω)·R i (15)

[0053] Where ω∈[0,1] represents the weight coefficient, Dp i and R iThese represent the normalized end-to-end delay Dp, respectively. i and normalized SFC reliability

[0054] We need to maximize the value of obj:

[0055] maxobj (16)

[0056] st:(11)-(15).

[0057] To address the problem (16), the SFC deployment problem is modeled as an MDP, and an algorithm based on DDQN is proposed to solve the problem.

[0058] First, establish the MDP model.

[0059] The SFC deployment problem is a sequential decision problem. Therefore, it can be modeled as a Markov decision process, represented as a tuple (S, A, R), where S represents the state of the environment, A represents the agent's action, and R represents the reward.

[0060] (a) State: State S t The topology of the satellite network TS t And calculations of all satellites

[0061] Resource capacity Cr t Composition. Represented as:

[0062] S t =(TS) t ,Cr t (17)

[0063] (b) Actions: The agent's deployment of SFC requests is divided into two phases. In the first phase, a primary satellite node is selected for each VNF. In the second phase, a backup satellite node is selected for each VNF. In both phases, the agent's action space is the set V of all satellite nodes.

[0064] (c) Rewards: Rewards are divided into phased rewards and overall rewards. For phased rewards, each time the agent selects a satellite node for a VNF, it uses that VNF ​​as the endpoint VNF, calculates the end-to-end latency and reliability using equations (1) and (9) respectively, and calculates the reward value using equation (15). For overall rewards, when the agent has completed the selection of master and backup nodes for all VNFs, the sum of the rewards of all previous actions is used as the overall reward. In addition, if the agent's action violates the constraints, a penalty value P = -50 is given as the reward value for the action.

[0065] Then, the proposed HRSFCD algorithm is presented.

[0066] In the action space, the allocation decision of satellite nodes is a discrete action. Therefore, this paper adopts the DDQN algorithm, which handles discrete actions, as the basic algorithm. The agent interacts with the satellite network environment to train the Q network. During the training process, the deployment of an SFC is divided into two phases. In the first phase, the Q network selects the primary satellite node for each VNF in turn. In the second phase, the Q network selects the backup satellite node for each VNF in turn.

[0067] Compared with existing technologies, this invention comprehensively considers software reliability and hardware reliability in a dynamic network environment, and uses the same Q network to complete the deployment of master nodes and backup nodes respectively. Attached Figure Description

[0068] Figure 1 This is an example image from SFC.

[0069] Figure 2 This is a structural diagram of the SFC reliability components.

[0070] Figure 3 This is a diagram of the HRSFCD algorithm framework. Detailed Implementation

[0071] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0072] Satellite network model

[0073] Satellite networks are represented by a connected graph G(V,E), where V={v1,v2,...v... |V|} represents the set of all satellite nodes, and |V| represents the number of satellite nodes. This represents the set of available computing resources for a satellite in time slot t, where Indicates v k Available computing resource capacity in time slot t. Ph = {Ph1, Ph2, ..., Ph} |V|} represents the hardware failure probability of all satellite nodes. E represents the set of physical links between satellite nodes. This represents the matrix indicating the shortest propagation delay between satellite nodes in time slot t. An example of an SFC is shown below. Figure 1 As shown.

[0074] Service Function Chain Model

[0075] Define SR = {SR1, SR2, ..., SR} |Sr|} represents the set of SFCs requests, where |Sr| represents the number of SFCs. (Using tuples) <VNF i D i Cd i Ps i > Indicates a request for SRi ∈SR. Indicates the formation of a request SR i The set of VNFs, where f i,j Let D represent the j-th VNF. i Indicates SR i The maximum tolerable end-to-end latency. This represents the computational resource requirements of each VNF. This represents the probability of software failure for each VNF.

[0076] (1) Optimization problem model

[0077] Satellite network applications such as real-time communication are sensitive to latency and reliability. High latency will lead to a decline in service quality, and low reliability will increase the risk of service interruption. Therefore, end-to-end latency and reliability are used as indicators for optimization problems.

[0078] First, establish the end-to-end latency model for the main path.

[0079] In real-world scenarios, the probability of system failure is not high, so the probability of using backup nodes is also low. Therefore, only the primary path is considered, which is the path consisting of the primary satellite nodes of all VNFs and the links between them. Satellite nodes are selected for each VNF sequentially according to its sequence number. The first VNF ​​deployed is called the starting VNF. This indicates that there are one or more endpoint VNFs. Based on this, the SFC request SR is... i Main path end-to-end delay Dp i Defined as the maximum value of the shortest delay between the starting VNF and the ending VNFs. The formula is as follows:

[0080]

[0081] Where C represents the set of endpoint VNFs. It is a binary variable. This indicates that the VNF software Deployed on the main hardware node superior.

[0082] Secondly, establish an SFC reliability model.

[0083] Put SR i The reliability is defined as SR i The probability of no failure, using Indicated. Because it is SR i All VNF software selects one primary hardware node and one backup hardware node respectively, so SR i reliability Reliability of the main path and the reliability of backup paths Relevant. Since using lasers as inter-satellite links has become a trend in inter-satellite communication, and lasers have high directionality and short wavelengths in space, satellite laser communication can achieve high speed, wide bandwidth, high precision, and high security without electromagnetic spectrum limitations. Therefore, only the reliability of the satellite nodes needs to be considered. For example... Figure 2 As shown, the reliability of SFC consists of the reliability of the main path, the reliability of the backup path under single-point failure, and the reliability of the backup path under multi-point failure. Multi-point failure refers to the simultaneous failure of two or more VNF software or hardware nodes.

[0084] (a) Calculation

[0085] Reliability of the main path This refers to the probability that all VNF software and the host hardware node of all VNF software are functioning correctly. (SR) i The probability that all VNF software works properly is expressed in Rs. i The calculation formula is as follows:

[0086]

[0087] Where 1-Ps i,j Indicates VNF software f i,j Software reliability. SR i The probability that all the main hardware nodes hosting all VNF software are functioning correctly is expressed by Rh. i The calculation formula is as follows:

[0088]

[0089] Among them 1-Ph k Represents node v k Hardware reliability. This indicates that when different VNFs are deployed to the same node v k At that time, v k The hardware reliability only participates in one computation. Therefore, the main path reliability... The calculation method is as follows:

[0090]

[0091] (b) Calculation

[0092] VNF software f i,j Deployed on satellite node v k The above is referred to as a VNF instance. If v k It is f i,j The primary node, when f i,j Software malfunction or v k Hardware failure caused instance If it fails to function properly, then the primary node v should take over. k The processed data stream will be forwarded to f i,j Processing is performed on the backup node. At this time, it is handled by f. i,j The backup nodes, the master nodes of other VNF software, and the links between the nodes constitute the SR. i A backup path, which will be used because of the instance Backup path used in case of failure This indicates that, since a backup node was selected for each VNF software, under the single point of failure scenario mentioned in this article, SR... i There is 1 main path and |VNF i | Backup path. SR i Backup path reliability Refers to SR i The sum of the reliability of all backup paths.

[0093] For backup path There are two reasons why it works. First, examples. VNF software f i,j A failure occurred, but the primary hardware node v k Normal. In this case, the path The hardware nodes involved in the operation can be divided into two categories: software and hardware. i,j The backup hardware node and the primary hardware node where other VNF software resides are considered. All hardware nodes are traversed, and if a node belongs to at least one of the two categories mentioned above, its hardware reliability is included in the calculation. Therefore, the backup path... The probability that all the hardware on it works properly is expressed as in Used to determine whether it belongs to the main hardware node where other VNF software resides. It is a binary variable. f i,j It is deployed to the backup node in time slot t. Above, in this formula, is used to determine whether it is software f. i,j Backup hardware nodes. SR i All VNF software works normally, and the probability is expressed as in equation (2). Therefore, in this case, the backup path The reliability is expressed as:

[0094]

[0095] Second, examples VNF software f i,j Normal, but the main hardware node v k Fault. In this case, the path The hardware nodes involved in the operation can be divided into two categories: master nodes and virtual nodes. k The backup hardware node for the VNF software, and the primary hardware node where other VNF software resides. Because v k The error occurred, so it iterated through all elements except v. k All hardware nodes other than those listed above, if they belong to at least one of the two node categories mentioned above, will have their hardware reliability included in the calculation. Therefore, the backup path... The probability that all the hardware on it works properly is expressed as in Used to determine whether it belongs to the main hardware node where other VNF software resides. The value used to determine whether something belongs to the master node is v. k Backup hardware nodes for VNF software. SR i All VNF software works normally, and the probability is expressed as in equation (2). Therefore, the backup path in this case is... Reliability The calculation formula is as follows:

[0096]

[0097] Therefore, backup path reliability Represented as:

[0098]

[0099] Furthermore, SR i Reliability of all backup paths Represented as:

[0100]

[0101] Finally, SR i reliability The calculation formula is:

[0102]

[0103] Then, the deployment of the service function chain is subject to some limitations.

[0104] (a) The computing resources of satellite nodes cannot be over-allocated by VNFs.

[0105]

[0106] (b) Each VNF may select only one primary satellite node and one backup satellite node.

[0107]

[0108] (c) For any f i,j ∈VNF i f is not allowed i,j The primary node and backup node are the same.

[0109]

[0110] (d) The end-to-end delay of the SFC must not exceed its delay constraint.

[0111]

[0112] Finally, an optimization problem model is established.

[0113] The research of this invention aims to optimize the end-to-end latency and reliability of the main path in SFC, so the optimization objective is defined as:

[0114] obj:ω·Dp i +(1-ω)·R i (15)

[0115] Where ω∈[0,1] represents the weight coefficient, Dp i and R i These represent the normalized end-to-end delay Dp, respectively. i and SFC reliability

[0116] We need to maximize the value of obj:

[0117] maxobj (16)

[0118] st:(11)-(15).

[0119] (2) Algorithm Design

[0120] First, establish the MDP model.

[0121] After the agent selects a satellite node for the VNF based on the environmental state, the available computing resources of the node change. The agent then makes its next action based on the new environment, repeating the above process until all tasks are completed. Therefore, the deployment problem of SFCs is modeled as a Markov decision process, represented as a tuple (S, A, R), where S represents the state of the environment, A represents the agent's action, and R represents the reward.

[0122] (a) State: State S t The topology of the satellite network TS t And calculations of all satellites

[0123] Resource capacity Cr t Composition. Represented as:

[0124] S t =(TS) t ,Cr t (17)

[0125] (b) Actions: The agent's deployment of SFC requests is divided into two phases. In the first phase, a primary satellite node is selected for each VNF. In the second phase, a backup satellite node is selected for each VNF. In both phases, the agent's action space is the set V of all satellite nodes.

[0126] (c) Rewards: Rewards are divided into phased rewards and overall rewards. For phased rewards, each time the agent selects a satellite node for a VNF, it uses that VNF ​​as the endpoint VNF, calculates the end-to-end latency and reliability using equations (1) and (9) respectively, and calculates the reward value using equation (15). For overall rewards, when the agent has completed the selection of master and backup nodes for all VNFs, the sum of the rewards of all previous actions is used as the overall reward. In addition, if the agent's action violates the constraints, a penalty value P = -50 is given as the reward value for the action.

[0127] Then, the proposed HRSFCD algorithm is presented.

[0128] In the action space, the allocation decision of satellite nodes is a discrete action. Therefore, this paper adopts the DDQN algorithm, which handles discrete actions, as the basic algorithm. The HRSFCD algorithm framework is as follows: Figure 3 As shown, the agent interacts with the satellite network environment to train the Q network. During the training process, the deployment of an SFC is divided into two phases. In the first phase, the Q network selects a primary satellite node for each VNF in turn. In the second phase, the Q network selects a backup satellite node for each VNF in turn.

[0129] Algorithm 1 HRSFCD Algorithm

[0130] Input: G(V,E), M t , Ph = {Ph1, Ph2, ..., Ph} |V|}, <VNF i D i Cd i Ps i >

[0131] Output: Q network

[0132]

[0133] Lines 2 to 15 of the algorithm complete the first stage, selecting the primary satellite node for each VNF. Lines 16 to 27 of the algorithm complete the second stage, selecting the backup satellite node for each VNF.

Claims

1. A method for high-reliability service function chaining deployment based on dual deep Q network, characterized in that: Includes the following steps: Step 1: Establish the satellite network model and the service function chain (SFC) model; Step 2: Establish an optimization problem model that includes the end-to-end delay of the main path, SFC reliability, and constraints; where the end-to-end delay of the main path is the maximum value of the shortest delay between the starting VNF and the ending VNFs; SFC reliability includes... The reliability is defined as The probability of no failure, using This indicates that SFC reliability is calculated using the primary path reliability and backup path reliability. reliability The calculation formula is: （9） Reliability of the main path This refers to the probability that all VNF software and the master hardware node hosting all VNF software are functioning correctly; backup path reliability. It refers to The sum of the reliability of all backup paths, including two cases, the first being the instance. VNF software Failure occurred but the main hardware node Normal; the second is VNF software Normal but main hardware node Fault; The optimization problem model is as follows: The objective of optimizing the end-to-end latency and reliability of the main path in SFC is defined as follows: (15) in Indicates the weighting coefficient. and These represent the normalized end-to-end delays. and SFC reliability ; Maximize the value of obj: ; The limitations imposed on SFC are as follows: (a) The computing resources of a satellite node cannot be over-allocated by the VNF; (b) Each VNF may select only one primary satellite node and one backup satellite node; (c) For any ∈ Not allowed The primary satellite node and the backup satellite node are the same; (d) The end-to-end delay of the SFC must not exceed its delay constraint; Step 3: Model the SFC deployment problem as a Markov Decision Process (MDP) and design the state space, including the topology of the satellite network and the available computing resources of the nodes. Design the action space, including selecting the primary and backup satellite nodes for the Virtual Network Function (VNF); The reward function is determined based on the end-to-end latency of the main path and the reliability of SFC, and then the reward value is determined by formula (15). The high-reliability SFC deployment algorithm based on DDQN is adopted. In the main path stage, the main satellite node is selected for each VNF in sequence, and in the backup path stage, the backup satellite node is selected for each VNF in sequence.

2. The method for deploying a highly reliable service function chain based on a dual-depth Q-network according to claim 1, characterized in that: The implementation steps for step 1 are as follows: Step 1.1: Establish a satellite network model Satellite networks are represented by connected graphs Indicates; among which Represents the set of all satellite nodes. Indicates the number of satellite nodes; ={ ,..., } indicates in A time slot is a set of available computing resources for a satellite, in which... express exist Available computing resource capacity for a time slot; This represents the probability of hardware failure for all satellite nodes. Represents the set of physical links between satellite nodes; Indicates in Time slots are matrices representing the shortest propagation delays between satellite nodes. Step 1.2: Establish the SFC model definition For the set of requests from SFCs, Represent the number of SFCs; using tuples Indicates a request ; Indicates that a request is constituted The set of VNFs, where, This represents the j-th VNF; express The maximum tolerable end-to-end delay; This represents the computational resource requirements of each VNF; This represents the probability of software failure for each VNF.

3. The method for deploying a highly reliable service function chain based on a dual-depth Q-network according to claim 1, characterized in that: Step 2 includes: Step 2.1: Establish the end-to-end delay model for the main path; The path consists of the master satellite nodes of all VNFs and the links between them. Satellite nodes are selected sequentially for each VNF according to its sequence number. The first VNF ​​deployed is called the starting VNF. This indicates that there are one or more endpoint VNFs; the SFC request... Main path end-to-end delay Defined as the maximum value of the shortest delay between the starting VNF and the ending VNFs; the formula is as follows: (1) in Describes the set of endpoint VNFs; It is a binary variable. =1 indicates that the VNF software is used. Deployed on the main hardware node superior; Step 2.2: Establish the SFC reliability model, which includes the following: Step 2.2.1: Establish the main path reliability model; Reliability of the main path This refers to the probability that all VNF software and the host hardware node of all VNF software are functioning correctly; The probability that all VNF software works properly is used The calculation formula is as follows: (2) in VNF software Software reliability; The probability that all the main hardware nodes containing all VNF software are functioning normally is used The calculation formula is as follows: (3) in Represents a node Hardware reliability; This indicates that different VNFs are deployed to the same node. hour, Hardware reliability only participates in one computation; main path reliability The calculation method is as follows: (4) Step 2.2.2: Establish a backup path reliability model VNF software Deployed on satellite nodes The above is referred to as a VNF instance. ;if yes The main satellite node, when Software malfunction or Hardware failure caused instance If it cannot function properly, then the main satellite node will... The processed data stream will be forwarded to Processing is performed on the backup satellite nodes; at this time, by The backup satellite nodes, the main satellite nodes of other VNF software, and the links between the nodes constitute the system. A backup path, which will be used because of the instance Backup path used in case of failure This indicates that a backup satellite node was selected for each VNF software, therefore, under the proposed single point of failure premise, There is 1 main path and | | Backup paths; Backup path reliability It refers to The sum of the reliability of all backup paths; For backup path There are two reasons that cause it to work; first, the example VNF software Failure occurred but the main hardware node Normal; Path The hardware nodes involved in the work are divided into two categories: software and hardware. Backup hardware nodes and the primary hardware nodes where other VNF software resides; traverse all hardware nodes, and if a node belongs to at least one of the above two types of nodes, include its hardware reliability in the calculation; backup path. The probability that all the hardware on it works properly is expressed as: ,in Used to determine whether it belongs to the main hardware node where other VNF software resides; It is a binary variable. =1 means exist The time slots were deployed to backup satellite nodes. The above is used to determine whether it is software. Backup hardware nodes; All VNF software works normally, and the probability is expressed as in equation (2); therefore, in this case, the backup path The reliability is expressed as: (5) second, VNF software Normal but main hardware node In case of failure, path The hardware nodes involved in the operation are divided into two categories: the main satellite node and the... The backup hardware node for the VNF software, and the primary hardware node where other VNF software resides; due to Fault, so iterate through all... For all hardware nodes other than those mentioned above, if they belong to at least one of the two types of nodes, the hardware reliability of the primary satellite node will be included in the calculation; backup path The probability that all the hardware on it works properly is expressed as: ,in Used to determine whether it belongs to the main hardware node where other VNF software resides. Used to determine whether it belongs to the main satellite node Backup hardware nodes for VNF software; All VNF software functions normally, and the probability is expressed as in equation (2); backup path Reliability The calculation formula is as follows: (6) Therefore, backup path reliability Represented as: (7) and then, Reliability of all backup paths Represented as: (8) Step 2.2.3: Limitations on SFC Deployment; (a) The computing resources of a satellite node cannot be over-allocated by the VNF; (10) (b) Each VNF may select only one primary satellite node and one backup satellite node; (11) (12) (c) For any ∈ Not allowed The primary satellite node and the backup satellite node are the same; (13) (d) The end-to-end delay of the SFC must not exceed its delay constraint; (14)。 4. The method for deploying a highly reliable service function chain based on a dual-depth Q-network according to claim 1, characterized in that: Step 3 includes: Step 3.1 Establish the MDP model; After the agent selects a satellite node for the VNF based on the environmental state, the available computing resources of the satellite node change. The agent then makes the next action based on the new environment, repeating this process until all processing is complete. The deployment problem of SFCs is modeled as a Markov decision process, represented as a tuple. Where S represents the state of the environment and A represents the action of the agent. Indicates a reward; (a) State: State The topology of satellite networks and the computing resource capacity of all satellites Composition; represented as: (17) (b) Actions: The deployment of SFC requests by the agent is divided into two phases; in the first phase, the primary satellite node is selected for each VNF, and in the second phase, the backup satellite node is selected for each VNF. In both phases, the action space of the agent is the set V of all satellite nodes. (c) Rewards: Rewards are divided into phased rewards and overall rewards. For phased rewards, the agent selects a satellite node for each VNF, takes that VNF ​​as the endpoint VNF, calculates the end-to-end delay and reliability using equations (1) and (9) respectively, and calculates the reward value using equation (15). For overall rewards, when the agent has completed the selection of the primary satellite node and backup satellite node for all VNFs, the sum of the rewards of all previous actions is used as the overall reward. In addition, if the agent's action violates the constraints, a penalty value is given as the reward value of the action. Step 3.2 HRSFCD algorithm; In the action space, the allocation decision of satellite nodes is a discrete action, and the DDQN algorithm, which handles discrete actions, is used as the basic algorithm. The agent interacts with the satellite network environment to train the Q network. During the training process, the deployment of an SFC is divided into two stages. In the first stage, the Q network selects the master satellite node for each VNF in turn. In the second stage, the Q network selects the backup satellite node for each VNF in turn.

Images