Service function chain deployment method, device and equipment

Abstract

The invention provides a service function chain deployment method, device and equipment, and relates to the field of network communication. The method comprises the following steps: acquiring a state space of a service function chain in a current time slot according to information of a preset physical network environment and information of the service function chain to be deployed; and determining a pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from an action space corresponding to a policy network by adopting the policy network in the preset decision process model, completing the training of the current training round, obtaining a state space corresponding to the service function chain again, and performing the training of the next training round. And determining a pre-deployment action of each VNF node in the service function chain in the preset training round as a target deployment action until training of the preset training round is completed, deploying each VNF node to a target physical node corresponding to the target deployment action, and completing deployment of all VNF nodes in the service function chain.

Description

Technical Field

[0001] The present invention relates to the field of network communications, and in particular to a service function chain deployment method, device and equipment. Background Art

[0002] In recent years, mobile communications have developed rapidly. The growth in the number of users and the diversification of communication services have led to an exponential increase in data traffic. At the same time, the limited resources of basic hardware facilities, poor scalability, and high network maintenance costs have led to the inability of traditional management methods and deployment solutions to meet the growing traffic demand. In order to improve the existing defects of the above-mentioned networks and accelerate the deployment of 5G services, the upgrade and innovation of network architecture is imperative. As a result, service function chain deployment technology based on network function virtualization (NFV) and software defined network (SDN) came into being and is considered to be an ideal network model to solve network rigidity and improve operational efficiency.

[0003] At present, in the study of service function chaining (SFC) deployment problems, modeling the problem as an integer linear programming (ILP) model is a common method. Integer linear programming is an extension of linear programming, in which some or all decision variables in the model are restricted to integer values. This method faces the challenge of a sharp increase in computational complexity when dealing with complex network topologies and large-scale SFC deployment. The complexity here is reflected in two aspects: one is the increase in computing resources required, and the other is that the solution algorithm tends to converge more slowly. In order to find a feasible solution to the optimization problem within an acceptable time and space cost, heuristic algorithms are designed to reduce the complexity of the problem and speed up the convergence of the algorithm. However, heuristic algorithms are usually constructed based on intuition or experience. These algorithms are usually based on problem-specific knowledge and quickly find a more satisfactory but not necessarily optimal solution by heuristically exploring the solution space. Summary of the invention

[0004] The purpose of the present invention is to provide a service function chain deployment method, device and equipment to address the deficiencies in the above-mentioned prior art, so as to adopt the policy network in the preset decision process model to optimally deploy all VNF nodes on the service function chain.

[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a service function chain deployment method, the method comprising:

[0007] According to the information of the preset physical network environment and the information of the service function chain to be deployed, a state space for the service function chain in the current time slot is obtained, wherein the state space includes: network feature information and chain feature information;

[0008] Adopting the policy network in the preset decision process model, according to the state space of the service function chain in the current time slot, determining the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network, and saving the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are some VNF nodes of the service function chain;

[0009] If the number of the multiple sets of policy data reaches a preset number threshold, adjusting the policy network according to the multiple sets of policy data;

[0010] Using the adjusted parameter policy network, the pre-deployment actions of the remaining VNF nodes in the service function chain are determined until the pre-deployment actions of all VNF nodes in the service function chain are determined, and the training of the current training round is completed;

[0011] Re-acquire the state space corresponding to the service function chain, perform training for the next training round, and complete the training for the preset training round, and determine the pre-deployment action of each VNF node in the service function chain within the preset training round;

[0012] Determine the pre-deployment action of each VNF node in the preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

[0013] In an optional implementation, determining the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network includes:

[0014] Determine, from the action space corresponding to the policy network, a prediction result of each VNF node in the service function chain in the current training round, the prediction result comprising: a plurality of prediction action values, the plurality of prediction action values ​​being used to characterize a prediction probability value of each VNF node being deployed on a plurality of physical nodes in the preset physical network environment;

[0015] Based on the multiple predicted action values, determine the pre-deployment action of each VNF node.

[0016] In an optional implementation manner, the acquiring, according to information of a preset physical network environment and information of a service function chain to be deployed, a state space for the service function chain in a current time slot includes:

[0017] Acquire a structural topology diagram of the preset physical network environment, wherein the structural topology diagram records information of the multiple physical nodes and information of physical links between the physical nodes, the information of the physical nodes at least includes: computing resource information of the physical nodes, and the information of the physical links at least includes: bandwidth information of the physical links;

[0018] Obtain a link topology map of the service function chain, wherein the link topology map records information of all VNF nodes in the service function chain and information of virtual links between VNF nodes;

[0019] According to the structure topology map and the link topology map, respectively obtain network feature information and link feature information for the service function chain in the current time slot;

[0020] According to the network characteristic information and the chain characteristic information of the current time slot, the state space of the service function chain in the current time slot is acquired.

[0021] In an optional implementation manner, if the number of the multiple sets of policy data reaches a preset number threshold, adjusting parameters of the policy network according to the multiple sets of policy data includes:

[0022] Calculate the first reward value of each VNF node according to the pre-deployment result of the pre-deployment action of each VNF node in the multiple VNF nodes corresponding to the multiple groups of policy data;

[0023] Calculate the second reward value of each VNF node respectively according to the resource consumption information of the pre-deployment action of each VNF node under the pre-deployment behavior;

[0024] According to the pre-deployment physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node, the penalty value of each VNF node is calculated respectively, wherein the penalty value of each VNF node is used to characterize the penalty value of each VNF node and the previous adjacent VNF ​​node mapped to the same deployment node;

[0025] According to the first reward value, the second reward value and the penalty value of the multiple VNF nodes, the target parameters of the multiple groups of policy data are calculated, and the policy network is adjusted according to the target parameters.

[0026] In an optional embodiment, the method further comprises:

[0027] Using the evaluation network in the preset decision process model, determining the state value of the current training round according to the state space of the service function chain in the current time slot;

[0028] The calculating the loss parameters of the multiple sets of policy data according to the first reward values, the second reward values, and the penalty values ​​of the multiple VNF nodes includes:

[0029] Calculate the expected reward value according to the first reward value, the second reward value and the penalty value of the plurality of VNF nodes;

[0030] Calculating an advantage estimation parameter according to the expected value of the reward and the state value;

[0031] Calculate target parameters of the multiple sets of strategy data according to the advantage estimation parameter, the expected value of the reward, and the state value.

[0032] In an optional implementation, calculating the expected reward value according to the first reward value, the second reward value, and the penalty value of the plurality of VNF nodes includes:

[0033] Calculate target reward values ​​of the multiple VNF nodes respectively according to the first reward values, the second reward values ​​and the penalty values ​​of the multiple VNF nodes;

[0034] The expected reward value is calculated based on the target reward values ​​of the multiple VNF nodes, the preset discount rates of the multiple VNF nodes and the state values.

[0035] In an optional implementation, the method further includes: using a policy network in a preset decision process model, determining a pre-deployment action of each virtual network function VNF node in the service function chain within the training round from an action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and saving the pre-deployment action, and obtaining multiple sets of policy data before obtaining the multiple sets of policy data.

[0036] The network performance of the structure topology map and the link topology map is digitized to construct a service chain deployment problem with the joint goal of maximizing deployment success rate and minimizing resource consumption;

[0037] The service chain deployment problem is adopted to construct the preset decision process model.

[0038] In an optional implementation manner, the network performance data is digitized according to the structure topology map and the link topology map to construct a service chain deployment problem with maximum deployment success rate and minimum resource consumption as joint goals, including:

[0039] Performing network performance digitization on the structure topology map and the link topology map to obtain mathematical expressions of deployment resources corresponding to VNF nodes, mathematical expressions of mapping bandwidths corresponding to virtual links, deployment decision variables of the VNF nodes, and mapping decision variables of the virtual links;

[0040] Constructing a deployment resource constraint expression according to a mathematical expression of deployment resources corresponding to the VNF node and a deployment decision variable of the VNF node;

[0041] Constructing a mapping bandwidth constraint expression according to a mathematical expression of a mapping bandwidth corresponding to the virtual link and a mapping decision variable of the virtual link;

[0042] According to the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, and the deployment success expression corresponding to the VNF node, a service chain deployment problem with maximum deployment success rate and minimization of resource consumption as the joint goals is constructed.

[0043] In a second aspect, an embodiment of the present application further provides a service function chain deployment device, the device comprising:

[0044] An acquisition module, configured to acquire a state space for the service function chain in a current time slot according to information of a preset physical network environment and information of a service function chain to be deployed, wherein the state space includes network feature information and chain feature information;

[0045] A determination module is used to adopt a policy network in a preset decision process model, determine the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and save the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are some VNF nodes of the service function chain;

[0046] A parameter adjustment module, configured to adjust the parameters of the policy network according to the multiple sets of policy data if the number of the multiple sets of policy data reaches a preset number threshold;

[0047] The determination module is further used to determine the pre-deployment actions of the remaining VNF nodes in the service function chain using the adjusted parameter policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined to be completed, thereby completing the training of the current training round;

[0048] The determination module is further used to reacquire the state space corresponding to the service function chain, perform training for the next training round, and complete the training for the preset training round, and determine the pre-deployment action of each VNF node in the service function chain within the preset training round;

[0049] The determination module is also used to determine the pre-deployment action of each VNF node in the preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

[0050] In the third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the steps of the service function chain deployment method as described in any one of the first aspects.

[0051] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the service function chain deployment method as described in any one of the first aspects are executed.

[0052] The beneficial effects of this application are:

[0053] The embodiments of the present application provide a method, apparatus and device for deploying a service function chain, including: obtaining a state space for the service function chain in the current time slot according to information of a preset physical network environment and information of a service function chain to be deployed, wherein the state space includes network feature information and chain feature information; using a policy network in a preset decision process model, according to the state space of the service function chain in the current time slot, determining a pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from an action space corresponding to the policy network, and saving the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are part of the VNF nodes of the service function chain; if multiple When the number of group policy data reaches the preset number threshold, the policy network is adjusted according to the multiple groups of policy data; the pre-deployment actions of the remaining VNF nodes in the service function chain are determined using the adjusted policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined and the training of the current training round is completed; the state space corresponding to the service function chain is re-acquired, and the next training round of training is performed until the training of the preset training round is completed, and the pre-deployment action of each VNF node in the service function chain within the preset training round is determined, and finally the pre-deployment action of each VNF node within the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action.

[0054] The method of the present application deploys each VNF node in the service function chain to be deployed in turn by adopting the policy network in the preset decision process model to complete the deployment of all VNF nodes. First, in the current training round, the pre-deployment action of each VNF node is determined and saved. If it is determined that the policy data of the preset number threshold is saved, the policy network is adjusted, and the pre-deployment actions of the remaining VNF nodes are continued to be determined and saved to complete the training of the current training round until the pre-deployment action of each VNF node in the service function chain in the preset training round is determined, and the pre-deployment action of each VNF node in the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action. The target physical node is the optimal deployment node, and the deployment of all VNF nodes on the service function chain is completed. The policy network in the preset decision process model is adopted to deploy the service function chain, which can improve the efficiency and stability of the service function chain deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 One of the flow diagrams of a service function chain deployment method provided in an embodiment of the present application;

[0057] Figure 2 A second flow chart of a service function chain deployment method provided in an embodiment of the present application;

[0058] Figure 3 A third flow chart of a service function chain deployment method provided in an embodiment of the present application;

[0059] Figure 4 A fourth flow chart of a service function chain deployment method provided in an embodiment of the present application;

[0060] Figure 5 A fifth flow chart of a service function chain deployment method provided in an embodiment of the present application;

[0061] Figure 6 A sixth flow chart of a service function chain deployment method provided in an embodiment of the present application;

[0062] Figure 7 A flow chart of a service function chain deployment method provided in an embodiment of the present application is shown in FIG7;

[0063] Figure 8 A schematic diagram of functional modules of a service function chain deployment device provided in an embodiment of the present application;

[0064] Fig. 9 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0066] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0067] In addition, the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0068] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0069] Service function chain deployment refers to creating virtual network function VNF instances on physical nodes according to service requests in the NFV environment or using already deployed VNF instances to carry the service functions required by the service. This process requires planning the connection sequence and traffic forwarding path between each virtual network function to ensure that the user's service quality and performance requirements are met.

[0070] In order to deploy a service function chain that satisfies minimum resource consumption and maximizes service rate, and improve the efficiency and stability of service function chain deployment, an embodiment of the present application provides a service function chain deployment method, which obtains the state space of the service function chain in the current time slot based on the information of the preset physical network environment and the information of the service function chain to be deployed, adopts the policy network in the preset decision process model, and determines the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and saves the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are part of the VNF nodes of the service function chain; if the multiple groups of policy data If the number of VNF nodes reaches the preset number threshold, the policy network is adjusted according to the multiple sets of policy data; the pre-deployment actions of the remaining VNF nodes in the service function chain are determined by the adjusted policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined and the training of the current training round is completed; the state space corresponding to the service function chain is re-acquired, and the training of the next training round is performed until the training of the preset training round is completed, and the pre-deployment action of each VNF node in the service function chain within the preset training round is determined, and finally the pre-deployment action of each VNF node in the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action, so that the deployed service function chain meets the minimum resource consumption and the maximum service rate.

[0071] The following is a detailed explanation of a service function chain deployment method provided by an embodiment of the present application through specific examples in conjunction with the accompanying drawings. A service function chain deployment method provided by an embodiment of the present application can be implemented by a computer device pre-installed with: a detection algorithm or detection software for a preset service function chain deployment method, by running the algorithm or software. The computer device can be, for example, a server or a terminal, and the terminal can be a user computer. Figure 1 This is one of the flow charts of a service function chain deployment method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0072] S101. Obtain a state space for the service function chain in the current time slot based on information of a preset physical network environment and information of the service function chain to be deployed.

[0073] In this embodiment, the state space includes: network feature information and chain feature information. The network feature information is feature information obtained by extracting features from information of a preset physical network environment, and the chain feature information is feature information obtained by extracting features from information of a service function chain to be deployed.

[0074] The preset physical network environment can be a physical network environment in a power network service scenario, a physical network environment in a smart metropolitan area network scenario, and a physical network environment in a smart logistics management scenario. The information of the preset physical network environment includes: information of physical servers in the preset physical network environment, information of virtual network function VNF instances in the physical servers, wherein physical network devices such as firewalls, caches, and load balancers are abstracted into virtual network functions, namely VNF instances, for deployment of service function chains. The information of the service function chain to be deployed includes: information of all VNF instances in the service function chain to be deployed.

[0075] Based on the information of the preset physical network environment and the information of the service function chain to be deployed, the state space of the service function chain in the current time slot is determined, wherein there is at least one VNF node in each service function chain. When each service function chain is deployed, each VNF node is deployed in turn until all VNF nodes are deployed.

[0076] S102. Using the policy network in the preset decision process model, according to the state space of the service function chain in the current time slot, determine the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network, and save the pre-deployment action to obtain multiple sets of policy data.

[0077] Among them, multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are part of the VNF nodes of the service function chain.

[0078] S103: If the number of the multiple sets of policy data reaches a preset number threshold, the policy network is adjusted according to the multiple sets of policy data.

[0079] The policy network in the preset decision process model can be the policy network in the proximal policy optimization algorithm (PPO) in deep reinforcement learning. The proximal policy optimization algorithm is an offline policy algorithm that can reuse the sampled data to improve the training speed and reduce the complexity of service function chain deployment. In addition, the proximal policy optimization algorithm can avoid the problem of unstable training caused by excessive policy updates by limiting the size of policy updates. It has a faster convergence effect and can effectively improve the efficiency and stability of service function chain deployment.

[0080] The policy network in the preset decision process model is adopted, and according to the state space of the service function chain in the current time slot, the pre-deployment action of each VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, and then the pre-deployment action of each VNF node is saved to obtain multiple groups of policy data. The multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are some VNF nodes of the service function chain. Since the policy network needs to determine the pre-deployment action of each VNF node in the service function chain in turn, each time the pre-deployment action of a VNF node is determined, the pre-deployment action corresponding to the VNF node is saved to obtain a group of policy data.

[0081] If the number of the saved multiple sets of policy data reaches a preset number threshold, namely, the K value, the policy network is adjusted according to the K sets of policy data.

[0082] The action space refers to the set of all possible actions that the agent can perform in a given environment. The entire SFC is deployed by deploying VNF nodes to physical nodes and linking them, so the action space is the normalized probability of VNF nodes being deployed to each physical node. k The size should be (1*N), that is, a k =[P1,P2,…,P N ], where N is the number of physical nodes. Finally, the Categorical sampling function is used to sample and select specific pre-deployed actions from multiple predicted action values ​​in the action space.

[0083] S104. Use the adjusted parameter policy network to determine the pre-deployment actions of the remaining VNF nodes in the service function chain until the pre-deployment actions of all VNF nodes in the service function chain are determined, completing the training of the current training round.

[0084] Specifically, after adjusting the parameters of the policy network through K groups of policy data, if there are still pre-deployment actions of the remaining VNF nodes that have not been determined, the adjusted policy network will continue to be used to determine the pre-deployment actions of the remaining VNF nodes until the pre-deployment actions of all VNF nodes are determined, which means that the current training round is completed.

[0085] It should be noted that if the adjusted policy network is used to determine the pre-deployment actions of the remaining VNF nodes, and K groups of policy data are saved, the adjusted policy network will continue to be adjusted according to the re-saved K groups of policy data until the pre-deployment actions of all VNF nodes are determined.

[0086] S105. Re-acquire the state space corresponding to the service function chain, and perform training for the next training round until the preset training round is completed, and determine the pre-deployment actions of each VNF node in the service function chain within the preset training round.

[0087] S106. Determine the pre-deployment action of each VNF node in the preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

[0088] After the current training round is completed, the state space of the service function chain in the next training round is obtained based on the information of the preset physical network environment and the information of the service function chain to be deployed, and the training of the next training round is started.

[0089] Specifically, a policy network after parameter adjustment is adopted, and according to the state space of the service function chain in the next training round, the pre-deployment action of each VNF node in the service function chain in the next training round is re-determined from the action space, and the pre-deployment action is saved to obtain multiple sets of policy data for the next training round.

[0090] Similarly, when the number of multiple groups of policy data in the next training round reaches a preset number threshold, the adjusted policy network is adjusted again according to the multiple policy data of the next training round, and then the adjusted policy network is used to determine the pre-deployment actions of the remaining VNF nodes in the next training round until the pre-deployment actions of all VNF nodes in the next training round are determined. At the beginning of a new training round, the pre-deployment actions of all VNF nodes continue to be determined until the preset training round is performed, and the pre-deployment actions of each VNF node in the service function chain in the preset training round are determined, wherein the preset training round indicates stopping the training round, for example, when the training reaches 50 rounds, the training is stopped, that is, the determination of the pre-deployment action is stopped.

[0091] Then, the pre-deployment action of each VNF node in the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action, completing the deployment of each VNF node in the service function chain to be deployed.

[0092] In summary, the present application provides a service function chain deployment method, including:

[0093] According to the information of the preset physical network environment and the information of the service function chain to be deployed, the state space of the service function chain in the current time slot is obtained, and the state space includes: network feature information and chain feature information; the policy network in the preset decision process model is used, and according to the state space of the service function chain in the current time slot, the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, and the pre-deployment action is saved to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are part of the VNF nodes of the service function chain; if the number of the multiple groups of policy data reaches a preset number threshold value, the policy network is adjusted according to the multiple sets of policy data; the pre-deployment actions of the remaining VNF nodes in the service function chain are determined by the adjusted policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined and the training of the current training round is completed; the state space corresponding to the service function chain is re-acquired, and the training of the next training round is performed until the training of the preset training round is completed, and the pre-deployment action of each VNF node in the service function chain within the preset training round is determined, and finally the pre-deployment action of each VNF node within the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action.

[0094] The method of the present application deploys each VNF node in the service function chain to be deployed in turn by adopting the policy network in the preset decision process model to complete the deployment of all VNF nodes. First, in the current training round, the pre-deployment action of each VNF node is determined and saved. If it is determined that the policy data of the preset number threshold is saved, the policy network is adjusted, and the pre-deployment action of the remaining VNF nodes is continued to be determined and saved to complete the training of the current training round until the pre-deployment action of each VNF node in the service function chain in the preset training round is determined, and the pre-deployment action of each VNF node in the preset training round is determined as the target deployment action, and each VNF node is deployed to the target physical node corresponding to the target deployment action to complete the deployment of all VNF nodes on the service function chain. The policy network in the preset decision process model is adopted to deploy the service function chain, which can improve the efficiency and stability of the service function chain deployment.

[0095] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 2 The second flow chart of a service function chain deployment method provided in the embodiment of the present application. Figure 2 As shown, the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, including:

[0096] S201. Determine the prediction result of each VNF node in the service function chain in the current training round from the action space corresponding to the policy network.

[0097] Among them, the prediction results include: multiple prediction action values, and the multiple prediction action values ​​are used to characterize the prediction probability values ​​of each VNF node being deployed on multiple physical nodes in a preset physical network environment.

[0098] S202. Determine the pre-deployment action of each VNF node based on multiple predicted action values.

[0099] In this embodiment, the prediction result of each VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, that is, the predicted probability value of each VNF node being deployed on multiple physical nodes in the preset physical network environment is obtained, and then the pre-deployment action of each VNF node is determined based on multiple predicted action values.

[0100] Specifically, for a VNF node, the action space output corresponding to the policy network is the action mean of a VNF node, so that multiple predicted action values ​​obey Gaussian distribution, and the standard deviation of Gaussian distribution decays with the network update frequency. The pre-deployed action a of the VNF node is randomly sampled from multiple predicted action values ​​of Gaussian distribution. k , and determine to select the pre-deployed action a from the action values ​​that obey the Gaussian distribution k Probability Then the set of policy data corresponding to the VNF node includes the state space of a training round, the pre-deployment action a of the VNF node k and the probability of pre-deployed actions

[0101] The network structure of the policy network adopts a multi-layer perceptron (MLP) structure, which includes an input layer, several hidden layers and an output layer. The hidden layer of the policy network consists of three fully connected layers, each with 256 neurons. The ReLU activation function is used between the fully connected layers to increase the nonlinear factors and improve the neural network's ability to express the model. The input layer of the policy network inputs the state space of each training round, and the output layer is the action mean of each training round. The probability distribution of the output action is then generated through the Softmax function, that is, multiple predicted action values ​​of each VNF node in each training round are obtained.

[0102] In the method provided in the embodiment of the present application, the prediction result of each VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, and the prediction result includes: multiple predicted action values, and the multiple predicted action values ​​are used to characterize the predicted probability value of each VNF node deployed on multiple physical nodes in a preset physical network environment. According to the multiple predicted action values, the pre-deployment action of each VNF node is determined.

[0103] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 3 The third flow chart of a service function chain deployment method provided in the embodiment of the present application. Figure 3 As shown, according to the information of the preset physical network environment and the information of the service function chain to be deployed, the state space of the service function chain in the current time slot is obtained, including:

[0104] S301, obtaining a structural topology diagram of a preset physical network environment.

[0105] The structural topology diagram records information of multiple physical nodes and information of physical links between physical nodes. The information of physical nodes at least includes computing resource information of physical nodes, and the information of physical links at least includes bandwidth information of physical links.

[0106] In this embodiment, the preset physical network environment is modeled according to the information of the preset physical network environment to obtain a structural topology diagram of the preset physical network environment. The structural topology diagram of the preset physical network environment is an undirected graph and can be expressed as: S =(N S ,L S ), where N S It consists of all physical nodes, each of which represents the location of one or more VNF instances. S It is represented as the set of physical links between all physical nodes. For each physical node n∈N S , define sum n The total computing resources of the physical node are the computing resource information of the physical node. (n,m) ∈L S is the actual physical link between physical node n and physical node m (n,m∈N S ),definition The total bandwidth capacity of the link, that is, the bandwidth information of the physical link is recorded.

[0107] S302. Obtain the link topology diagram of the service function chain.

[0108] The link topology diagram records the information of all VNF nodes in the service function chain and the information of virtual links between VNF nodes.

[0109] According to the information of the service function chain, a link topology graph of the service function chain is constructed. Specifically, the service function chain is abstracted into a link topology graph, i.e., an undirected graph: G V =(N V ,L V ), where N V is the set of all VNF instances in the service function chain, i.e., it records the information of all VNF nodes in the service function chain. V The collection of all virtual links in the service function chain records the information of virtual links between VNF nodes. For a service function chain request r, Represents the set of VNF nodes that make up the service function chain request. Similarly, Indicates the set of virtual links connecting VNFs.

[0110] S303. According to the structure topology map and the link topology map, respectively obtain network characteristic information and chain characteristic information for at least one server function chain in the current time slot.

[0111] S304. Obtain the state space of the service function chain in the current time slot according to the network characteristic information and chain characteristic information of the current time slot.

[0112] Specifically, the state space refers to the set of all possible states that an agent may encounter in a given environment. The state is a representation of the environment and can include various observations, information, and features to describe the current situation of the environment. The state space includes: network feature information and chain feature information.

[0113] Each physical node has several attributes, such as the capacity of computing resources and the total bandwidth of adjacent physical links. In order to facilitate the VNF node to select a suitable deployment physical node, the characteristics of each physical node in the structure topology diagram are extracted to obtain the network characteristic information of a VNF node in an iteration cycle. The extraction process of network characteristic information is as follows:

[0114] (a) The remaining available capacity of the physical node res n : The remaining computing resources have a great impact on its availability, and physical nodes with higher capacity can deploy more VNF instances.

[0115] (b) Physical node entry and exit degree n : The in-degree of a physical node represents the sum of the number of links connected to the physical node. A physical node with more adjacent links is more likely to find paths to other physical nodes.

[0116] (c) The physical link bandwidth of the physical node and Indicates the sum of the link bandwidths connecting the physical node. A larger value means more available bandwidth.

[0117] The features of each VNF node in the link topology are extracted to obtain the chain feature information of the service function chain in the current time slot. The chain feature information extraction process is as follows:

[0118] (a) Average distance avg dst (n s ): When deploying VNF nodes, consider the deployment locations of other VNF nodes in the same service function chain. By selecting physical nodes close to the deployed VNF nodes, the cost of link bandwidth can be reduced. The bandwidth between adjacent VNF ​​nodes is used as the path, and the FloydWarshall algorithm is used to calculate and find the shortest path.

[0119] For a service function chain, avg dst (n s )The formula is as follows:

[0120]

[0121] |N s | indicates the number of VNF nodes deployed in the service function chain. DST(n s , n m ) indicates that n is obtained by using the FloydWarshall algorithm s to n m When the VNF node has not been deployed in the service function chain, avg dst (n s )=0.

[0122] (b) The first VNF ​​node identification bit head: In a service function chain, the first VNF ​​node is different from the subsequent VNF ​​nodes. Deploying it to a location with abundant physical node resources as much as possible will help the deployment of subsequent VNF ​​nodes. head = 1 indicates the first VNF ​​node in the service function chain.

[0123] (c) Article r Service Function Chain: In the actual network, the service function chain follows the principle of first come first served. The service function chain that arrives later may be unable to provide service or may have to wait for service due to insufficient node computing resources.

[0124] In summary, the state space s of the service function chain in the current time slot k It can be expressed as:

[0125]

[0126] In the method provided in the embodiment of the present application, a structural topology map of a preset physical network environment is obtained, wherein the structural topology map records information of multiple physical nodes and information of physical links between physical nodes, and the information of physical nodes at least includes: computing resource information of physical nodes, and the information of physical links at least includes: bandwidth information of physical links; a link topology map of a service function chain is obtained, wherein the link topology map records information of all VNF nodes in the service function chain and information of virtual links between VNF nodes; network feature information and link feature information for the current time slot of the service function chain are obtained according to the structural topology map and the link topology map respectively; a state space of the service function chain in the current time slot is obtained according to the network feature information and chain feature information of the current time slot, and the state space of the current time slot includes the network feature information and chain feature information of the current time slot, which is used to input a policy network and determine a pre-deployment action of each virtual network function VNF node in the service function chain in the current training round.

[0127] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 4 The fourth flow chart of a service function chain deployment method provided in an embodiment of the present application. Figure 4 As shown, if the number of multiple sets of policy data reaches a preset number threshold, the policy network is adjusted according to the multiple sets of policy data, including:

[0128] S401. Calculate the first reward value of each VNF node according to the pre-deployment results of the pre-deployment actions of each VNF node in the multiple VNF nodes corresponding to the multiple groups of policy data.

[0129] In this embodiment, a reward function is used to calculate the first reward value, the second reward value and the penalty value of each VNF node, wherein the reward function is a function that evaluates and gives feedback on the behavior results of the intelligent agent, i.e., the VNF node in the environment. It defines the reward value obtained by the intelligent agent when taking different actions in different states, and is used to guide the intelligent agent to select a better behavior strategy during the learning process.

[0130] Among them, if each VNF node is deployed in a training round, the pre-deployment result is obtained, and the first reward value of each VNF node is calculated It is expressed as follows:

[0131]

[0132] Among them, the first reward value It is represented as a Boolean value indicating whether each VNF node is successfully deployed to the server node, thereby obtaining the first reward value of each VNF node in a training round.

[0133] S402. Calculate the second reward value of each VNF node based on the resource consumption information of the pre-deployment behavior of the pre-deployment action of each VNF node.

[0134] Specifically, when a VNF mapping fails, it is considered that the service function chain corresponding to the VNF section has failed to be deployed, and the next service function chain begins to be deployed.

[0135] When all VNF nodes of a service function chain are successfully deployed, considering minimizing resource consumption, the resource consumption should also be reflected in the reward function. Then, the second reward value of each VNF node is calculated based on the resource consumption information of the pre-deployment behavior of the pre-deployment action of each VNF node. It is expressed as follows:

[0136]

[0137] Among them, the second reward value It is the ratio of the bandwidth resource consumption theoretically required by the service function chain to the actual bandwidth consumption after deployment, and its value range is between 0 and 1, so as to obtain the second reward value of each VNF node in a training round.

[0138] S403. Calculate the penalty value of each VNF node according to the pre-deployment physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node.

[0139] Among them, the penalty value of each VNF node is used to characterize the penalty value of each VNF node and the previous adjacent VNF ​​node mapped to the same deployment node.

[0140] Specifically, multiple VNF nodes can be deployed on the same physical node to minimize costs, but this approach will lead to unbalanced resource utilization and increase the risk of servers under high load. In order to increase the reliability of the service function chain and prevent the service function chain from suffering a heavy blow to the network due to a single point failure, different physical nodes will be used to deploy the VNF nodes in the service function chain in a distributed manner.

[0141] Therefore, the reward function includes a penalty for mapping adjacent VNF ​​nodes to the same physical node. The penalty value of each VNF node in a training round is calculated based on the pre-deployed physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node. It is expressed as follows:

[0142]

[0143] Among them, the penalty value Defined as the penalty for deploying adjacent VNF ​​nodes to the same physical node, It means that VNF ​​node i is mapped to logistics node n, and VNF node i and VNF node j are adjacent VNF ​​nodes, so as to obtain the penalty value of each VNF node in a training round.

[0144] S404. Calculate target parameters of multiple sets of policy data based on the first reward value, the second reward value, and the penalty value of multiple VNF nodes, and adjust the policy network according to the target parameters.

[0145] The target parameters of multiple sets of policy data are calculated through the first reward value, the second reward value and the penalty value of multiple VNF nodes, and then the policy network is adjusted according to the target parameters.

[0146] In the method provided in the embodiment of the present application, a first reward value of each VNF node is calculated respectively according to the pre-deployment result of the pre-deployment action of each VNF node in the multiple VNF nodes corresponding to the multiple sets of policy data under the pre-deployment behavior; a second reward value of each VNF node is calculated respectively according to the resource consumption information under the pre-deployment behavior of the pre-deployment action of each VNF node; a penalty value of each VNF node is calculated respectively according to the pre-deployment physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node, wherein the penalty value of each VNF node is used to characterize the penalty value of each VNF node and the previous adjacent VNF ​​node mapped to the same deployment node; target parameters of multiple sets of policy data are calculated according to the first reward value, the second reward value and the penalty value of the multiple VNF nodes, and the policy network is adjusted according to the target parameters, and the optimal deployment position of each VNF node is found through the adjusted policy network.

[0147] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 5 The fifth flow chart of a service function chain deployment method provided in an embodiment of the present application.

[0148] like Figure 5 As shown, the method also includes:

[0149] S501. Using the evaluation network in the preset decision process model, determine the state value of a training round according to the state space of the service function chain in a training round.

[0150] In this embodiment, the evaluation network in the preset decision process model can be the evaluation network in the proximal strategy optimization algorithm in deep reinforcement learning. The network structure of the evaluation network also adopts a multi-layer perceptron (MLP) structure, including an input layer, several hidden layers and an output layer. The hidden layer of the evaluation network is composed of four fully connected layers, each layer has 256 neurons, and the ReLU activation function is used between the fully connected layers to increase the nonlinear factor. The input layer of the evaluation network inputs the state space at each moment in each iteration cycle, and the output layer is the state value corresponding to the state space at each moment.

[0151] By inputting the state space of the service function chain in the current time slot into the evaluation network, the state value of the current training round can be obtained.

[0152] Based on the first reward value, the second reward value and the penalty value of the multiple VNF nodes, the loss parameters of the multiple sets of strategy data are calculated, including:

[0153] S502. Calculate the expected reward value based on the first reward value, the second reward value and the penalty value of multiple VNF nodes.

[0154] Optionally, the target reward values ​​of multiple VNF nodes are calculated respectively based on the first reward values, second reward values ​​and penalty values ​​of multiple VNF nodes.

[0155] The expected value of the reward is calculated according to the target reward values ​​of multiple VNF nodes, the preset discount rates of multiple VNF nodes, and the state values.

[0156] For example, according to the first reward value of each VNF node Second reward value and penalty value Calculate the target reward value r of each VNF node k , which is expressed as follows:

[0157]

[0158] Then, according to the target reward values ​​of multiple VNF nodes, the preset discount rates of multiple VNF nodes, and the state values, the expected reward value corresponding to the current time slot is calculated. It is expressed as follows:

[0159]

[0160] There are multiple moments in the current training round, and each moment determines the pre-deployment action of a VNF node. K represents the Kth moment, γ represents the preset discount rate, and V(s k ) represents the state space s at the Kth moment k The corresponding state value, rk Represents the target reward value of the VNF node at the Kth moment.

[0161] It should be noted that after the target reward value is determined, a set of data corresponding to the current time slot is obtained, that is, a set of policy data corresponding to each VNF node. If there are K moments in the current training round, K sets of policy data are recorded and expressed as: s k represents the state space at the kth moment, a k represents the pre-deployed action at the kth moment, Indicates selecting action a from the action values ​​that obey the Gaussian distribution k The probability of k Represents the target reward value at the kth moment.

[0162] S503: Calculate advantage estimation parameters according to the expected value of the reward and the state value.

[0163] Specifically, the advantage estimation parameter The calculation expression is as follows:

[0164]

[0165] S504: Calculate target parameters of multiple sets of strategy data according to advantage estimation parameters, expected reward values, and state values.

[0166] Specifically, the objective function is expressed as follows:

[0167]

[0168] Among them, S e Expressed as the entropy of the policy model, is the policy gradient objective function, is the objective function of the evaluation network, c1 and c2 are constant coefficients used to adjust the weights of each part of the network objective function. and The function can be calculated using the following formula:

[0169]

[0170]

[0171]

[0172] in, is the ratio of the new and old strategies, ε is the cutoff constant, and the clip function is The upper and lower bounds are set. The value of is limited to the interval [1-ε, 1+ε], thereby reducing the policy update amplitude and preventing the policy from being over-updated. Finally, by maximizing the target parameter Update the network parameters θ to adjust the policy network parameters, and use the collected K groups of data to continuously update the network parameters θ n update After that, the parameter θ old Update to θ, that is, after adjusting the policy network parameters for a training round, the parameter θ old Updated to θ, which is used to save the policy data of the next preset number threshold and continue to adjust the parameters of the adjusted policy network.

[0173] In the method provided in the embodiment of the present application, an evaluation network in a preset decision process model is used to determine the state value of the current training round according to the state space of the service function chain in the current time slot; the expected reward value is calculated according to the first reward value, the second reward value and the penalty value of multiple VNF nodes; the advantage estimation parameter is calculated according to the expected reward value and the state value; the target parameters of multiple groups of policy data are calculated according to the advantage estimation parameter, the expected reward value and the state value, so that the policy network is adjusted according to the target parameters to obtain the optimal deployment position of each VNF node.

[0174] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 6 The sixth flow chart of a service function chain deployment method provided in the embodiment of the present application. Figure 6 As shown, the policy network in the preset decision process model is used, and according to the state space of the service function chain in the current time slot, the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round is determined from the action space corresponding to the policy network, and the pre-deployment action is saved. Before obtaining multiple sets of policy data, the method also includes:

[0175] S601. Digitalize the network performance of the structure topology map and the link topology map, and construct a service chain deployment problem with the joint goals of maximizing deployment success rate and minimizing resource consumption.

[0176] S602. Use the service chain deployment problem to build a preset decision process model.

[0177] In this embodiment, for service function chain deployment, the VNF nodes in the link topology map need to be matched with the physical nodes in the structure topology map to realize the deployment and path selection of the service function chain. The structure topology map and the link topology map are integrated to construct a service chain deployment problem with maximizing service rate and minimizing resource consumption as the joint optimization goals. By adopting weighted comprehensive optimization, the balanced optimization of the two key performance indicators of maximizing service rate and minimizing resource consumption is achieved through compromise.

[0178] Then, according to the constructed service chain deployment problem, a preset decision process model is constructed so that the service function chain deployed by the preset decision process model meets the requirements of maximizing service rate and minimizing resource consumption.

[0179] The present application embodiment also provides another possible implementation of the service function chain deployment method. Figure 7 FIG7 is a flow chart of a service function chain deployment method provided in an embodiment of the present application. Figure 7 As shown in the figure, the network performance is digitized according to the structure topology diagram and link topology diagram, and a service chain deployment problem with the joint goal of maximizing deployment success rate and minimizing resource consumption is constructed, including:

[0180] S701. Digitalize the network performance of the structure topology diagram and the link topology diagram to obtain the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, the deployment decision variables of the VNF node, and the mapping decision variables of the virtual link.

[0181] In this embodiment, for M service function chains SFC, define The computing resources required for instantiating the ith VNF in the rth SFC and deploying it on the physical node n are expressed as follows: The link between two VNF instances selects the shortest path for link mapping. Definition The link bandwidth required to map the link (k, l) in the rth SFC to the actual physical link (n, m) in the network topology is obtained, and the mathematical expression of the mapping bandwidth corresponding to the virtual link is obtained: Introducing deployment decision variables and mapping decision variables To represent the deployment of the rth SFC, the expression is as follows:

[0182]

[0183]

[0184] S702. Construct a deployment resource constraint expression based on the mathematical expression of the deployment resources corresponding to the VNF node and the deployment decision variables of the VNF node.

[0185] Among them, the network performance data of the structure topology diagram and link topology diagram are digitized. The network performance data mainly includes bandwidth data and network resource data. Taking the network performance data into consideration, the following constraints are proposed:

[0186] For any physical node n∈N S , you need to ensure that the resources of the instantiated VNF do not exceed the total available resources sumn , the deployment resource constraint expression is as follows:

[0187]

[0188] S703: Construct a mapping bandwidth constraint expression according to a mathematical expression of a mapping bandwidth corresponding to a virtual link and a mapping decision variable of the virtual link.

[0189] Considering the limitation of link bandwidth resources, the bandwidth required to map the virtual link (k, l) to the physical link (n, m) cannot exceed its available capacity. The mapping bandwidth constraint expression is as follows:

[0190]

[0191] S704. Based on the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, and the deployment success expression corresponding to the VNF node, a service chain deployment problem with the joint goals of maximizing the deployment success rate and minimizing the resource consumption is constructed.

[0192] In addition, considering the rationality of link mapping, it is required that each virtual link (k, l) must and can only be mapped to a continuous physical link (n, m), otherwise it will cause incomplete network and link confusion. Similarly, each VNF node must and can only be mapped to a physical node once, that is, the computing resources allocated to a certain VNF node must be completely provided by a physical node n. In addition, if a service function chain is deployed to the underlying server, if the physical node resources are insufficient to deploy to the specified VNF node, it means that the service function chain is denied service. Therefore, the deployment success expression corresponding to the VNF node is expressed as follows:

[0193]

[0194] Finally, according to the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expression of the deployment resource corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, and the deployment success expression corresponding to the VNF node, the service chain deployment problem with the maximum deployment success rate and the minimum resource consumption as the joint goal is constructed, which is expressed as follows:

[0195]

[0196] Among them, A1 and A2 are constant weight functions.

[0197] In the method provided in the embodiment of the present application, the network performance of the structure topology map and the link topology map is digitized to obtain mathematical expressions of deployment resources corresponding to VNF nodes, mathematical expressions of mapping bandwidth corresponding to virtual links, deployment decision variables of VNF nodes, and mapping decision variables of virtual links; according to the mathematical expressions of deployment resources corresponding to VNF nodes and the deployment decision variables of VNF nodes, a deployment resource constraint expression is constructed; according to the mathematical expressions of mapping bandwidth corresponding to virtual links and the mapping decision variables of virtual links, a mapping bandwidth constraint expression is constructed; according to the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expressions of deployment resources corresponding to VNF nodes, the mathematical expressions of mapping bandwidth corresponding to virtual links, and the deployment success expression corresponding to VNF nodes, a service chain deployment problem with maximum deployment success rate and minimum resource consumption as joint goals is constructed, so that the deployed service function chain meets the requirements of maximum deployment success rate and minimum resource consumption.

[0198] The following is a corresponding explanation of the service function chain deployment device and electronic device provided by any of the above embodiments of the present application. The specific implementation process and the technical effects produced are the same as those of the corresponding method embodiments mentioned above. For the sake of brief description, the parts not mentioned in this embodiment can refer to the corresponding contents in the method embodiments.

[0199] Figure 8 A schematic diagram of the functional modules of a service function chain deployment device provided in an embodiment of the present application. Figure 8 As shown, the service function chain deployment device 100 includes:

[0200] The acquisition module 110 is used to acquire the state space of the service function chain in the current time slot according to the information of the preset physical network environment and the information of the service function chain to be deployed, wherein the state space includes: network feature information and chain feature information;

[0201] The determination module 120 is used to adopt the policy network in the preset decision process model, determine the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and save the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are part of the VNF nodes of the service function chain;

[0202] A parameter adjustment module 130, configured to adjust the parameters of the policy network according to the multiple sets of policy data if the number of the multiple sets of policy data reaches a preset number threshold;

[0203] The determination module 120 is further used to determine the pre-deployment actions of the remaining VNF nodes in the service function chain using the adjusted parameter policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined and the training of the current training round is completed;

[0204] The determination module 120 is further used to reacquire the state space corresponding to the service function chain, perform training for the next training round, and complete the training for the preset training round to determine the pre-deployment action of each VNF node in the service function chain within the preset training round;

[0205] The determination module 120 is also used to determine the pre-deployment action of each VNF node in a preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

[0206] Optionally, the determination module 120 is also used to determine the prediction results of each VNF node in the service function chain in the current training round from the action space corresponding to the policy network, and the prediction results include: multiple predicted action values, and the multiple predicted action values ​​are used to characterize the predicted probability values ​​of each VNF node being deployed on multiple physical nodes in a preset physical network environment; based on the multiple predicted action values, determine the pre-deployment action of each VNF node.

[0207] Optionally, the acquisition module 110 is also used to obtain a structural topology map of a preset physical network environment, wherein the structural topology map records information of multiple physical nodes and information of physical links between physical nodes, and the information of physical nodes at least includes: computing resource information of physical nodes, and the information of physical links at least includes: bandwidth information of physical links; obtain a link topology map of the service function chain, wherein the link topology map records information of all VNF nodes in the service function chain and information of virtual links between VNF nodes; obtain network feature information and chain feature information for the service function chain in the current time slot according to the structural topology map and the link topology map respectively; obtain the state space of the service function chain in the current time slot according to the network feature information and chain feature information of the current time slot.

[0208] Optionally, the parameter adjustment module 130 is also used to calculate the first reward value of each VNF node according to the pre-deployment results of the pre-deployment actions of each VNF node in the multiple VNF nodes corresponding to the multiple sets of policy data under the pre-deployment behavior; calculate the second reward value of each VNF node according to the resource consumption information under the pre-deployment behavior of the pre-deployment actions of each VNF node; calculate the penalty value of each VNF node according to the pre-deployment physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node, wherein the penalty value of each VNF node is used to characterize the penalty value of each VNF node and the previous adjacent VNF ​​node mapped to the same deployment node; calculate the target parameters of the multiple sets of policy data according to the first reward value, the second reward value and the penalty value of the multiple VNF nodes, and adjust the parameters of the policy network according to the target parameters.

[0209] Optionally, the determination module 120 is further configured to use an evaluation network in a preset decision process model to determine the state value of the current training round according to the state space of the service function chain in the current time slot;

[0210] Optionally, the device further comprises:

[0211] The calculation module is used to calculate the expected reward value according to the first reward value, the second reward value and the penalty value of multiple VNF nodes; calculate the advantage estimation parameter according to the expected reward value and the state value; calculate the target parameters of multiple sets of strategy data according to the advantage estimation parameter, the expected reward value and the state value.

[0212] Optionally, the calculation module is also used to calculate the target reward values ​​of multiple VNF nodes respectively according to the first reward values, second reward values ​​and penalty values ​​of multiple VNF nodes; and calculate the expected value of the reward according to the target reward values ​​of multiple VNF nodes, the preset discount rates and state values ​​of multiple VNF nodes.

[0213] Optionally, the device further comprises:

[0214] The construction module is used to digitize the network performance of the structure topology map and the link topology map, and to construct a service chain deployment problem with the joint goals of maximizing the deployment success rate and minimizing resource consumption; the service chain deployment problem is used to construct a preset decision process model.

[0215] Optionally, the construction module is also used to digitize the network performance of the structural topology map and the link topology map, and obtain the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, the deployment decision variables of the VNF node, and the mapping decision variables of the virtual link; construct a deployment resource constraint expression based on the mathematical expression of the deployment resources corresponding to the VNF node and the deployment decision variables of the VNF node; construct a mapping bandwidth constraint expression based on the mathematical expressions of the mapping bandwidths corresponding to multiple virtual links and the mapping decision variables of the virtual links; construct a service chain deployment problem with the joint goals of maximizing deployment success rate and minimizing resource consumption based on the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, and the deployment success expression corresponding to the VNF node.

[0216] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0217] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0218] Fig. 9 This is a schematic diagram of an electronic device provided in an embodiment of the present application, which can be used for service function chain deployment. Fig. 9 As shown, the electronic device 200 includes: a processor 210 , a storage medium 220 , and a bus 230 .

[0219] The storage medium 220 stores machine-readable instructions executable by the processor 210. When the electronic device is running, the processor 210 communicates with the storage medium 220 via the bus 230, and the processor 210 executes the machine-readable instructions to perform the steps of the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0220] Optionally, the present application further provides a storage medium 220, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method embodiment are executed. The specific implementation method and technical effect are similar and will not be repeated here.

[0221] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0222] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0223] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0224] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0225] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A service function chain deployment method, characterized in that: The method comprises: According to the information of the preset physical network environment and the information of the service function chain to be deployed, a state space for the service function chain in the current time slot is obtained, wherein the state space includes: network feature information and chain feature information; Adopting the policy network in the preset decision process model, according to the state space of the service function chain in the current time slot, determining the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network, and saving the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are some VNF nodes of the service function chain; If the number of the multiple sets of policy data reaches a preset number threshold, adjusting the policy network according to the multiple sets of policy data; Using the adjusted parameter policy network, the pre-deployment actions of the remaining VNF nodes in the service function chain are determined until the pre-deployment actions of all VNF nodes in the service function chain are determined, and the training of the current training round is completed; Reacquire the state space corresponding to the service function chain, perform training for the next training round, and complete the training for the preset training round, and determine the pre-deployment action of each VNF node in the service function chain within the preset training round; Determine the pre-deployment action of each VNF node in the preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

2. The method according to claim 1, characterized in that The determining, from the action space corresponding to the policy network, the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round includes: Determine, from the action space corresponding to the policy network, a prediction result of each VNF node in the service function chain in the current training round, the prediction result comprising: a plurality of prediction action values, the plurality of prediction action values ​​being used to characterize a prediction probability value of each VNF node being deployed on a plurality of physical nodes in the preset physical network environment; Based on the multiple predicted action values, determine the pre-deployment action of each VNF node.

3. The method according to claim 1, characterized in that The acquiring, according to the information of the preset physical network environment and the information of the service function chain to be deployed, a state space of the service function chain in the current time slot includes: Acquire a structural topology diagram of the preset physical network environment, wherein the structural topology diagram records information of a plurality of physical nodes and information of physical links between the physical nodes, the information of the physical nodes at least includes: computing resource information of the physical nodes, and the information of the physical links at least includes: bandwidth information of the physical links; Obtain a link topology map of the service function chain, wherein the link topology map records information of all VNF nodes in the service function chain and information of virtual links between VNF nodes; According to the structure topology map and the link topology map, respectively obtain network feature information and link feature information for the service function chain in the current time slot; According to the network characteristic information and the chain characteristic information of the current time slot, the state space of the service function chain in the current time slot is acquired.

4. The method according to claim 1, characterized in that: If the number of the plurality of sets of policy data reaches a preset number threshold, adjusting parameters of the policy network according to the plurality of sets of policy data includes: Calculate the first reward value of each VNF node according to the pre-deployment result of the pre-deployment action of each VNF node in the multiple VNF nodes corresponding to the multiple groups of policy data; Calculate the second reward value of each VNF node respectively according to the resource consumption information of the pre-deployment action of each VNF node under the pre-deployment behavior; According to the pre-deployment physical node corresponding to the pre-deployment action of each VNF node and the actual deployment node of the previous adjacent VNF ​​node, the penalty value of each VNF node is calculated respectively, wherein the penalty value of each VNF node is used to characterize the penalty value of each VNF node and the previous adjacent VNF ​​node mapped to the same deployment node; According to the first reward value, the second reward value and the penalty value of the multiple VNF nodes, the target parameters of the multiple groups of policy data are calculated, and the policy network is adjusted according to the target parameters.

5. The method according to claim 4, characterized in that The method further comprises: Using the evaluation network in the preset decision process model, determining the state value of the current training round according to the state space of the service function chain in the current time slot; The calculating the loss parameters of the multiple sets of policy data according to the first reward values, the second reward values, and the penalty values ​​of the multiple VNF nodes includes: Calculate the expected reward value according to the first reward value, the second reward value and the penalty value of the plurality of VNF nodes; Calculating an advantage estimation parameter according to the expected value of the reward and the state value; Calculate target parameters of the multiple sets of strategy data according to the advantage estimation parameter, the expected value of the reward, and the state value.

6. The method according to claim 5, characterized in that The step of calculating the expected reward value according to the first reward value, the second reward value and the penalty value of the plurality of VNF nodes comprises: Calculate target reward values ​​of the multiple VNF nodes respectively according to the first reward values, the second reward values ​​and the penalty values ​​of the multiple VNF nodes; The expected reward value is calculated based on the target reward values ​​of the multiple VNF nodes, the preset discount rates of the multiple VNF nodes and the state values.

7. The method according to claim 3, characterized in that Before the method adopts the policy network in the preset decision process model, determines the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and saves the pre-deployment action to obtain multiple sets of policy data, the method also includes: The network performance of the structure topology map and the link topology map is digitized to construct a service chain deployment problem with the joint goal of maximizing deployment success rate and minimizing resource consumption; The service chain deployment problem is adopted to construct the preset decision process model.

8. The method according to claim 7, characterized in that The network performance data is digitized according to the structure topology map and the link topology map to construct a service chain deployment problem with maximum deployment success rate and minimum resource consumption as joint goals, including: Performing network performance digitization on the structure topology map and the link topology map to obtain mathematical expressions of deployment resources corresponding to VNF nodes, mathematical expressions of mapping bandwidths corresponding to virtual links, deployment decision variables of the VNF nodes, and mapping decision variables of the virtual links; Constructing a deployment resource constraint expression according to a mathematical expression of deployment resources corresponding to the VNF node and a deployment decision variable of the VNF node; Constructing a mapping bandwidth constraint expression according to a mathematical expression of a mapping bandwidth corresponding to the virtual link and a mapping decision variable of the virtual link; According to the deployment resource constraint expression, the mapping bandwidth constraint expression, the mathematical expression of the deployment resources corresponding to the VNF node, the mathematical expression of the mapping bandwidth corresponding to the virtual link, and the deployment success expression corresponding to the VNF node, a service chain deployment problem with maximum deployment success rate and minimization of resource consumption as the joint goals is constructed.

9. A service function chain deployment device, characterized in that: The device comprises: An acquisition module, configured to acquire a state space for the service function chain in a current time slot according to information of a preset physical network environment and information of a service function chain to be deployed, wherein the state space includes network feature information and chain feature information; A determination module is used to adopt a policy network in a preset decision process model, determine the pre-deployment action of each virtual network function VNF node in the service function chain in the current training round from the action space corresponding to the policy network according to the state space of the service function chain in the current time slot, and save the pre-deployment action to obtain multiple groups of policy data; the multiple groups of policy data correspond to multiple VNF nodes respectively, and the multiple VNF nodes are some VNF nodes of the service function chain; A parameter adjustment module, configured to adjust the parameters of the policy network according to the multiple sets of policy data if the number of the multiple sets of policy data reaches a preset number threshold; The determination module is further used to determine the pre-deployment actions of the remaining VNF nodes in the service function chain using the adjusted parameter policy network until the pre-deployment actions of all VNF nodes in the service function chain are determined to be completed, thereby completing the training of the current training round; The determination module is further used to reacquire the state space corresponding to the service function chain, perform training for the next training round, and complete the training for the preset training round, and determine the pre-deployment action of each VNF node in the service function chain within the preset training round; The determination module is also used to determine the pre-deployment action of each VNF node in the preset training round as the target deployment action, and deploy each VNF node to the target physical node corresponding to the target deployment action.

10. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the service function chain deployment method as described in any one of claims 1 to 8.

Images