A service function chain mapping resource intelligent allocation method suitable for a universal platform

Abstract

The application provides a service function chain mapping resource intelligent allocation method suitable for a generalization platform, steps are as follows: an SFC mapping model is established, the SFC mapping process in the generalization platform is divided into two-layer structures of a physical topology resource layer and a virtual network function request layer, and abstract parameter representation is carried out on the two-layer structures; a processing process of a service request in the generalization platform is analyzed, and a minimum mathematical model of processing time of joint allocation of node calculation and link bandwidth communication resources is established; the minimum mathematical model is modeled as a Markov process, and a three-tuple containing a state, an action and a reward is defined; the three-tuple is combined with an asynchronous advantage action evaluation algorithm, a master network is trained as a template, a plurality of sub-networks are generated by using a multi-thread technology for parallel training, and an optimal resource allocation strategy is found. The node calculation resources and the link bandwidth communication resources are intelligently allocated, and the processing rate of the SFC request is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile communication, and particularly relates to a service function chain mapping resource intelligent allocation method suitable for a universal platform. BACKGROUND

[0002] With the rapid development of communication network technology, the number of network users is increasing year by year, and the network function service business required by users is becoming more and more diversified. In the traditional network architecture, there is strong coupling between network functions and special hardware devices, which cannot provide flexible and customizable network services for users. Network function virtualization (Network Functions Virtualization, NFV) virtualizes network functions into virtual network functions (Virtualized Network Function, VNF) and deploys them on a universal hardware platform, and provides customized network services for users in the form of a service function chain (Service Function Chain, SFC), providing a new network function service business processing system and improving the flexibility and scalability of network services. In NFV, SFC refers to a logical chain service request for processing network services according to the logical order of VNF in the service request (SFC Request, SFCR) issued by the user, guiding network traffic to pass through network links in sequence to the corresponding network function node for processing. Among them, SFC mapping is a key process for completing SFCR processing, which refers to the process of instantiating VNF contained in SFCR in a universal network platform and making traffic pass through corresponding network functions in sequence. In the SFC mapping process, the main network resources involved are node computing resources and link bandwidth communication resources, and how much of the two resources is allocated on the SFCR fundamentally affects the response efficiency of the SFCR. However, the existing mapping resource intelligent allocation method mostly only optimizes the allocation of single node computing resources or link bandwidth communication resources, which is easy to cause low utilization of the other resource and resource allocation imbalance problem, and due to the complexity of SFC mapping action, there is a slow convergence efficiency problem. Therefore, it is necessary to design a SFC mapping resource intelligent allocation method that optimizes the joint node computing resources and link bandwidth communication resources in a universal platform, reasonably allocates computing and communication related resources, and speeds up the convergence efficiency of the algorithm, so as to improve the SFC mapping rate and resource utilization, speed up the processing efficiency of SFCR, and reduce the total processing time of processing SFC.

[0003] The application patent with the application number 201911399761.8 discloses a service function chain low-cost intelligent deployment method based on environment perception. The steps are as follows: taking the wireless rate accessible to the user wireless end as the basis for SFC resource allocation, jointly considering SFC deployment and wireless subcarrier allocation, and establishing an SFC deployment cost minimization model under the user delay requirement, wireless rate demand and resource constraint; the above optimization problem is converted into a Markov decision process with continuous state space and high-dimensional action space under discrete time; finally, considering the high dimensionality of the MDP problem, the joint optimization strategy of SFC deployment and subcarrier allocation is obtained by using the DDPG reinforcement algorithm, so as to reduce resource consumption and reduce the cost of SFC deployment. This method can effectively reduce the total cost of SFC deployment while guaranteeing the user delay requirement and wireless rate demand; and can make the subcarrier allocation and SFC deployment strategy with the lowest deployment cost according to the current environment at each time slot. The above method allocates corresponding computing and virtual link bandwidth resources to SFC according to the SFC flow rate, which can reduce the resource consumption of the core network and reduce the SFC deployment cost. However, this also means that the SFC with low flow rate can allocate less resources, resulting in long waiting service time for part of the users and reducing the user service quality. Moreover, the DDPG used belongs to the deterministic policy gradient algorithm, which has the characteristics that the result feedback obtained by each training is better than the last result, but the deterministic policy means that the state-action policy space cannot be well explored, which is not conducive to the exploration of the virtual function mapping action policy when facing real-time service requests, and it is easy to fall into local policy optimization. SUMMARY

[0004] In view of the problems of imbalance between node computing resources and link bandwidth communication resources in service function chain mapping in a universal platform, resource waste or resource shortage, inability to guarantee service quality, and inability to support new business demands, and the slow convergence efficiency of the existing allocation method due to the complexity of resource allocation actions in the SFC mapping process, the present application proposes a service function chain mapping resource intelligent allocation method suitable for a universal platform (referred to as SA3C), which converts the mapping resource allocation process into a Markov decision model, takes node resource allocation and link resource allocation as action strategies, takes the total processing time as the reward value, and based on the asynchronous training method in the Asynchronous Advantage Actor-Critic (A3C) algorithm, uses the training master network as a template to generate multiple sub-networks for parallel training, thereby effectively accelerating the convergence efficiency of the algorithm.

[0005] To achieve the above purpose, the technical scheme of the present application is as follows: a service function chain mapping resource intelligent allocation method suitable for a universal platform, the steps of which are as follows:

[0006] Step 1: Establish an SFC mapping model, dividing the SFC mapping process in the generalized platform into a two-layer structure of physical topology resource layer and virtual network function request layer, and abstracting the parameters of the two-layer structure.

[0007] Step 2: Analyze the service request processing process in the generalized platform and establish a mathematical model that minimizes the processing time of the joint allocation of node computing and link bandwidth communication resources;

[0008] Step 3: Model the minimized mathematical model as a Markov process, defining a triple containing state, action, and reward;

[0009] Step 4: Combine the Markov process with the graph convolutional neural network to optimize the triples and transform the high-dimensional adjacency information of the state into a low-dimensional vector representation.

[0010] Step 5: Combine the optimized triplet from Step 4 with the asynchronous dominant action evaluation algorithm. Based on the asynchronous training method in the asynchronous dominant action evaluation algorithm, use the main training network as a template and use multi-threading technology to generate multiple sub-networks for parallel training to find the optimal resource allocation strategy.

[0011] Preferably, the physical topology resource layer is represented by an undirected graph G = {N, L}, where N = {n1, n2... n}. m Let} represent the set of physical service nodes, m represent the maximum number of physical service nodes, and n1, n2...n m There are m physical service nodes; L = {l i,j =(n i ,n j The set of physical links is represented by |i,j≤m}. i,j For the i-th physical service node n i With the j-th physical service node n j The physical link between them; the available computing resources of the physical service nodes C = [c1, c2...c m ], where c i For physical service node n i Available computing resources; physical link bandwidth communication resources B = {b i,j |i,j≤m},b i,j For physical service node n i With physical service node n j Physical link bandwidth communication resources between them, if b i,j =0 indicates that the available link bandwidth communication resources are 0 or that there is no physical link connection between the two points;

[0012] The virtual network function request is a set of user service request SFCs = {f1, f2, ...}, where each f represents a specific SFC request. Each SFC request is represented by a quadruple f =<V f E f Data f ,P f > indicates that V f ={v1,v2...v v} represents the virtual network function request sequence, v represents the virtual network function requested by the user, and E f ={e1,e2,...e e} represents the set of virtual links, where e u ={v p ,v q} represents virtual network function v p and virtual network function v q Virtual links between, Data f P represents the size of the data to be transmitted in the SFC request f. f ={p1,p2...p5} represents the set of processing priorities for SFC request f, with priority p1 being the lowest and priority p5 being the highest.

[0013] Preferably, the minimized mathematical model is:

[0014]

[0015] Where C1 represents the minimum total processing time for processing the set of virtual network function requests, indicating that each VNF must be mapped to a physical service node; C2 represents that the computing resources allocated to a VNF when mapped to a physical service node are limited by the capacity of the available resources of that physical node; C3 represents that the bandwidth resources allocated to a virtual link when mapped to a physical link are limited by the capacity of the available bandwidth resources of that physical link; where I is an identity matrix of all 1s, |N| represents the sequence of physical service nodes, and |V f | represents the sequence of virtual network function requests for the f-th SFC request. This indicates the mapping status of the f-th SFC request in time slot t on the physical service node, D. f Let C(t) be the total processing time of the node for the f-th SFC request, C(t) be the binary matrix formed by the virtual network function mapping, and B(t) be the binary matrix formed by the virtual link mapping. Mapping the i-th virtual function in the f-th SFC request to the physical service node n within time slot t in the physical topology. j The mapping state, For the physical service node that is successfully mapped at time slot t, c is allocated node computing resources. jData j (t) represents the node computing resource available for the current jth physical service node, Data e,f (t) represents the successful mapping of the e th virtual link in the f th SFC to the physical service node n in the physical topology i Data n (t) represents the link mapping state between the physical service node n j and the physical link (n i , n j ) between them, b f (t) represents the link bandwidth communication resource allocated for the successfully mapped physical link at time slot t, i,j Data n (t) represents the available bandwidth communication resource of the physical link.

[0016] Preferably, the total processing time D f requested by the f th SFC during the mapping process is the sum of the physical service node processing total time f Data n (t) represents the available bandwidth communication resource of the physical link. and the link processing total time Data n (t) represents the available bandwidth communication resource of the physical link.

[0017] The physical service node total processing time of the f th SFC request is:

[0018] wherein v c represents the virtual network function contained in the f th SFC request, N f represents the sequence of physical service nodes to which the virtual function in the f th SFC request is mapped, and the time the f th SFC request stays in a physical node

[0019] The processing rate of the f th SFC request in the physical service node at time slot t is proportional to the computing resource allocated for the physical node, and:

[0020] wherein, is the conversion factor, and Data n (t) represents the node computing resource allocated for the successfully mapped physical service node at time slot t, and c c represents the node computing resource available for the physical service node; Data f represents the transmission data size of the SFC request f;

[0021] The link processing total processing time of the f th SFC request is:

[0022] wherein the time the f th SFC request stays in a physical link

[0023] The transmission rate of the f th SFC request in the physical link at time slot t proportional to the bandwidth resource allocated to the physical link, and

[0024] wherein η is a conversion factor, and η > 1, is the link bandwidth communication resource allocated to the physical link successfully mapped at time slot t, and b p,q represents the available bandwidth communication resource of the physical link.

[0025] Preferably, the entire process of processing the SFC request is divided into several time slots according to processing time, and the set of time slots is represented by T = {1, 2,... t}, and one time slot t is the SFC mapping processing period; in the time slot t when the virtual function is mapped into the physical topology, the mapping state of the service node is represented by , and when , it represents that the i-th virtual network function v i in the f-th SFC request is successfully mapped into the j-th physical service node in the physical topology, and if the mapping fails, when the virtual network function in the f-th SFC request is mapped, the binary matrix formed by the virtual network function mapping is represented by , and the link mapping state is represented by , and when , it represents that the e-th virtual link in the f-th SFC request is successfully mapped on the physical link (n p , n q ) between the physical service nodes n p and n q , and if the mapping fails, the mapping state is when the virtual link in the f-th SFC request is mapped, the binary matrix formed by the virtual link mapping is represented by .

[0026] Preferably, in the mapping process, the priority p i also affects the order of mapping, and the priority of each SFC request is p3 by default if no setting is made, and the higher the priority, the greater the proportion of the allocated computing resources and link bandwidth communication resources, and the shorter the total processing time in the case of the same data volume.

[0027] Preferably, the Markov process is a triple M = <S, A, R>, wherein S is a finite state space, A is a finite action space, and R is a reward space;

[0028] The state space S is composed of the system state in the physical topology formed by the mapping of each SFC request into the universal platform at time slot t, so the system state of the universal platform at time slot t is s t={s1(t),s2(t)...s SFCs (t)};

[0029] Among them, state This represents the physical topology resource information at time slot t. The x-th SFC request is characterized by |SFCs|; |SFCs| represents the number of SFC requests at time slot t.

[0030] The action space A represents the allocation method of various resources during mapping of SFC requests in the current generalized platform system state. The action of time slot t is: a t ={a c (t),a b (t)};

[0031] Among them, a c (t) represents the allocation method of computing resources for physical nodes, a b (t) represents the allocation method of physical link bandwidth communication resources;

[0032] When in time slot t, based on state s t Take action a at that time t It will enter the next state s t+1 And receive an instant reward r t The instantaneous rewards for each time slot t constitute the reward space R = (r1, r2, r3...), where r1, r2, r3... represent the instantaneous feedback reward values ​​obtained by actions a1, a2, and a3 taken according to the corresponding states s1, s2, and s3 in the 1st, 2nd, and 3rd time periods, respectively; the negative of the processing time is set as the reward function.

[0033] Preferably, the method for optimizing triples by combining Markov processes with graph convolutional neural networks is as follows: obtain the generalized platform system state s of time slot t. t ={s1(t),s2(t)...s SFCs (t)}, the status of the xth SFC request Physical topology resource layer There are |N m | Node x, SFC request Functional sequence | V x |Form a|N m |*|V x |dimensional feature matrix X, forming an |N m |*|N m The adjacency matrix A is used to perform forward propagation of the ReLU activation function and the softmax normalization function through two layers of graph convolution operations: in It is a metric matrix; I is the identity matrix. It is a symmetric matrix, W S It is a weight parameter matrix in a finite state space;

[0034] The finite action space a t ={a c (t),a b Combining (t)} with a graph convolutional neural network, the high-dimensional action space is represented by a low-dimensional vector space, and the modified formula for the finite action set is: Where a wi =wi(a c ,a b ), |V x | represents the number of functions in the function sequence of the x-th SFC request, W A It is a weight parameter matrix with a finite action space.

[0035] Preferably, the implementation method of step five is as follows: a single SFC request is used as an agent trained in a sub-network. The parameter values ​​of the optimal resource allocation strategy are obtained in the sub-network and synchronized with the main network parameters through differential updates; the sub-network interacts with the physical topology resource layer to learn and update the sub-network parameters θ. μ′ With θ Q′ And using differential updates to update the parameters θ of the main network, the policy network, and the value network. μ With θ Q Perform synchronization;

[0036] Define the set of subnetworks as M, and use parameter vectors. Generate an action selection policy function π(s) based on state s. t Select mapping action a t Using parameter vectors Generate a state-value function V(s) to evaluate the current state s. t The advantages and disadvantages; for the network architecture of the asynchronous dominant action evaluation algorithm, both the main network and the sub-network maintain their own network's random action strategy π(s) = π(s) t ,a t |θ μ ) and state value function V(s t )=V(s t |θ Q ).

[0037] Preferably, the differential update method is implemented as follows:

[0038] Define strategy π(s) tState value function

[0039] Where γ∈(0,1) is the discount factor. Represents the action policy π(s) t The mathematical expectation of V(s) t+1 ) represents state s t+1 State value function;

[0040] Define the action policy π(s) t The action-value function Q(s) t ,a t ), used to evaluate the magnitude of the value corresponding to the current action relative to the average value, and:

[0041] The parameters of the policy network and value network in the main network are θ μ With θ Q This indicates that the parameters of the policy network and value network in the sub-network are expressed as θ. μ′ With θ Q′ This indicates that the parameter θ in the policy network Actor... μ The update follows the policy gradient as follows:

[0042]

[0043] Where δ is the entropy hyperparameter, which is set relatively large at the beginning and gradually decreases during the training phase; H() represents the entropy of the policy; and dθ μ The parameter θ represents the policy network. μ The differential, The parameter θ represents the policy network. μ The total differential, Represents the policy network parameters θ μ′ The total differential, π(s) t |θ μ′ ) indicates that the state is s t The policy network parameter is θ μ′ The action policy function generated at that time, dθ μ′ Represents the subnetwork parameters θ μ′ The derivative of α represents the learning rate of the policy parameter;

[0044] Parameter θ in the value network Critic Q The update follows the policy gradient:

[0045]

[0046] Where β represents the learning rate of the value parameter, dθ Q The parameter θ represents the value network. Q The differential, dθ Q′Represents the value network parameters θ Q′ The differential, V(s) t |θ Q′ ) represents state s t Value network parameters θ Q′ State value function at time The parameter θ represents the value network. Q′ The partial derivatives of .

[0047] Compared with existing technologies, the beneficial effects of this method are:

[0048] (1) This invention analyzes the relationship between resource allocation and service processing efficiency during SFC mapping in a generalized platform and expresses it in a formula, providing theoretical support for formulating the optimal allocation formula to allocate node computing resources and link bandwidth communication resources more reasonably; and models the resource allocation of the SFC mapping process as a Markov decision process, introduces the A3C method to accelerate the convergence efficiency of asynchronous training, effectively improves the SFC mapping rate and resource utilization, reduces the total processing time of SFC, and maps more SFCs in the limited resources of the generalized platform; in parallel training, each SFC request is treated as an agent for training a sub-network to find the optimal resource allocation strategy and synchronizes with the main network parameters through differential updates.

[0049] (2) This invention combines the finite state space in the triplet of the Markov decision process with a graph convolutional neural network, and represents the high-dimensional adjacency information of the finite state space as a low-dimensional vector, which effectively reduces the complexity of the input finite state space and improves the processing efficiency of the method.

[0050] (3) By intelligently allocating node computing resources and link bandwidth communication resources, this invention effectively improves the processing rate of SFC requests. Compared with the Actor-Critic (AC) method and the Policy Gradient (PG) method, it improves the mapping rate by at least 6.72%, the resource utilization rate by 9.85%, and reduces the total processing time by at least 10.72%. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 The SFC mapping model diagram provided for an example of the present invention.

[0053] Figure 2 A diagram of the SFC mapping resource allocation model based on A3C is provided for an example of the present invention.

[0054] Figure 3 This is a comparison chart of mapping rates provided by the present invention.

[0055] Figure 4 A comparison chart of total processing time provided for this invention.

[0056] Figure 5 Resource utilization comparison chart provided for this invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] In SFC mapping within a generalized platform, the main network resources involved are node computing resources and link bandwidth communication resources. The allocation of these two resources to the SFCR fundamentally affects the platform's response efficiency to user services. However, most existing machine learning methods only optimize the allocation of single node computing resources or link bandwidth communication resources, easily leading to low utilization of the other resource and resource allocation imbalance. Furthermore, due to the complexity of SFC mapping, convergence efficiency is slow. Therefore, a network resource allocation method for generalized SFC platforms needs to be designed that can jointly optimize node computing resources and link bandwidth communication resources and accelerate algorithm convergence efficiency. This invention proposes an intelligent allocation method for service function chain mapping resources applicable to generalized platforms. The specific workflow steps are as follows:

[0059] Step 1: Establish an SFC mapping model, dividing the SFC mapping process in the generalized platform into a two-layer structure of physical topology resource layer and virtual network function request layer, and abstracting the parameters of the two-layer structure.

[0060] The SFC mapping model diagram is shown below. Figure 1 As shown, the specific content is as follows:

[0061] The physical topology resource layer can be represented by an undirected graph G = {N, L}, where N = {n1, n2, ... n}. m} represents the set of physical service nodes, m represents the maximum number of physical service nodes, and it consists of general-purpose high-performance servers used to receive, process, and encapsulate user network service requests, n1, n2...n m There are m physical service nodes; L = {l i,j =(ni ,n j The set of physical links (i,j≤m) represents the set of physical links that can transmit user network service requests to various physical service nodes. i,j For the i-th physical service node n i With the j-th physical service node n j The physical link between them. Each physical service node can deploy multiple VNF instances to provide network services to users. Different VNF ​​instances can fulfill different network service requirements, and VNFs can be used. i ={vnf1,vnf2...} represents the sequence of network functions that a physical service node can provide, used for comparison when mapping virtual functions to physical service nodes; and the available computing resources of the physical service node are represented by C = [c1,c2...c...]. m ] indicates that c i For physical service node n i Available computing resources. Physical link bandwidth communication resources are represented by B = {b}. i,j |i,j≤m} represents the physical service node n i With physical service node n j Physical link bandwidth communication resources between them are used by b i,j This means that if b i,j =0 indicates that the available link bandwidth communication resources are 0 or that there is no physical link connection between the two points.

[0062] The set of virtual network function requests is represented by SFCs = {f1, f2, ...}, which is the set of user service requests. Each f represents a specific SFC request, and can be represented by a quadruple f =<V f E f Data f ,P f > indicates that V f ={v1,v2...v v} represents the virtual network function request sequence, v represents the virtual network function requested by the user, and E f ={e1,e2,...e i} represents the set of virtual links, where e u ={v p ,v q} represents virtual network function v p and virtual network function v q The virtual link between them represents the direction of data flow or business flow. f P represents the amount of data to be transmitted in the SFC request f of this service. f={p1,p2...p5} represents the set of processing priorities for SFC request f, with p1 being the lowest priority and p5 being the highest.

[0063] Step 2: Analyze the service request processing process in the generalized platform and establish a mathematical model to minimize the processing time of the joint allocation of node computing and link bandwidth communication resources.

[0064] The specific implementation method is as follows: The entire process of handling service requests is divided into several time slots according to the processing time, denoted by T = {1, 2, ... t}, with one time slot t as the SFC mapping processing cycle. Within the time slot t that maps virtual functions to the physical topology, using... Indicates the service node mapping status, when The time indicates the i-th virtual network function v in the f-th SFC request. i Successfully mapped to the j-th physical service node in the physical topology; if mapping fails, then... Once all virtual network functions in the f-th SFC request have been mapped, they will be available. This represents a binary matrix formed by mapping its virtual network functions. (Used...) This indicates the link mapping status, when When the e-th virtual link in the f-th SFC request is successfully mapped to the physical service node n in the physical topology, it indicates that the request has been successfully mapped. p With physical service node n q Physical links between (n) p ,n q If mapping fails, then the mapping state... After the virtual links in the f-th SFC request are all mapped, they are available. This represents the binary matrix formed by its virtual link mapping. During the mapping process, the processing rate of the f-th SFC request in the physical service node at time slot t is proportional to the computing resources allocated to that physical node, which can be described by formula (1):

[0065]

[0066] in, As the transformation factor, and For the physical service node that is successfully mapped at time slot t, the node computing resources allocated to it are determined, and c c This represents the remaining computing resources of the physical service node. Therefore, the time that the f-th SFC request stays on this physical node can be expressed by formula (2):

[0067]

[0068] Therefore, the total processing time for the node making the f-th SFC request is:

[0069]

[0070] Among them, v c N represents the virtual network functionality contained in the f-th SFC request. f This represents the sequence of physical service nodes mapped to the virtual function in the f-th SFC request.

[0071] Similarly, the transmission rate of the physical link in time slot t for the f-th SFC request. It is proportional to the bandwidth resources allocated to the physical link, and can be described by formula (4):

[0072]

[0073] Where η is the transformation factor, and η > 1. For a physical link that is successfully mapped at time slot t, the allocated link bandwidth communication resources are determined, and b p,q This represents the remaining bandwidth communication resources of the physical link. Therefore, the time that the f-th SFC request stays on this physical link can be expressed by formula (5):

[0074]

[0075] Therefore, the total processing time for the f-th SFC request is:

[0076]

[0077] Therefore, the total processing time D for the f-th SFC request is f Total processing time for physical service nodes Sum of total link processing time It can be expressed by formula (7).

[0078]

[0079] During the mapping process, priority p i This will also affect the order of mapping. If no priority is set for each SFC request, the default priority is p3. The higher the priority, the greater the proportion of computing resources and link bandwidth communication resources allocated, and the shorter the total processing time for the same amount of data.

[0080] In summary, the generalized platform SFC resource allocation problem can be divided into a mathematical model that minimizes the processing time of joint allocation of node resources and link bandwidth communication resources, and satisfies the following constraints:

[0081]

[0082] Where C1 is the minimum processing time for the entire set of virtual network function requests, and is also the optimization objective, it represents that each VNF must be mapped to a physical service node, where I is an identity matrix of all 1s, |N| represents the sequence of physical service nodes, and |V f | represents the sequence of virtual network function requests (VNFs) for the f-th SFC request; C2 indicates that the computing resources allocated to the VNF when mapped to a physical service node are limited by the capacity of the available resources on that physical node. j C1 represents the available computing resources of the j-th physical service node; C2 represents the bandwidth resources allocated to the virtual link when it is mapped to the physical link, which are limited by the capacity of the available bandwidth resources of the physical link.

[0083] Step 3: Model the minimization mathematical model of the generalized platform service function chain mapping resource allocation process as a Markov process, and define a triple containing state, action, and reward.

[0084] Specifically, the objective of this invention is to map a greater number of SFC requests within the limited resources of a general-purpose platform. Based on the analysis of the impact of node computing resources and link bandwidth communication resources on the SFC request processing rate in step one, the processing time of SFC is adjusted by allocating physical service node computing resources and link bandwidth communication resources to SFC requests, thereby improving SFC processing efficiency. The mapping of the environment state during the above resource allocation process is a dynamically changing process and can be modeled as a Markov Decision Process (MDP). The MDP modeled from this resource allocation process can be defined as a triple M = <S, A, R>, where S is a finite state space, A is a finite action space, and R is a reward space.

[0085] The state space S consists of the system states in the physical topology formed by mapping each SFC request at time slot t to the generalized platform. Therefore, the generalized platform system state at time slot t can be represented as:

[0086] s t ={s1(t),s2(t)...s |SFCs| Equation (9)

[0087] in This represents the physical topology resource information at time slot t. This indicates the characteristics of the x-th SFC request. |SFCs| represents the number of SFC requests at time slot t.

[0088] Action space A represents the allocation method of various resources during mapping of SFC requests in the current generalized platform system state. Therefore, the action space of time slot t can be represented as:

[0089] a t ={a c (t),a b Equation (10)

[0090] Among them, a c (t) represents the allocation method of computing resources for physical nodes, a b (t) represents the allocation method of physical link bandwidth communication resources. When in time slot t, based on state s... t Take action a at that time t It will enter the next state s t+1 And receive an instant reward r t The instantaneous rewards for each time slot t constitute the reward space R = (r1, r2, r3...), where r1, r2, r3... represent the instantaneous feedback reward values ​​obtained from actions a1, a2, and a3 taken according to the corresponding states s1, s2, and s3 in the 1st, 2nd, and 3rd time periods, respectively. Since the optimization objective of this invention is to minimize the SFC request processing time, the negative of the processing time is set as the reward function, i.e.:

[0091]

[0092] Step 4: Combine Markov processes with graph convolutional neural networks to optimize the triplet model and transform the high-dimensional adjacency information of the state into a low-dimensional vector representation.

[0093] To improve the training convergence efficiency of the intelligent resource allocation method for service function chain mapping in a generalized platform, a graph convolutional neural network method is combined with a triplet finite state space and action space. High-dimensional adjacency information of the state is represented by a low-dimensional vector, improving the input efficiency and mapping action response rate of the intelligent allocation method. Specifically, this involves obtaining the generalized platform system state s in time slot t. t ={s1(t),s2(t)...s |SFCs| (t)}, with the xth SFC request status For example, physical topology resource layer There are |N m | Node x, SFC request The functional sequence is |V x |, forming a |N m |*|V x |dimensional feature matrix X, forming an |N m |*|N m |The adjacency matrix A.

[0094] The forward propagation of the ReLU activation function and the softmax normalization function is completed through two layers of graph convolution operations:

[0095]

[0096] in It is a metric matrix. I is the identity matrix. It is a symmetric matrix.

[0097] The finite action space a t ={a c (t),a b By combining (t)} with a graph convolutional neural network, the high-dimensional action space is represented by a low-dimensional vector space. The specific modified formula for the finite action set is as follows: Where a wi =wi(a c ,a b ), |V x | represents the number of functions in the function sequence of the x-th SFC request, W A It is a weight parameter matrix with a finite action space.

[0098] Step 5: Combine the optimized MDP triples from Step 4 with A3C to propose the SA3C method. Based on the asynchronous training method in the asynchronous advantage action evaluation algorithm, using the main training network as a template, multiple sub-networks are generated for parallel training using multi-threading technology. The optimized parameters of the sub-networks after training are obtained, and they are synchronized with the main network parameters through differential updates, accelerating parameter convergence efficiency. This invention treats a single SFC request as an agent for training a sub-network, interacting with the physical topology resource layer to learn and update the sub-network parameters θ. μ′ With θ Q′ And using differential updates to update the parameters θ of the main network, policy network, and value network. μ With θ Q Synchronization is performed to find the optimal resource allocation strategy.

[0099] The SFC mapping resource allocation model based on A3C is shown in the figure below. Figure 2 As shown, the specific content is as follows: Define the set of subnetworks as M, and use parameter vectors. Generate an action selection policy function π(s) based on state s. t Select mapping action a t Using parameter vectors Generate a state-value function V(s) to evaluate the current state s. tThe advantages and disadvantages. For the A3C network architecture, both the main network and the sub-networks maintain their own network's random action strategy π(s) = π(s). t ,a t |θ μ ) and state value function V(s t )=V(s t |θ Q ).

[0100] When using A3C, you need to first define the strategy π(s) t The state-value function V(s) t Its essence is the cumulative expected discount reward, which can be expressed as formula (13).

[0101]

[0102] Where γ∈(0,1) is the discount factor, representing the degree of influence of future decisions on the current state. Represents the action policy π(s) t The mathematical expectation of V(s) t+1 ) represents state s t+1 The state value function. Secondly, the action policy π(s) needs to be defined. t The action-value function Q(s) t ,a t The value corresponding to the current action is defined to evaluate the magnitude of the value relative to the average value, and can be described by formula (14):

[0103]

[0104] In A3C, both the main network and sub-networks are essentially actor-critic network structures, requiring updates to the policy network and value network parameters during updates. Therefore, the policy network and value network parameters in the main network are related by θ. μ With θ Q This indicates that the parameters of the policy network and value network in the sub-network are expressed as θ. μ′ With θ Q′ Representation. Parameter θ in the Actor policy network. μ The update follows the policy gradient formula as shown in equation (15).

[0105]

[0106] Where δ is the entropy hyperparameter, which is set relatively large at the beginning and gradually decreases during the training phase; H() represents the entropy of the policy; and dθ μ Represents the policy network parameters θ μ The differential, Represents the policy network parameters θ μThe total differential, Represents the policy network parameters θ μ′ The total differential, π(s) t |θ μ′ ) indicates that the state is s t The policy network parameter is θ μ′ The action policy function generated at that time, dθ μ′ Represents the policy network parameters θ μ′ The derivative of α represents the learning rate of the policy parameters. The parameters θ in the Critic value network... Q The update follows the policy gradient formula (16) as shown:

[0107]

[0108] Where β represents the learning rate of the value parameter. dθ Q Represents the value network parameters θ Q The differential, dθ Q′ Represents the value network parameters θ Q′ The differential, V(s) t |θ Q′ ) represents state s t Value network parameters θ Q′ The state-value function at time, Represents the policy network parameters θ Q′ The partial derivatives of .

[0109] The SA3C method trains the sub-network parameters and synchronizes them with the main network through differential updates. The effectiveness of the proposed method is validated in terms of mapping rate, total processing time, and resource utilization. It is compared with the Actor-Critic (AC) and Policy Gradient (PG) methods. The simulated network topology used has 24 nodes and 45 links. The specific details are as follows:

[0110] (1) Mapping rate

[0111] The mapping rate at which an SFC request is successfully mapped to the physical network for processing can be defined by equation (17), where Num(SFCs) represents the total number of current SFC requests. This indicates the number of successful mappings. The mapping rates for the three methods are compared below. Figure 3 As shown, from Figure 3As can be seen, in the initial stage, the mapping rate of all three methods gradually decreases from 100%. This is because as the number of SFC requests increases, the computing resources and link bandwidth communication resources of nodes in the physical network are gradually occupied, causing some SFCs to fail to map due to insufficient resources. However, the mapping rate of the SA3C method decreases significantly less than that of the AC and PG algorithms. When mapping ends at 100 SFC requests, the mapping rate of SA3C is 75.74%, which is 6.72% higher than that of AC and 16.7% higher than that of the PG algorithm. This is because the SA3C method uses multiple sub-networks for parallel learning, reducing training time and accelerating convergence efficiency.

[0112]

[0113] (2) Total processing time

[0114] The total processing time for the current set of SFC requests can be calculated using equation (7) in step two, as shown in equation (18). A comparison of the total processing times for the three methods is provided below. Figure 4 As shown. From Figure 4 As can be seen, the total processing time of all three methods shows an upward trend. This is because as the number of SFC requests increases, the amount of data to be processed also gradually increases, and the time spent processing the data also increases. However, the upward trend of the total processing time of the SA3C method is significantly smaller than that of the AC and PG methods. When processing 100 SFC requests, the total processing time required by SA3C is 1025.86 units of time, which is 10.72% lower than the total processing time of AC and 18.43% lower than the PG algorithm. This is because SA3C feeds each SFC into a sub-network for learning and training to achieve the optimization effect of minimizing the total processing time.

[0115]

[0116] (3) Resource utilization rate

[0117] When an SFC request is mapped to the physical network, it requires certain resources to be processed. The resource utilization rate of the current set of SFC requests can be defined by formula (18), where Represents the computing resources of physical nodes. Indicates link bandwidth communication resources. This represents the sum of physical node and link bandwidth resources allocated in the SFC request. The resource utilization rates of the three methods are compared below. Figure 5 As shown, from Figure 5As can be seen, the resource utilization of the three methods shows an upward trend. This is because as the number of SFC requests to be processed increases, the available physical node computing resources and physical link bandwidth communication resources gradually decrease, leading to a gradual increase in resource utilization. Since SFC needs to select nodes with the corresponding VNFs for mapping, if there are no VNF nodes to map, the SFC mapping fails and will not occupy the resources of other nodes without deployed VNFs. Therefore, although the resource utilization shows an upward trend, it generally does not reach full capacity.

[0118] from Figure 5 As can be seen, when mapping 100 SFC requests, the resource utilization rate of the SA3C algorithm is 69.31%, which is 9.85% higher than that of AC and 17.08% higher than that of the PG algorithm. This is because SA3C uses the allocation of physical node computing resources and physical link bandwidth communication resources as a joint optimization objective, adjusts the weight of resource allocation during SFC mapping, speeds up the processing rate of SFC, and thus releases resources to process more SFCs.

[0119]

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent allocation of service function chain mapping resources applicable to a general-purpose platform, characterized in that, The steps are as follows: Step 1: Establish an SFC mapping model, dividing the SFC mapping process in the generalized platform into a two-layer structure of physical topology resource layer and virtual network function request layer, and abstracting the parameters of the two-layer structure. Step 2: Analyze the service request processing process in the generalized platform and establish a mathematical model that minimizes the processing time of the joint allocation of node computing and link bandwidth communication resources; The minimized mathematical model is: Where C1 is the minimum total processing time for processing the set of virtual network function requests, indicating that each VNF must be mapped to a physical service node; C2 indicates that the computing resources allocated to a VNF when mapped to a physical service node are limited by the capacity of the available resources of that physical service node; C3 indicates that the bandwidth resources allocated to a virtual link when mapped to a physical link are limited by the capacity of the available bandwidth resources of that physical link; I is an identity matrix consisting entirely of 1s; N represents the set of physical service nodes; |N| represents the sequence of physical service nodes; |V f | represents the sequence of virtual network function requests for the f-th SFC request. This indicates the mapping status of the f-th SFC request in time slot t on the physical service node, D. f Let C(t) be the total processing time of the node for the f-th SFC request, C(t) be the binary matrix formed by the virtual network function mapping, and B(t) be the binary matrix formed by the virtual link mapping. Mapping the i-th virtual function in the f-th SFC request to the physical service node n within time slot t in the physical topology. j The mapping state, For the physical service node that is successfully mapped at time slot t, c is allocated node computing resources. j This indicates the available computing resources of the j-th physical service node. This indicates that the e-th virtual link in the f-th SFC has been successfully mapped to the physical service node n in the physical topology. i With n j Physical links (n) between i ,n j The link mapping status on ) For the physical link successfully mapped at time slot t, the allocated link bandwidth communication resources, b i,j This represents the available bandwidth communication resources of the physical link; L represents the set of physical links, and SFCs is the set of user service requests. Step 3: Model the minimized mathematical model as a Markov process, defining a triple containing state, action, and reward; Step 4: Combine the Markov process with the graph convolutional neural network to optimize the triples and transform the high-dimensional adjacency information of the state into a low-dimensional vector representation. Step 5: Combine the optimized triplet from Step 4 with the asynchronous dominant action evaluation algorithm. Based on the asynchronous training method in the asynchronous dominant action evaluation algorithm, use the main training network as a template and use multi-threading technology to generate multiple sub-networks for parallel training to find the optimal resource allocation strategy.

2. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 1, characterized in that, The physical topology resource layer is represented by an undirected graph G = {N, L}, where N = {n1, n2, ... n}. m Let} represent the set of physical service nodes, m represent the maximum number of physical service nodes, and n1, n2...n m There are m physical service nodes; L = {l i,j =(n i ,n j The set of physical links is represented by |i,j≤m}. i,j For the i-th physical service node n i With the j-th physical service node n j The physical link between them; the available computing resources of the physical service nodes C = [c1, c2...c m ], where c i For physical service node n i Available computing resources; physical link bandwidth communication resources B = {b i,j |i,j≤m},b i,j For physical service node n i With physical service node n j Physical link bandwidth communication resources between them, if b i,j =0 indicates that the available link bandwidth communication resources are 0 or that there is no physical link connection between the two points; The virtual network function request is a set of user service request SFCs = {f1, f2, ...}, where each f represents a specific SFC request. Each SFC request is represented by a quadruple f =<V f E f Data f ,P f > indicates that V f ={v1,v2...v v } represents the virtual network function request sequence, v represents the virtual network function requested by the user, and E f ={e1,e2,...e e } represents the set of virtual links, where e u ={v p ,v q } represents virtual network function v p and virtual network function v q Virtual links between, Data f P represents the size of the data to be transmitted in the SFC request f. f ={p1,p2...p5} represents the set of processing priorities for SFC request f, with priority p1 being the lowest and priority p5 being the highest.

3. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 1, characterized in that, During the mapping process, the total processing time D for the f-th SFC request is... f Total processing time for physical service nodes Total link processing time sum; The total processing time of the physical service node for the SFC request in clause f is: Among them, v c N represents the virtual network functionality contained in the f-th SFC request. f This represents the sequence of physical service nodes mapped to the virtual function in the f-th SFC request, and the time the f-th SFC request stays on a single physical service node. The processing rate of an SFC request in the physical service node at time slot t is proportional to the computing resources allocated to that physical service node, and: in, As the transformation factor, and For the physical service node that is successfully mapped at time slot t, the node computing resources allocated to it are determined, and c c This indicates the available node computing resources for the physical service node; Data f Indicates the amount of data to be transmitted in SFC request f; The total processing time for the SFC request in clause f is: Among them, the time that the fth SFC request stays on a physical link Article f requests the transmission rate in the physical link at time slot t. It is proportional to the bandwidth resources allocated to the physical link, and: Where η is the transformation factor, and η > 1. For a physical link that is successfully mapped at time slot t, the allocated link bandwidth communication resources are determined, and b p,q This indicates the available bandwidth communication resources of the physical link.

4. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 3, characterized in that, The entire process of processing SFC requests is divided into several time slots according to processing time, denoted by T = {1, 2, ... t}, with one time slot t being the SFC mapping processing cycle; within the time slot t where the virtual function is mapped to the physical topology, using... Indicates the service node mapping status, when The time indicates the i-th virtual network function v in the f-th SFC request. i Successfully mapped to the j-th physical service node in the physical topology; if mapping fails, then... After all the virtual network functions in the f-th SFC request have been mapped, use Represents a binary matrix formed by mapping virtual network functions; using Indicates the link mapping status, when When the e-th virtual link in the f-th SFC request is successfully mapped to the physical service node n in the physical topology, it indicates that the request has been successfully mapped. p With physical service node n q Physical links (n) between p ,n q If mapping fails, then the mapping state... After the virtual links in the f-th SFC request are all mapped, use It represents the binary matrix formed by its virtual link mapping.

5. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 3 or 4, characterized in that, During the mapping process, priority p i It also affects the order of mapping. If no priority is set for each SFC request, the default priority is p3. The higher the priority, the greater the proportion of computing resources and link bandwidth communication resources allocated, and the shorter the total processing time for the same amount of data.

6. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 3 or 4, characterized in that, The Markov process is a triple M = <S, A, R>, where S is a finite state space, A is a finite action space, and R is a reward space. The state space S is composed of the system states in the physical topology formed by mapping each SFC request at time slot t to the generalized platform. Therefore, the generalized platform system state at time slot t is: s t ={s1(t),s2(t)...s |SFCs| (t)}; Among them, state This represents the physical topology resource information at time slot t. The x-th SFC request is characterized by |SFCs|; |SFCs| represents the number of SFC requests at time slot t. The action space A represents the allocation method of various resources during mapping of SFC requests in the current generalized platform system state. The action of time slot t is: a t ={a c (t),a b (t)}; Among them, a c (t) represents the allocation method of computing resources for physical service nodes, a b (t) represents the allocation method of physical link bandwidth communication resources; When in time slot t, based on state s t Take action a at the time t It will enter the next state s t+1 And receive an instant reward r t The instantaneous rewards for each time slot t constitute the reward space R = (r1, r2, r3...), where r1, r2, r3... represent the instantaneous feedback reward values ​​obtained by actions a1, a2, and a3 taken according to the corresponding states s1, s2, and s3 in the 1st, 2nd, and 3rd time periods, respectively; the negative of the processing time is set as the reward function.

7. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 6, characterized in that, Combining Markov processes with graph convolutional neural networks, the method for optimizing triples is as follows: Obtain the generalized platform system state s for time slot t. t ={s1(t),s2(t)...s |SFCs| (t)}, the status of the xth SFC request Physical topology resource layer There are |N m | Node x, SFC request Functional sequence | V x |Form a|N m |*|V x |dimensional feature matrix X, forming an |N m |*|N m The adjacency matrix A is used to perform forward propagation of the ReLU activation function and the softmax normalization function through two layers of graph convolution operations: in It is a metric matrix; I is the identity matrix. It is a symmetric matrix, W S It is a weight parameter matrix in a finite state space; The finite action space a t ={a c (t),a b Combining (t)} with a graph convolutional neural network, the high-dimensional action space is represented by a low-dimensional vector space, and the modified formula for the finite action set is: Where a wi =wi(a c ,a b ), |V x | represents the number of functions in the function sequence of the x-th SFC request, W A It is a weight parameter matrix with a finite action space.

8. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 6, characterized in that, The implementation method of step five is as follows: a single SFC request is used as an agent trained in a sub-network. The parameter values ​​of the optimal resource allocation strategy are obtained in the sub-network and synchronized with the main network parameters through differential updates. The sub-network interacts with the physical topology resource layer to learn and update the sub-network parameters θ. μ′ With θ Q′ And using differential updates to update the parameters θ of the main network, the policy network, and the value network. μ With θ Q Perform synchronization; Define the set of subnetworks as M, and use parameter vectors. Generate an action selection policy function π(s) based on state s. t Select mapping action a t Using parameter vectors Generate a state-value function V(s) to evaluate the current state s. t The advantages and disadvantages; for the network architecture of the asynchronous dominant action evaluation algorithm, both the main network and the sub-network maintain their own network's random action strategy π(s) = π(s) t ,a t |θ μ ) and state value function V(s t )=V(s t |θ Q ).

9. The intelligent allocation method for service function chain mapping resources applicable to a generalized platform according to claim 8, characterized in that, The implementation method of the differential update is as follows: Define strategy π(s) t State value function Where γ∈(0,1) is the discount factor. Represents the action policy π(s) t The mathematical expectation of V(s) t+1 ) represents state s t+1 State value function; Define the action policy π(s) t The action-value function Q(s) t ,a t ), used to evaluate the magnitude of the value corresponding to the current action relative to the average value, and: The parameters of the policy network and value network in the main network are θ μ With θ Q This indicates that the parameters of the policy network and value network in the sub-network are expressed as θ. μ′ With θ Q′ This indicates that the parameter θ in the policy network Actor... μ The update follows the policy gradient as follows: Where δ is the entropy hyperparameter, which is set relatively large at the beginning and gradually decreases during the training phase; H() represents the entropy of the policy; and dθ μ The parameter θ represents the policy network. μ The differential, The parameter θ represents the policy network. μ The total differential, Represents the policy network parameters θ μ′ The total differential, π(s) t |θ μ′ ) indicates that the state is s t The policy network parameter is θ μ′ The action policy function generated at that time, dθ μ′ Represents the subnetwork parameters θ μ′ The derivative of α represents the learning rate of the policy parameter; Parameter θ in the value network Critic Q The update follows the policy gradient: Where β represents the learning rate of the value parameter, dθ Q The parameter θ represents the value network. Q The differential, dθ Q′ Represents the value network parameters θ Q′ The differential, V(s) t |θ Q′ ) represents state s t Value network parameters θ Q′ State value function at time The parameter θ represents the value network. Q′ The partial derivatives of .

Images