Feature-fused business function chain embedding method

Abstract

The invention discloses a feature-fused business function chain embedding method. The method comprises the following steps of S1, feature aggregation; s2, embedding the virtual node into the physical node to be embedded; step S3: carrying out routing selection; and S4, updating the PPO model. According to the method and the device, the end-to-end transmission delay in the service function chain embedding process caused by satellite motion can be optimized and reduced in a dynamic network environment.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication, and particularly to a method for embedding service function chains with fusion features. Background Art

[0002] The high-speed movement of satellites results in a highly dynamic topology of Space-Terrestrial Integrated Networks (STINs), making it extremely difficult to optimize the deployment of Service Function Chains (SFCs) embedded in STINs. Currently, existing SFC deployment methods mainly target static network environments and do not consider the impact of network dynamics on SFC deployment. Therefore, existing deployments cannot adapt to the time-varying packet transmission process in a dynamic network environment, cannot meet the requirements of making full use of network resources and operational complexity, and thus limit the flexibility of the network and the delivery efficiency of services.

[0003] Given the limitedness and uncertainty of satellite network resources, as well as the rapid growth of the number of SFCs, exploring more efficient and flexible methods to reduce the end-to-end transmission delay in the process of embedding service function chains has become an urgent research need. Summary of the Invention

[0004] To this end, the present invention provides a method for embedding service function chains with fusion features, which can solve the problem that the end-to-end transmission delay in the process of embedding service function chains in the space-ground integrated network often causes service blocking.

[0005] To achieve the objectives of the present invention, the following technical solutions are adopted:

[0006] A method for embedding service function chains with fusion features includes the following steps:

[0007] Step S1: Feature aggregation, including encoding the features of physical nodes into feature vectors;

[0008] Step S2: Embedding virtual nodes into physical nodes to be embedded, where the embedding of virtual nodes into physical nodes to be embedded is carried out in a predetermined order;

[0009] Step S3: Route selection, finding the physical network edges corresponding to the edge set formed between the already embedded virtual nodes, and embedding the virtual links into the physical links;

[0010] Step S4: Update of the PPO model.

[0011] The method for embedding the service function chain with fused features, wherein the physical node is a satellite node, and step S1 includes: encoding the neighbor link information of all satellite nodes into feature vectors.

[0012] The method for embedding the service function chain with fused features, wherein each physical node n s The aggregated feature L after fusion agg is expressed as:

[0013]

[0014] where Neighbor(·) represents the set of neighbor nodes of a certain physical node, v represents a neighbor node in the set of neighbor nodes of the physical node, respectively represent the bandwidth allocated between the physical link of a certain physical node and a neighbor node and the transmission delay between a certain physical node and a neighbor node. Denote the set of fused features of all physical nodes as L agg .

[0015] The method for embedding the service function chain with fused features, wherein step S2 includes:

[0016] Embedding a node in the virtual network through the PPO model. The output policy π of the PPO model at time t t is an index of a node in the physical network, and this index corresponds to the physical network node to which the virtual network node is to be embedded.

[0017] In the method for embedding the service function chain with fused features, in step S2:

[0018] The PPO model updates the policy through policy gradients and gradually learns how to select the optimal embedding method to embed the virtual node onto the appropriate physical node;

[0019] where the goal of the PPO model algorithm is to maximize the long-term cumulative reward, expressed as:

[0020]

[0021] where, is the reward at time step t, is the discount factor. The above formula is re-expressed as:

[0022]

[0023] where is the input state information, which is composed of the physical network, the virtual network, the features obtained by aggregation, the indicator vectors of the physical nodes and virtual nodes that have been embedded, and the one-hot code information of the virtual node currently being embedded, expressed as:

[0024]

[0025] wherein, respectively represent the states of the physical network and the virtual network at time t, respectively represent the embedding indication vectors of the physical node and the virtual node at time t, O c，t represents the one-hot vector corresponding to the virtual node c that is being embedded at time t;

[0026] After this stage, a mapping corresponding to a certain virtual node to a physical node is obtained, that is, node_embedding[c]=π t .

[0027] The service function chain embedding method with the fused features, wherein step S3 includes:

[0028] First, obtain the set of edges formed by the already embedded virtual nodes corresponding to time t and obtain the set of paths formed by the already embedded entity nodes corresponding to time t For each edge in its corresponding mapping between entity nodes, if it exists, is obtained by the Dijkstra algorithm, specifically expressed as:

[0029] edge_embedding[(n source ,n dest )]=Dij(node_embedding[n source , node_embedding[n dest )

[0030] wherein, n source is the index of the source virtual node, and n dest is the index of the target virtual node;

[0031] Check whether the returned path of this algorithm is in , if it is, continue with the embedding of the next edge; if not, reject the SFC request.

[0032] The service function chain embedding method with the fused features, wherein step S3 further includes:

[0033] If all virtual nodes and edges are successfully embedded, the SFC request is regarded as successful, and the status information is updated:

[0034]

[0035] The method for embedding the service function chain with fused features, wherein step S4 includes:

[0036] Perform gradient backpropagation and parameter update of the PPO model, and the state information involved in each step executed in steps S2 and S3 Policy π t and reward r t , next state etc. are all put into a buffer When t reaches the threshold t u , it is regarded that the buffer is full, and batch policy gradient backpropagation and parameter update are started for the data in the buffer:

[0037] θ←θ + Δθ, where Δθ is the change amount of the parameter.

[0038] A computer-readable storage medium stores a computer program that can be used to execute the method for embedding the service function chain with fused features as described above.

[0039] An electronic device includes a memory, a processor, and a computing program stored on the memory and executable on the processor. When the processor executes the program, the method for embedding the service function chain with fused features as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the service function chain embedding method of the present invention;

[0041] Figure 2 Schematic diagram of the average delay of different algorithms when the number of SFC requests is 20;

[0042] Figure 3 Schematic diagram of the deployment success rate of different algorithms when the number of SFC requests is 20;

[0043] Figure 4 Schematic diagram of the average delay of different algorithms when the number of SFC requests is 100;

[0044] Figure 5 Schematic diagram of the deployment success rate of different algorithms when the number of SFC requests is 100. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following will combine with the attached Figures 1-5 , and the specific embodiments of the present invention will be described in detail. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention. The described embodiments are exemplary and are only used to explain the present invention and cannot be understood as a limitation of the present invention.

[0046] As used in this specification, "an embodiment" or "some embodiments" etc. mean that in one or more embodiments of the present invention, the specific features, structures or characteristics described in connection with the embodiment are included. Thus, the terms "include", "comprise", "have" and their variants in this specification all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0047] A method for embedding a service function chain with fusion features of the present invention includes:

[0048] Step S1: Feature aggregation, encoding the features of all satellite nodes (mainly link features, such as neighbor link information, including the bandwidth allocated between links, link delay, etc.) into feature vectors. The satellite nodes are called physical nodes.

[0049] Assume that a node can access the information of one-hop neighbors, aggregate the feature vectors of all nodes and their neighbor nodes to form context information, which is part of the input state of the PPO model (Proximal Policy Optimization, a reinforcement learning model);

[0050] Each physical node n s The aggregated feature L after fusion agg Can be expressed as:

[0051]

[0052] Where Neighbor(·) represents the set of neighbor nodes of a certain physical node, v represents a neighbor node in the set of neighbor nodes of the physical node, respectively represent the bandwidth allocated between the physical link of a certain physical node and a neighbor node and the transmission delay between a certain physical node and a neighbor node. Denote the set of fused features of all physical nodes as

[0053] Step S2: Embed the virtual node into the physical node to be embedded

[0054] This step embeds a node in the virtual network (service function chain) through the PPO model (the node embedding is carried out in a pre-specified order). The output policy π of the PPO model at time t t Is an index of a node in the physical network, and this index corresponds to the physical network node into which the virtual network node is to be embedded. Determined by the PPO model, at the current virtual node n V , only when Can it be embedded into the physical node n s If there is no underlying physical node that satisfies the constraint, the SFC request is rejected; where, Indicates the demand of the virtual node n for the resource type r (preferably, the resource type r is CPU and memory). Indicates the available amount of the resource type r on the physical node n at time t.

[0055] This constraint ensures that the resource demand of each virtual node does not exceed the available resources of the corresponding physical node at the current moment.

[0056] Specifically, the PPO algorithm updates the policy through policy gradients, gradually learning how to select the optimal embedding method to embed the virtual node onto the appropriate physical node. The goal of the PPO algorithm is to maximize the long-term cumulative reward, expressed as:

[0057]

[0058] Where is the reward at time step t, is the discount factor, in another representation:

[0059]

[0060] Where is the input state information, which is composed of the characteristics of the physical network, the virtual network, the indication vectors of the already embedded physical nodes and virtual nodes, and the one-hot code information of the currently embedding virtual node, and can be expressed as:

[0061]

[0062] Where represent the states of the physical network and the virtual network at time t respectively, represent the embedding indication vectors of the physical node and the virtual node at time t respectively, O c，t represents the one-hot vector corresponding to the virtual node c that is currently being embedded, L agg Indicates each physical node n s The aggregated feature after fusion. After this stage, the mapping corresponding to a certain virtual node to the physical node is obtained, that is, node_embedding[c]=π t .

[0063] After that, update the state After all virtual nodes are embedded, enter the subsequent edge embedding, that is, enter the routing selection stage of step S3.

[0064] Step S3: Routing selection. This step uses a single-source shortest path algorithm, such as the Dijkstra algorithm, to find the physical network edge embedding corresponding to the edge set formed between the already embedded virtual nodes. The virtual link (nV , m V ) is embedded into the physical link (n s , m s ). This embedding process must satisfy this constraint condition, otherwise the SFC request is rejected. represents the bandwidth required by the virtual link (n, m), represents the available bandwidth on the physical link (n, m) at time t. This constraint condition ensures that the bandwidth required by each virtual link does not exceed the available bandwidth of the corresponding physical link at the current time. After rejecting the SFC request, first interrupt the edge embedding operation, then return to step S2 to select a physical node for embedding the corresponding virtual node, and then enter step S3 to start edge embedding from the interruption point.

[0065] If all the virtual links corresponding to this virtual node satisfy the constraint condition, select the next virtual node and repeat step S3;

[0066] Specifically, first obtain the set of edges formed by the already embedded virtual nodes corresponding to time t and obtain the set of paths formed by the already embedded physical nodes corresponding to time t For each edge in its corresponding mapping between physical nodes, if it exists, can be obtained by the Dijkstra algorithm, specifically expressed as:

[0067] edge_embedding[(n source , n dest )] = Dij(node_embedding[n source , node_embedding[n dest )

[0068] where n source is the index of the source virtual node, and n dest is the index of the target virtual node. The above formula means inputting the already embedded physical nodes into the model to obtain the returned path.

[0069] Check whether the returned path of this algorithm is in . If it is, continue with the embedding of the next edge; if not, reject the SFC request, first interrupt the edge embedding operation, return to step S2 to select a physical node for embedding the corresponding virtual node, and then enter step S3 to start edge embedding from the interruption point. After all the edges of the current virtual node are successfully embedded, then enter step S3 to embed the next virtual node; if all virtual nodes and edges are successfully embedded, the SFC request is considered successful, and update the status information:

[0070]

[0071] Among them, update(·) updates the status information according to the action situation For example, when the embedding is successful it is necessary to update the corresponding resource changes and node occupancy, as well as the embedded entity nodes and virtual nodes Vector update, O c，t and change to the next node to be embedded. When the embedding fails, the nodes and links that have been embedded before the currently failed node remain unchanged. The edge embeddings of the virtual nodes that have been completed for the currently failed virtual node are revoked, and the physical nodes embedded for this virtual node in step S2 are correspondingly revoked; in particular, if t reaches a certain threshold t u and the SFC request is still not completed, it will preferentially enter the next step, that is, step S4.

[0072] It should be noted that any embedding failure during step S3 will cause the SFC request to be rejected. At this time, the status information also needs to be updated. At this time, the edge embeddings and physical node embeddings related to the nodes that failed to be embedded in this SFC request are revoked.

[0073] Step S4: PPO model update. In this step, the gradient backpropagation and parameter update of the PPO model are performed. The status information involved in each step executed in steps S2 and S3 policy π t reward r t next state etc. will all be put into a buffer When t reaches the threshold t u it is regarded that the buffer is full, and the batch policy gradient backpropagation and parameter update of the data in the buffer are started:

[0074] θ←θ + Δθ

[0075] where Δθ is the change amount of the parameter.

[0076] It should be noted that during the training process of the PPO reinforcement learning model, the SFC requests are not single, but in batches (that is, a batch of SFC requests need to be processed at one time). Optionally, the structure of the physical network may be dynamic, and the appearance of dynamics will affect the content of the update function.

[0077] This embodiment considers a space-ground integrated network composed of 24 satellites and 10 ground stations. Using the above embedding method for embedding calculation and comparing with other algorithms, better results can be obtained. The link parameters of the STINs are generated by the Satellite Tool Kit (STK). AttachedFigure 2 and Figure 3 respectively show the performance of Feature Encoding + Proximal Policy Optimization, Proximal Policy Optimization, and Deep Q-Network (DQN) algorithms in terms of average latency and success rate when the number of SFC requests is 20. It can be observed that when applying PPO to the Virtual Network Embedding (VNE) task, compared with the traditional Deep Q-Network algorithm, it can achieve lower latency, higher success rate, and faster convergence speed. The method of combining feature encoding operation and proximal policy optimization proposed in the present invention is superior to the method of only using proximal policy optimization or Deep Q-Network, which indicates that the feature encoding stage effectively extracts the neighborhood information of nodes, thereby improving the effect of the training stage.

[0078] To further verify the superiority of the algorithm proposed in the present invention, the number of SFC requests within the same time interval is increased to 100, and the corresponding results are shown in Figure 4 and Figure 5 . The increase in the number of requests increases the network load, resulting in performance degradation. When only using the Deep Q-Network method, the training performance shows obvious fluctuations, and the success rate only increases slightly; although the performance of using the proximal policy optimization method alone is better, its convergence speed is still not as good as the method proposed in the present invention.

[0079] The present invention also provides a computer-readable storage medium, which stores a computer program that can be used to execute the service function chain embedding method with fused features as described above.

[0080] The present invention also provides an electronic device, including a memory, a processor, and a computing program stored on the memory and executable on the processor. When the processor executes the program, it implements the service function chain embedding method with fused features as described above. Through the present invention, it is possible to optimize and reduce the end-to-end transmission delay in the service function chain embedding process caused by satellite movement in a dynamic network environment.

[0081] The above is only a preferred embodiment of the present invention, and it does not impose any form of limitation on the present invention. Although the present invention has been disclosed above with a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to it as equivalent embodiments within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for embedding a service function chain with fusion features, characterized in that The steps include: Step S1: feature aggregation, including encoding the features of the physical nodes into feature vectors; Step S2: embedding the virtual node into the physical node to be embedded, wherein the embedding of the virtual node into the physical node to be embedded is performed in a predetermined order; Step S3: route selection, finding the physical network edge corresponding to the edge set formed between the embedded virtual nodes, and embedding the virtual link into the physical link; Step S4: PPO model update.

2. The method for embedding a service function chain of fusion features according to claim 1 is characterized in that The physical node is a satellite node, and step S1 includes: encoding neighbor link information of all satellite nodes into a feature vector.

3. The method for embedding a service function chain of fusion features according to claim 2, characterized in that: Each physical node n s The aggregated feature L after fusion agg Expressed as: Among them, Neighbor(·) represents the neighbor node set of a physical node, v represents a neighbor node in the neighbor node set of the physical node, They represent the bandwidth allocated between the physical link of a physical node and a neighboring node and the transmission delay between a physical node and a neighboring node respectively. The set of fusion features of all physical nodes is denoted as L agg .

4. The method for embedding a service function chain of fusion features according to claim 1 is characterized in that Step S2 includes: The PPO model is used to embed a node in the virtual network. The output strategy π of the PPO model at time t t is a node index in the physical network, and the index corresponds to the physical network node where the virtual network node is to be embedded.

5. The method for embedding a service function chain of fusion features according to claim 4 is characterized in that In step S2: The PPO model updates the strategy through policy gradient, gradually learning how to select the optimal embedding method, embedding the virtual node into the appropriate physical node, and obtaining the mapping corresponding to a virtual node to the physical node.

6. The method for embedding a service function chain of fusion features according to claim 1 is characterized in that Step S4 includes: Perform gradient backpropagation and parameter update of the PPO model.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program can be used to execute the service function chain embedding method of fusion features as described in any one of claims 1-6.

8. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the service function chain embedding method of fusion features as described in any one of claims 1-6 is implemented.

Images