Virtual Network Mapping Method, Device and Equipment Based on UAV Cluster Scenario

Abstract

The present application provides a virtual network mapping method, apparatus, and device based on an unmanned aerial vehicle (UAV) cluster scenario. When a virtual network embedding request is received, a topological graph sequence of the UAV cluster network is obtained; spatio-temporal dynamic features between each topological graph in the topological graph sequence are extracted based on a pre-constructed feature extraction network; the spatio-temporal dynamic features are input into a pre-trained node policy generation network to generate a node mapping policy; the virtual nodes to be embedded are mapped to the UAV nodes of the UAV cluster network based on the node mapping policy; after all the virtual nodes are successfully mapped, the virtual links to be embedded are mapped to the shortest physical links. The embodiments of the present invention can meet the virtual mapping requirements in a network dynamic scenario; and a two-stage mapping method is adopted to map the virtual nodes and virtual links in sequence, which can ensure that both the nodes and the links are successfully mapped and improve the mapping success rate.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular, to a virtual network mapping method, device, and equipment based on an unmanned aerial vehicle (UAV) cluster scenario. Background Art

[0002] VNE (Virtual Network Embedding) is a virtual network resource allocation technology. On the basis of meeting the quality of service requirements of a virtual network, by mapping virtual nodes to physical nodes and virtual links to physical links, more flexible, efficient, and reliable network services can be realized. High-efficiency virtual network mapping can improve the utilization rate of physical network resources.

[0003] In the prior art, when mapping a virtual network to a physical network, some are based on heuristic or meta-heuristic algorithms, and a series of heuristic strategies and rules are used to gradually search for and optimize the mapping scheme. However, heuristic algorithms usually make decisions based on experience and heuristic rules, and may not be able to guarantee finding the global optimal solution. Others use reinforcement learning algorithms, but existing online reinforcement learning algorithms are all based on the static topology of the physical network and do not consider the dynamic change characteristics of the physical network, and cannot meet the virtual mapping requirements of dynamic network scenarios. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide a virtual network mapping method, device, and equipment based on an unmanned aerial vehicle (UAV) cluster scenario to meet the virtual mapping requirements of dynamic network scenarios.

[0005] In a first aspect, a virtual network mapping method based on an unmanned aerial vehicle (UAV) cluster scenario is provided. The method includes:

[0006] When receiving a virtual network embedding request, obtaining a topological graph sequence of the UAV cluster network; the virtual network embedding request includes at least virtual nodes and virtual links to be embedded; the topological graph sequence is composed of topological graphs captured at each snapshot moment;

[0007] Extracting spatio-temporal dynamic features between the topological graphs in the topological graph sequence based on a pre-constructed feature extraction network;

[0008] Inputting the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy;

[0009] Mapping the virtual nodes to be embedded to the UAV nodes in the UAV cluster network based on the node mapping policy;

[0010] After all virtual nodes are successfully mapped, the shortest physical links between the mapped UAV nodes are determined based on the Floyd algorithm, and the virtual links to be embedded are mapped into the shortest physical links.

[0011] Optionally, the feature extraction network consists of a graph convolutional network and a Transformer-XL network; the spatio-temporal dynamic features between each topological graph in the topological graph sequence are extracted based on the pre-constructed feature extraction network, including:

[0012] Extract the spatial features of each topological graph one by one based on the graph convolutional network;

[0013] Determine whether the number of topological graphs in the topological graph sequence exceeds a preset number threshold;

[0014] If it exceeds the preset number threshold, the spatial features of the preset number threshold of the latest captured topological graphs are jointly input into the Transformer-XL network to extract the spatio-temporal dynamic features between the preset number threshold of topological graphs;

[0015] If it does not exceed the preset number threshold, all the spatial features extracted by the graph convolutional network are jointly input into the Transformer-XL network to extract the spatio-temporal dynamic features between each topological graph.

[0016] Optionally, the graph convolutional network includes two hidden layers; extracting the spatial features of each topological graph one by one based on the graph convolutional network includes:

[0017] For each topological graph, determine the first adjacency matrix of the UAV nodes in the topological graph and the attribute feature matrix of the UAV cluster network;

[0018] Input the first adjacency matrix and the attribute feature matrix into the first hidden layer to obtain the initial spatial features;

[0019] Input the initial spatial features into the second hidden layer to extract higher-dimensional feature information, and obtain the final spatial features of the topological graph.

[0020] Optionally, the node policy generation network adopts a deep reinforcement learning network based on the A3C algorithm. The deep reinforcement learning network based on the A3C algorithm includes an Actor network and a Critic network; the pre-training process of the node policy generation network includes:

[0021] Obtain the current environmental states of multiple UAV cluster networks; the current environmental states at least include the CPU capacity, remaining CPU capacity, bandwidth capacity, remaining bandwidth capacity, and maximum network latency of the UAV nodes;

[0022] Based on the current environmental state and virtual mapping requests, select the UAV nodes to be mapped in the UAV cluster network through the Actor network to obtain the node mapping strategy;

[0023] Calculate the gradient update direction of the Actor network through the policy gradient algorithm, and calculate the advantage function value of the Actor network along the gradient update direction until the advantage function value is minimized;

[0024] Evaluate the value of the node mapping strategy generated by the Actor network in the current environmental state based on the Critic network;

[0025] Train the Critic network by minimizing the TD error until the TD error is minimized.

[0026] Optionally, mapping the virtual nodes to be embedded into the UAV nodes of the UAV cluster network based on the node mapping strategy includes:

[0027] Sort the virtual nodes to be embedded in descending order according to the CPU demand of the virtual nodes in the virtual embedding request;

[0028] Map the virtual node ranked first to the UAV node in the UAV cluster network according to the node mapping strategy;

[0029] Judge whether the virtual node ranked first is successfully mapped;

[0030] If the mapping is successful, continue to map the next virtual node to the UAV node in the UAV cluster network according to the sorting until all virtual nodes are successfully mapped.

[0031] Optionally, judging whether the virtual node ranked first is successfully mapped includes:

[0032] Obtain the CPU demand and network latency of the virtual node ranked first; and the CPU capacity and network latency of the UAV node to which the virtual node ranked first is mapped;

[0033] Judge whether the CPU capacity and network latency of the UAV node are greater than the CPU demand and network latency of the virtual node ranked first;

[0034] If it is greater, the mapping is successful.

[0035] Optionally, the method further includes:

[0036] Judge whether all virtual links are successfully mapped;

[0037] If any virtual link mapping fails, reject the virtual network embedding request.

[0038] In a second aspect, a virtual network mapping device based on a drone swarm scenario is provided. The device includes:

[0039] An acquisition unit, configured to acquire a sequence of topology graphs of the drone swarm network when receiving a virtual network embedding request; the virtual network embedding request includes at least virtual nodes and virtual links to be embedded; the sequence of topology graphs is composed of topology graphs captured at each snapshot moment;

[0040] An extraction unit, configured to extract spatio-temporal dynamic features between the topology graphs in the sequence of topology graphs based on a pre-constructed feature extraction network;

[0041] A generation unit, configured to input the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy;

[0042] A node mapping unit, configured to map the virtual nodes to be embedded to the drone nodes in the drone swarm network based on the node mapping policy;

[0043] A link mapping unit, configured to, after all the virtual nodes are successfully mapped, determine the shortest physical links between the mapped drone nodes based on the Floyd algorithm, and map the virtual links to be embedded to the shortest physical links.

[0044] In a third aspect, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0045] The memory is used to store a computer program;

[0046] The processor, when executing the program stored on the memory, implements the method steps described in any one of the first aspect.

[0047] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method steps described in any one of the first aspect.

[0048] The virtual network mapping method, device and equipment based on the UAV cluster scenario provided by the present invention, when receiving a virtual network embedding request, obtain the topological graph sequence of the UAV cluster network; extract the spatio-temporal dynamic features between the topological graphs in the topological graph sequence based on a pre-constructed feature extraction network; input the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy; map the virtual nodes to be embedded to the UAV nodes in the UAV cluster network based on the node mapping policy; after all the virtual nodes are successfully mapped, determine the shortest physical links between the mapped UAV nodes based on the Floyd algorithm, and map the virtual links to be embedded to the shortest physical links. The embodiment of the present invention generates a node mapping policy and the shortest mapping path based on the spatio-temporal dynamic features, which can meet the virtual mapping requirements in the network dynamic scenario; and adopts a two-stage mapping method to map the virtual nodes and virtual links in sequence, which can ensure that both the nodes and the links are successfully mapped and improve the mapping success rate.

[0049] To make the above objects, features and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0051] Figure 1 Shows the flowchart of the virtual network mapping method based on the UAV cluster scenario provided by the embodiment of the present invention;

[0052] Figure 2 Shows the schematic diagram of the model structure of the feature extraction network and the node policy generation network provided by the embodiment of the present invention;

[0053] Figure 3 Shows the result comparison diagram of the long-term average revenue of four algorithms in the simulation experiment provided by the embodiment of the present invention;

[0054] Figure 4 Shows the result comparison diagram of the long-term average revenue-cost ratio of four algorithms in the simulation experiment provided by the embodiment of the present invention;

[0055] Figure 5 Shows the result comparison diagram of the request acceptance rate of four algorithms in the simulation experiment provided by the embodiment of the present invention;

[0056] Figure 6Shows a schematic structural diagram of a virtual network mapping device based on an unmanned aerial vehicle (UAV) cluster scenario provided by an embodiment of the present invention;

[0057] Figure 7 Shows a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the present invention claimed, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] Considering that existing online reinforcement learning algorithms are all based on the static topology of the physical network and do not consider the dynamic change characteristics of the physical network, they cannot meet the mapping requirements of dynamic network scenarios.

[0060] Based on this, embodiments of the present invention provide a virtual network mapping method and device based on an unmanned aerial vehicle (UAV) cluster scenario, which will be described below through embodiments.

[0061] An embodiment of the present invention provides a virtual network mapping method based on an unmanned aerial vehicle (UAV) cluster scenario, as Figure 1 shown, the method includes the following steps:

[0062] Step S101: When receiving a virtual network embedding request, obtain a topological graph sequence of the UAV cluster network.

[0063] In this step, the virtual network embedding request at least includes the virtual nodes to be embedded, virtual links, and attribute information of the virtual network. The attribute information of the virtual network includes one or more of the following: the number of virtual nodes, the distribution range of the CPU capacity of the virtual nodes, and the distribution range of the bandwidth of the virtual links.

[0064] In an embodiment of the present invention, the topological graph sequence is composed of topological graphs captured at each snapshot moment. Specifically, the topological graph sequence is a topological graph captured at preset snapshot intervals between the current virtual network embedding request moment and the previous virtual network embedding request moment.

[0065] At each snapshot moment, capture the topology structure information and network status information of the UAV cluster network corresponding to that moment. The topology structure includes UAV nodes and the edges between UAV nodes. The network status information includes the status of UAV nodes and the edges between nodes, such as the number of UAV nodes, the number of UAV physical links, the distribution range of the CPU capacity of UAV nodes, the distribution range of physical link bandwidths, and the distribution range of the maximum delay of UAV nodes.

[0066] In one example, the sequence of topology graphs collected by snapshots of the UAV cluster is represented as ; where T represents the snapshot moment; represents the topology graph collected at snapshot moment T;

[0067] where ; is the set of UAV nodes at snapshot moment ; is the set of UAV links at snapshot moment ; is the network status set of the UAV cluster at snapshot moment .

[0068] Step S102: Extract the spatio-temporal dynamic features between each topology graph in the sequence of topology graphs based on a pre-constructed feature extraction network.

[0069] In this step, the pre-constructed feature extraction network consists of a graph convolutional network and a Transformer-XL network. The spatial features between UAV nodes are extracted through the graph convolutional network; the dynamic features of UAV nodes changing over time are extracted through the Transformer-XL network to obtain the spatio-temporal dynamic features of the UAV cluster network. Transformer generally processes text sequence data. Therefore, in the embodiments of the present invention, the topology graph of the UAV is encoded into a topology graph sequence format by taking snapshots, so as to extract the dynamic characteristics of the topology graphs between different moments through the Transformer-XL network.

[0070] Step S103: Input the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy.

[0071] The spatio-temporal dynamic features reflect the dynamic characteristics of the UAV cluster network in time and space, and a node mapping policy can be generated based on the dynamic features, greatly improving the accuracy of node mapping.

[0072] Step S104: Map the virtual node to be embedded to the UAV nodes in the UAV cluster network based on the node mapping policy.

[0073] The specific mapping method will be described in the following embodiments and will not be elaborated here.

[0074] Step S105: After all virtual nodes are successfully mapped, determine the shortest physical links between the mapped UAV nodes based on the Floyd algorithm, and map the virtual links to be embedded into the shortest physical links.

[0075] In this step, the Floyd algorithm is a prior art and will not be elaborated here. In one example, for instance, there are four UAV nodes A, B, C, and D in total, and three virtual nodes a, b, and c. The generated node mapping strategy is a - A, b - B, c - D. The paths between the mapped UAV nodes A, B, and D are A - B - D and A - B - C - D; the shortest path A - B - D is selected through the Floyd algorithm, and the links of the virtual nodes are mapped into this shortest path A - B - D, thereby reducing the consumption of bandwidth resources.

[0076] The present invention can extract the spatio - temporal dynamic features of the network in the UAV cluster network scenario through the feature extraction network; generating the node mapping strategy and the shortest mapping path based on the spatio - temporal dynamic features can meet the virtual mapping requirements in the network dynamic scenario; and adopting a two - stage mapping method to map virtual nodes and virtual links sequentially can ensure that both nodes and links are successfully mapped, improving the mapping success rate.

[0077] Based on the above - mentioned embodiments, as Figure 2 shown, the feature extraction network consists of a graph convolutional network (GCN) and a Transformer - XL network; extracting the spatio - temporal dynamic features between each topological graph in the topological graph sequence based on the pre - constructed feature extraction network includes:

[0078] Step S1021: Extract the spatial features of each topological graph one by one based on the graph convolutional network.

[0079] In this step, the graph convolutional network includes two hidden layers.

[0080] In a feasible implementation manner, extracting the spatial features of each topological graph includes:

[0081] Step A: For each topological graph, determine the first adjacency matrix of the UAV nodes in the topological graph and the attribute feature matrix of the UAV cluster network.

[0082] In this step, for each topological graph , based on the node set and the link set determine the adjacency matrix ; based on the network state set determine the attribute feature matrix .

[0083] Step B: Input the first adjacency matrix and the attribute feature matrix into the first hidden layer to obtain the initial spatial features.

[0084] Step C: Input the initial spatial features into the second hidden layer to extract higher-dimensional feature information, and obtain the final spatial features of the topological graph.

[0085] In one example, the layer-by-layer propagation formula of the graph convolutional network is:

[0086] (1);

[0087] Where, represents the input vector of the graph convolutional network, that is, the attribute feature matrix, and its dimension is ; is the number of UAV nodes in the UAV cluster network, is the dimension of multiple state information of the UAV node, is the parameter of the graph convolutional network, and its dimension is , is the number of neurons in the hidden layer of the graph convolutional network; is the normalized adjacency matrix of the UAV node; is the degree matrix; is the output spatial feature, and its dimension is .

[0088] Where, represents the normalization process of the adjacency matrix and the degree matrix, and the normalized matrix is set as , then the layer-by-layer propagation formula of the graph convolutional network can be simplified as:

[0089] (2);

[0090] Where, represents the matrix after normalizing the adjacency matrix and the degree matrix; represents the -th layer spatial feature matrix; represents -th layer output spatial feature matrix; represents the parameter of the graph convolutional network.

[0091] In the embodiments of the present invention, by introducing two hidden layers, the graph convolutional network (GCN) can learn more complex and abstract node feature representations. The first hidden layer can capture information of direct neighbors, while the second layer can further integrate information from more distant neighbors, which helps to understand long-range dependencies in the network structure. Additionally, compared with deeper networks, two hidden layers can reduce the number of parameters to a certain extent, thereby reducing the risk of overfitting. The two hidden layers require less computational resources, including memory and computational time, which is particularly important for processing large-scale graph data. Due to the relatively simple structure, the GCN with two hidden layers can converge to a satisfactory solution faster, accelerating the speed of feature extraction.

[0092] Step S1022: Determine whether the number of topological graphs in the topological graph sequence exceeds a preset number threshold.

[0093] In this step, in a topological graph sequence, there may be 20 or even more topological graphs; inputting a large number of topological graphs into the Transformer-XL network will greatly reduce the efficiency of feature extraction and increase the computational amount. Therefore, a preset number threshold is set to control the amount of data input into the Transformer-XL network.

[0094] In an example, the preset number threshold is set to 5, that is, the maximum input is the spatial features of 5 sequences of topological graphs.

[0095] Step S1023: If it exceeds the preset number threshold, jointly input the spatial features of the preset number threshold of the most recently captured topological graphs into the Transformer-XL network to extract the spatio-temporal dynamic features Zt between the preset number threshold of topological graphs.

[0096] In this step, for example, there are topological graph sequences collected at 10 snapshot moments. In chronological order, the spatial features of the topological graphs at the most recent five moments are collected as the input to the Transformer-XL network.

[0097] Step S1024: If it does not exceed the preset number threshold, jointly input all the spatial features extracted by the graph convolutional network into the Transformer-XL network to extract the spatio-temporal dynamic features Zt between each topological graph.

[0098] Continuing the previous example, if there are less than 5 topological graphs, for example, 4, the spatial features of these 4 topological graphs can be directly input into the Transformer-XL network.

[0099] Among them, the Transformer-XL network is a neural network with a multi-head attention mechanism. In the embodiments of the present invention, it is set to four attention heads, and its expression formula is: (3);

[0100] Among them, - represents an attention head; represents a fully connected layer that connects four attention heads to a 512 - dimensional feature space.

[0101] (4);

[0102] (5);

[0103] Among them, represents the - th attention head; represents the query vector of the - th attention head; represents the key vector of the - th attention head; represents the value vector of the - th attention head; represents the dimension of the key vector; , , respectively represent the weights of each vector.

[0104] Through formula 4 of this attention head, it can be seen that the working principle of each attention head is as follows: First, calculate the similarity score between the query and the key vector, and scale it by dividing by the dimension of the key vector. Then, use the softmax function to transform these scores into an attention weight in the form of a probability distribution. Finally, apply these weights to the value vector to obtain the output after weighted summation.

[0105] Compared with the traditional Transformer, Transformer - XL introduces a memory mechanism. This mechanism allows the Transformer - XL network to automatically learn the dependencies between different positions when processing sequence data without the need to pre - define a fixed window size or distance limit. It can capture long - range dependencies across windows by retaining the hidden states of previous windows; and by retaining the hidden state data of previous windows, it does not need to repeat the calculation for each sequence, greatly reducing the computational amount, improving the feature extraction efficiency, thereby improving the embedding efficiency of the virtual network and reducing the large - scale virtual request migration.

[0106] On the basis of the above - mentioned embodiments, the node policy generation network adopts a deep reinforcement learning network based on the A3C algorithm. As Figure 2 shown, the deep reinforcement learning network based on the A3C algorithm includes an Actor network and a Critic network; the pre - training process of the node policy generation network includes:

[0107] Step a: Obtain the current environmental state of multiple UAV cluster networks.

[0108] The current environmental state includes at least the CPU capacity, remaining CPU capacity, bandwidth capacity, remaining bandwidth capacity, and maximum network latency of UAV nodes.

[0109] Step b: Based on the current environmental state and virtual mapping requests, select the UAV nodes to be mapped in the UAV cluster network through the Actor network to obtain a node mapping strategy.

[0110] In this step, all possible node mapping actions in the current environmental state are obtained based on the virtual mapping requests; for example, virtual nodes a, b, c; UAV nodes include A, B, C, D. Mapping three virtual nodes to four UAV nodes, there are 24 possible node mapping strategies. Through this Actor network, the reward values of each node mapping strategy can be obtained. Apply the node mapping strategy corresponding to the larger reward value to the actual scenario, obtain the next environmental state after application, and then continue to train for all possible node mapping actions in the next environmental state until the training is completed.

[0111] As Figure 2 shown, after the training is completed, during actual deployment and application, the dynamic characteristics of the UAV cluster network output by the Transformer-XL network can be used as the input of this Actor network. The dynamic characteristics of the UAV cluster network are equivalent to the environmental state of this UAV cluster network.

[0112] Step c: Calculate the gradient update direction of the Actor network through the policy gradient algorithm, and calculate the advantage function value of the Actor network along the gradient update direction until the advantage function value is minimized.

[0113] Among them, the policy gradient algorithm is as follows:

[0114] (6);

[0115] In the formula: is the gradient parameter of the Actor network; is the current environmental state all actions of the node mapping strategy; is the logarithm of the policy distribution likelihood ratio about the parameter gradient, also known as the score function; is the decay parameter. The decay parameter β is set to be large at the beginning and gradually shrinks during the training phase;

[0116] In order to prevent premature convergence to local optimal solutions and encourage exploration, an entropy regularization term is added to this policy gradient algorithm. This term is multiplied by a small decay parameter and subtracted from the policy gradient, thus gradually reducing its influence during training.

[0117] The Actor network is mainly used to generate node mapping strategies; the goal of the Critic network is to learn the value function and is mainly used to estimate the long-term reward when following the policy in a given state . The Critic network learns by minimizing the square of the TD error. Except for the output layer, these two networks share a similar structure.

[0118] Step d: Evaluate the value of the node mapping strategy generated by the Actor network in the current environmental state based on the Critic network;

[0119] Step e: Train the Critic network by minimizing the TD error until the TD error is minimized.

[0120] This TD error is calculated as follows:

[0121] (7);

[0122] In the formula: represents the estimated value function in state ; are the parameters of the Critic network; is the learning rate of the Critic network; is the immediate reward; is the discount factor, which is used to adjust the importance of future rewards.

[0123] The A3C algorithm combines the updates of these two networks and uses a multi-threaded asynchronous update method to learn in parallel in different environmental instances, thus improving the learning efficiency and stability.

[0124] Based on the above embodiments, mapping the virtual nodes to be embedded into the drone nodes of the drone cluster network based on the node mapping strategy includes:

[0125] Sort the virtual nodes to be embedded in descending order according to the CPU demand of the virtual nodes in the virtual embedding request;

[0126] Map the virtual node ranked first to the drone node of the drone cluster network according to the node mapping strategy;

[0127] Judge whether the virtual node ranked first is successfully mapped.

[0128] In a feasible real-time manner, determining whether the first virtual node is successfully mapped includes:

[0129] Obtaining the CPU demand and network latency of the first virtual node; and the CPU capacity and network latency of the drone node to which the first virtual node is mapped;

[0130] Determining whether the CPU capacity and network latency of the drone node are greater than the CPU demand and network latency of the first virtual node;

[0131] If it is greater, the mapping is successful.

[0132] If the mapping is successful, continue to map the next virtual node to the drone nodes in the drone cluster network according to the sorting and the node mapping strategy until all virtual nodes are successfully mapped.

[0133] If one virtual node fails to be mapped, reject the virtual network embedding request.

[0134] Based on the above embodiments, the method further includes:

[0135] Determining whether all virtual links are successfully mapped.

[0136] In a feasible implementation manner, determine whether the link is successfully mapped by determining whether the bandwidth and maximum network latency of the physical link between drone nodes meet the bandwidth and maximum network latency of the virtual link. If both are met, the link mapping is successful.

[0137] If any virtual link fails to be mapped, reject the virtual network embedding request.

[0138] After generating the node mapping strategy and the link mapping path, when mapping nodes through the node mapping strategy and mapping virtual links through the shortest path, by determining again whether the mapping is successful, the accuracy of the mapping is further ensured.

[0139] Finally, in order to verify the performance of the node mapping strategy generated by the deep reinforcement learning network after extracting the dynamic features of the network through the feature extraction network composed of the graph convolutional network GCN and the Transformer-XL network provided by the embodiments of the present invention, it is illustrated by a group of experiments. For simplicity, the feature extraction network composed of GCN and the Transformer-XL network is hereinafter referred to as GCFormer.

[0140] The network is generated by the simulation software GT-ITM to conduct simulation experiments, and the initialization experimental data required for the simulation experiments is set: the experimental topology includes 550 physical links and 100 physical nodes. The CPU capacity of each physical node is uniformly distributed within (50, 100), and the bandwidth is uniformly distributed between (20, 50). Two virtual network request sets are generated, and each request set contains 1000 virtual embedding requests. One set is used for training, and the other set is used for testing. The number of nodes in each virtual request is uniformly distributed between (2, 10), and the CPU of the virtual request is uniformly distributed between (0, 50). The bandwidth resources of each virtual link are evenly distributed between (0, 50). The connection probability between virtual links is set to 0.5. Among them, the arrival time of virtual embedding requests follows a Poisson distribution. On average, 4 virtual embedding requests arrive within 100 time units. The duration of each virtual embedding request follows an exponential distribution, and the average duration is 1000 time units.

[0141] On this basis, the simulation effects of the heuristic algorithms NodeRank algorithm, RLVNE algorithm, and GCN-VNE algorithm are compared. As Figure 3 , Figure 4 , Figure 5 shown, they are the evaluation indicators of the four algorithms respectively; Figure 3 is the long-term average revenue indicator; Figure 4 is the long-term revenue-cost ratio indicator; Figure 5 is the request acceptance rate indicator.

[0142] It can be found that the heuristic NodeRank algorithm gives priority to heavyweight nodes, so the initial performance is reasonable. However, from the perspective of long-term performance, it is difficult to effectively characterize dynamic multi-domain networks, and the performance is poor. RLVNE and GCN-VNE are superior to NodeRank in all evaluation indicators, which fully shows that the introduction of AI algorithms significantly improves the performance of VNE. In addition, GCFormer performs better than RLVNE and GCNVNE in the long term. This is because the embodiments of the present invention can better understand the dynamic characteristics of nodes in a dynamic environment and can better focus on the attributes of the dynamic physical environment.

[0143] In summary, in the dynamic network scenario, GCFormer has better performance and better reduces the fragmentation of physical resources.

[0144] Based on the same inventive concept, a virtual network mapping device based on an unmanned aerial vehicle cluster scenario is provided. As Figure 6 shown, the device includes:

[0145] An acquisition unit 601, configured to acquire the topological graph sequence of the unmanned aerial vehicle cluster network when receiving a virtual network embedding request.

[0146] The virtual network embedding request includes at least the virtual nodes and virtual links to be embedded; the topological graph sequence is composed of topological graphs captured at each snapshot moment.

[0147] An extraction unit 602, configured to extract spatio-temporal dynamic features between each topological graph in the topological graph sequence based on a pre-constructed feature extraction network.

[0148] A generation unit 603, configured to input the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy.

[0149] A node mapping unit 604, configured to map the virtual nodes to be embedded to the drone nodes in the drone cluster network based on the node mapping policy.

[0150] A link mapping unit 605, configured to, after all the virtual nodes are successfully mapped, determine the shortest physical links between the mapped drone nodes based on the Floyd algorithm, and map the virtual links to be embedded to the shortest physical links.

[0151] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, as Figure 7 shown, including a processor 701, a communication interface 702, a memory 703, and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0152] The memory 703 is used to store a computer program;

[0153] The processor 701, when executing the program stored on the memory 703, implements the steps of the virtual network mapping method based on the drone cluster scenario.

[0154] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above electronic device and other devices.

[0156] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0157] The aforementioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] The computer program product for performing virtual network mapping in the UAV cluster scenario provided by the embodiments of the present invention includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the methods described in the foregoing method embodiments. For specific implementation, reference can be made to the method embodiments, which will not be elaborated herein.

[0159] The device for virtual network mapping in the UAV cluster scenario provided by the embodiments of the present invention may be specific hardware on the device, or software or firmware installed on the device, etc. For the device provided by the embodiments of the present invention, the implementation principle and the resulting technical effects are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference can be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can all refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated herein.

[0160] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

[0161] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional unit in the embodiments provided by the present invention may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0163] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0164] It should be noted that: similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0165] Finally, it should be noted that: the above-described embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than to limit it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A virtual network mapping method based on the scenario of UAV clusters, characterized in that, The method includes: When receiving a virtual network embedding request, obtaining a sequence of topology graphs of the UAV cluster network; at least a virtual node and a virtual link to be embedded are included in the virtual network embedding request; the sequence of topology graphs is composed of topology graphs captured at each snapshot moment; Extracting the spatio-temporal dynamic features between the topology graphs in the sequence of topology graphs based on a pre-constructed feature extraction network; The feature extraction network is composed of a graph convolutional network and a Transformer-XL network; the extracting the spatio-temporal dynamic features between the topology graphs in the sequence of topology graphs based on a pre-constructed feature extraction network includes: Successively extracting the spatial features of each topology graph based on the graph convolutional network; the graph convolutional network includes two hidden layers; The extracting the spatial features of each topology graph includes: For each topology graph, determining a first adjacency matrix of the UAV nodes in the topology graph and an attribute feature matrix of the UAV cluster network; Inputting the first adjacency matrix and the attribute feature matrix into the first hidden layer to obtain initial spatial features; Inputting the initial spatial features into the second hidden layer to extract higher-dimensional feature information, and obtaining the final spatial features of the topology graph; Judging whether the number of topology graphs in the sequence of topology graphs exceeds a preset number threshold; If it exceeds the preset number threshold, jointly inputting the spatial features of the preset number threshold of the most recently captured topology graphs into the Transformer-XL network to extract the spatio-temporal dynamic features between the preset number threshold of the topology graphs; If it does not exceed the preset number threshold, jointly inputting all the spatial features extracted by the graph convolutional network into the Transformer-XL network to extract the spatio-temporal dynamic features between the topology graphs; The Transformer-XL network introduces an attention mechanism, which is set to four attention heads, and its representation formula is: ; Among them, - represents an attention head; represents a fully connected layer that connects four attention heads to a 512-dimensional feature space; Inputting the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping policy; The node policy generation network adopts a deep reinforcement learning network based on the A3C algorithm, and the deep reinforcement learning network based on the A3C algorithm includes an Actor network and a Critic network; the pre-training process of the node policy generation network includes: Obtaining the current environmental states of multiple UAV cluster networks; the current environmental states at least include the CPU capacity, remaining CPU capacity, bandwidth capacity, remaining bandwidth capacity and maximum network delay of the UAV nodes; Based on the current environmental states and the virtual mapping request, selecting the UAV nodes to be mapped in the UAV cluster network through the Actor network to obtain a node mapping policy; Calculating the gradient update direction of the Actor network through a policy gradient algorithm, and calculating the advantage function value of the Actor network along the gradient update direction until the advantage function value is the smallest; Evaluating the value of the node mapping policy generated by the Actor network in the current environmental state based on the Critic network; Training the Critic network by minimizing the TD error until the TD error is minimized; Mapping the virtual nodes to be embedded into the UAV nodes of the UAV cluster network based on the node mapping policy; After all virtual nodes are successfully mapped, determine the shortest physical links between the mapped UAV nodes based on the Floyd algorithm, and map the virtual links to be embedded into the shortest physical links.

2. The method according to claim 1, characterized in that, The mapping of the virtual nodes to be embedded into the UAV nodes of the UAV cluster network based on the node mapping strategy includes: Sort the virtual nodes to be embedded in descending order according to the CPU requirements of the virtual nodes in the virtual embedding request; Map the virtual node ranked first to the UAV node of the UAV cluster network according to the node mapping strategy; Determine whether the virtual node ranked first is successfully mapped; If the mapping is successful, continue to map the next virtual node to the UAV node of the UAV cluster network according to the sorting until all virtual nodes are successfully mapped.

3. The method according to claim 2, characterized in that, The determination of whether the virtual node ranked first is successfully mapped includes: Obtain the CPU requirements and network latency of the virtual node ranked first; and the CPU capacity and network latency of the UAV node to which the virtual node ranked first is mapped; Determine whether the CPU capacity and network latency of the UAV node are greater than the CPU requirements and network latency of the virtual node ranked first; If it is greater, the mapping is successful.

4. The method according to claim 1, wherein The method further includes: Determine whether all virtual links are successfully mapped; If any virtual link mapping fails, reject the virtual network embedding request.

5. A virtual network mapping device based on an unmanned aerial vehicle cluster scenario, characterized in that, Applied to the virtual network mapping method based on the UAV cluster scenario according to any one of claims 1-4, the device includes: An acquisition unit, configured to acquire a topological graph sequence of the UAV cluster network when receiving a virtual network embedding request; at least the virtual nodes and virtual links to be embedded are included in the virtual network embedding request; the topological graph sequence is composed of topological graphs captured at each snapshot moment; An extraction unit, configured to extract the spatio-temporal dynamic features between the topological graphs in the topological graph sequence based on a pre-constructed feature extraction network; A generation unit, configured to input the spatio-temporal dynamic features into a pre-trained node policy generation network to generate a node mapping strategy; A node mapping unit, configured to map the virtual nodes to be embedded to the UAV nodes of the UAV cluster network based on the node mapping strategy; A link mapping unit, configured to determine the shortest physical links between the mapped UAV nodes based on the Floyd algorithm after all virtual nodes are successfully mapped, and map the virtual links to be embedded into the shortest physical links.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store a computer program; The processor is configured to implement the method steps according to any one of claims 1-4 when executing the program stored on the memory.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method steps according to any one of claims 1-4 are implemented.

Images