Wireless access network slice deployment method and device, storage medium and computer equipment

Abstract

The invention provides a radio access network slice deployment method and device, a storage medium and computer equipment, and is applied to a 5G network, and the method comprises the steps: obtaining the state information of a target RAN slice request when the target RAN slice request is received, and extracting the current state feature information of a physical network through a pre-trained GAT network, the physical network receives the target RAN slice request to obtain a state set corresponding to the target RAN slice request, and comprises a plurality of physical nodes; and according to the state set corresponding to the target RAN slice request, adopting a pre-trained DDPG algorithm to select and execute a deployment action, so as to deploy an RAN slice corresponding to a required VNF to a physical node in a physical network according to the VNF required by the RAN slice request, and a reward function corresponding to the DDPG algorithm being used for minimizing end-to-end delay and deployment cost. In this way, the end-to-end time delay and deployment cost of RAN slice deployment are effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of wireless network technology, and in particular to a method, apparatus, storage medium and computer equipment for deploying wireless access network slices. Background Art

[0002] The power industry needs to accurately implement differential protection, resource scheduling, security monitoring, automatic control, and intelligent operation and maintenance functions in the entire process of generation, transmission, transformation, distribution, use, and management. It is urgent to build a 5G power virtual private network with large traffic bandwidth, low transmission latency, and wide connection of IoT terminals. It is also mobile, highly reliable, and highly secure. RAN (Radio Access Network) slices are composed of several virtual network functions (VNFs). According to different business needs, VNFs are connected in a certain order to provide users with specific network services. RAN slices divide a physical network into multiple virtual logical subnets with different characteristics to meet the application scenarios of high bandwidth, high security, low latency, and ultra-high reliability of the power grid, and provide customized services for users' differentiated needs, thereby formulating the best solution for smart grid communications.

[0003] In the process of RAN slice deployment, end-to-end latency is an important indicator for measuring user QoS. However, most current studies focus on deployment costs and network overhead, without considering reducing the end-to-end latency and deployment costs of RAN slice deployment. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect that the prior art does not consider reducing the end-to-end latency and deployment cost of RAN slice deployment.

[0005] In a first aspect, the present application provides a method for deploying a wireless access network slice, which is applied to a 5G network. The method includes:

[0006] When receiving a target RAN slice request, obtaining state information of the target RAN slice request, and extracting current state feature information of a physical network through a pre-trained GAT network to obtain a state set corresponding to the target RAN slice request, where the physical network includes a plurality of physical nodes;

[0007] According to the state set corresponding to the target RAN slice request, the pre-trained DDPG algorithm is used to select and execute deployment actions to deploy the RAN slices corresponding to the required VNFs to the physical nodes in the physical network according to the VNFs required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

[0008] In one embodiment, the step of extracting the current state feature information of the physical network through the pre-trained GAT network includes:

[0009] Determine the features of each physical node in the F dimension and apply linear transformation to obtain new features of each physical node in the F' dimension;

[0010] According to the new features of each physical node in the F' dimension, the attention coefficient between the physical node and each neighboring physical node is calculated, and then normalized to obtain the attention weight;

[0011] For each physical node, the attention weight of the physical node is used to perform weighted summation on each neighboring physical node of the physical node and its new features in the F' dimension to obtain the current state feature information of the physical node.

[0012] In one embodiment, the training process of the DDPG algorithm includes:

[0013] Obtain N deployment experience samples in the 5G network, each deployment experience sample including current state feature information of the physical network, current deployment action, current instant reward, and next state feature information;

[0014] In each training round, M deployment experience samples are randomly selected from each deployment experience sample, and the corresponding target Q value is calculated, and the parameters of the current Critic network are updated by minimizing the loss function. The policy gradient of the current Actor network is calculated according to the action value function calculated by the updated current Critic network, and the parameters of the current Actor network are updated by minimizing the policy gradient. The parameters of the target Actor network and the target Critic network are updated by soft update, and the next training round is entered until the trained DDPG algorithm meets the training end conditions.

[0015] In one embodiment, the expression corresponding to the current deployment action is:

[0016] a t =μ(s t |θ μ )+V t

[0017] In the formula, μ(s t |θ μ ) indicates that the target Actor network calculates the action in the current state, V t For random actions.

[0018] In one embodiment, the reward function is:

[0019]

[0020] Where ω,σ∈[0,1] represent the weight factors of the end-to-end delay and deployment cost of the deployment, respectively. k represents the end-to-end delay of the kth RAN slice instance deployment, E k represents the RAN slice deployment cost;

[0021] C1-C7 are constraints. Constraint C1 is a binary variable for whether the VNF is mapped to a physical node. Constraint C2 is a binary variable for whether a virtual link is mapped to a physical link. Constraint C3 means that a VNF can only be mapped to one physical node. Constraint C4 means that the VNFs in the same RAN slice request need to be deployed on different nodes. Constraint C5 means that the deployment delay does not exceed the tolerable delay. Constraints C6 and C7 mean that the total computing resources consumed by all VNFs and the total bandwidth resources consumed do not exceed the computing resources of the physical node and the bandwidth resources of the link, respectively.

[0022] In one embodiment, the expression corresponding to the deployment cost is:

[0023]

[0024] In the formula, m1 and m2 represent the unit cost of computing resources and bandwidth resources respectively, C k and B k Represent computing resources and bandwidth resources respectively, Indicates that in the kth RAN slice request When mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, express The computing resources consumed, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links 1 if yes, 0 otherwise. Indicates that the virtual link Mapping to physical links The number of hops, Indicates that the virtual link Connected and The bandwidth consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0025] In one embodiment, the expression corresponding to the end-to-end delay is:

[0026]

[0027] In the formula, represents the processing delay of the kth RAN slice request, represents the total transmission delay, Indicates that in the kth RAN slice request If it is mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, Indicates that in the kth slice request, Processing delay at physical node m, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links If yes, it is 1, otherwise it is 0. Indicates that the virtual link will be Connected and The transmission delay consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0028] In a second aspect, the present application provides a radio access network slice deployment device, which is applied to a 5G network, and the device includes:

[0029] A state set determination module is used to obtain state information of the target RAN slice request when receiving the target RAN slice request, and extract current state feature information of the physical network through a pre-trained GAT network to obtain a state set corresponding to the target RAN slice request, where the physical network includes multiple physical nodes;

[0030] The RAN slice deployment module is used to select and execute deployment actions according to the state set corresponding to the target RAN slice request using the pre-trained DDPG algorithm, so as to deploy the RAN slice corresponding to the required VNF to the physical node in the physical network according to the VNF required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

[0031] In a third aspect, the present application provides a storage medium: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the wireless access network slice deployment method as described in any of the above embodiments.

[0032] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0033] Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the steps of the wireless access network slice deployment method in any of the above embodiments are performed.

[0034] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0035] In the wireless access network slice deployment method, device, storage medium and computer equipment provided in this application, the state characteristic information of the physical network is extracted through the pre-trained GAT network, which can more accurately understand the current state of the physical network to help optimize resource allocation. The DDPG algorithm is used to make decisions based on the state set corresponding to the target RAN slice request and select the optimal deployment action. The DDPG algorithm aims to minimize the reward function of end-to-end delay and deployment cost, which can minimize deployment and operating costs. The 5G network covers a variety of application scenarios, each of which has different requirements for network performance and service quality. This method can flexibly customize the deployment of network resources according to different RAN slice requests, and support diverse service requirements and application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A schematic diagram of a process for deploying a wireless access network slice provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a 5G network system model provided in an embodiment of the present application;

[0039] Figure 3 A model structure diagram of GAT-DDPG provided in an embodiment of the present application;

[0040] Figure 4 A comparison diagram of end-to-end delay provided in an embodiment of the present application;

[0041] Figure 5 A comparison chart of deployment costs provided for embodiments of the present application;

[0042] Figure 6 A schematic diagram of the structure of a wireless access network slice deployment device provided in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] This application provides a wireless access network slice deployment method, which is applied to 5G networks, such as Figure 1 As shown, the method includes:

[0046] S101: When a target RAN slice request is received, the state information of the target RAN slice request is obtained, and the current state feature information of the physical network is extracted through a pre-trained GAT network to obtain a state set corresponding to the target RAN slice request, where the physical network includes multiple physical nodes.

[0047] RAN (Radio Access Network) slicing refers to the logical segmentation of the radio access network according to different business needs and service requirements in the 5G network. A target RAN slice request indicates the resources and configuration that need to be allocated in the network for a specific business or service. State information refers to the specific parameters and requirements of the target RAN slice request, such as service level agreement (SLA), bandwidth requirements, latency requirements, etc. State information is the basic data for the system to understand and process RAN slice requests.

[0048] GAT (Graph Attention Network) is a graph attention network used to process graph data. Its characteristic is that it can learn and model the relationship between nodes in the graph. Here, the pre-trained GAT network is used to analyze and extract the state feature information of each node in the physical network. The current state feature information of the physical network refers to the operating status, load conditions, network topology and other information of each node in the physical network. Through the GAT network, this information can be extracted in the form of state features to form a description of the overall physical network state.

[0049] In this step, a RAN slice request is received from the service requester. The request may contain service requirement parameters, such as bandwidth and QoS level requirements. Parse and extract key status information in the target RAN slice request, which is the basic conditions and restrictions for specifying service requirements. The pre-trained GAT network extracts the current status feature information of each node by learning the attention mechanism of the nodes of the physical network, which may include the node's load, connection status, distance, etc. The status information of the target RAN slice request and the status feature information of the physical network are combined to form a comprehensive status set. This status set describes the potential resources and configuration options that can meet the target RAN slice request under the current network conditions.

[0050] In one example, based on Figure 2 The system model shown in the figure is for a RAN slice request S k , the state set includes the state characteristic information s of the physical network p and RAN slice request status information k Among them, the state information of the physical network is extracted through the GAT network, including the CPU resources of each node in the physical network and CPU clock frequency Bandwidth resources of each physical link And transmission delay Slice request status information including the tolerable latency of RAN slices Required VNF Set and virtual link set There is also a collection of computing resources required by each VNF and the bandwidth resource set of the virtual link Therefore, the state set can be expressed as the formula:

[0051]

[0052] S102: According to the state set corresponding to the target RAN slice request, a pre-trained DDPG algorithm is used to select and execute a deployment action, so as to deploy the RAN slice corresponding to the required VNF to the physical node in the physical network according to the VNF required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

[0053] VNF (Virtualized Network Function) is a network function that is implemented in software and can run in a virtualized environment. For example, network functions such as firewalls, routers, and load balancers can be implemented in the form of VNFs. The DDPG algorithm (Deep Deterministic Policy Gradient Algorithm) is a deep reinforcement learning algorithm for continuous action space that combines the policy gradient method and Q-learning. It performs well in optimizing continuous control problems and is suitable for dynamic resource allocation and optimization problems in this scenario. In reinforcement learning, the reward function defines the goal of the system at each time step. In this case, the reward function is designed to minimize end-to-end latency and deployment cost to provide optimal network resource configuration.

[0054] In this step, based on the pre-trained DDPG algorithm, the optimal deployment action is selected according to the current state set, that is, according to the decision of the DDPG algorithm, the required VNF is deployed to the most suitable node selected in the physical network. At each time step, the effect of the current deployment is evaluated according to the reward function. The goal of the reward function is to minimize the end-to-end delay and deployment cost.

[0055] In the above embodiment, by extracting the state feature information of the physical network through the pre-trained GAT network, the current state of the physical network can be understood more accurately to help optimize resource allocation. The DDPG algorithm is used to make decisions based on the state set corresponding to the target RAN slice request and select the optimal deployment action. The DDPG algorithm aims to minimize the reward function of end-to-end delay and deployment cost, which can minimize deployment and operating costs. The 5G network covers a variety of application scenarios, each of which has different requirements for network performance and service quality. This method can flexibly customize the deployment of network resources according to different RAN slice requests, and support diverse service requirements and application scenarios.

[0056] In one embodiment, the step of extracting the current state feature information of the physical network through the pre-trained GAT network includes:

[0057] Determine the features of each physical node in the F dimension and apply linear transformation to obtain new features of each physical node in the F' dimension;

[0058] According to the new features of each physical node in the F' dimension, the attention coefficient between the physical node and each neighboring physical node is calculated, and then normalized to obtain the attention weight;

[0059] For each physical node, the attention weight of the physical node is used to perform weighted summation on each neighboring physical node of the physical node and its new features in the F' dimension to obtain the current state feature information of the physical node.

[0060] Specifically, the GAT network is used to extract the state features of the physical network represented as a directed acyclic graph. When extracting features, the GAT network follows the message passing paradigm and introduces an attention mechanism. First, given a feature set of N nodes The feature dimension of each node is F. The goal of GAT is to output a new node feature set That F′ is the dimension of the output features, which may be different from the original dimension F.

[0061] When calculating the attention coefficient, for nodes i and j, a linear transformation is first applied to their features, that is, the learnable parameter W is used to transform the features from F dimensions into F′-dimensional new features, and the obtained and Then, calculate the attention coefficient e ij , which represents the attention value of node i relative to node j. The calculation formula of attention coefficient is as follows

[0062]

[0063] in, Indicates that the shared parameter W is used Perform linear transformation; [||] represents concatenation of the transformed results of nodes i and j; a(·) represents mapping the concatenated high-dimensional features to a real number. Get all e ij After that, the softmax(·) function is used for normalization to get the final attention weight a ij :

[0064]

[0065] Among them, LeakyReLU(·) represents the activation function. After obtaining the attention weights, GAT uses these weights to perform weighted summation on the features of neighboring nodes to update the feature vector of the node. The formula is expressed as:

[0066]

[0067] Among them, σ(·) represents the activation function.

[0068] In one embodiment, the training process of the DDPG algorithm includes:

[0069] Obtain N deployment experience samples in the 5G network, each deployment experience sample including current state feature information of the physical network, current deployment action, current instant reward, and next state feature information;

[0070] In each training round, M deployment experience samples are randomly selected from each deployment experience sample, and the corresponding target Q value is calculated, and the parameters of the current Critic network are updated by minimizing the loss function. The policy gradient of the current Actor network is calculated according to the action value function calculated by the updated current Critic network, and the parameters of the current Actor network are updated by minimizing the policy gradient. The parameters of the target Actor network and the target Critic network are updated by soft update, and the next training round is entered until the trained DDPG algorithm meets the training end conditions.

[0071] Specifically, if Figure 3 As shown in the figure, the DDPG algorithm includes two stages in the training process, the learning stage and the model training stage. In the learning stage, the experience samples (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool D to improve training efficiency and stability. Each sample contains the current state s t , Action taken a t , the reward r t and the state s at the next moment t+1 , used for subsequent training. In the model training phase, the data in the experience pool is used to train and update the network. Then the target Q value is calculated, and the parameters of the online critic network are updated by minimizing the loss function, which are expressed as:

[0072]

[0073] y=r(s t ,a t )+γQ′(s t+1 ,μ′(s t+1 |Q μ′ )|θ Q′ )

[0074] Among them, θ Q represents the parameters of the online critic network, θ μ′ and θ Q′ Represent the parameters of the target Actor network and the target Critic network respectively. Q(s t ,a t |θ Q ) represents the online action value function of the agent, and y represents the target Q value.

[0075] Next, according to the action value function calculated by the Critic network, the policy gradients of the Actor network and the Critic network are calculated, and the parameters of the Actor network and the Critic network are updated by minimizing the policy gradient and minimizing the loss function respectively. The policy gradients of the Actor network and the Critic network can be expressed as follows:

[0076]

[0077] Among them, μ(s t |θ μ ) represents the online exploration strategy function of the agent, θ μ represents the parameters of the online Actor network. Therefore, the network parameters θ μ and θ Q The expressions are as follows:

[0078]

[0079] Among them, ε and χ represent the learning rates of the Actor network and the Critic network respectively. Finally, the target Actor network parameter θ is updated by soft update μ′ and target critic network parameters θ Q′ , the formula is as follows:

[0080]

[0081] In one embodiment, the expression corresponding to the current deployment action is:

[0082] a t =μ(s t |θ μ )+V t

[0083] In the formula, μ(s t |θ μ ) indicates that the target Actor network calculates the action in the current state, V t For random actions.

[0084] In this embodiment, μ(s t |θ μ ) provides action selection based on learning strategies, which helps to utilize the best strategies learned; V t Providing a certain degree of randomness gives the agent a chance to explore, avoids being trapped in local optimal solutions, and helps discover new and potentially better strategies.

[0085] In one embodiment, the reward function is:

[0086]

[0087] Where ω,σ∈[0,1] represent the weight factors of the end-to-end delay and deployment cost of the deployment, respectively. k represents the end-to-end delay of the kth RAN slice instance deployment, E k represents the RAN slice deployment cost;

[0088] C1-C7 are constraints. Constraint C1 is a binary variable for whether the VNF is mapped to a physical node. Constraint C2 is a binary variable for whether a virtual link is mapped to a physical link. Constraint C3 means that a VNF can only be mapped to one physical node. Constraint C4 means that the VNFs in the same RAN slice request need to be deployed on different nodes. Constraint C5 means that the deployment delay does not exceed the tolerable delay. Constraints C6 and C7 mean that the total computing resources consumed by all VNFs and the total bandwidth resources consumed do not exceed the computing resources of the physical node and the bandwidth resources of the link, respectively.

[0089] The size of the reward value is used to evaluate the quality of the currently selected action. Therefore, it is necessary to design a suitable reward function to continuously optimize the agent's strategy. If the constraints of C1-C7 are met, it means that the currently selected action can successfully deploy the VNF and give the agent a positive reward value. Otherwise, the reward value is 0.

[0090] Specifically, based on Figure 2 The system model shown in the figure constructs the underlying physical network as an undirected graph Gp = (Np, Lp), where N p Represents a collection of physical nodes, providing computing and storage resources. Each physical node can deploy one or more VNFs. p Represents the set of physical links. For a physical node n, the computing resources it has are expressed as The physical link l between nodes m and n mn ∈L p The bandwidth resource is Transmission delay representation

[0091] The RAN slice request set is represented as S = {S1, S2, ..., S K}, where K represents the number of RAN slice requests. Each RAN slice instance consists of multiple ordered VNFs and virtual links. The kth slice request can be represented by a weighted directed graph as in represents the set of VNFs required for the k-th RAN slice request, Represents a collection of virtual links, represents the maximum tolerable delay of this service request. The i-th VNF in the k-th slice request is represented as The required computing resources are expressed as Through the virtual link Connected and The bandwidth consumed is expressed as Transmission delay

[0092] For RAN slice request It is necessary to map the required VNFs to physical nodes and map virtual links to physical links. Using binary variables Indicates that in the kth slice request Whether it is mapped to the physical node m, and the expression is defined as follows:

[0093]

[0094] Furthermore, the VNF in each RAN slice instance request can only be mapped to one physical node, so:

[0095]

[0096] Using binary variables Indicates the virtual link in the kth slice request Whether to map to a physical link The formula is defined as follows:

[0097]

[0098] The present invention realizes the reasonable deployment of RAN slices by selecting appropriate deployment nodes and physical links for VNFs in RAN slices, so as to minimize the end-to-end delay and deployment cost of deployment. The optimization goal can be expressed as follows:

[0099] U k =ωT k +σE k

[0100] Where ω,σ∈[0,1], respectively represent the weight factors of the end-to-end delay and deployment cost of the deployment. Therefore, the optimization problem of the present invention can be described as follows:

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] In this embodiment, for the constraints, C1 and C2 ensure the correctness and connectivity of network functions, avoiding link breaks or function deployment errors in the virtual network; C3 ensures that each VNF can only be mapped to one physical node, simplifying the network structure and management, and reducing resource conflicts; C4 ensures that the VNFs in the same RAN slice request are deployed on different nodes, enhancing redundancy and reliability, as well as fault isolation capabilities; C5 ensures that the end-to-end delay of the deployment does not exceed the acceptable threshold. By controlling the deployment delay, the user experience can be optimized and the service quality can be avoided; C6 and C7 limit the consumption of computing resources and bandwidth resources respectively, avoiding over-allocation or under-allocation of resources, thereby ensuring the scalability and performance stability of the network. The reward function in the form of an exponential function weights the end-to-end delay and deployment cost of the deployment, and can decrease rapidly mathematically. This design encourages the optimization algorithm to tend to reduce delay and cost when selecting a deployment strategy, because the reward decreases exponentially with the increase of these indicators. By adjusting the weight factors ω and σ, the optimization goals between delay and cost can be balanced.

[0110] In one embodiment, the expression corresponding to the deployment cost is:

[0111]

[0112] In the formula, m1 and m2 represent the unit cost of computing resources and bandwidth resources respectively, C k and B k Represent computing resources and bandwidth resources respectively, Indicates that in the kth RAN slice request When mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, express The computing resources consumed, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links 1 if yes, 0 otherwise. Indicates that the virtual link Mapping to physical links The number of hops, Indicates that the virtual link Connected and The bandwidth consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0113] Specifically, the present invention defines the deployment cost as the cost caused by the computing resources required to deploy the RAN slice instance and the bandwidth resource consumption. First, the computing cost is related to the total CPU resources consumed, which can be expressed as:

[0114]

[0115] For the kth RAN slice request, the bandwidth cost is defined as the bandwidth resource consumed by mapping the virtual link to the physical link, which can be expressed as:

[0116]

[0117] in, Indicates that the virtual link Mapping to physical links Therefore, the RAN slice deployment cost can be expressed as:

[0118] E k =m1C k +m2B k

[0119] Among them, m1 and m2 represent the unit cost of computing resources and bandwidth resources respectively.

[0120] In this embodiment, m1C k Represents the computing resource cost consumed by deploying each VNF, ensuring that each VNF deployed on a physical node is cost-effective because the actual usage and unit price of resources are taken into account; m2B k Taking into account the bandwidth consumption of VNFs connected through virtual links helps to optimize the allocation of network resources and ensure that the bandwidth resources used are not over-consumed while avoiding waste.

[0121] In one embodiment, the expression corresponding to the end-to-end delay is:

[0122]

[0123] In the formula, represents the processing delay of the kth RAN slice request, represents the total transmission delay, Indicates that in the kth RAN slice request If it is mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, Indicates that in the kth slice request, Processing delay at physical node m, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links If yes, it is 1, otherwise it is 0. Indicates that the virtual link will be Connected and The transmission delay consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0124] Specifically, the end-to-end delay is defined as the processing delay and transmission delay of RAN slice deployment. The processing delay is mainly determined by the CPU resources of the physical node. Therefore, in the kth slice request, The processing delay at physical node m can be expressed as:

[0125]

[0126] in, Indicates the CPU clock frequency of the physical node where the current VNF ​​is deployed. express The computing resources consumed, then for the kth RAN slice request, the calculation formula for the processing delay can be expressed as:

[0127]

[0128] The total transmission delay is the sum of the transmission delays of each hop link, which can be expressed as:

[0129]

[0130] Therefore, the end-to-end delay for the kth RAN slice instance deployment is expressed as:

[0131]

[0132] In this embodiment, the expression of end-to-end delay combines processing delay and transmission delay, and can comprehensively evaluate the overall time from RAN slice request to data transmission completion.

[0133] The GAT-DDPG algorithm proposed in this invention is compared in terms of end-to-end delay and deployment cost. Figure 4Among them, the ASR algorithm has the lowest end-to-end delay, followed by the GAT-DDPG algorithm proposed in the present invention. This is because the ASR algorithm only considers the optimization of deployment delay, without considering other factors, and will give priority to nodes and links with low latency. The algorithm proposed in the present invention and the DDPG algorithm jointly consider the end-to-end delay and deployment cost of RAN slice deployment, so they will not consume too many resources in exchange for lower deployment delay, but instead achieve a trade-off between the two. Therefore, the end-to-end delay is lower than that of the ASR algorithm. Figure 5 In terms of deployment cost, the GAT-DDPG algorithm proposed in the present invention has the lowest deployment cost. This is because the GAT-DDPG algorithm and the DDPG algorithm proposed in the present invention take into account the optimization of deployment cost, and will minimize the consumption of node computing resources and link bandwidth resources, and will not exchange excessive resource consumption for weak benefits, thus minimizing the deployment cost.

[0134] The following describes a wireless access network slice deployment device provided in an embodiment of the present application. The wireless access network slice deployment device described below and the wireless access network slice deployment method described above can be referred to each other. The present application provides a wireless access network slice deployment device, which is applied to a 5G network, such as Figure 6 As shown, the device comprises:

[0135] The state set determination module 201 is used to obtain the state information of the target RAN slice request when receiving the target RAN slice request, and extract the current state feature information of the physical network through the pre-trained GAT network to obtain the state set corresponding to the target RAN slice request, and the physical network includes multiple physical nodes;

[0136] The RAN slice deployment module 202 is used to select and execute deployment actions according to the state set corresponding to the target RAN slice request using a pre-trained DDPG algorithm, so as to deploy the RAN slice corresponding to the required VNF to the physical node in the physical network according to the VNF required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

[0137] In one embodiment, the state set determination module 201 includes:

[0138] A feature transformation unit, used to determine the feature of each physical node in the F dimension and apply a linear transformation to obtain a new feature of each physical node in the F' dimension;

[0139] An attention weight determination unit is used to calculate the attention coefficient between the physical node and each neighboring physical node according to the new features of each physical node in the F' dimension, and then perform normalization processing to obtain the attention weight;

[0140] The state feature information determination unit is used to, for each physical node, use the attention weight of the physical node to perform weighted summation on each neighboring physical node of the physical node and its new features in the F' dimension to obtain the current state feature information of the physical node.

[0141] In one embodiment, the RAN slice deployment module 202 includes:

[0142] A deployment experience sample unit is used to obtain N deployment experience samples in the 5G network, each deployment experience sample includes current state feature information of the physical network, current deployment action, current instant reward and next state feature information;

[0143] The DDPG algorithm training unit is used to randomly extract M deployment experience samples from each deployment experience sample in each training round, calculate the corresponding target Q value, and update the parameters of the current Critic network by minimizing the loss function. According to the action value function calculated by the updated current Critic network, the policy gradient of the current Actor network is calculated, and the parameters of the current Actor network are updated by minimizing the policy gradient. The parameters of the target Actor network and the target Critic network are updated by soft update, and the next training round is entered until the trained DDPG algorithm meets the training end conditions.

[0144] In one embodiment, the expression corresponding to the current deployment action is:

[0145] a t =μ(s t |θ μ )+V t

[0146] In the formula, μ(s t |θ μ ) indicates that the target Actor network calculates the action in the current state, V t For random actions.

[0147] In one embodiment, the reward function is:

[0148]

[0149] Where ω,σ∈[0,1] represent the weight factors of the end-to-end delay and deployment cost of the deployment, respectively. k represents the end-to-end delay of the kth RAN slice instance deployment, E k represents the RAN slice deployment cost;

[0150] C1-C7 are constraints. Constraint C1 is a binary variable for whether the VNF is mapped to a physical node. Constraint C2 is a binary variable for whether a virtual link is mapped to a physical link. Constraint C3 means that a VNF can only be mapped to one physical node. Constraint C4 means that the VNFs in the same RAN slice request need to be deployed on different nodes. Constraint C5 means that the deployment delay does not exceed the tolerable delay. Constraints C6 and C7 mean that the total computing resources consumed by all VNFs and the total bandwidth resources consumed do not exceed the computing resources of the physical node and the bandwidth resources of the link, respectively.

[0151] In one embodiment, the expression corresponding to the deployment cost is:

[0152]

[0153] In the formula, m1 and m2 represent the unit cost of computing resources and bandwidth resources respectively, C k and B k Represent computing resources and bandwidth resources respectively, Indicates that in the kth RAN slice request When mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, express The computing resources consumed, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links 1 if yes, 0 otherwise. Indicates that the virtual link Mapping to physical links The number of hops, Indicates that the virtual link Connected and The bandwidth consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0154] In one embodiment, the expression corresponding to the end-to-end delay is:

[0155]

[0156] In the formula, represents the processing delay of the kth RAN slice request, represents the total transmission delay, Indicates that in the kth RAN slice request If it is mapped to physical node m, it is 1, otherwise it is 0. represents the set of VNFs required for the k-th RAN slice request, Indicates that in the kth slice request, Processing delay at physical node m, N p Represents a collection of physical nodes, Represents the virtual link in the i-th slice request Mapping to physical links If yes, it is 1, otherwise it is 0. Indicates that the virtual link will be Connected and The transmission delay consumed, L p Represents a collection of physical links, Represents a collection of virtual links.

[0157] In one embodiment, the present application also provides a storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the steps of the wireless access network slice deployment method as described in any one of the above embodiments.

[0158] In one embodiment, the present application also provides a computer device, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the wireless access network slice deployment method as described in any one of the above embodiments.

[0159] Indicatively, Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 7 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the radio access network slice deployment method of any of the above embodiments.

[0160] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0161] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0162] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, "one", "one", "said", "the" and "it" may also include plural forms, unless the context clearly indicates another way. A plurality refers to at least two cases, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the relevant listed items.

[0163] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0164] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for deploying a wireless access network slice, characterized in that: Applied to a 5G network, the method comprises: When receiving a target RAN slice request, obtaining state information of the target RAN slice request, and extracting current state feature information of a physical network through a pre-trained GAT network to obtain a state set corresponding to the target RAN slice request, wherein the physical network includes a plurality of physical nodes; According to the state set corresponding to the target RAN slice request, a pre-trained DDPG algorithm is used to select and execute a deployment action, so as to deploy the RAN slice corresponding to the required VNF to the physical node in the physical network according to the VNF required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

2. The method for deploying wireless access network slices according to claim 1, characterized in that: The step of extracting the current state feature information of the physical network through the pre-trained GAT network includes: Determine the feature of each physical node in the F dimension and apply linear transformation to obtain a new feature of each physical node in the F' dimension; According to the new features of each physical node in the F' dimension, the attention coefficient between the physical node and each neighboring physical node is calculated, and then normalized to obtain the attention weight; For each of the physical nodes, the attention weight of the physical node is used to perform weighted summation on each neighboring physical node of the physical node and its new features in the F' dimension to obtain the current state feature information of the physical node.

3. The method for deploying wireless access network slices according to claim 1, characterized in that: The training process of the DDPG algorithm includes: Obtain N deployment experience samples in the 5G network, each of the deployment experience samples including current state feature information, current deployment action, current instant reward, and next state feature information of the physical network; In each training round, M deployment experience samples are randomly selected from each of the deployment experience samples, and the corresponding target Q value is calculated, and the parameters of the current Critic network are updated by minimizing the loss function. The policy gradient of the current Actor network is calculated according to the action value function calculated by the updated current Critic network, and the parameters of the current Actor network are updated by minimizing the policy gradient. The parameters of the target Actor network and the target Critic network are updated by soft updating, and the next training round is entered until the trained DDPG algorithm meets the training end conditions.

4. The method for deploying wireless access network slices according to claim 3, characterized in that: The expression corresponding to the current deployment action is: In the formula, Indicates that the target Actor network calculates the action in the current state. For random actions.

5. The method for deploying wireless access network slices according to claim 1, characterized in that: The reward function is: In the formula, , represent the weight factors of the end-to-end delay and deployment cost of deployment, Indicates The end-to-end latency of a RAN slice instance deployment, represents the RAN slice deployment cost; C1-C7 are constraints. Constraint C1 is a binary variable for whether the VNF is mapped to a physical node. Constraint C2 is a binary variable for whether a virtual link is mapped to a physical link. Constraint C3 means that a VNF can only be mapped to one physical node. Constraint C4 means that the VNFs in the same RAN slice request need to be deployed on different nodes. Constraint C5 means that the deployment delay does not exceed the tolerable delay. Constraints C6 and C7 mean that the total computing resources consumed by all VNFs and the total bandwidth resources consumed do not exceed the computing resources of the physical node and the bandwidth resources of the link, respectively.

6. The method for deploying wireless access network slices according to claim 5, characterized in that: The expression corresponding to the deployment cost is: In the formula, and Respectively represent the unit price of computing resources and bandwidth resources, and Represent computing resources and bandwidth resources respectively, Indicated in VNF in RAN slice request Mapping to physical nodes 1 if yes, 0 otherwise. Indicates The set of VNFs required for a RAN slice request, Represents VNF The computing resources consumed, Represents a collection of physical nodes, Indicates Virtual links in slice requests Mapping to physical links 1 if yes, 0 otherwise. Indicates that the virtual link Mapping to physical links The number of hops, Indicates that the virtual link Connected VNFs and VNF The bandwidth consumed, Represents a collection of physical links, Represents a collection of virtual links.

7. The method for deploying wireless access network slices according to claim 5, characterized in that: The expression corresponding to the end-to-end delay is: In the formula, Indicates The processing delay of a RAN slice request, represents the total transmission delay, Indicated in VNF in RAN slice request Mapping to physical nodes If yes, it is 1, otherwise it is 0. Indicates The set of VNFs required for a RAN slice request, Indicated in In slice requests, VNF On physical nodes The processing delay, Represents a collection of physical nodes, Indicates Virtual links in slice requests Mapping to physical links If yes, it is 1, otherwise it is 0. Indicates that the virtual link will be Connected VNFs and VNF The transmission delay consumed, Represents a collection of physical links, Represents a collection of virtual links.

8. A wireless access network slice deployment device, characterized in that: Applied to 5G network, the device comprises: A state set determination module, configured to, when receiving a target RAN slice request, obtain state information of the target RAN slice request, and extract current state feature information of a physical network through a pre-trained GAT network to obtain a state set corresponding to the target RAN slice request, wherein the physical network includes a plurality of physical nodes; The RAN slice deployment module is used to select and execute deployment actions according to the state set corresponding to the target RAN slice request using a pre-trained DDPG algorithm, so as to deploy the RAN slice corresponding to the required VNF to the physical node in the physical network according to the VNF required by the RAN slice request. The reward function corresponding to the DDPG algorithm is used to minimize the end-to-end delay and deployment cost.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the wireless access network slice deployment method as described in any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the wireless access network slice deployment method as described in any one of claims 1 to 7 are performed.

Images