A 5G end-to-end slice deployment method and system

Abstract

A 5G end-to-end network slice deployment method and system includes: modeling the physical network and slices based on an undirected graph, describing various attributes of nodes and edges, and various deployment constraints; describing each step of the slice deployment process using a heterogeneous graph, constructing an encoder based on a heterogeneous graph neural network to automatically extract global information from the slice deployment environment; feeding the extracted global information into a decoder based on a dual network architecture to output deployment decisions; and training the encoder and decoder parameters using a dual two-layer deep Q-learning algorithm, with the trained encoder and decoder jointly forming a DRL agent to output an approximately optimal slice deployment strategy. This invention combines heterogeneous graph neural networks and reinforcement learning algorithms, constructing an encoder based on a heterogeneous graph neural network to automatically extract global information from the slice deployment environment and feeding it into a decoder based on a dual network architecture to output deployment decisions, thereby realizing the deployment of 5G end-to-end network slices.

Description

Technical Field

[0001] This invention belongs to the field of network slicing technology, and specifically relates to a 5G end-to-end slicing deployment method and system. Background Technology

[0002] Network slicing is a crucial component of 5G technology, enabling customized services for various vertical industries. A slice is essentially multiple virtual networks deployed on a shared underlying physical network. Each slice has independent logical topology, security rules, and performance characteristics. The introduction of network slicing can significantly improve the utilization and flexibility of an operator's physical infrastructure, thereby better supporting service needs across diverse application scenarios. Implementing network slicing faces several key challenges, with slice deployment being particularly critical. During the network slicing preparation phase, slice tenants design slice templates based on their needs, including the Virtual Network Functions (VNFs) required for deployment and the connections between them to support slice instantiation. Slice deployment refers to the arrangement of multiple slices—composed of VNFs and their connections—on the underlying network owned by the mobile network operator (MNO) based on slice templates provided by multiple tenants. Efficient deployment can simultaneously meet the needs of slice tenants and maximize the utilization of physical resources, thereby bringing the greatest benefit to the MNO within a limited resource budget. Therefore, slice deployment is one of the key technologies for realizing network slicing.

[0003] Limitations and shortcomings of existing technologies: The deployment of network slices has been theoretically proven to be NP-hard and difficult to approximate. In recent years, many solutions to the slice deployment problem have been proposed. Heuristic methods are usually designed for deployment scenarios with certain characteristics, but they lack general applicability to slice deployment across different categories of scenarios. Some studies utilize deep reinforcement learning (DRL) to construct deployment schemes, but they often rely on manually selected node and edge features, which may lose information about the intrinsic topological structure of slices and the physical network. Some literature proposes using graph neural networks to automatically generate state representations of the slice deployment environment, including both attribute and topological information. However, these methods process slices and the underlying network separately, ignoring the mapping relationship between them, resulting in incomplete information and low efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a 5G end-to-end slicing deployment method and system to solve the problems of poor universality, information loss, and incomplete information in existing technologies.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a 5G end-to-end slicing deployment method, comprising:

[0007] We model physical networks and slices using undirected graphs, describing various attributes of nodes and edges, as well as deployment constraints.

[0008] Each step in the slice deployment process is described using a heterogeneous graph, and an encoder is built based on the heterogeneous graph neural network to automatically extract global information from the slice deployment environment.

[0009] The extracted global information is fed into a decoder based on a dual network architecture to output deployment decisions;

[0010] The dual two-layer deep Q-learning algorithm is used to train the encoder and decoder parameters. The trained encoder and decoder together form a DRL agent and output an approximately optimal slice deployment strategy.

[0011] Optional description of physical network and slice modeling and deployment constraints:

[0012] Model the physical network as an undirected graph Model the slice network as an undirected graph. Each physical node Has node type attribute Node position attributes Node security level attributes and node resource capacity attributes Each slice node v has the VNF type attribute that the node represents. Deployment location requirements attributes Security level requirement attribute SR(v) and physical resource requirement attribute C(v); for each physical link It has physical bandwidth capacity attribute and the link delay attribute on this link Each slice virtual link e vv′ It has its bandwidth requirement attribute B(e) vv′ ) and the latency requirement attribute D(e) of the link vv′ ).

[0013] Optional deployment constraints: The deployment constraints that must be followed during the slice deployment process include basic constraints on slice node mapping, location constraints, security constraints, node capacity constraints, link mapping constraints, latency constraints, and link bandwidth capacity constraints.

[0014] Optionally, global information extraction of the slice deployment environment based on heterogeneous graph neural networks:

[0015] The slice deployment process is modeled using a heterogeneous graph. This graph includes two types of nodes: slice nodes and physical nodes, and four types of connection relationships: "physical connection," "virtual connection," "embedded," and "bearer." "Physical connection" represents a physical link, "virtual connection" represents a virtual link between slice nodes, "embedded" represents a mapping relationship where a slice node is deployed to a physical node, and "bearer" represents a bearer relationship where a slice node is deployed on a physical node. Before extracting global information, the encoding vectors of each node and edge in the heterogeneous graph are initialized. The remaining resources on physical nodes, the resources required by slice nodes, the remaining bandwidth resources and latency on physical links, and the bandwidth and latency requirements of slice virtual links are used as the initial encoding vectors for each type of node and edge. Then, different types of graph convolutional neural networks are applied to the different types of connection relationships in the heterogeneous graph to extract features from the subgraphs formed by different types of connection relationships, obtaining the encoding vectors of each physical node and the target slice node in the current deployment environment. The target slice node refers to the slice node awaiting deployment decision in the current deployment step.

[0016] Optional deployment decision output based on dual network architecture:

[0017] Based on the encoded vectors obtained in the current deployment environment, a decoder based on a dual network architecture is constructed to output the deployment decision for the current target slice node. This dual network architecture can be divided into two multilayer perceptrons (MLPs), V and A, where V is used to predict the value V(s; θ) of the current state s. V ), θ v These are the learnable parameters of the MLP; A is used to predict the advantage value A(s,a; θ) of all feasible candidate physical nodes for the current target slice node. A ), where 'a' represents a candidate physical node, and θ A These are the learnable parameters of the MLP; feasible candidate physical nodes refer to physical nodes that, for the current target slice node, satisfy the basic constraints of VNF mapping, location constraints, security constraints, and node capacity constraints; the current state value predicted by MLPV and the advantage values ​​of all feasible candidate physical nodes predicted by MLPA are combined according to the following formula to obtain the score of each physical node in the current deployment environment:

[0018]

[0019] Where A(s) represents the set of all feasible candidate physical nodes for the current target slice node; the candidate physical node with the highest score will be used as the final deployment decision for the current target slice node.

[0020] Optional, reinforcement learning agent training:

[0021] The agent consists of an encoder based on a heterogeneous graph neural network and a decoder based on a dual network architecture, as described above. First, the learnable parameters in the encoder and decoder are initialized to form a trained neural network Q with parameters θ. Simultaneously, a target neural network Q′ with the same structure and initialization parameters, also with parameters θ, is constructed. - Then, for each step in each training iteration, the agent extracts global information from the current deployment environment through the encoder and feeds it into the decoder to obtain the scores of all feasible candidate physical nodes in the current deployment environment; then, it obtains the reward for this deployment using the following formula:

[0022]

[0023] Among them, R t This represents the corresponding reward. This is the slice that is currently being deployed. refer to The revenue-to-cost ratio after deployment; the current deployment state S after receiving the reward. t The process will transition to the next state S. t+1 and the state transition quadruple (S t A t ,R t ,S t+1 Store in the experience pool In order to update the parameters of the Q-neural network, from... A fixed number of quadruplets are drawn from the sample. For each drawn quadruplet (S) j A j ,R j ,S j+1 Calculate the corresponding target Q value y. j By minimizing y j The predicted value of Q is Q(S). j A j The mean square error between θ and θ' is used to update the parameters θ of the Q network; for updating the parameters of the target neural network Q′, the update period N is set. - , per N - Step to assign the value of θ to θ - .

[0024] Optionally, each training iteration refers to performing a deployment traversal on the set of slices to be deployed, while each deployment step refers to the deployment decision for a waiting slice node within the currently being deployed slices. After obtaining the scores of all feasible candidate physical nodes, deployment action A is selected based on a greedy strategy. t A t This represents the selected candidate physical node.

[0025] Secondly, the present invention provides a 5G end-to-end slicing deployment system, characterized in that it includes:

[0026] The modeling module is used to model physical networks and slices based on undirected graphs, describing various attributes of nodes and edges and various deployment constraints;

[0027] The information extraction module is used to describe each step of the slice deployment process with a heterogeneous graph and build an encoder based on the heterogeneous graph neural network to automatically extract global information from the slice deployment environment.

[0028] The information input processing module is used to send the extracted global information into the decoder based on the dual network architecture to output deployment decisions;

[0029] The training output module is used to train the encoder and decoder parameters using the dual two-layer deep Q-learning algorithm. The trained encoder and decoder together form a DRL agent and output an approximately optimal slice deployment strategy.

[0030] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a 5G end-to-end slicing deployment method.

[0031] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a 5G end-to-end slicing deployment method.

[0032] Compared with the prior art, the present invention has the following technical effects:

[0033] This invention combines heterogeneous graph neural networks and reinforcement learning algorithms. An encoder based on the heterogeneous graph neural network is used to automatically extract global information from the slice deployment environment and feed it into a decoder based on a dual network architecture to output deployment decisions, thereby realizing the deployment of 5G end-to-end network slices. This invention has the following advantages:

[0034] This invention uses heterogeneous graph neural networks for automated feature extraction in slice deployment environments, which can extract information about the physical network, the slice network, and the mapping relationship between the two, making it comprehensive and efficient in feature extraction.

[0035] This invention employs a dual network to construct a decoder, which combines state value prediction and advantage value prediction for each candidate physical node to obtain a score for each candidate physical node. This design can effectively handle the problem of changes in the action space caused by multiple constraints, thereby improving the accuracy of decision-making.

[0036] This invention uses reinforcement learning algorithms to optimize and update learnable parameters in the encoder and decoder, and can automatically update parameters according to different application scenarios, thus having versatility and adaptability.

[0037] Simulation results on a 5G end-to-end simulated physical network, including access, edge, and core nodes, show that compared with heuristic methods and feature extraction schemes based on isomorphic graph neural networks, this invention can bring mobile network operators a higher slice deployment reception rate, thereby greatly improving their revenue. Attached Figure Description

[0038] Figure 1 This is a flowchart of a 5G slicing deployment method based on heterogeneous graph neural networks and reinforcement learning proposed in this invention.

[0039] Figure 2 This is a schematic diagram of the heterogeneous graph model established for the slice deployment process of this invention and the heterogeneous graph neural network architecture constructed therefrom.

[0040] Figure 3 A chart comparing the cumulative revenue of different plans.

[0041] Figure 4 A comparison chart of the cumulative "revenue-expense ratio" for different options.

[0042] Figure 5 This is a comparison chart of the utilization rate of physical nodes for different schemes. Detailed Implementation

[0043] This invention proposes a 5G slicing deployment method based on heterogeneous graph neural networks and reinforcement learning, such as... Figure 1 As shown, the specific workflow steps are as follows.

[0044] Physical network and slice network modeling and deployment constraint description

[0045] Model the physical network as an undirected graph This represents the set of all physical nodes contained in the physical network. This represents the set of all physical links contained in a physical network. The sliced ​​network is modeled as an undirected graph. ε represents the set of all slice nodes (VNFs) contained in the slice network, and ε represents the set of all slice links contained in the slice network.

[0046] Each physical node Has node type attribute Where AN represents access node, EN represents edge node, and CN represents core network node; node location attributes Among them, loc1, etc. represent The geographical region to which it belongs, and only physical nodes of type AN have this attribute; node security level attribute This represents the level of security that the physical node can provide; node resource capacity attribute. This represents the amount of remaining available resources on that physical node;

[0047] Each slice node v has the VNF type attribute that the node represents. Where RF represents access network function, UF represents user plane network function, and CF represents control plane network function; deployment location requirements attributes. Among them, loc1 represents the geographical area that v requires to be deployed, and only RF type slice nodes have this attribute; security level requirement attribute This represents the required security level for the slice node; physical resource requirements attribute. This represents the amount of resources required by the slice node.

[0048] Each physical link It has physical bandwidth capacity attribute Represents the amount of remaining available bandwidth resources on this physical link; the link latency attribute on this link. This represents the inherent latency of the physical link.

[0049] Each slice virtual link e vv′ It has its bandwidth requirement attribute This represents the amount of bandwidth resources required by the sliced ​​link; the latency requirement attribute of the link. This represents the maximum latency that the slice link can tolerate.

[0050] Slice deployment refers to mapping sliced ​​networks to physical networks. Slice nodes need to be mapped to physical nodes. After the mapping decision of the slice nodes is determined, the slice links need to be mapped to the physical path between the two slice nodes that make up the link and the physical nodes to which they are mapped. The physical path can consist of one or more physical links.

[0051] The deployment constraints that must be followed during slice deployment include basic constraints on slice node mapping, location constraints, security constraints, node capacity constraints, link mapping constraints, latency constraints, and link bandwidth capacity constraints. These are described in detail below:

[0052] The basic constraint of slice node mapping is that a slice node can only be deployed to one physical node.

[0053] Location constraints mean that slice nodes can only be deployed within their required geographical area;

[0054] Security constraints mean that slice nodes can only be deployed on physical nodes with a security level higher than required.

[0055] Node capacity constraints mean that slice nodes can only be deployed on physical nodes whose remaining capacity exceeds their resource requirements;

[0056] Link mapping constraints mean that once a slice link is deployed on a path consisting of multiple physical links, all physical links on that path must allocate the resources required by the slice link.

[0057] The latency constraint means that the sum of the latencies of multiple physical links that support a certain slice link must not exceed the maximum latency that the slice link can withstand.

[0058] Link bandwidth capacity constraint means that the sum of the bandwidth requirements of all slice links on a physical link must not exceed the bandwidth capacity of that physical link.

[0059] Global Information Extraction for Slice Deployment Environment Based on Heterogeneous Graph Neural Network

[0060] This invention employs a sequential decision-making approach for slice deployment. For each slice to be deployed, deployment decisions are made sequentially for each slice node that constitutes it. Once both ends of a slice link have been decided, a shortest path mapping strategy is used to map the slice link onto a physical path. To obtain the deployment decision for each slice node, this invention first extracts global information about the slice deployment environment based on a heterogeneous graph neural network, thereby enabling the decoding of the deployment decision based on the extracted global information.

[0061] Modeling the slice deployment process using heterogeneous graphs, such as Figure 2 As shown in the diagram, this heterogeneous graph includes two types of nodes: slice nodes and physical nodes, and four types of connection relationships: "physical connection," "virtual connection," "embedded," and "bearer." "Physical connection" represents a physical link, "virtual connection" represents a virtual link between slice nodes, "embedded" represents a mapping relationship where a slice node is deployed on a physical node, and "bearer" represents a bearer relationship where a slice node is deployed on a physical node.

[0062] Before extracting global information, the encoding vectors of each node and edge in the heterogeneous graph are first initialized. The initial feature vector of a physical node represents its remaining available resources. The initial feature vector of a slice node represents its required resources, x. v =C(v). The initial feature vector of the physical link is its remaining bandwidth resources and its latency. The initial feature vector of a sliced ​​link is its required bandwidth resources and its latency, e vv′ =B(e) vv′ )||D(evv′ ).

[0063] Furthermore, different types of graph convolutional neural networks are applied to different types of connection relationships in the heterogeneous graph to extract features from the subgraphs formed by different types of connection relationships, so as to obtain the encoding vector of each physical node and the encoding vector of the target slice node in the current deployment environment. Here, the target slice node refers to the slice node waiting for deployment decision in the current deployment step.

[0064] Apply a K1-layer NNConv graph convolutional neural network to the subgraphs corresponding to the "physical connectivity" relationships, so that each The attributes and topological information of the physical nodes within its K1-hop neighborhood are obtained. Each convolutional operation can be represented by the following formula: in, Represents physical nodes The encoding vector at layer l, Indicates and A set of physical nodes with "physical connections"; W1 represents learnable parameters; NN() represents a neural network. Represents physical nodes and The encoding vector of the physical link between them.

[0065] A K2-layer NNConv graph convolutional neural network is applied to the subgraph corresponding to the "virtual connection" relationship, enabling each slice node v to obtain the attribute and demand information of the slice nodes in its K2-hop neighborhood on that slice. Each convolutional operation can be represented by the following equation: in, Let represent the encoding vector of slice node v at layer l, Nei(v) represent the set of slice nodes with "virtual connections" to v, W2 represent the learnable parameters, NN() represent the neural network, and e vv′ This represents the encoding vector of the slice link between slice nodes v and v′.

[0066] Apply a layer of SAGEConv graph convolutional neural network to the subgraph corresponding to the "embedding" relationship, so that each physical node We obtain the slice node information it carries. Each convolution operation can be represented by the following formula: in, Indicates and A set of slice nodes that have an "embedded" relationship. express The encoded vector after undergoing K1 layers of NNConv convolutions, W3 and W4 represent the encoded vector of v after undergoing K2 layers of NNConv convolution, and both W3 and W4 represent learnable parameters.

[0067] After the heterogeneous graph neural network convolution is completed, the encoding vector of each physical node and the encoding vector of the target slice node in the current deployment environment can be obtained.

[0068] Deployment decision output based on dual network architecture

[0069] Based on the encoded vectors obtained in the current deployment environment, a decoder based on a dual network architecture is constructed to output the deployment decision for the current target slice node. This dual network architecture can be divided into two multilayer perceptrons (MLPs), V and A. V is used to predict the value V(s; θ) of the current state s. V ), θ V These are the learnable parameters of the MLP. A is used to predict the advantage value A(s,a; θ) of all feasible candidate physical nodes for the current target slice node. A ), where 'a' represents a candidate physical node, and θ A These are the learnable parameters of the MLP. Among them, feasible candidate physical nodes refer to physical nodes that, for the current target slice node, satisfy the basic constraints of VNF mapping, location constraints, security constraints, and node capacity constraints.

[0070] To predict state value, the representation of the current physical network is first obtained by calculating the mean of the encoded vectors of all physical nodes. Then, this representation vector is concatenated with the encoded vector of the target slice node as input to MLPV. The output of this MLP represents a scalar value representing the current state value.

[0071] To predict the advantage value of each candidate physical node, the encoding vector of each candidate physical node and the encoding vector of the target slice node are concatenated to form a multi-dimensional vector group as input to MLPA. The MLP will output the advantage value of each candidate physical node if it is selected.

[0072] The current state value predicted by MLPV and the advantage values ​​of all feasible candidate physical nodes predicted by MLPA are combined according to the following formula to obtain the score of each physical node in the current deployment environment.

[0073]

[0074] Here, A(s) represents the set of all feasible candidate physical nodes for the current target slice node. The candidate physical node with the highest score will be used as the final deployment decision for the current target slice node.

[0075] Reinforcement learning agent training

[0076] The agent consists of an encoder based on a heterogeneous graph neural network and a decoder based on a dual network architecture, as described above. A reinforcement learning algorithm is used to train the neural network parameters it contains.

[0077] First, the encoder and decoder, each containing learnable parameters, are randomly initialized to form a neural network Q with parameters θ. Simultaneously, a target neural network Q′ with the same structure and initialization parameters, also with parameters θ, is constructed. - .

[0078] Then, for each step in each training iteration, the agent extracts global information from the current deployment environment through the encoder and feeds it into the decoder to obtain the scores of all feasible candidate physical nodes in the current deployment environment. Here, each training iteration refers to performing a deployment traversal on each of the set of slices to be deployed, while each deployment step refers to making a deployment decision for a specific slice node waiting to be deployed in the currently being deployed slice.

[0079] After obtaining the scores of all feasible candidate physical nodes, deployment action A is selected based on the greedy strategy. t A t This represents the selected candidate physical node.

[0080] The greedy strategy selects actions based on the free exploration control factor ∈, randomly selecting with a probability of ∈ and choosing the candidate physical node with the highest score with a probability of 1-∈. ∈ decreases linearly with increasing training iterations, referring to its preset minimum value.

[0081] A t After execution, the reward for this deployment decision will be obtained according to the following formula:

[0082]

[0083] Among them, R t This represents the corresponding reward. This is the slice that is currently being deployed. slice The cost-benefit ratio after deployment. Among them, Representative slice If successfully deployed, the benefits it can bring are the sum of the node resources and bandwidth resources it requires. Representative slice The overhead of deployment is the sum of the physical nodes and bandwidth required for its deployment; These are decision variables, representing slices in this deployment decision. Is the slice node v deployed on a physical node? superior; These are decision variables, representing slices in this deployment decision. slice link e vv′ Whether it is deployed on the physical link Above, where e vv′ The deployment decision is determined by the deployment positions of its two end slice nodes v and v′, e vv′ It is mapped to the shortest physical path between the two physical nodes where v and v′ are deployed.

[0084] After receiving the reward, the current deployment state is S. t The process will transition to the next state S. t+1 If A t If successful, the physical node allocates the required resources to the slice node and embeds the slice link between the slice node and the already deployed slice nodes according to the shortest path strategy, transferring the target slice node to the next undeployed slice node; if A t If the deployment fails, the slice deployment fails, the resources allocated to the slice on the physical network are released, and the target slice is moved to the next undeployed slice in the slice set.

[0085] Furthermore, the state transition quadruple (S) t A t ,R t ,S t+1 Store it in experience pool B.

[0086] To update the parameters of the Q-neural network, from A fixed number of quadruplets are drawn from the sample. For each drawn quadruplet (S) j A j ,R j ,S j+1 Calculate the corresponding target Q value y. j If S j+1 If it is a terminated state, then y j =R j Otherwise, y j =R j +γQ(S j+1 argmax Aj+1 Q(S j+1 A j+1 ;θ);θ - ), where γ is a preset loss factor.

[0087] By minimizing y j The predicted value Q(S) of the Q network j A j The mean square error between θ and θ is used to update the Q-network parameters θ.

[0088] For parameter updates of the target neural network Q′, the update period N is set. - , per N - Step to assign the value of θ to θ - .

[0089] The above process will continue to iterate until the number of training iterations reaches the preset maximum value.

[0090] Simulation environment:

[0091] An end-to-end physical network is constructed, including an access ring, a aggregation ring, and a core ring. For slice networks, the BA network model is used to generate their topology, and the size of the slice network is uniformly distributed between [10, 20]. Table I shows the attribute settings for nodes and edges in the physical network and slice networks. During deployment, a slice is considered successfully deployed once all the requirements for nodes and edges in the slice network are met.

[0092] Table I shows the attribute settings for nodes and edges in the physical network and sliced ​​network during the simulation experiment.

[0093]

[0094]

[0095] To demonstrate the superiority of this invention, the following comparative scheme is set up:

[0096] HomoGCN: Employs a homogeneous graph neural network to extract global features of the slice deployment environment and trains a reinforcement learning agent to make slice deployment decisions.

[0097] GSH: A slice deployment heuristic algorithm that deploys slice nodes on previously used physical nodes whenever possible, without occupying new physical nodes.

[0098] LFH: Slice deployment heuristic algorithm, which deploys slice nodes on physical nodes with the most remaining resources whenever possible.

[0099] Simulation results:

[0100] Comparison of cumulative revenue and expenses

[0101] Figure 3 The cumulative revenue results of different schemes for deploying the same slice sequence on the same physical network are shown. It can be seen that the cumulative revenue generated for mobile operators by this invention is the highest. Furthermore, only after the corresponding slice is successfully deployed... Figure 3 The curve in the present invention only rises when the curve rises the most times, thus the present invention achieves the highest slice acceptance rate. For a given scheme, the cumulative "revenue-to-cost ratio" is defined as the total number of slices successfully deployed. Corresponding revenue-to-expense ratio sum. Figure 4 The comparison results of different solutions based on the cumulative "revenue-to-cost ratio" evaluation metric are presented. It can be seen that the deployment solution corresponding to this invention generates the highest cumulative "revenue-to-cost ratio".

[0102] Physical node utilization comparison

[0103] Figure 5 This paper demonstrates the utilization of physical network nodes after deploying the same slice sequence on the same physical network using different schemes. It can be seen that, because this invention achieves the highest reception rate for this slice sequence, it also boasts the highest utilization rate for different types of physical nodes, including access nodes, edge nodes, core nodes, or the combined utilization rate of different types of nodes.

[0104] In another embodiment of the present invention, a 5G end-to-end slicing deployment system is provided, which can be used to implement the above-described 5G end-to-end slicing deployment method. Specifically, the system includes:

[0105] The modeling module is used to model physical networks and slices based on undirected graphs, describing various attributes of nodes and edges and various deployment constraints;

[0106] The information extraction module is used to describe each step of the slice deployment process with a heterogeneous graph and build an encoder based on the heterogeneous graph neural network to automatically extract global information from the slice deployment environment.

[0107] The information input processing module is used to send the extracted global information into the decoder based on the dual network architecture to output deployment decisions;

[0108] The training output module is used to train the encoder and decoder parameters using the dual two-layer deep Q-learning algorithm. The trained encoder and decoder together form a DRL agent and output an approximately optimal slice deployment strategy.

[0109] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0110] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a 5G end-to-end slicing deployment method.

[0111] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the 5G end-to-end slicing deployment method in the above embodiments.

[0112] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A 5G end-to-end slicing deployment method, characterized in that, include: We model physical networks and slices using undirected graphs, describing various attributes of nodes and edges, as well as deployment constraints. Each step in the slice deployment process is described using a heterogeneous graph, and an encoder is built based on the heterogeneous graph neural network to automatically extract global information from the slice deployment environment. The extracted global information is fed into a decoder based on a dual network architecture to output deployment decisions; The dual two-layer deep Q-learning algorithm is used to train the encoder and decoder parameters. The trained encoder and decoder together form a DRL agent and output the optimal slice deployment strategy. Global information extraction of slice deployment environment based on heterogeneous graph neural network: The slice deployment process is modeled using a heterogeneous graph. This graph includes two types of nodes: slice nodes and physical nodes, and four types of connection relationships: "physical connection," "virtual connection," "embedded," and "bearer." "Physical connection" represents a physical link, "virtual connection" represents a virtual link between slice nodes, "embedded" represents a mapping relationship where a slice node is deployed to a physical node, and "bearer" represents a bearer relationship where a slice node is deployed on a physical node. Before global information extraction, the encoding vectors of each node and edge in the heterogeneous graph are initialized. The remaining resources on physical nodes, the resources required by slice nodes, the remaining bandwidth resources and latency on physical links, and the bandwidth and latency requirements of slice virtual links are used as the initial encoding vectors for each type of node and edge. Then, different types of graph convolutional neural networks are applied to the different types of connection relationships in the heterogeneous graph to extract features from the subgraphs formed by different types of connection relationships, obtaining the encoding vectors of each physical node and the target slice node in the current deployment environment. The target slice node refers to the slice node awaiting deployment decision in the current deployment step. Deployment decision output based on dual network architecture: Based on the encoded vectors obtained in the current deployment environment, a decoder based on a dual network architecture is constructed to output deployment decisions for the current target slice node; this dual network architecture can be divided into two multilayer perceptron (MLP) layers. and , Used to predict the current state value , These are the learnable parameters of the MLP; Used to predict the dominance of all feasible candidate physical nodes for the current target slice node. , Represents a candidate physical node. These are the learnable parameters of the MLP; feasible candidate physical nodes refer to physical nodes that, for the current target slice node, satisfy the basic constraints of VNF mapping, location constraints, security constraints, and node capacity constraints; the MLP The predicted current state value and MLP The advantage values ​​of all predicted feasible candidate physical nodes are combined using the following formula to obtain the score of each physical node in the current deployment environment: in, This represents the set of all feasible candidate physical nodes for the current target slice node; the candidate physical node with the highest score will be used as the final deployment decision for the current target slice node. Reinforcement learning agent training: The intelligent agent consists of an encoder based on a heterogeneous graph neural network and a decoder based on a dual network architecture, as described above. First, the learnable parameters in the encoder and decoder are initialized to form the trained neural network. Its parameters are Simultaneously, construct a target neural network with the same structure and initialization parameters. Its parameters are Then, for each step in each training iteration, the agent extracts global information from the current deployment environment through the encoder and feeds it into the decoder to obtain the scores of all feasible candidate physical nodes in the current deployment environment; then, it obtains the reward for this deployment using the following formula: in, This represents the corresponding reward. This is the slice that is currently being deployed. refer to The revenue-to-cost ratio after deployment; the current deployment status after receiving the reward. Will move to the next state and the state transition quadruple Experience Pool In order to update the neural network The parameters, from A fixed number of quadruplets are drawn from the sample. For each drawn quadruplet... Calculate its corresponding target Q value By minimizing and Predicted value The mean squared error between them is used to perform... Network parameters Update; for the target neural network If the parameters are updated, then set the update cycle. ,Every Step General Value Assignment .

2. The 5G end-to-end slicing deployment method according to claim 1, characterized in that, Physical network and slice modeling: Model the physical network as an undirected graph Model the slice network as an undirected graph Each physical node Has node type attribute Node position attribute Node security level attributes Node resource capacity attribute Each slice node It has the VNF type attribute represented by this node. Deployment location requirements attributes Security level requirements attributes Physical resource requirements attributes Each physical link It has physical bandwidth capacity attribute and the link delay attribute on this link Each slice virtual link It has its bandwidth requirement attribute and the latency requirements of this link .

3. The 5G end-to-end slicing deployment method according to claim 2, characterized in that, Deployment constraints: The deployment constraints that must be followed during the deployment of slices include basic constraints on slice node mapping, location constraints, security constraints, node capacity constraints, link mapping constraints, latency constraints, and link bandwidth capacity constraints.

4. A 5G end-to-end slicing deployment system, characterized in that, include: The modeling module is used to model physical networks and slices based on undirected graphs, describing various attributes of nodes and edges and various deployment constraints; The information extraction module is used to describe each step of the slice deployment process with a heterogeneous graph and build an encoder based on the heterogeneous graph neural network to automatically extract global information from the slice deployment environment. The information input processing module is used to send the extracted global information into the decoder based on the dual network architecture to output deployment decisions; The training output module is used to train the encoder and decoder parameters using the dual two-layer deep Q-learning algorithm. The trained encoder and decoder together form a DRL agent and output the optimal slice deployment strategy. Global information extraction of slice deployment environment based on heterogeneous graph neural network: The slice deployment process is modeled using a heterogeneous graph. This graph includes two types of nodes: slice nodes and physical nodes, and four types of connection relationships: "physical connection," "virtual connection," "embedded," and "bearer." "Physical connection" represents a physical link, "virtual connection" represents a virtual link between slice nodes, "embedded" represents a mapping relationship where a slice node is deployed to a physical node, and "bearer" represents a bearer relationship where a slice node is deployed on a physical node. Before global information extraction, the encoding vectors of each node and edge in the heterogeneous graph are initialized. The remaining resources on physical nodes, the resources required by slice nodes, the remaining bandwidth resources and latency on physical links, and the bandwidth and latency requirements of slice virtual links are used as the initial encoding vectors for each type of node and edge. Then, different types of graph convolutional neural networks are applied to the different types of connection relationships in the heterogeneous graph to extract features from the subgraphs formed by different types of connection relationships, obtaining the encoding vectors of each physical node and the target slice node in the current deployment environment. The target slice node refers to the slice node awaiting deployment decision in the current deployment step. Deployment decision output based on dual network architecture: Based on the encoded vectors obtained in the current deployment environment, a decoder based on a dual network architecture is constructed to output deployment decisions for the current target slice node; this dual network architecture can be divided into two multilayer perceptron (MLP) layers. and , Used to predict the current state value , These are the learnable parameters of the MLP; Used to predict the dominance of all feasible candidate physical nodes for the current target slice node. , Represents a candidate physical node. These are the learnable parameters of the MLP; feasible candidate physical nodes refer to physical nodes that, for the current target slice node, satisfy the basic constraints of VNF mapping, location constraints, security constraints, and node capacity constraints; the MLP The predicted current state value and MLP The advantage values ​​of all predicted feasible candidate physical nodes are combined using the following formula to obtain the score of each physical node in the current deployment environment: in, This represents the set of all feasible candidate physical nodes for the current target slice node; the candidate physical node with the highest score will be used as the final deployment decision for the current target slice node. Reinforcement learning agent training: The intelligent agent consists of an encoder based on a heterogeneous graph neural network and a decoder based on a dual network architecture, as described above. First, the learnable parameters in the encoder and decoder are initialized to form the trained neural network. Its parameters are Simultaneously, construct a target neural network with the same structure and initialization parameters. Its parameters are Then, for each step in each training iteration, the agent extracts global information from the current deployment environment through the encoder and feeds it into the decoder to obtain the scores of all feasible candidate physical nodes in the current deployment environment; then, it obtains the reward for this deployment using the following formula: in, This represents the corresponding reward. This is the slice that is currently being deployed. refer to The revenue-to-cost ratio after deployment; the current deployment status after receiving the reward. Will move to the next state and the state transition quadruple Experience Pool In order to update the neural network The parameters, from A fixed number of quadruplets are drawn from the sample. For each drawn quadruplet... Calculate its corresponding target Q value By minimizing and Predicted value The mean squared error between them is used to perform... Network parameters Update; for the target neural network If the parameters are updated, then set the update cycle. ,Every Step General Value Assignment .

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a 5G end-to-end slicing deployment method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a 5G end-to-end slicing deployment method as described in any one of claims 1 to 3.

Images