Dynamic network slice resource allocation method and device, storage medium and electronic equipment

Abstract

The present disclosure provides a dynamic network slice resource allocation method and device, a storage medium and an electronic device, comprising: constructing a Markov decision model of a core network side and an access network side respectively; determining an optimization objective of a dynamic network slice resource allocation problem; based on the optimization objective, using first and second learning models based on the constructed Markov decision models of the core network and the access network side, training first and second strategy models of the core network side and the access network side respectively; generating a dynamic network slice resource allocation result based on the first and second strategy models; and performing dynamic network slice resource allocation according to the dynamic network slice resource allocation result. This method realizes the reasonable allocation of dynamically changing network slice resources, so that the network can provide more flexible services for different businesses, thereby improving the network service quality, increasing the user experience and improving the utilization rate of resources.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a dynamic network slicing resource allocation method and apparatus, storage medium and electronic device. Background Technology

[0002] In recent years, mobile networks have faced numerous challenges in handling high-speed data transmission, high user device density, and stringent latency requirements. To address the complex communication service demands of the future, 5G (the fifth generation) mobile communication technology has emerged. In the 5G era, the types of services and application scenarios provided by the internet are becoming increasingly diversified, necessitating on-demand networking for various service scenarios and the flexible deployment of multiple network functions.

[0003] Network slicing, a key technology in 5G, is a logical combination of service functions that supports the communication network requirements of specific use cases. It allows for customized customization of network functions based on business scenario needs, enabling the network to provide more targeted services for different services.

[0004] However, current understanding of network slicing technology is limited to macro-level concepts and architectures. There is still no reasonable solution for the resource management and configuration process of dynamically changing network slicing services.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a dynamic network slice resource allocation method, apparatus, storage medium, and electronic device to achieve the rational allocation of dynamically changing network slice resources.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a method for dynamic network slice resource allocation is provided, comprising:

[0009] Markov decision models are constructed for both the core network side and the access network side; the optimization objective for the dynamic network slice resource allocation problem is determined; based on the optimization objective, a first strategy model for the core network side is trained using a first learning model, based on the constructed Markov decision model for the core network side; based on the optimization objective, a second strategy model for the access network side is trained using a second learning model, based on the constructed Markov decision model for the access network side; dynamic network slice resource allocation results are generated based on the first and second strategy models; and dynamic network slice resource allocation is executed based on the dynamic network slice resource allocation results.

[0010] In one embodiment of this disclosure, constructing Markov decision models for the core network side and the access network side respectively includes: acquiring first computing resources and first communication resources on the core network side; acquiring second computing resources and second communication resources on the access network side; expressing the dynamic allocation of the first computing resources and first communication resources on the core network side as a first infinite-time Markov decision model; and expressing the dynamic allocation of the second computing resources and second communication resources on the access network side as a second infinite-time Markov decision model.

[0011] In one embodiment of this disclosure, determining the optimization objective of the dynamic network slice resource allocation problem includes: determining the deployment cost of the core network side based on the first computing resources and the first communication resources of the core network side; determining the revenue of the access network side based on the second computing resources and the second communication resources of the access network side; and determining maximizing the revenue of the access network side and minimizing the deployment cost of the core network side as the optimization objective based on the revenue and the deployment cost.

[0012] In one embodiment of this disclosure, obtaining the first computing resources and the first communication resources on the core network side further includes: determining the first computing resources on the core network side based on the CPU resources of each network node; and determining the first communication resources on the core network side based on the link bandwidth resources of each network node.

[0013] In one embodiment of this disclosure, obtaining the second computing resources and the second communication resources on the access network side further includes: determining the second computing resources on the access network side based on the CPU resources of each distributed processing unit; and determining the second communication resources on the access network side based on the number of physical resource blocks (PRBs) in each radio remote unit (RRU).

[0014] In one embodiment of this disclosure, according to the optimization objective, a first policy model for the core network side is trained using a first learning model based on the constructed Markov decision model for the core network side, wherein the first learning model is a deep reinforcement learning model.

[0015] In one embodiment of this disclosure, based on the optimization objective, a second policy model for the access network side is trained using a second learning model, based on the constructed Markov decision model for the access network side, wherein the second learning model is a deep reinforcement learning model.

[0016] In one embodiment of this disclosure, the deep reinforcement learning model is a double-Q learning model.

[0017] In one embodiment of this disclosure, generating a dynamic network slice resource allocation result based on a first strategy model and a second strategy model includes: inputting a randomly generated allocation result of computing and communication resources on the access network side that satisfies preset constraints into a first strategy model on the core network side to obtain a first allocation result including computing and communication resources on the core network side; inputting the first allocation result into a second strategy model to obtain a second allocation result including computing and communication resources on the access network side; and generating a dynamic network slice resource allocation result based on the first allocation result and the second allocation result.

[0018] According to another aspect of this disclosure, a dynamic network slice resource allocation apparatus is provided, comprising: a modeling module for constructing Markov decision models for the core network side and the access network side respectively; an optimization module for determining the optimization objective of the dynamic network slice resource allocation problem; a first training module for training a first policy model for the core network side based on the constructed Markov decision model for the core network side and using a first learning model, according to the optimization objective; a second training module for training a second policy model for the access network side based on the constructed Markov decision model for the access network side and using a second learning model, according to the optimization objective; a result generation module for generating a dynamic network slice resource allocation result based on the first policy model and the second policy model; and an execution module for executing the dynamic network slice resource allocation according to the dynamic network slice resource allocation result.

[0019] In one embodiment of this disclosure, the modeling module constructs Markov decision models for the core network side and the access network side respectively, including: acquiring first computing resources and first communication resources on the core network side; acquiring second computing resources and second communication resources on the access network side; expressing the dynamic allocation of the first computing resources and first communication resources on the core network side as a first infinite-time Markov decision model; and expressing the dynamic allocation of the second computing resources and second communication resources on the access network side as a second infinite-time Markov decision model.

[0020] In one embodiment of this disclosure, the optimization module determines the optimization objective of the dynamic network slice resource allocation problem as follows: based on the first computing resources and the first communication resources on the core network side, determine the deployment cost on the core network side; based on the second computing resources and the second communication resources on the access network side, determine the benefit on the access network side; and based on the benefit and the deployment cost, determine maximizing the benefit on the access network side and minimizing the deployment cost on the core network side as the optimization objective.

[0021] In one embodiment of this disclosure, the modeling module is further configured to: determine the first computing resources on the core network side based on the CPU resources of each network node; and determine the first communication resources on the core network side based on the link bandwidth resources of each network node.

[0022] In one embodiment of this disclosure, the modeling module is further configured to: obtain the second computing resources and the second communication resources on the access network side, and further include: determining the second computing resources on the access network side based on the CPU resources of each distributed processing unit; and determining the second communication resources on the access network side based on the number of physical resource blocks (PRBs) in each radio remote unit (RRU).

[0023] In one embodiment of this disclosure, the first training module trains a first policy model on the core network side based on the constructed Markov decision model on the core network side according to the optimization objective, using a first learning model, wherein: the first learning model is a deep reinforcement learning model.

[0024] In one embodiment of this disclosure, the second training module trains a second policy model for the access network side based on the constructed Markov decision model on the access network side and a second learning model, according to the optimization objective, wherein the second learning model is a deep reinforcement learning model.

[0025] In one embodiment of this disclosure, the deep reinforcement learning model is a double-Q learning model.

[0026] In one embodiment of this disclosure, the result generation module generates dynamic network slice resource allocation results based on a first strategy model and a second strategy model, including: inputting a randomly generated allocation result of computing and communication resources on the access network side that satisfies preset constraints into the first strategy model on the core network side to obtain a first allocation result including computing and communication resources on the core network side; inputting the first allocation result into the second strategy model to obtain a second allocation result including computing and communication resources on the access network side; and generating dynamic network slice resource allocation results based on the first allocation result and the second allocation result.

[0027] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described dynamic network slicing resource allocation method by executing the executable instructions.

[0028] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described dynamic network slicing resource allocation method.

[0029] The dynamic network slice resource allocation method provided in the embodiments of this disclosure achieves reasonable allocation of dynamically changing network slice resources by jointly optimizing the network slice resource management at both the core network side and the access network side. This makes the network more flexible in providing services for different services, thereby improving network service quality, enhancing user experience, and increasing resource utilization.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0032] Figure 1 This diagram illustrates a dynamic network architecture according to an embodiment of the present disclosure.

[0033] Figure 2 This diagram illustrates a flowchart of a dynamic network slice resource management method according to an embodiment of the present disclosure;

[0034] Figure 3 A flowchart illustrating the method for constructing Markov decision models for the core network side and the access network side respectively in the dynamic network slicing resource management method of this disclosure embodiment is shown.

[0035] Figure 4 A flowchart illustrating the optimization objective method for determining the dynamic network slice resource allocation problem in the dynamic network slice resource management method of this disclosure is shown in an embodiment of the present disclosure.

[0036] Figure 5 The flowchart illustrates a method for generating dynamic network slice resource allocation results based on a first strategy model and a second strategy model in a dynamic network slice resource management method according to an embodiment of the present disclosure.

[0037] Figure 6 This diagram shows an overall flowchart of a dynamic network slice resource management method according to an embodiment of the present disclosure;

[0038] Figure 7 This diagram illustrates a dynamic network slice resource management device according to an embodiment of the present disclosure; and

[0039] Figure 8 This diagram illustrates a structural block diagram of a computer device for dynamic network slice resource management according to an embodiment of the present disclosure. Detailed Implementation

[0040] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0041] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0042] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0043] To address the technical problems existing in the aforementioned related technologies, this disclosure provides a dynamic network slice resource management method to solve at least one or all of the aforementioned technical problems.

[0044] Figure 1 A schematic diagram of a network architecture according to an embodiment of this disclosure is shown. Figure 1 As shown, the network architecture includes a core network side and an access network side.

[0045] In the embodiments disclosed herein, such as Figure 1The core network side shown comprises physical nodes and physical links consisting of several network node servers and interconnected links. These physical nodes and links on the core network side are virtualized and mapped to the core network slice Virtual Network Function (VNF) of the network slice.

[0046] In the disclosed embodiment, the algorithm for virtualizing core network side resources is as follows:

[0047] The physical and virtual networks in the core network are described using graph theory: the physical network can be represented as an undirected weighted graph. ,in and These represent the set of physical nodes and the set of physical links, respectively. For example, the set of general-purpose servers in a network can be considered as the set of physical nodes. , . This represents a set of attributes for a physical node, such as CPU, memory, and disk. In this embodiment, to simplify model complexity, various resources on a node are unified as computing resources. In this embodiment, the demand for server computing resources is related to the amount of data that the Virtual Network Function (VNF) needs to process. This represents the physical link attribute, i.e., the physical link bandwidth. In this embodiment of the disclosure, to simplify the model complexity, the link bandwidth resources between nodes are treated as communication resources on the core network side.

[0048] Based on the combined logic of business requests, a virtual network topology is generated, representing the virtual network requests as an undirected weighted graph. ,in, This represents the set of VNFs, i.e., the set of virtual nodes; Represents a set of virtual links. This represents the set of attributes of a virtual network node, i.e., the attribute set of a VNF, including the amount of data that the VNF needs to process. This represents the bandwidth of the virtual link. The mapping process from a virtual network to a physical network can be represented as follows: .

[0049] In this embodiment of the disclosure, slice requirements are mapped to the physical network: slice requirements are Virtual Network Resources (VNRs), and different slice requirements correspond to different types of slices, with the slice type set as follows: Each type of slice consists of different Virtual Network Functions (VNFs). The VNF is composed of .like Figure 1 In the network slice, it consists of VNF1-4 in network slice 1 and VNF5-8 in network slice 2.

[0050] In the embodiments disclosed herein, such as Figure 1 The access network side shown contains several network infrastructure components, which may include several terminals. These terminals communicate with their corresponding Distributed Units (DUs) through each Remote Radio Unit (RRU).

[0051] The terminal can be a wireless terminal, such as a mobile phone, game console, tablet computer, e-book reader, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) player, smart home device, AR (Augmented Reality) device, VR (Virtual Reality) device, or other mobile terminal. Alternatively, the terminal 120 can also be a personal computer (PC), such as a laptop computer or desktop computer. In this embodiment, multiple terminals communicate with their corresponding distributed processing units (DUs) via remote radio units (RRUs).

[0052] The physical nodes (DUs) on the access network side and the physical resource blocks (PRBs) in each RRU are virtualized and mapped to the access network slice virtual network function (VNF) of the network slice. In the disclosed embodiment, the process of virtualizing access network side resources is as follows:

[0053] like Figure 1The physical computing resource pool of the Centralized Unit (CU) shown, supported by virtualization and collaborative processing technologies, aggregates and shares the computing resources of each Distributed Unit (DU) (DU1-DU3). This provides each Remote Radio Unit (RRU) (RRU1-RRU3) with the computing resources needed to process baseband data, enabling more effective collaboration and forming a dynamically allocated virtual computing resource pool. All computing resources are centrally formed into a virtual computing resource pool through virtualization operations. In this embodiment, to simplify the model's complexity, the computing resources of each distributed processing unit can be attributes such as CPU, memory, and disk.

[0054] like Figure 1 As shown, the PRB resources occupied by each user terminal on the access network side are different at different times. Therefore, each corresponding RRU requires a different amount of communication resources at different times. In this embodiment of the disclosure, a time-varying random channel model is considered. This model assumes that there are multiple remote radio units (RRUs) in a specific area. The total bandwidth of Hz is divided into multiple Physical Resource Blocks (PRBs), and the bandwidth of each PRB is... These PRBs are shared by all RRUs. In this embodiment of the disclosure, to simplify the complexity of the model, the number of PRBs in each RRU is used as the communication resource on the access network side.

[0055] In this embodiment of the disclosure, it is assumed that the entire network provides I access network slice services to U users, and the user set is... Each RRU can provide services for multiple types of access network slices. For a finite set of channel states, and , For users exist Constant access to access network slicing The channel gain at that time, where, Indicates the channel state is The probability of accessing each access network slice. When a user requests access to each access network slice, it is assumed that the channel state within each time slot remains fixed, but the channel state between different time slots changes randomly, and the channel states between each time slot are independent of each other. This indicates that the user is accessing the access network slice. The average channel gain, This indicates the time slot. The corresponding spectral efficiency.

[0056] In summary, network slicing can include at least two parts: core network slicing and access network slicing. Terminals connect to the entities of each network slice by selecting the Virtual Network Function (VNF) of the access network slice and the Virtual Network Function (VNF) of the core network slice, as well as the routing path, thereby accessing the data network. Through specific Virtual Network Function (VNF) and parameter configuration, customized services are provided, thus forming a network slicing architecture from the access network to the core network.

[0057] The embodiments disclosed herein can be applied to mobile communication systems such as 5G communication systems or Long Term Evolution (LTE) systems.

[0058] Through the above method, this invention considers network slicing including both the core network side and the access network side. During the deployment and orchestration of network slices, the resource allocation involved includes both core network slices and access network slices. The resource allocation on the core network side and the access network side is not isolated. Due to constraints such as network slice rate requirements and latency requirements, the resources on the core network side and the access network side will restrict and affect each other. By jointly considering the resource management of the core network side and the radio access network side, the overall effectiveness of network slicing is improved, thereby increasing resource utilization.

[0059] The following will describe in more detail each step of the dynamic network slice resource management method in this example embodiment, with reference to the accompanying drawings and embodiments.

[0060] Figure 2 This illustration shows a flowchart of a dynamic network slice resource management method according to an embodiment of the present disclosure. The method provided in this embodiment can be executed by any electronic device with computing power, such as a terminal and / or a server. In the following illustrative examples, a server cluster is used as the execution entity for illustration.

[0061] like Figure 2 As shown, the dynamic network slice resource management method provided in this embodiment may include the following steps:

[0062] Step S210: Construct Markov decision models for the core network side and the access network side, respectively.

[0063] In this embodiment of the disclosure, this step transforms the resource allocation problem of network slices into location and interconnection decisions involving multiple VFNs, thereby turning the management of network slices into a constraint and optimization problem and simplifying the management of dynamic network slices.

[0064] Step S220: Determine the optimization objective for the dynamic network slice resource allocation problem.

[0065] In this embodiment of the disclosure, the optimization objective is to maximize the overall utility of network slices over a long time scale, that is, to maximize the benefits of network slices while ensuring the system transmission rate, and to minimize the deployment cost, in order to manage resources for dynamic network slices. In other embodiments, the optimization objective may also be other objectives that achieve reasonable resource allocation, and this disclosure does not limit this.

[0066] Step S230: Based on the optimization objective, and using the first learning model, train the first policy model of the core network side using the constructed Markov decision model on the core network side.

[0067] In this embodiment of the disclosure, to achieve the optimization goal of maximizing the overall utility of network slices over a long time scale, it is necessary to repeatedly learn and train the weight function to improve the network prediction performance. In this embodiment of the disclosure, the network prediction performance can be improved by learning and training the Markov decision model of the core network.

[0068] For the same reasons as in step S230, step S240 is executed. Based on the optimization objective and the constructed Markov decision model on the access network side, a second policy model on the access network side is trained using the second learning model. By training the Markov decision model of the access network, the performance of network prediction can be improved.

[0069] In this embodiment, steps S230 and S240 can be executed in parallel or sequentially; this disclosure does not restrict the execution order. In this embodiment, the two Markov decision models described above can be trained offline based on a deep reinforcement learning model. In some embodiments, the two Markov decision models described above can be trained offline based on a double-Q learning model.

[0070] Step S250: Based on the first and second policy models, a dynamic network slice resource allocation result is generated. In this embodiment of the disclosure, based on the optimization of network slice resource management at both the core network side and the access network side, the training results in steps S230 and S240 are combined. This allows for real-time adjustment of network slice resource allocation on the access network side to adapt to the dynamic attributes of network slices on the core network side. In other embodiments, the policy model on the access network side may be executed first, followed by the policy model on the core network side, thereby allowing for real-time adjustment of network slice resource allocation on the core network side to adapt to the dynamic attributes of network slices on the access network side.

[0071] Step S260: Perform dynamic network slice resource allocation based on the dynamic network slice resource allocation result.

[0072] The method disclosed herein achieves a reasonable allocation of network slice resources by jointly optimizing the management of network slice resources on both the core network side and the access network side. This enables the network to provide more flexible services for different businesses, thereby improving network service quality, enhancing user experience, and increasing resource utilization.

[0073] Figure 3 The flowchart illustrates step S210 of the dynamic network slice resource management method according to an embodiment of the present disclosure, which is a method for constructing Markov decision models on the core network side and the access network side, respectively. Figure 3 As shown, it includes:

[0074] Step S310: Obtain the first computing resources and the first communication resources on the core network side. In this embodiment of the disclosure, in order to simplify the complexity of the model, the CPU resources of the network node server on the core network side are used as the first computing resources on the core network side, and the inter-network node link bandwidth resources on the core network side are used as the first communication resources on the core network side.

[0075] Step S320: Obtain the second computing resources and the second communication resources on the access network side. In this embodiment of the present disclosure, in order to simplify the complexity of the model, the CPU resources of each distributed processing unit on the access network side are used as the computing resources on the access network side, and the PRB resources on the access network side are used as the communication resources on the access network side.

[0076] Step 330: On the core network side, the dynamic allocation of the first computing resources and the first communication resources is expressed as a first infinite-time Markov decision model.

[0077] In this embodiment of the disclosure, considering the influencing factors in the actual environment, the core network physical network topology may change due to network hardware failures, server power outages, etc. If virtual network nodes are mapped to underlying servers that have stopped working, it will not only lead to a reduction in service quality, but may also affect the allocation of computing and communication resources, making network slices unable to provide services. Therefore, when allocating network slice resources on the core network side, it is necessary to consider the dynamic changes in the physical network topology and clarify the real-time topology status to achieve reasonable resource allocation.

[0078] Therefore, considering the influence of objective factors in actual networks, physical nodes and links between nodes may fail due to equipment failure or excessive load in this embodiment of the disclosure. To indicate the working state of a node, let This indicates that node n is functioning normally; otherwise, the node is invalid. Indicate the working state of the link between nodes n and n', let This indicates that the link between nodes n and n' is in a normal state; otherwise, the link is invalid.

[0079] In this embodiment of the disclosure, a binary node association factor is defined. VNF With physical nodes The mapping relationship is valid if and only if VNR VNF is required in China And the VNF Mapped to node hour Assume that the data stream arriving at each VNF mapped to the physical node undergoes a queuing process, let... Indicates mapping to nodes VNF In the time slot Initial queue length, Slice In the time slot VNF within Similar to the access network side, the number of arriving data packets also follows a Gaussian distribution. Furthermore, they are independently and identically distributed across different time slots. For VNF Required data processing rate. Node VNF The dynamic update process of the queue can be represented as follows:

[0080] (1)

[0081] In this step, the dynamic allocation of the first computing resources and the first communication resources on the core network side is expressed as a first infinite-time Markov decision model. This model is applicable to the quantitative analysis of uncertain events on the core network side of complex communication systems and changes in their global queue state, node state, and link state, and can minimize the overhead of dynamic allocation of core network side resources.

[0082] Step 330: On the access network side, the dynamic allocation of the second computing resources and the second communication resources is expressed as a second infinite-time Markov decision model.

[0083] In this embodiment of the disclosure, a corresponding queuing queue is constructed for data packets of each network slice service. A discrete-time queuing system is considered on the access network side, with a fixed length for each time slot. Multiple different slices can be requested for access within any time slot. Indicates user Access network slice In the time slot The number of data packets arriving within the region follows a Gaussian distribution. Furthermore, they are independently and identically distributed across different time slots. This model constructs a corresponding queuing queue for each network slice. In the time slot The initial queue length is And there are ,in For users slices In the time slot The queue length. The dynamic update process can be represented as:

[0084] (2)

[0085] Among them, time slot The number of packets leaving the queue of slice i is denoted as ,in For the network in time slots Internal allocation to slice The number of PRBs, The size of the data packets in the slice queue. For slices In the time slot The number of data packets arriving within. Let Indicates the system in time slots Global queue status information within. For time slots Global channel state information within.

[0086] In this step, the dynamic allocation of the second computing resources and the second communication resources on the access network side is expressed as a second infinite-time Markov decision model. This model is suitable for quantitative analysis of uncertain events on the access network side and changes in the global queue state and global channel state in complex communication systems, and can maximize the benefits of dynamic allocation of access network side resources.

[0087] Figure 4 The flowchart illustrates step S220 of the dynamic network slice resource management method according to an embodiment of the present disclosure, which is a method for determining the optimization objective of the dynamic network slice resource allocation problem.

[0088] In this embodiment, the optimization objective can be set to maximize the overall utility of network slices over a long time scale, maximizing the average benefit of network slices while ensuring system transmission rate, and minimizing the average deployment cost. The benefit comes from service rate (access network side), and the cost comes from VNF deployment cost (core network side). In this step, a higher rate corresponds to better service provided to users, while simultaneously reducing the system overhead on the core network side.

[0089] like Figure 4As shown, it includes: Step 410, determining the deployment cost of the core network side based on the first computing resources and the first communication resources of the core network side.

[0090] In this embodiment of the disclosure, a core network VNF is defined. Required computing resources With the required data processing rate The relationship is linear. In this embodiment of the disclosure, through... Let represent the correlation coefficient between VNF computing resource requirements and data processing rate, then VNF The computational resource requirements are expressed as follows:

[0091] (3)

[0092] VNF From mapping to node On The data packets leaving the queue are represented as

[0093] (4)

[0094] in Indicates mapping to nodes VNF The queue length.

[0095] make Represents physical nodes , The bandwidth consumption between them can be expressed as

[0096] (5)

[0097] in, The association factor for binary nodes represents VNF. With physical nodes The mapping relationship, This represents the size of the data packets in the slice queue.

[0098] S420 determines the revenue of the access network side based on the second computing resources and the second communication resources on the access network side.

[0099] In this embodiment of the disclosure, since only the service rate gain of the access network-side VNF is considered, the overhead of the access network-side VNF computing resources is not taken into account, thus... Indicates time slot The PRB allocation results for internal communication resources satisfy

[0100] (6)

[0101] Where Z represents the total number of PRBs in the network.

[0102] All network slices in the entire network in time slots The sum rate within can be expressed as

[0103] (7)

[0104] in, This indicates the time slot. The corresponding spectral efficiency, Bandwidth for each PRB.

[0105] In this embodiment, the computing resource set on the access network side can consist of multiple CPU cores. All computing resources are centrally formed into a virtual computing resource pool through virtualization operations. It is assumed that the total number of CPU cores in the computing resource pool is Y. Each CPU core has the same data bandwidth processing capability. Mbps. (This likely refers to a specific speed or unit of measurement, possibly Mbps.) Indicates time slot Internal computing resource allocation results satisfy

[0106] (8)

[0107] That is, time slot The internally allocated computing resources shall not exceed the total computing resources.

[0108] Step 430: Based on the benefits and deployment costs, determine the optimization objectives as maximizing the benefits on the access network side and minimizing the deployment costs on the core network side.

[0109] In this embodiment of the disclosure, the benefits on the access network side and the overhead on the core network side are respectively expressed as follows:

[0110] (9)

[0111] (10)

[0112] (11)

[0113] in, , , These represent the unit price of service rate and the server node, respectively. The unit price of computing resources and the unit price of communication resources between nodes.

[0114] Therefore, the overall average network utility of dynamic network slicing can be expressed as:

[0115] (12)

[0116] Therefore, the optimization objective of this invention, maximizing the average network utility, can be expressed as:

[0117] (13)

[0118] In this disclosure embodiment, one or more of the following constraints are also defined:

[0119] (13a) indicates that the allocation of computing resources must not exceed the total computing resources;

[0120] (13b) indicates that the total link bandwidth occupied by the data stream of each slice in the access network during transmission should not exceed the bandwidth limit of the access network;

[0121] (13c) indicates that the PRB allocation must meet the minimum service rate requirement of the slice;

[0122] (13d) indicates mapping to the same node. The total computing resources required for a VNF cannot exceed that of the node. Total computing resources;

[0123] (13e) indicates that when VNF Mapped to node And VNF Mapped to node At that time, the mapping from virtual links to physical links is realized, and the link bandwidth required by any virtual network function cannot exceed the maximum available bandwidth limit provided between any two nodes. The maximum available bandwidth provided between any two physical nodes;

[0124] (13f) indicates that for each slice requirement, it should be ensured that the required VNF can be mapped to a physical node, where, Slice Is VNF required? If needed ;otherwise .

[0125] This step aims to improve service rates on the access network side while reducing system overhead on the core network side, thereby improving network resource allocation and enhancing user experience.

[0126] In some embodiments, the optimization objective can also be set as the ratio of service rate gains to system overhead; a larger ratio indicates a more optimized allocation decision. In other embodiments, the optimization objective can be other objectives, which are not limited herein.

[0127] Figure 5 This illustration shows step S250 of the dynamic network slice resource management method according to an embodiment of the present disclosure, a flowchart of a method for generating dynamic network slice resource allocation results based on a first policy model and a second policy model. In this embodiment, the policy models on the core network side and the access network side are executed sequentially; the policy model on the core network side can be executed first, or the policy model on the access network side can be executed first. In this embodiment, the policy model on the core network side is executed first, and then the allocation result generated on the core network side is input into the policy model on the access network side. This achieves the adjustment of the VNF resource allocation method of the access network slice according to the dynamic changes of the VNFs in the core network slice, making the network resource allocation more reasonable and thus improving resource utilization.

[0128] like Figure 5 As shown, in step S510, a randomly generated allocation result of computing and communication resources on the access network side that satisfies preset constraints is input into the first strategy model to obtain a first allocation result including computing and communication resources on the core network side.

[0129] In this embodiment of the disclosure, the constraint can be at least one of (13a)-(13c). In some embodiments, the constraint can be any condition related to the optimization objective.

[0130] In this embodiment of the disclosure, the randomly generated computing and communication resources on the access network side that meet the constraints can be the allocation method of the CPU resources of each distributed processing unit on the access network side and the number of physical resource blocks (PRBs) in each radio remote unit (RRU).

[0131] In this embodiment of the disclosure, the first policy model can be generated by optimizing the first Markov decision model on the core network side based on a deep reinforcement learning model. In some embodiments, the deep reinforcement learning model can be a double-Q learning model.

[0132] In this embodiment of the disclosure, the first allocation result obtained includes the allocation result of computing resources and communication resources on the core network side, which may be the VNF migration result and VNF resource allocation result formed based on the network node CPU resources and network node link bandwidth resources on the core network side.

[0133] Step S520: Input the first allocation result into the second strategy model to obtain the second allocation result, which includes the computing resources and communication resources on the access network side.

[0134] In this embodiment of the disclosure, the second policy model can be generated by optimizing the second Markov decision model on the access network side based on a deep reinforcement learning model. In some embodiments, the deep reinforcement learning model can be a double-Q learning model.

[0135] In this embodiment of the disclosure, the second allocation result includes the computing resources and communication resources on the access network side, which may be the allocation method of the CPU resources of each distributed processing unit on the access network side and the number of physical resource blocks (PRBs) in each radio frequency remote unit (RRU).

[0136] Step S530: Generate dynamic network slice resource allocation results based on the first allocation result and the second allocation result.

[0137] In this embodiment of the disclosure, the dynamic network slice resource allocation result may include the VNF migration result on the core network side, the VNF resource allocation result on the core network side, the PRB allocation result on the access network side, and the CPU resource allocation result on the access network side.

[0138] Using the methods described above, the deployment and orchestration of core network slices and access network slices can be adjusted according to the characteristics of the core network and access network, thereby achieving optimized allocation of network resources.

[0139] Figure 6 This diagram illustrates an overall flowchart of a dynamic network slice resource management method according to an embodiment of the present disclosure. Figure 6 As shown, it includes:

[0140] Step S610: Construct Markov decision models for the core network side and the access network side, respectively.

[0141] Step S620: On the core network side, the dynamic allocation of the first computing resources and the first communication resources is expressed as a first infinite-time Markov decision model.

[0142] Step S630: On the access network side, the dynamic allocation of the second computing resources and the second communication resources is expressed as a second infinite-time Markov decision model.

[0143] Steps S610-S630 and Figures 2-3 Step S210 is similar and will not be repeated here.

[0144] Step S640: Determine the optimization objective for the dynamic network slice resource allocation problem. Step S640 and... Figure 2 Step S220 is similar and will not be repeated here.

[0145] Step S650: Based on the optimization objective, and using the first learning model, train the first policy model of the core network side using the constructed Markov decision model on the core network side.

[0146] In this embodiment of the disclosure, the first computing resource on the core network side is the network node CPU resource, and the first communication resource is the link bandwidth resource between nodes.

[0147] In this embodiment of the disclosure, the dynamic allocation of CPU resources on servers and link bandwidth resources between servers in the underlying physical network, as well as the migration results of VNFs, can be described as a constrained Markov decision problem. A quadruple is defined, consisting of system state, migration behavior, state transition probability, and cost function. The system is then processed in time slots. The state is defined as In the time slot Action definition ,in, For time slots The two-dimensional transfer action vector within the VNF consists of two parts: , , , , Indicates in time slot slices VNF Perform the migration; otherwise, return 0. VNF The target node to migrate; For time slots The set of mapping actions for each VNF within the VNF.

[0148] The process of mapping the state space to the action space is defined as follows: It is a stability strategy, that is According to the strategy time slot The expected cumulative CPU resource allocation within is as follows:

[0149] (14)

[0150] Expected cumulative bandwidth resource allocation is as follows

[0151] (15)

[0152] in, This is a discount factor, representing the degree of decay of the reward function value, indicating the extent to which future rewards influence current behavioral choices. The core network-side optimization objective is to find suitable VNF migration results. and VNF resource allocation results To minimize deployment costs, the stochastic optimization model can be expressed as:

[0153] (16)

[0154] Therefore, in this embodiment of the disclosure, based on the definition and description of the Markov decision problem described above, taking the double-Q learning model as an example, the first policy model is defined as follows:

[0155] State (Environment): Slice In the time slot The state is defined as ,in For global queue status, For node status, This indicates the link status.

[0156] Action: Satisfaction A set of VNF migration and resource allocation actions.

[0157] Rewards: The agent selects and executes actions from the set of migration behaviors, which affects the environment. The environment then feeds back the reward function generated by the current action to the agent, which uses this function to determine the merits of the selected action. Similarly, when a VNF in a slice selects an action from the action set, a reward function is needed to determine the merits of the selected action. This reward function can be defined as the utility of each slice. If the action selection does not satisfy constraints (13a)-(13f), the feedback value is set to a fixed -1.

[0158] (17)

[0159] The training process is as follows:

[0160] The first step is to input the current system state (global queue state, node state, and link state) into the main network of the first strategy model to generate precise values ​​of the predicted core network-side computing and communication resource allocation results. Select the core network side computing and communication resource allocation results corresponding to the function with the minimum action value;

[0161] The second step is to store the current global queue state, node state, and link state c in the system state, as well as the next system state (global queue state, node state, and link state) c' and the resource allocation utility reward r corresponding to the next system state into the experience replay pool.

[0162] The third step involves randomly selecting a system state sample c' from the experience replay pool and inputting it into the target Q network to obtain the predicted values ​​of the core network-side computing and communication resource allocation results; and

[0163] The fourth step is to calculate the loss function of the target Q-network during training, obtain the current loss, and update the parameter weights of the target Q-network using gradient descent. This continues until the loss no longer decreases, i.e., until the first policy model is generated.

[0164] Step 660: Based on the optimization objective, and using the constructed Markov decision model on the access network side, train the second policy model on the access network side using the second learning model.

[0165] In this embodiment of the disclosure, the second computing resource on the access network side is the CPU resource of each distributed processing unit, and the second communication resource is the number of physical resource blocks (PRBs) in each radio frequency remote unit (RRU).

[0166] In this embodiment of the disclosure, the system is time-slotted in the access network. The state is defined as In the time slot Action definition The process of mapping the state space to the action space is defined as follows: This is a result of a stable strategy, namely Based on the initial state c and the policy result ,in Representing the set of all possible policy outcomes, the expected cumulative slice and rate can be obtained as follows:

[0167] (18)

[0168] Similarly, discount factor This represents the degree of decay of the reward function value, indicating the extent to which future rewards influence current behavioral choices. The optimization objective on the access network side is to find a suitable allocation of PRBs and computing resources to maximize network slice revenue, while satisfying the minimum service rate constraints and network bandwidth resource constraints for each network slice. The stochastic optimization model can be expressed as:

[0169] (19)

[0170] Therefore, in this embodiment of the disclosure, based on the definition and description of the Markov decision problem described above, taking the double-Q learning model as an example, the second policy model is defined as follows:

[0171] State (Environment): Define the state of the access network side network system as follows. This includes global queue status and global channel status information.

[0172] Actions: The action set is defined as a series of vectors, each vector representing a slice. The allocation of PRBs and computing resources on all RRUs satisfies The set of PRB and computing resource allocation actions. This represents the number of CPU cores that can be selected for each slice. This represents the number of optional PRBs for each slice.

[0173] Reward: Considering that the optimization objective of this algorithm is to maximize the sum of system utilities, the reward function is defined as the sum of system utilities obtained after selecting their respective PRBs and computational resources when constraints (13a)-(13f) are satisfied; otherwise, it is defined as a negative feedback, i.e.

[0174] (20)

[0175] The training process is as follows:

[0176] The first step is to input the current system state (global queue state and global channel state) into the main network of the second strategy model to generate precise values ​​of the predicted access network-side computing and communication resource allocation results. Select the access network side computing and communication resource allocation results corresponding to the function with the maximum action value;

[0177] The second step is to store the current global queue state and global channel state c in the system state, as well as the next system state (global queue state and global channel state) c' and the resource allocation utility reward r corresponding to the next system state into the experience replay pool.

[0178] The third step involves randomly selecting a system state sample c' from the experience replay pool and inputting it into the target Q network to obtain the predicted values ​​of the access network computing and communication resource allocation results; and

[0179] The fourth step is to calculate the loss function of the target Q-network during training, obtain the current loss, and update the parameter weights of the target Q-network using gradient descent. This continues until the loss no longer decreases, i.e., until the second strategy model is generated.

[0180] Step 670: Input a randomly generated allocation result of computing and communication resources on the access network side that satisfies preset constraints into the first strategy model on the core network side to obtain the first allocation result including computing and communication resources on the core network side.

[0181] In this embodiment of the disclosure, the online decision-making process of the first strategy model can be as follows:

[0182] Input: Underlying physical network topology Virtual network topology PRB and CPU resource allocation methods ;

[0183] Step (1): for t=1,2,…T do;

[0184] Step (2): Monitor the current time slot Global state of the core network side This includes global queue status information. Global node status and global link status ;

[0185] Step (3): if or ;

[0186] Step (4): When the condition is satisfied... All Based on the migration to other nodes, calculate the appropriate VNF migration results. and VNF communication and computing resource allocation results ;

[0187] Step (5): else;

[0188] Step (6): Directly calculate the appropriate VNF migration results and VNF communication and computing resource allocation results ;

[0189] Step (7): Based on appropriate actions Perform VNF migration and allocate communication and computing resources;

[0190] Step (8): t = t + 1;

[0191] Step (9): end for;

[0192] Output: VNF transition results and VNF resource allocation results .

[0193] In step (2), the core network status is monitored, and the currently monitored network status is used as the input to the main network. In steps (3)-(5), to improve network reliability, time slots are used... If a physical node or link fails, the system needs to migrate the VNF mapped to the failed physical node. In this process, the appropriate VNF migration result is selected with the goal of maximizing the overall utility of the network slice. Steps (5)-(6) indicate that if there is no failed node or link, the appropriate migration result is directly selected.

[0194] Step 680: Input the allocation method of computing resources and communication resources of the first allocation result into the second strategy model to obtain the second allocation result including computing resources and communication resources on the access network side.

[0195] In this embodiment of the disclosure, the online decision-making process of the second strategy model can be as follows:

[0196] Input: VNF transition results and VNF resource allocation results ;

[0197] Step (1): for t=1,2,…T do;

[0198] Step (2): Monitor the current time slot The global state c(t) on the access network side includes global queue state information. Global channel state information ;

[0199] Step (3): Calculate the appropriate PRB and calculate resource allocation actions. ;

[0200] Step (4): Based on appropriate actions Adjust the PRB and computing resource allocation of the wireless access network slice;

[0201] Step (5): t = t + 1;

[0202] Step (6): end for;

[0203] Output: PRB allocation results CPU resource allocation results .

[0204] The appropriate VNF migration results and VNF resource allocation results obtained through the first strategy model are used as inputs to the second strategy model. Step (3) is to obtain the state based on the latest output results of the main network. The appropriate action Step (4) according to The algorithm jointly adjusts the slice calculation and link resources on the access network side to ensure service quality in real time. The entire algorithm continues until the end of the slice's lifecycle, yielding the final VNF migration result. VNF resource allocation results .

[0205] Step 690: Generate dynamic network slice resource allocation results based on the first allocation result and the second allocation result.

[0206] In this embodiment of the disclosure, the obtained VNF migration results are processed according to the first strategy model and the second strategy model. VNF resource allocation results and PRB allocation results CPU resource allocation results This is the result of resource allocation in dynamic network slicing.

[0207] By using the above-mentioned method of jointly allocating communication and computing resources on the core network side and the access network side, the communication and computing resources in the dynamic network slice are dynamically and jointly allocated, which optimizes the overall utility of the network slice, increases the average benefit of network service rate, reduces the system deployment cost, and thus improves the quality of network service and enhances user experience.

[0208] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0209] Figure 7 A schematic diagram of a dynamic network slice resource management device according to an embodiment of this disclosure is shown. Figure 7 As shown, it includes:

[0210] Modeling module 710 is used to construct Markov decision models for the core network side and the access network side, respectively;

[0211] Optimization module 720 is used to determine the optimization objective for the dynamic network slice resource allocation problem;

[0212] The first training module 730 is used to train the first policy model of the core network side based on the constructed Markov decision model of the core network side according to the optimization objective and using the first learning model.

[0213] The second training module 740 is used to train the second policy model of the access network side based on the constructed Markov decision model of the access network side and the second learning model, according to the optimization objective; and

[0214] The strategy generation module 750 is used to generate dynamic network slice resource allocation results based on the first strategy model and the second strategy model; and

[0215] The execution module 760 is used to perform dynamic network slice resource allocation based on the dynamic network slice resource allocation results.

[0216] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0217] The following reference Figure 8 To describe an electronic device 800 according to this embodiment of the present invention. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0218] like Figure 8 As shown, the electronic device 800 is manifested in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0219] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 810 can perform actions such as... Figure 2 As shown in the diagram, S210 constructs Markov decision models for both the core network side and the access network side; S220 determines the optimization objective for the dynamic network slice resource allocation problem; S230, based on the optimization objective and the constructed Markov decision model for the core network side, trains a first policy model for the core network side using a first learning model; S240, based on the optimization objective and the constructed Markov decision model for the access network side, trains a second policy model for the access network side using a second learning model; S250, based on the first policy model and the second policy model, generates dynamic network slice resource allocation results; and S260, based on the dynamic network slice resource allocation results, performs dynamic network slice resource allocation.

[0220] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202, and may further include a read-only memory unit (ROM) 8203.

[0221] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0222] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0223] Electronic device 800 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0224] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0225] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0226] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0227] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0228] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0229] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0230] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0231] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0232] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0233] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0234] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for dynamic network slice resource allocation, characterized in that, include: Markov decision models are constructed for the core network side and the access network side, respectively; The optimization objectives for the dynamic network slicing resource allocation problem are determined, including maximizing the benefits on the access network side and minimizing the deployment costs on the core network side. According to the optimization objective, based on the constructed Markov decision model of the core network side, the first policy model of the core network side is trained using the first learning model. According to the optimization objective, based on the constructed Markov decision model on the access network side, the second policy model on the access network side is trained using the second learning model; Based on the first and second strategy models, dynamic network slice resource allocation results are generated. as well as Based on the dynamic network slice resource allocation results, perform dynamic network slice resource allocation.

2. The dynamic network slice resource allocation method according to claim 1, characterized in that, The Markov decision models constructed for the core network side and the access network side respectively include: Acquire the first computing resources and the first communication resources on the core network side; Acquire the second computing resources and second communication resources on the access network side; On the core network side, the dynamic allocation of the first computing resources and the first communication resources is expressed as a first infinite-time Markov decision model. On the access network side, the dynamic allocation of the second computing resources and the second communication resources is expressed as a second infinite-time Markov decision model.

3. The dynamic network slice resource allocation method according to claim 2, characterized in that, The optimization objectives for the dynamic network slice resource allocation problem include: Based on the first computing resources and the first communication resources on the core network side, the deployment cost of the core network side is determined. Based on the second computing resources and the second communication resources on the access network side, determine the revenue of the access network side; Based on the aforementioned benefits and deployment costs, the optimization objectives are determined to be maximizing the benefits on the access network side and minimizing the deployment costs on the core network side.

4. The dynamic network slice resource allocation method according to any one of claims 2 or 3, characterized in that, The acquisition of the first computing resources and the first communication resources on the core network side also includes: The first computing resource on the core network side is determined based on the CPU resources of each network node; and The first communication resource on the core network side is determined based on the link bandwidth resources of each network node.

5. The dynamic network slice resource allocation method according to any one of claims 2 or 3, characterized in that, The acquisition of the second computing resources and second communication resources on the access network side also includes: The second computing resources on the access network side are determined based on the CPU resources of each distributed processing unit; and The second communication resource on the access network side is determined based on the number of Physical Resource Blocks (PRBs) in each Radio Remote Unit (RRU).

6. The dynamic network slice resource allocation method according to claim 2, characterized in that, According to the optimization objective, based on the constructed Markov decision model of the core network side, a first policy model of the core network side is trained using a first learning model, wherein: The first learning model is a deep reinforcement learning model.

7. The dynamic network slice resource allocation method according to claim 2, characterized in that, According to the optimization objective, based on the constructed Markov decision model on the access network side, a second policy model on the access network side is trained using a second learning model, wherein: The second learning model is a deep reinforcement learning model.

8. The dynamic network slice resource allocation method according to any one of claims 6 or 7, characterized in that, The deep reinforcement learning model is a double-Q learning model.

9. The dynamic network slice resource allocation method according to claim 2, characterized in that, Based on the first and second strategy models, the dynamic network slice resource allocation results are generated as follows: The allocation result of computing and communication resources on the access network side, which is randomly generated and meets preset constraints, is input into the first strategy model on the core network side to obtain the first allocation result including computing and communication resources on the core network side. The first allocation result is input into the second strategy model to obtain a second allocation result that includes the computing and communication resources on the access network side; and Based on the first allocation result and the second allocation result, a dynamic network slice resource allocation result is generated.

10. A dynamic network slicing resource allocation device, characterized in that, include: The modeling module is used to construct Markov decision models for the core network side and the access network side, respectively. An optimization module is used to determine the optimization objective for the dynamic network slice resource allocation problem. The optimization objective is used to maximize the benefits on the access network side and minimize the deployment costs on the core network side. The first training module is used to train the first policy model of the core network side based on the constructed Markov decision model of the core network side according to the optimization objective and using the first learning model. The second training module is used to train the second policy model of the access network side based on the constructed Markov decision model of the access network side according to the optimization objective and using the second learning model. and The result generation module is used to generate dynamic network slice resource allocation results based on the first strategy model and the second strategy model. as well as The execution module is used to perform dynamic network slice resource allocation based on the dynamic network slice resource allocation result.

11. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the dynamic network slice resource allocation method of any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic network slice resource allocation method according to any one of claims 1 to 9.

Images