A resource scheduling method, system, device and medium for a 5G access network slice

Abstract

The application discloses a resource scheduling method, system and device for 5G access network slices and a medium, and comprises the following steps: acquiring state information of each access network slice; inputting the state information of each access network slice into a trained competitive deep Q network model; and performing air interface resource scheduling between each access network slice according to an output result of the trained competitive deep Q network model. The method, system, device and medium can realize air interface resource scheduling between each access network slice, improve the utilization rate of air interface resources, and ensure the service quality of services.

Description

Technical Field

[0001] This invention belongs to the field of communication technology and relates to a resource scheduling method, system, device and medium for 5G access network slicing. Background Technology

[0002] After the development of R15-R17, 5G communication technology, based on technologies such as Network Functions Virtualization (NFV) and Software-Defined Networking (SDN), has acquired the ability to customize resources. Operators leverage 5G network slicing technology to isolate multiple virtual end-to-end networks on a unified infrastructure. Based on the latency, bandwidth, reliability, and security requirements of vertical industry services, shared physical resources are scheduled onto logical network slices, providing end-to-end logically isolated communication networks for vertical industry users. 5G network slicing includes access network slicing, bearer network slicing, and core network slicing. Among these, access network slicing, due to its diverse and numerous terminal types and limited air interface resources, is the focus of slicing design and radio resource allocation research.

[0003] With the integration of power systems and 5G, 5G slicing technology has solved communication problems in many power business scenarios, promoting the digital transformation of power services. However, with the construction of new power systems, the number of user-side terminals is gradually increasing, and the interaction of control information between the system and user-side terminals is becoming more frequent. This has led to a significant increase in data communication demands for power control services. The differentiated communication needs of power control services place higher demands on the real-time performance, reliability, and resource allocation flexibility of communication networks. Furthermore, the scarcity of wireless air interface resources means that if 5G access network slices are not customized according to the communication needs and characteristics of the services, the adaptability of slices to system services will be insufficient, affecting the slices' ability to support power control services. In addition, in actual slice operation, resource scheduling between slices is needed to ensure that slices carrying power control services can quickly obtain the necessary resources and avoid resource waste. Therefore, it is necessary to study slice resource allocation methods for power control services to ensure service quality and air interface resource utilization, but existing technologies do not provide specific solutions. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a resource scheduling method, system, device and medium for 5G access network slices. This method, system, device and medium can realize the scheduling of air interface resources between access network slices, improve the utilization rate of air interface resources and ensure the quality of service.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In one aspect, the present invention provides a resource scheduling method for 5G access network slicing, comprising:

[0007] Obtain the status information of each access network slice;

[0008] The state information of each access network slice is input into the trained competition depth Q network model, and air interface resource scheduling between each access network slice is performed based on the output of the trained competition depth Q network model.

[0009] The resource scheduling method for 5G access network slicing described in this invention is further improved in that:

[0010] The status information of the access network slice includes the number of power control services accessed within the access network slice, the amount of air interface resources required by the access network slice, the latency requirements of the power control services within the access network slice, and the amount of air interface resources allocated to the access network slice.

[0011] The reward function of the competitive deep Q-network model during training is:

[0012]

[0013] Among them, B m,g (τ) represents the amount of RB resources already acquired in the access network slice, λ is the reward coefficient, and B m,h (τ) represents the RB resource quantity information of the access network slicing requirement.

[0014] The action value function of the competitive deep Q-network model during the training process is:

[0015]

[0016] Among them, w evaluate V(s) represents the parameters of the value network. m (τ); w evaluate A(s) is a scalar related to the state. m (τ),a m (τ); w evaluate ) are vectors related to states and actions, A represents the action space of the competing deep Q-network model, and a m (τ) represents the discrete air interface resource allocation value in A, s m (τ) represents the state of access network slice m at time τ.

[0017] The loss function of the competitive deep Q-network model during training is:

[0018]

[0019] Among them, K m N represents the number of empirical samples used for training. m(τ) represents the empirical samples used for training, γ represents the discount factor of the competitive deep Q-network model, and w target R represents the parameters of the target network. m (τ) represents the result of the reward function calculation.

[0020] Before obtaining the status information of each access network slice, the process also includes:

[0021] Design of access network slicing based on k-means algorithm;

[0022] When a power control service is initiated, an access network slice is allocated to carry the power control service.

[0023] In a second aspect, the present invention provides a resource scheduling system for 5G access network slicing, comprising:

[0024] The acquisition module is used to acquire the status information of each access network slice;

[0025] The scheduling module is used to input the status information of each access network slice into the trained competition depth Q network model, and to perform air interface resource scheduling among the access network slices based on the output of the trained competition depth Q network model.

[0026] The resource scheduling system for 5G access network slicing described in this invention is further improved in that:

[0027] Also includes:

[0028] The design module is used to design access network slices based on the k-means algorithm.

[0029] The allocation module is used to allocate access network slices to carry power control services when power control services are initiated.

[0030] In three aspects, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the resource scheduling method for the 5G access network slice.

[0031] In four aspects, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the resource scheduling method for 5G access network slicing.

[0032] The present invention has the following beneficial effects:

[0033] In specific operation, the resource scheduling method, system, device, and medium for 5G access network slicing described in this invention inputs the status information of each access network slice into a trained competition-depth Q-network model to match the communication requirements of power control services. Then, based on the output results of the trained competition-depth Q-network model, air interface resource scheduling is performed between each access network slice to achieve intelligent and efficient scheduling of air interface resources, ensuring the air interface resource requirements of power control services and improving the utilization rate of air interface resources. Attached Figure Description

[0034] The accompanying drawings, which form part of this specification, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0035] Figure 1 Flowchart for access network slicing design;

[0036] Figure 2 This is a flowchart of the method of the present invention;

[0037] Figure 3 Flowchart for training a competitive deep Q-network model;

[0038] Figure 4 This is a system structure diagram of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] The present invention will now be described in further detail with reference to the accompanying drawings:

[0042] Example 1

[0043] The resource scheduling method for 5G access network slicing described in this invention includes the following steps:

[0044] 1) Design access network slices based on k-means algorithm.

[0045] refer to Figure 1 The specific process of step 1) is as follows:

[0046] 11) Calculate the communication requirements of services that need to be carried by 5G slicing to obtain the bandwidth, service latency and reliability of power control services, and use bandwidth, service latency and reliability as three indicators for power control services.

[0047] in, There are two ways to obtain the length of the data packet: a) use Wireshark to capture the data packets of the actual running service terminal and calculate the length of the service data packet; b) estimate the length of the service data packet by analyzing the requirements of the service communication data unit structure in the protocol; the service delay is obtained according to the standard.

[0048] 12) Standardize the bandwidth, latency, and reliability of power control services to obtain standardized samples.

[0049] Specifically, the deviation standardization method is used for standardization, that is: Remove the units from different performance indicators and convert them to pure numerical values ​​to facilitate calculations for indicators with different units. The numerical range of the standardized indicators is [y]. min ,y max ] = [0, 1].

[0050] 3) Clustering algorithms are used to perform cluster analysis on all power control-related services to obtain the optimal number of clusters and cluster centers.

[0051] In this embodiment, the k-means algorithm is used for cluster analysis. Specifically, k samples are randomly selected from all standardized samples as centroids. The distances between the remaining sample points and the centroids are calculated. The sample points are then divided into k clusters based on the distances between the centroids and the remaining sample points. The centroids are recalculated, and the above process is repeated until the required number of calculations is met. The squared error for different k values ​​is calculated. M is cluster C i The total number of samples, x i For cluster C i One sample, p Ci For cluster C iThe center point is considered to be the optimal number of clusters when the decrease in SSE tends to level off as the value of k increases.

[0052] 14) Analyze the clustering results, and use an access network slice to carry the services contained in each cluster. According to the service communication index requirements, set some slice parameters. The partial slice parameters include the maximum number of UEs that can access the network, latency, network availability, a certain achievable rate for a single UE, and the proportion of UEs communicating at the same time. Among them, the certain achievable rate for a single UE is set to the maximum value of the bandwidth requirement for power control services in this category, the latency is set to the minimum value of the latency requirement for services in this category, the network availability is set to the maximum value of the reliability requirement for services in this category, and the remaining slice parameters are determined according to the specific deployment of power control services in the slice.

[0053] 2) After a power control service is initiated, the base station allocates an access network slice to be carried based on the clustering results of the power control service, and assigns the power control service to the access network slice.

[0054] 3) Obtain the status information of each access network slice.

[0055] The access network slice status information includes the number of power control services accessing the access network slice, the amount of air interface resources required by the access network slice, the latency requirements of the power control services within the access network slice, and the amount of air interface resources allocated to the access network slice. The amount of air interface resources required by the access network slice is calculated based on the communication requirement indicators of the power control services within the slice.

[0056] 4) Input the state information of each access network slice into the trained competition depth Q network model, and perform dynamic scheduling of air interface resources among each access network slice according to the output of the competition depth Q network model to ensure that the air interface resources allocated to each access network slice match the required air interface resources. Here, air interface resources refer to resource blocks (RBs) defined in the 5G air interface time-frequency domain.

[0057] refer to Figure 2 The specific steps for dynamic resource scheduling are as follows:

[0058] S1) Determine the goal of resource scheduling: to ensure that power control services within the access network slice can obtain sufficient air interface resources, while ensuring that the access network slice has a high resource utilization rate.

[0059] S2) The allocation of multiple access network slice resources is modeled as a Markov decision process, and the state space is s. m (τ)={m,B a (τ),B m,g (τ),B m,h(τ)}, this state space represents the state of slice m at time τ, where m represents the access network slice identification code, B a (τ) represents the current availability of RB resources in the network, B m,g (τ) represents the amount of RB resources already acquired by the slice, B m,h (τ) represents the RB resource quantity information of the slice requirement; the action space is a. m (τ)={a1,a2,...a k The action space represents k discrete air interface resource allocation values, the specific values ​​of which are changed according to the actual business calculation requirements. The reward function is:

[0060]

[0061] The reward function represents the value returned by the environment after the network takes an action. It needs to maximize the goal of resource scheduling. λ is the reward coefficient, which is used to discount the reward.

[0062] During training, the S3 agent (base station) acquires the state of each access network slice and outputs an action to receive a reward, and further updates the parameters of the competitive deep Q network model.

[0063] S4) The present invention makes improvements in action selection: it checks the available resources in the current access network slice and blocks some illegal actions to speed up the training process.

[0064] S5) After a period of training, once the agent converges, it can accurately allocate the required air interface resources to each access network slice.

[0065] like Figure 3 As shown, the training steps of the neural network in the competitive deep Q-network model are as follows:

[0066] 41) Select a batch of experience samples N from the experience playback library. m (τ) serves as the input to both the value network and the target network, N m The empirical sample size of (τ) is K. m .

[0067] 42) During training, taking a value network as an example, the update of its action value function is as follows:

[0068]

[0069] Among them, w evaluate V(s) represents the parameters of the value network. m (τ); w evaluate A(s) is a scalar related to the state. m (τ),a m (τ); wevaluate ) are vectors related to states and actions, A represents the action space of the competing deep Q-network model, and a m (τ) represents a discrete air interface resource allocation value selected from A, s m (τ) represents the state of access network slice m at time τ, and the action value function output by the target network can be derived by analogy.

[0070] The loss function during training is:

[0071]

[0072] During training, the parameters of the competing deep Q-network model are updated using the gradient descent algorithm, specifically: α is the learning rate of the network. For L(w) evaluate Regarding w evaluate The gradient.

[0073] Example 2

[0074] refer to Figure 4 The resource scheduling system for 5G access network slicing described in this invention includes:

[0075] Design module 3 is used to design access network slices based on the k-means algorithm;

[0076] Allocation module 4 is used to allocate access network slices to carry power control services when power control services are initiated.

[0077] Module 1 is used to obtain the status information of each access network slice;

[0078] The scheduling module 2 is used to input the status information of each access network slice into the trained competition depth Q network model, and to perform air interface resource scheduling among each access network slice according to the output results of the trained competition depth Q network model.

[0079] Example 3

[0080] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a resource scheduling method for a 5G access network slice. The memory may include main memory, such as high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, an extended industry-standard architecture bus, etc. The bus may be classified as an address bus, a data bus, a control bus, etc. The memory stores the program; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0081] Example 4

[0082] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a resource scheduling method for a 5G access network slice. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0083] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A resource scheduling method for 5G access network slicing, characterized in that, include: Obtain the status information of each access network slice; The state information of each access network slice is input into the trained competition depth Q network model, and air interface resource scheduling between each access network slice is performed based on the output of the trained competition depth Q network model. The status information of the access network slice includes the number of power control services accessed within the access network slice, the amount of air interface resources required by the access network slice, the latency requirements of the power control services within the access network slice, and the amount of air interface resources allocated to the access network slice. The reward function of the competitive deep Q-network model during training is: in, This represents the amount of RB resources already acquired by the access network slice, where λ is the reward coefficient. Information on the amount of RB resources required for access network slicing; The action value function of the competitive deep Q-network model during the training process is: in, Parameters representing the value network, For state-dependent scalars, A vector relating to state and action. This represents the action space of a competitive deep Q-network model. express Discrete air interface resource allocation values, This represents the state of access network slice m at time τ; The loss function of the competitive deep Q-network model during training is: in, This indicates the number of empirical samples used for training. This represents the empirical samples used in training. This represents the discount factor for the competitive deep Q-network model. The parameters representing the target network, This is the result of the reward function calculation.

2. The resource scheduling method for 5G access network slicing according to claim 1, characterized in that, Before obtaining the status information of each access network slice, the process also includes: Design of access network slicing based on k-means algorithm; When a power control service is initiated, an access network slice is allocated to carry the power control service.

3. A resource scheduling system for 5G access network slicing, characterized in that, include: The acquisition module (1) is used to acquire the status information of each access network slice; The scheduling module (2) is used to input the status information of each access network slice into the trained competitive deep Q network model, and to perform air interface resource scheduling between each access network slice according to the output result of the trained competitive deep Q network model. The status information of the access network slice includes the number of power control services accessed within the access network slice, the amount of air interface resources required by the access network slice, the latency requirements of the power control services within the access network slice, and the amount of air interface resources allocated to the access network slice. The reward function of the competitive deep Q-network model during training is: in, This represents the amount of RB resources already acquired by the access network slice, where λ is the reward coefficient. Information on the amount of RB resources required for access network slicing; The action value function of the competitive deep Q-network model during the training process is: in, Parameters representing the value network, For state-dependent scalars, A vector relating to state and action. This represents the action space of a competitive deep Q-network model. express Discrete air interface resource allocation values, This represents the state of access network slice m at time τ; The loss function of the competitive deep Q-network model during training is: in, This indicates the number of empirical samples used for training. This represents the empirical samples used in training. This represents the discount factor for the competitive deep Q-network model. The parameters representing the target network, This is the result of the reward function calculation.

4. The resource scheduling system for 5G access network slicing according to claim 3, characterized in that, Also includes: Design module (3) is used to design access network slices based on the k-means algorithm; The allocation module (4) is used to allocate access network slices to carry the power control service when a power control service is initiated.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the resource scheduling method for 5G access network slicing as described in any one of claims 1-2.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the resource scheduling method for 5G access network slicing as described in any one of claims 1-2.

Images