Mobile load balancing method and device, electronic device and storage medium

Abstract

The embodiments of the present disclosure provide a mobile load balancing method and apparatus, an electronic device, and a storage medium. The method is applied to a mobile network architecture based on network slicing, wherein the mobile network architecture includes N network slices, J remote radio units (RRUs), and U users, where N, J, and U are all positive integers greater than or equal to 1; wherein the method includes: obtaining satisfaction of each network slice; obtaining system satisfaction based on the satisfaction of each network slice; and if the system satisfaction does not meet the requirements, implementing a slice-level load control strategy to adjust the proportion of wireless resources allocated to each RRU to the network slice.

Description

Technical Field

[0001] The present disclosure relates to the field of wireless communication technologies, and in particular to a mobile load balancing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In 3GPP TS 36.902, Mobility Load Balancing (MLB) is defined as a key function in Self-Organizing Networks (SON). As the number of users continues to grow, load imbalances can occur when a network reaches a certain scale. Within the existing network architecture, mobile load balancing achieves a relatively balanced load across access points, thereby expanding system capacity and optimizing network performance. From the perspective of the entire network system, mobile load balancing balances the load between cells, improving overall network throughput and thus enhancing the network's data processing capabilities.

[0003] With the continuous development of wireless technology, network slicing has been identified as one of the key technologies for the Fifth Generation Mobile Communication System (5G). It will play a vital role in future mobile networks by providing flexible, on-demand network services to meet users' diverse communication needs. The introduction of network slicing technology enables highly flexible and efficient resource allocation, significantly improving network capacity, latency, and transmission speed.

[0004] In addition to these significant benefits, the introduction of network slicing also brings many challenges to the design of the Radio Access Network (RAN), including network function virtualization, network resource allocation, and mobility management. In particular, mobile load balancing is crucial for ensuring user service needs in changing communication environments (for example, channel conditions at different access points, slice resource availability, etc.). This is not only related to ensuring user Service Level Agreement (SLA) requirements, but also has a significant impact on slice deployment and radio resource management.

[0005] With the introduction of network slicing, traditional mobile load balancing mechanisms are no longer applicable. Summary of the Invention

[0006] The embodiments of the present disclosure provide a mobile load balancing method and apparatus, an electronic device, and a computer-readable storage medium. Taking into account the impact of network slicing on mobile load balancing, a mobile load balancing method for 5G RAN slicing is proposed.

[0007] Embodiments of the present disclosure provide a mobile load balancing method, which is applied to a network slicing-based mobile network architecture, wherein the mobile network architecture includes N network slices, J remote radio units (RRUs), and U users, where N, J, and U are all positive integers greater than or equal to 1. The method includes: obtaining a satisfaction level for each network slice; obtaining a system satisfaction level based on the satisfaction level of each network slice; and, if the system satisfaction level does not meet the requirements, implementing a slice-level load control strategy to adjust the proportion of radio resources allocated to each RRU to the network slice.

[0008] An embodiment of the present disclosure provides a mobile load balancing device, comprising: the device being applied to a mobile network architecture based on network slicing, the mobile network architecture comprising N network slices, J remote radio units (RRUs), and U users, where N, J, and U are all positive integers greater than or equal to 1. The device comprises: a slice satisfaction obtaining unit, configured to obtain the satisfaction of each network slice; a system satisfaction obtaining unit, configured to obtain the system satisfaction based on the satisfaction of each network slice; and a slice-level load control unit, configured to implement a slice-level load control strategy to adjust the proportion of radio resources allocated to each RRU to the network slice if the system satisfaction does not meet the requirements.

[0009] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the mobile load balancing method in the above embodiment is implemented.

[0010] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a storage device configured to store at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the mobile load balancing method as described in the above embodiment.

[0011] According to one aspect of the present disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the mobile load balancing method provided in various optional implementations of the above-described embodiments.

[0012] In the technical solutions provided in some embodiments of the present disclosure, the impact of network slicing on mobile load balancing is taken into consideration. Based on the traditional mobile load balancing mechanism, a mobile load balancing method for 5G RAN slicing is proposed, which provides a solution for mobile load balancing under the network slicing architecture. By performing slice-level load control, the huge challenges faced by RAN slice mobile load balancing can be solved. It can effectively reduce the number of dissatisfied users, achieve higher system satisfaction, ensure the SLA contract rate of the slice, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The flowchart of the mobile load balancing method according to an embodiment of the present disclosure is schematically shown.

[0014] Figure 2 A structural diagram of a network slicing-oriented RAN model according to an embodiment of the present disclosure is schematically shown.

[0015] Figure 3 The present invention schematically shows a structural diagram of a mobile load balancing decision module for 5G RAN slicing according to an embodiment of the present disclosure.

[0016] Figure 4 A schematic diagram of the process of mobile load balancing decision for RAN slices according to an embodiment of the present disclosure is shown.

[0017] Figure 5 The diagram schematically shows a DRL principle framework diagram according to an embodiment of the present disclosure.

[0018] Figure 6 The block diagram schematically shows a mobile load balancing device according to an embodiment of the present disclosure.

[0019] Figure 7 A schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0021] The features, structures or characteristics described in the present disclosure may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0022] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all content and steps, nor must they be executed in the order described. For example, some steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0023] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the terms "comprising", "including" and "having" are used to express open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0024] The mobile load balancing method provided in each embodiment of the present disclosure can be executed by any electronic device, which can be a server, a user equipment UE, or can be implemented through interaction between a server and a user equipment.

[0025] The server in the embodiments of the present disclosure may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. UE may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited thereto. UE and server may be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure.

[0026] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0027] Figure 1A flowchart of a mobile load balancing method according to an embodiment of the present disclosure is schematically shown. The method provided by the embodiment of the present disclosure can be applied to a mobile network architecture based on network slicing, wherein the mobile network architecture may include N network slices, J remote radio frequency units (RRUs), and U users, where N, J, and U are all positive integers greater than or equal to 1.

[0028] like Figure 1 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0029] In step S110, the satisfaction of each network slice is obtained.

[0030] In an exemplary embodiment, the N network slices may include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N, and the U users may include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U.

[0031] Among them, obtaining the satisfaction of each network slice may include: obtaining the achievable data rate of user u and the minimum service rate of user u; obtaining the satisfaction of user u with the service level agreement SLA requirements of network slice n based on the achievable data rate of user u and its minimum service data rate; determining user u whose satisfaction with the SLA requirements of network slice n is greater than 0.5 as a user who meets the slice SLA requirements; obtaining the number of users who meet the slice SLA requirements and the total number of users who access the network through slice n; obtaining the satisfaction of network slice n based on the number of users who meet the slice SLA requirements and the total number of users who access the network through slice n.

[0032] In an exemplary embodiment, the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J.

[0033] Obtaining the achievable data rate of user u may include: obtaining a received signal-to-interference-plus-noise ratio (SINR) received by user u from RRU j; calculating a corresponding spectrum efficiency based on the SINR received by user u from RRU j; obtaining corresponding wireless resources allocated by RRU j to user u; and obtaining the achievable data rate of user u based on the spectrum efficiency and the corresponding wireless resources allocated by RRU j to user u.

[0034] In step S120, the system satisfaction is obtained based on the satisfaction of each network slice.

[0035] In an exemplary embodiment, obtaining the system satisfaction according to the satisfaction of each network slice may include: taking the minimum value of the satisfaction of each network slice as the system satisfaction.

[0036] In step S130, if the system satisfaction does not meet the requirements, a slice-level load control strategy is implemented to adjust the proportion of wireless resources allocated to each RRU to the network slice.

[0037] In an exemplary embodiment, the N network slices may include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N, the J RRUs may include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J, and the U users may include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U. Implementing a slice-level load control strategy to adjust the proportion of radio resources allocated to each RRU to the network slice may include: obtaining the load of RRU j and the load of network slice n on RRU j; when it is determined based on the load of RRU j that RRU j is not overloaded and the load of network slice n on RRU j is less than a utilization threshold, reducing the proportion of radio resources allocated to network slice n by RRU j; and when it is determined based on the load of RRU j that RRU j is not overloaded and the load of network slice n on RRU j is greater than 1, increasing the proportion of radio resources allocated to network slice n by RRU j.

[0038] In an exemplary embodiment, the N network slices may include a network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N; the J RRUs may include an RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J; and the U users may include a user u, where u is a positive integer greater than or equal to 1 and less than or equal to U. Implementing a slice-level load control strategy to adjust the proportion of radio resources allocated to the network slice by each RRU may include: obtaining a current state of the network slice n, where the current state includes a satisfaction level of the network slice n, a proportion of radio resources allocated to the network slice n by RRU j, and a load of the network slice n on RRU j; inputting the current state of the network slice n into a deep reinforcement learning model to select a load control action for adjusting the proportion of radio resources allocated to the network slice n by RRU j based on the current state of the network slice n; updating the proportion of radio resources allocated to the network slice n by RRU j according to the selected load control action to obtain a next state of the network slice n; and calculating a first reward function based on the next state of the network slice n.

[0039] In an exemplary embodiment, calculating the first reward function based on the next state of the network slice n may include: obtaining the system satisfaction after implementing the slice-level load control strategy based on the updated proportion of wireless resources allocated to the network slice n by the RRU j; obtaining the service rate benefit and the first balancing overhead based on the updated proportion of wireless resources allocated to the network slice n by the RRU j; obtaining the first user utility based on the service rate benefit and the first balancing overhead; when the first constraint, the second constraint and the third constraint are met, determining the first reward function based on the system satisfaction after implementing the slice-level load control strategy and the first user utility.

[0040] In an exemplary embodiment, the method may further include: obtaining system satisfaction after implementing the slice-level load control strategy; if the system satisfaction after implementing the slice-level load control strategy still does not meet the requirements, implementing a cell-level load balancing strategy, the cell-level load balancing strategy including a first cell-level load balancing strategy, a second cell-level load balancing strategy and a third cell-level load balancing strategy.

[0041] Among them, the first cell-level load balancing strategy can be user switching between RRUs covered by the same slice; the second cell-level load balancing strategy can be switching between different slices and RRUs at the same time; the third cell-level load balancing strategy can be deploying new slices in lightly loaded RRUs.

[0042] The mobile load balancing method provided in the embodiments of the present disclosure takes into account the impact of network slicing on mobile load balancing. Based on the traditional mobile load balancing mechanism, a mobile load balancing method for 5G RAN slicing is proposed, which provides a solution for mobile load balancing under the network slicing architecture. By performing slice-level load control, the huge challenges faced by RAN slicing mobile load balancing can be solved. It can effectively reduce the number of dissatisfied users, achieve higher system satisfaction, ensure the SLA contract rate of the slice, and improve the user experience.

[0043] The following will be combined Figures 2 to 5 The method provided in the embodiment of the present disclosure is illustrated by way of example, but the present disclosure is not limited thereto.

[0044] Compared to traditional cellular networks, the introduction of network slicing introduces some complexity to mobile load balancing. A network slice can be considered an independent virtual network to provide corresponding services, but users of the same mobile network operator (MNO) will access different slices based on their SLA requirements. This poses the following challenges to the research of mobile load balancing in RAN slices:

[0045] (1) Changes in network structure. Specifically, under the RAN slicing architecture, user equipment (UE, corresponding to user equipment in the embodiment of the present disclosure) is associated with a network slice (also referred to as a slice) via a specific radio remote unit (RRU), thereby forming a three-layer association structure of UE-RRU-network slice, for example Figure 2 Therefore, when MLB occurs, the traditional MLB mechanism that simply considers Reference Signal Received Power (RSRP) and cell resource status is no longer applicable. Mobile load balancing for RAN slices also needs to consider cell load, slice load, slice service type, and slice SLA to improve user experience.

[0046] (2) Changes in load information granularity. In the 3GPP (3rd Generation Partnership Project) standardization work related to MLB, some slicing-related content has been introduced. It is clear that the impact of RAN slicing should be considered in the MLB enhancement mechanism. At the same time, it is pointed out that the data analysis function of network slicing should be fully utilized to perform slice-level load control.

[0047] Therefore, in the 5G system, mobile load balancing based on the RAN slicing architecture needs to consider not only cell-level load balancing, but also slice-level load control.

[0048] Taking into account the complexity brought to mobile load balancing by the introduction of network slicing technology, the purpose of the embodiments of the present disclosure is to propose a mobile load balancing method for 5G RAN slices, whose main functions include: providing a mobile load balancing decision module for 5GRAN slices, which can further include slice-level load control achieved by adjusting the proportion of wireless resources allocated to slices by RRUs, and cell-level load balancing achieved based on user switching. At the same time, by defining a system satisfaction index to measure the contract status of the slice SLA, slice-level load control and cell-level load balancing are performed by monitoring the system satisfaction decision. In order to improve system satisfaction and consider maximizing the system utility of mobile load balancing, a method for jointly optimizing system satisfaction and user utility function is proposed in slice-level load control and cell-level load balancing, respectively, to achieve the optimal wireless resource proportion allocation in slice-level load control and user switching in cell-level load balancing.

[0049] In the mobile load balancing process, ensuring user SLA requirements and achieving load balancing across the entire network are key criteria for verifying the MLB algorithm. MLB also affects radio resource allocation and slice deployment across the entire network. Therefore, the mobile load balancing decision module for the RAN slicing architecture must consider not only cell-level mobile load balancing but also slice-level load control.

[0050] The mobile load balancing method proposed in the embodiment of the present disclosure is as follows: Figure 3 As shown, a mobile load balancing decision module 300 for 5G RAN slicing is provided, which includes a system satisfaction monitoring module 310, a slice-level load control module 320 and a cell-level load balancing module 330.

[0051] First, we define system satisfaction indicators to measure the satisfaction of slice SLA.

[0052] Secondly, the system satisfaction monitoring module 310 monitors the system satisfaction. When the system satisfaction does not meet the requirements (for example, it is less than a preset threshold value, the size of the threshold value can be set according to the actual application scenario, for example, the threshold value range can be [0.9, 1], but the present disclosure is not limited to this. In the following examples, the threshold value of 0.9 is used as an example, that is, the system satisfaction is greater than or equal to 0.9 and the system satisfaction is considered to meet the requirements, and the system satisfaction is less than 0.9 and the system satisfaction is considered to not meet the requirements), it decides to perform slice-level load control, that is, adjust the proportion of wireless resources allocated to the slice by each RRU according to the real-time wireless resource demand of the slice, so as to balance the wireless resource utilization of each slice and the satisfaction of the slice SLA, thereby achieving the purpose of improving system satisfaction.

[0053] If, after slice-level load control, the system satisfaction monitoring module 310 determines that the system satisfaction is effectively improved and meets the corresponding requirements (for example, greater than or equal to the preset threshold), it will continue to monitor the system satisfaction; otherwise, it will decide to perform cell-level load balancing, that is, the edge users of the overloaded RRU will be switched by selecting the target RRU and slice to prevent local network overload, ensure the performance of the slice and the user's SLA requirements, and improve the overall system satisfaction.

[0054] The method provided in the embodiment of the present disclosure includes the following main steps: Figure 4 Next, a detailed introduction will be given.

[0055] Unlike traditional networks, the embodiments of the present disclosure can provide different balancing decisions, such as adjusting only the radio resources of a slice, performing user handover between RRUs, performing handover of both slices and RRUs, or even applying for deployment of new network slices on RRUs. Having different types of mobile load balancing decision modules may result in different balancing costs. Therefore, the embodiments of the present disclosure define a mobile load balancing decision module under a RAN slice architecture, which includes both cell-level load balancing and slice-level load control:

[0056] (1) Slice-level load control strategy: M NS , that is, adjusting the proportion of radio resources allocated to the slice by RRU.

[0057] (2) Cell-level load balancing may further include:

[0058] M BS : Inter-RRU handover, that is, user handover between RRUs under the same slice coverage, can be called the first cell-level load balancing strategy;

[0059] M NS-BS : Both slices and RRUs are switched, that is, switching between different slices (slices of the same type, slice SLA needs to be considered) and RRUs is performed simultaneously. This can be called the second cell-level load balancing strategy;

[0060] Among them, slices of the same type refer to slices that provide the same service type. Assuming that user u is switched from slice n to slice n' (n' is a positive integer greater than or equal to 1 and less than or equal to N), both slice n and slice n' can provide the service type required by user u, and the switched slice n' can meet the requirements of the slice SLA.

[0061] M New : Deploying corresponding slices in lightly loaded RRUs (this situation is a special switching under the RAN slicing architecture) can be called the third cell-level load balancing strategy.

[0062] In the embodiment of the present disclosure, in order to leave a certain margin, the load ρ j An RRU with a load greater than 0.95 is defined as an overloaded RRU, and an RRU with a load less than or equal to the set load threshold (the load threshold can range from [0.5, 0.6]) is determined to be a lightly loaded RRU. New slices can be deployed on the lightly loaded RRU.

[0063] When performing mobile load balancing for RAN slices, it is necessary to consider the system overhead generated by mobile load balancing (for example, signaling interaction and processing delay required for radio resource adjustment and handover). Different balancing decisions result in different overheads. Therefore, the embodiments of the present disclosure optimize the network through slice-level load control and cell-level load balancing to ensure slice performance and user SLA requirements while reducing balancing overhead.

[0064] In step S410, system satisfaction monitoring is performed.

[0065] First, a system satisfaction index is defined to measure the satisfaction of the slice SLA.

[0066] Consider a mobile network architecture based on network slicing, which consists of multiple end-to-end network slices, RRUs, and users (corresponding to UEs). These slices share physical resources in the core network (CN) and RAN. Each slice has different network functional modules, such as mobility management, access control, and security, to provide secure and differentiated services to UEs.

[0067] For example, consider Figure 2 The multiple RRUs shown ( Figure 2 Four RRUs are used as an example for illustration, but the embodiment of the present disclosure does not limit the number of RRUs) and multiple slices ( Figure 2 In the figure, slice 1, slice 2 and slice 3 are used as examples for illustration, but the embodiment of the present disclosure does not limit the number of slices) of the RAN model.

[0068] make and are the sets of RRUs, RAN slices, and users respectively, where J is a positive integer greater than or equal to 1, N is a positive integer greater than or equal to 1, and U is a positive integer greater than or equal to 1.

[0069] Assume that N RAN slices are deployed on demand in a cellular network consisting of J RRUs. Multiple RAN slices are deployed on the same physical infrastructure, sharing these physical resources and allocating radio resources based on the slices' radio resource requirements.

[0070] Among them, one or more RAN slices can be deployed on an RRU, and one RAN slice can also cover one or more RRUs at the same time.

[0071] In the embodiment of the present disclosure, any user u in the set of users is used as an example for illustration, where u is a positive integer greater than or equal to 1 and less than or equal to N. The minimum service rate of user u is used To describe its SLA requirements.

[0072] make is the set of all service types, and m is a positive integer greater than or equal to 1. is the service type of user u. For example, when the service type T m When the user's SLA requirements can be met,

[0073] Among them, service types may include eMBB (Enhanced Mobile Broadband), mMTC (massive Machine Type of Communication), uRLLC (ultra-reliable low latency communications), etc., but the present disclosure is not limited to this, and further subdivision can be performed according to different business indicators.

[0074] In the embodiment of the present disclosure, let c j,n represents the ratio of radio resources allocated by RRU j to slice n, where j is a positive integer greater than or equal to 1 and less than or equal to J, and n is a positive integer greater than or equal to 1 and less than or equal to N, then:

[0075]

[0076] When RRU j is not within the coverage of slice n, c j,n =0.

[0077] In the embodiment of the present disclosure, two elements (T n , C n ) to determine a specific slice, for example: slice n, where T n is the type of service that slice n can provide, C n It is a vector representing the proportion of radio resources allocated to slice n from all RRUs.

[0078] Among them, users can only access the slice n through the RRU within the coverage area of the slice. Figure 2 In the example, for example, user u can access slice 1 only via RRU 1 and RRU 2.

[0079] In the embodiment of the present disclosure, a connection indicator variable is defined When user u accesses the network through slice n of RRU j, otherwise A user can only access a slice through one RRU, which satisfies the following formula:

[0080]

[0081] Assume that the signal strength between each user and the surrounding RRUs can be known in advance through detection and transmitted to the connected RRU. is the path loss between RRU j and user u, is the channel gain between RRU j and user u, is the power allocated by RRU j to user u, σ 2 is the noise power. Then, in a certain state, the received signal-to-interference-noise ratio of user u to RRU j can be defined as:

[0082]

[0083] According to Shannon's formula and the above formula (3), the corresponding spectrum efficiency can be calculated As shown in the following formula (4):

[0084]

[0085] if Then RRU j will allocate corresponding wireless resources to user u, which is defined as (used to indicate the actual radio resources allocated by RRU j to user u).

[0086] According to the minimum service rate required by user u The minimum radio resources allocated to user u in RRU j that meet its rate requirements can be calculated for:

[0087]

[0088] In the above formula (5), B is the bandwidth of the subcarrier.

[0089] In the embodiment of the present disclosure, it is assumed that each RRU has the same total amount of wireless resources W. total , that is, the maximum value of wireless resources that can be provided to users, then:

[0090]

[0091] The radio resource utilization of an RRU can be calculated by the ratio of the radio resources occupied by users to the total radio resources of the RRU, that is, the radio resource utilization of RRU j Ω j It can be expressed as:

[0092]

[0093] Among them, the radio resource utilization rate of RRU j is Ω j It can also be expressed as the load ρ of RRU j j .

[0094] In the embodiment of the present disclosure, in order to measure the system satisfaction, let Sat u The satisfaction of user u (in the embodiment of the present disclosure, any user corresponds to a user equipment UE) with the slice SLA requirement (also referred to as user satisfaction) is calculated using the Sigmoid function to calculate the user u's satisfaction with the rate:

[0095]

[0096] Among them, in the above formula (8), represents the Sigmoid function, Γ u is the data rate achievable by user u (also called the achievable data rate, abbreviated as achievable data rate), Γ u is the data rate that user u can achieve based on the wireless resources allocated to him. Specifically, That is, the data rate Γ that user u can achieve u = spectrum efficiency and the actual radio resources allocated by RRU j to user u The product of is the minimum service data rate required by user u. If Sat u That is, the satisfaction value of user u with the slice SLA requirement is greater than 0.5, which means that the rate requirement of user u has been met; otherwise, it means that user u is not satisfied with the provided service.

[0097] Based on the above user satisfaction, the satisfaction λ of slice n can be calculated n (which can be called slice satisfaction), that is:

[0098]

[0099] In the above formula (9), U n is the total number of users accessing the network through slice n. The number of users meeting the slice SLA requirement is calculated for each user u accessing the network through slice n. The satisfaction Sat of each user u with the slice SLA requirement is calculated according to formula (8): u , Sat uUsers with a SLA greater than 0.5 are considered to meet the slice SLA requirements. The cumulative sum of users who meet the slice SLA requirements is used to obtain the number of users who meet the slice SLA requirements. Users with a SLA less than or equal to 0.5 are considered to not meet the slice SLA requirements. The cumulative sum of users who do not meet the slice SLA requirements is used to obtain the number of users who do not meet the slice SLA requirements. For example, assuming that the total number of users accessing the network through slice n is 100, and 90 of them have a SLA of u is greater than 0.5, then the corresponding λ n =90 / 100=0.9.

[0100] Considering the worst result of the slice on the system satisfaction, the system satisfaction λ is defined as:

[0101] λ=min{λ n} (10)

[0102] The above formula (10) means that the satisfaction λ1, λ2, ...λ of slice 1 to slice N are calculated respectively by formula (9) N , then the satisfaction from slice 1 to slice N is λ1, λ2, ...λ N The minimum value is taken as the system satisfaction.

[0103] In step S420, if the system satisfaction is monitored to be less than a preset threshold (for example, 0.9), step S430 is executed, that is, the system satisfaction monitoring module 310 continuously monitors the system satisfaction. When it is monitored that the system satisfaction does not meet the requirements, it decides to perform slice-level load control, or even subsequent cell-level load balancing; if the system satisfaction is greater than or equal to the preset threshold, it jumps back to the above step S410 to continue monitoring the system satisfaction.

[0104] In step S430, slice-level load control is performed.

[0105] By monitoring the system satisfaction, when the system satisfaction does not meet the requirements, the system satisfaction monitoring module 310 decides to perform slice-level load control, that is, adjusting the proportion of wireless resources allocated to the slice by each RRU according to the real-time wireless resource demand of the slice, so as to balance the wireless resource utilization of each slice and the satisfaction of the slice SLA, thereby achieving the purpose of improving system satisfaction.

[0106] The decision taken in slice-level load control is M NS : That is, adjust the proportion of wireless resources c allocated by RRU j to slice n j,n .

[0107] When the radio resource ratio c allocated to slice n by RRU j j,n ≠0, use ρ j,nrepresents the load of slice n on RRU j and is defined as:

[0108]

[0109] The above formula (11) is expressed as the ratio of the total radio resources required for the user to access the network through slice n of RRU j to the total radio resources allocated by RRUj to slice n.

[0110] Among them, when ρ j,n When ρ<1, it means that there are still some radio resources allocated by RRU j to slice n; when ρ j,n =1, it means that the radio resources allocated by RRU j to slice n can just meet the SLA requirements of the connected users; when ρ j,n >1, indicating that the wireless resources allocated by RRUj to slice n are insufficient, and the SLA requirements of some users will not be met.

[0111] Therefore, the load of the entire slice can be measured by the number of users who meet the SLA requirements, that is, the slice satisfaction can be calculated by the above formula (9).

[0112] In the method provided in the embodiment of the present disclosure, it is assumed that the rate required by the UE is fixed, that is, it is assumed that the minimum service data rate required by the user u is If is fixed, it is relatively simple to evaluate whether the user's SLA requirements are met. When judging whether the user's SLA requirements are met, the data rate that the user can achieve can be used, for example, the data rate Γ that user u can achieve u , by calculating the above formula (8), we can calculate whether the SLA requirement of user u is met without evaluating the specific data rate.

[0113] For a certain RRU j that is not overloaded (e.g., the load of RRU j is ρ j Less than or equal to 0.95), but the wireless resources allocated to slice n are insufficient, that is, ρ j,n > 1, slice n needs to obtain additional radio resources from RRU j It can be expressed as:

[0114]

[0115] RRU j dynamically updates the radio resource ratio c of each slice based on its own radio resource usage j，n First, analyze the wireless resource usage of each slice from each RRU, that is, calculate the load ρ of slice n on RRU j j，n , determine which RRU has insufficient radio resources allocated to the slice and needs to adjust the radio resource ratio c allocated to the slice j,n .

[0116] a. In the embodiment of the present disclosure, the usage of wireless resources in the RRU can be analyzed according to the above formula (7), that is, the wireless resource utilization rate Ω of RRU j is calculated. j or load ρ j , if the radio resources of RRU j are still available, for example, the load of RRU j is ρ j If it is less than or equal to 0.95, the proportion of radio resources allocated to the slice is adjusted first;

[0117] b. Then, the wireless resource usage of the slices covered by the cell can be analyzed according to the above formula (11), that is, the load ρ of slice n on RRU j can be calculated j,n , determine ρ j,n The slices with a utilization rate less than the utilization threshold (the range is [0.5, 0.7]) are slices with low radio resource utilization. Slices with low radio resource utilization are scaled down, that is, the proportion of radio resources allocated by RRU j to slices with low radio resource utilization is reduced. The specific reduction in the proportion of radio resources can be determined based on ρ j,n For example, by reducing the proportion of wireless resources allocated to the corresponding slice so that its ρ j,n Increased to 0.9; for ρ j,n > 1, RRU j increases the proportion of wireless resources allocated to the slice. The specific increase in the proportion of wireless resources can be determined based on ρ j,n For example, by increasing the proportion of wireless resources allocated to the corresponding slice so that its ρ j,n Reduced to 0.9.

[0118] For mobile load balancing for 5G RAN slicing, within a decision cycle, the utility obtained by user u when taking the corresponding action (called the first system utility) includes the service rate obtained by taking the balancing decision and the overhead caused by taking the balancing decision (called the first balancing overhead). ). The service rate obtained by user u after taking the corresponding balancing action is related to the system state, let To represent the service rate obtained after user u takes action in a certain decision cycle (Γ above u Indicates the actual radio resources allocated to user u at RRU j before user u takes action in a certain decision cycle Under the condition of , the data rate that user u can achieve), then the service rate benefit g u for:

[0119]

[0120] Wherein, in the above formula (16), δ is the service rate unit price.

[0121] At the same time, taking corresponding actions will bring about a certain amount of balancing overhead, which is called the first balancing overhead based on the analysis of slice-level load control. It can be defined as:

[0122]

[0123] Among them, β NS To adopt the equilibrium strategy M NS The unit price of expenses incurred, To select the equilibrium strategy M NS The previous connection indicator variable, To select the equilibrium strategy M NS The connection indicator variable after C n Indicates that the equilibrium strategy M is adopted NS The vector consisting of the ratio of wireless resources obtained from all RRUs in the first slice n, C′ n Indicates that the equilibrium strategy M is adopted NS The vector of the proportion of wireless resources obtained from all RRUs after the slice n. The above formula (17) means that if user u adopts the balanced strategy M NS Both before and after access the network through RRU j within the coverage of slice n, and user u adopts the balancing strategy M NS If the ratio of wireless resources allocated to slice n by all RRUs before and after does not change, the first balancing overhead is 0, that is, there is no balancing overhead; if user u adopts the balancing strategy M NS Both before and after access the network through RRU j within the coverage of slice n, and user u adopts the balancing strategy M NS The ratio of radio resources allocated to slice n by all RRUs before and after changes, so the first balanced overhead is

[0124] According to the above formulas (16) and (17), the utility function of user u can be obtained (which can be called the first user utility or the first system utility) is:

[0125]

[0126] After monitoring the decline in system satisfaction, slice-level load control is first performed, that is, by adjusting the proportion of wireless resources allocated to the slice by each RRU c j,n Therefore, for slice-level load control, the system satisfaction and the first user utility are jointly optimized, which can be expressed as:

[0127]

[0128] st(1),(2)

[0129]

[0130] In the above formula, c represents the proportion of radio resources allocated by RRU j to slice n. j,n ; Constraints (1) The sum of the proportions of radio resources allocated by each RRU to each slice cannot exceed the total amount of radio resources of the RRU (called the first constraint); constraint (2) It means that a user can only access a certain slice through one RRU (called the second constraint); constraint (20) means that each slice must meet the SLA requirements of all users associated with it (called the third constraint).

[0131] In order to solve the above optimization problem, the embodiment of the present disclosure solves the optimization problem through a deep reinforcement learning algorithm. Deep reinforcement learning (DRL) is a subfield of machine learning that combines reinforcement learning and deep learning. By integrating deep learning into the solution, DRL allows the agent to make decisions based on unstructured input data without manually designing the state space. The combination of deep learning and reinforcement learning can complement each other, enabling DRL to innovatively solve perception decision-making problems in complex models. Its principles are as follows: Figure 5 shown.

[0132] Deep reinforcement learning (DL) is a machine learning method based on data representation and learning. Reinforcement learning (RL) explores an unknown environment while building a model and learning an optimal strategy. Deep reinforcement learning (DRL) combines the perception capabilities of deep learning (DL) with the decision-making capabilities of reinforcement learning (RL). It can directly control based on input information and is an AI method that is closer to human thinking.

[0133] The principle of deep reinforcement learning is this: Deep Q-networks address fundamental instability issues in function approximation used in reinforcement learning (RL) by leveraging two techniques: experience replay and target networks. Experience replay enables RL agents to sample and train from previously observed data offline. This not only significantly reduces the number of interactions required with the environment but also allows for sampling from a batch of experiences, reducing variance in learning updates. Furthermore, by uniformly sampling from a large memory, temporal dependencies that can adversely affect RL algorithms are broken. Finally, from a practical perspective, batches of data can be efficiently processed in parallel using hardware, thereby increasing throughput.

[0134] Q learning mainly uses the Bellman equation to iteratively solve the Q function. Assuming the initial state is s, then repeatedly perform the following operations: select and execute an action a, retain the reward r and the new state s', and calculate the action value function Q π [s,a]=Q π [s,a]+α(r+γmax a' Q π [s',a']-Q π [s, a]), α and γ are both numbers greater than or equal to 0 and less than or equal to 1, let s = s', until the end.

[0135] Among them, the loss function can be r+γmax in the loss function a' Q π [s',a'] is the target Q value, Q π [s,a] is used as the predicted Q value, that is, the loss function is performed based on the target Q value and the predicted Q value.

[0136] The Q value update process is as follows:

[0137] 1) Use the current state s to calculate the Q value of all actions through the neural network, that is, Q π [s,a];

[0138] 2) Use the next state s' to calculate Q through the neural network π [s',a'], and get the maximum value max a' Q π [s',a'];

[0139] 3) Set the target Q value of action a to r+γmax a' Q π [s',a'], for other actions, set the target Q value to the Q value returned in step 1 to make the error 0;

[0140] 4) Use backpropagation to update the Q network weights.

[0141] Commonly used Q-learning requires a Q-table to store Q values. However, when the state space is large, the Q-table required for maintenance becomes very large, requiring significant storage and space consumption. Consequently, it is inefficient for problems with large state spaces. The disclosed embodiments employ a Deep Q Network (DQN) to solve the aforementioned optimization problem. By replacing tables with neural networks, DQN is more suitable for problems with large state spaces.

[0142] Slice-level load control mainly improves system satisfaction by updating the proportion of radio resources allocated to each RRU in real time. After monitoring a decrease in system satisfaction, decisions on slice-level load control are made based on the slice load and the radio resource utilization of each RRU.

[0143] At each decision moment, due to the changing conditions of the slice, the proportion of radio resources allocated to the slice by each RRU will be adjusted. j,n The Markov decision process is formulated by defining state, action, reward and next state. The definitions of each component are expressed as follows.

[0144] 1) System State: Indicates the network status ( is the set of all states). The state space is defined as the slice satisfaction, the proportion of radio resources allocated to the slice by each RRU, and the load of the slice, that is, the current state s of slice n n Defined as a tuple s n ={λ n ,c 1,n ,...,c j,n ,ρ 1，n ,...,ρ j,n}, where λ n is the satisfaction of slice n, which can be calculated using the above formula (9), c j,n is the resource ratio allocated to slice n by RRU j, i.e., the radio resource ratio, ρ j,n represents the load of slice n on RRU j, which can be calculated using the above formula (11).

[0145] 2) Action: Based on the current state s of slice n n An action is selected to learn the optimal radio resource configuration for each slice n.

[0146] The action space of a slice is defined as the ratio of radio resources allocated to the slice by the RRU. For example, assuming that the discrete action space of slice n is defined as a n={-0.6,-0.4,-0.2,0,0.2,0.4,0.6}. It should be understood that the values here are for illustration only and are not intended to be limiting. If the selected operation / action / action is negative, it means that the slice's radio resources should be reduced by that percentage. For example, if the selected operation a for slice n is -0.6, then the radio resources allocated to slice n should be reduced by 60%. Conversely, if the selected operation is positive, it means that the slice's radio resources should be increased by the corresponding percentage. For example, if the selected operation a for slice n is 0.6, then the radio resources allocated to slice n should be increased by 60%. A selected value of zero indicates that the slice's current radio resource configuration remains unchanged.

[0147] The operation selected on this slice is used to update the slice radio resource configuration at the RRU level. The mapping from the overall radio resource allocation in the RRU to the slice resources depends on the weight of the RRU for a specific slice. That is, in the embodiment of the present disclosure, different slices can be assigned corresponding priorities, and different priorities correspond to different slice weights. When the RRU reconfigures the radio resources allocated to each slice, it can give priority to the slice with the highest priority.

[0148] 3) Reward: Deep reinforcement learning aims to interact with the wireless environment by trying possible actions and reinforcing the trending ones. The objective of radio resource allocation for RAN slices should consider multiple variables, and the weighted sum of these variables can be regarded as the corresponding reward.

[0149] In reality, the radio resources of each RRU and slice are limited, but the user may be in the coverage of multiple RAN slices at the same time. Considering that the goal of the embodiment of the present disclosure is to maximize the sum of system utilities, when the constraints (1), (2) and (20) are met, the first reward function r1 is set to n Defined as selecting equilibrium strategy M NS The product of the system satisfaction after and the first system utility, otherwise, it is defined as a negative feedback, that is:

[0150]

[0151] 4) Next State: After receiving a reward for the selected action, the system enters the next state, where the proportion of radio resources allocated to the slice by the RRU, the slice satisfaction, and the slice load are modified. The next state parameters are affected by the action taken on the slice. After the parameters are updated, they are stored in memory and used to predict possible actions during training.

[0152] The radio resources of the slice are updated according to the selected action. When the selected action is positive, the radio resources allocated to the slice are increased, and when the action is negative, the corresponding radio resources are reduced. When the action is zero, the allocated radio resources remain unchanged. j，n As the radio resource configuration of slice n on RRU j, the radio resource configuration of the slice is updated at the decision moment, and the calculation formula is:

[0153]

[0154] Among them, a is the action taken to allocate resources for slice n at the decision time, a∈a n , that is, a is the discrete action space a from slice n n = a value selected from {-0.6, -0.4, -0.2, 0, 0.2, 0.4, 0.6}, c j,n is the ratio of radio resources allocated to slice n by RRU j before executing action a, c′ j，n is the ratio of radio resources allocated to slice n by RRU j after executing action a. Slice-level radio resource configuration is implemented at each decision moment.

[0155] It should be noted that at each decision moment, these values are updated to update the slice-level radio resources. Finally, the reconfigured slice allocates radio resources to its users. The DQN-based slice-level load control algorithm can be summarized as follows.

[0156] Table 1 Slice-level load control algorithm based on DQN

[0157]

[0158] In step S440, after slice-level load control is performed, the system satisfaction is continuously monitored.

[0159] In step S450, if the monitored system satisfaction is still less than the preset threshold after slice-level load control, step S460 is executed to perform cell-level load balancing; otherwise, jump back to step S410 to continue monitoring the system satisfaction.

[0160] In step S460, cell-level load balancing is performed.

[0161] After slice-level load control, the system satisfaction monitoring module 310 determines whether the system satisfaction is effectively improved and meets the corresponding requirements. If the system satisfaction meets the requirements, the system satisfaction will continue to be monitored; otherwise, cell-level load balancing will be performed, that is, the edge users of the overloaded RRU will be switched by selecting the target RRU and slice to prevent local network overload, ensure the performance of the slice and the user's SLA requirements, and improve the overall system satisfaction.

[0162] In the embodiment of the present disclosure, the load ρ j An RRU j with a value greater than 0.95 is determined to be an overloaded RRU, but the present disclosure is not limited thereto. An edge user refers to the same user being within the coverage of multiple RRUs at the same time. For example, assuming that user u is within the coverage of both RRU j and RRU j', when RRU j is overloaded, cell-level load balancing is performed, and user u can be switched from RRU j to RRU j'. That is, RRU j' is the target RRU selected by edge user u of the overloaded RRU j. At the same time, one or more RAN slices are deployed on RRU j'. During the handover, a slice that can provide the service type required by user u is selected from the one or more RAN slices deployed on RRU j' as the target slice n' after the handover.

[0163] In the embodiment of the present disclosure, there are three types of cell-level load balancing strategies for cell-level load balancing:

[0164] M BS : Inter-RRU switching, that is, switching between RRUs under the same slice coverage;

[0165] M NS-BS : Both slices and RRUs are switched, that is, switching between different slices (slices of the same type, slice SLA needs to be considered) and RRUs is performed simultaneously;

[0166] M New : Deploy corresponding slices on the lightly loaded RRU (this situation is a special switching under the RAN slicing architecture).

[0167] Similar to the method of calculating slice-level load, the cell load can be defined by the radio resource utilization of RRU j, so the load of RRU j ρ j Defined as:

[0168]

[0169] That is, the load ρ of RRU j j is the ratio of the radio resources required by users in RRU j to the total amount of radio resources in the RRU.

[0170] Among them, when ρ j When ρ<1, the sum of the wireless resources required by all users connected to RRU j is less than the total amount of wireless resources that the access point can provide. At this time, RRU j is not overloaded (in order to leave a certain margin, in the example, ρ j <0.95 is judged as not overloaded); when ρ j=1, that is, when the sum of the wireless resources required by all users associated with RRU j is equal to the total amount of wireless resources that RRU j can provide, then RRU j is just fully loaded; and when ρ j When ρ > 1, that is, when the sum of the wireless resources required by all users connected to RRU j is greater than the total amount of wireless resources that the access point can provide, RRU j is overloaded. In order to leave a certain margin, in the example, ρ j ≥0.95 is considered as overload.

[0171] Based on the above analysis, the load that needs to be transferred by the overloaded RRU j is ρ z for:

[0172]

[0173] in, It is the overload threshold and can be set according to the actual scenario.

[0174] The load that each adjacent cell can accommodate ρ i接纳 for:

[0175]

[0176] In the above formula (25), it is assumed that the neighboring cell of the overloaded RRU j is RRU i. The load of the neighboring cell RRU i can be calculated using the above formula (23) as follows: If the load of RRU i in the adjacent cell is ρ i If the load is neither overloaded nor fully loaded, the load that the adjacent cell RRU i can accept from the overloaded RRU j is ρ i接纳 .

[0177] For example, suppose is 0.95, and assuming that the overloaded RRU j has two adjacent cells, the load of one adjacent cell is 0.7, then the load that the adjacent cell can accommodate is 0.95-0.7=0.25; the load of the other adjacent cell is 0.8, then the load that the other adjacent cell can accommodate is 0.95-0.8=0.15.

[0178] For cell-level load balancing, the load of adjacent cells and the slice coverage are analyzed by formula (23), and appropriate users and adjacent cells are selected for mobile load balancing. Once the trigger condition is met, that is, the overload threshold is exceeded The user selects the appropriate target RRU and slice that matches the service type for switching.

[0179] Similar to slice-level load control, within a decision cycle, the utility obtained by the user by adopting the corresponding strategy (called the second system utility) includes the service rate obtained by adopting the balancing strategy and the overhead caused by adopting the balancing strategy (called the second balancing overhead). For cell-level load balancing, the service rate benefit g u It can be calculated by the above formula (16).

[0180] At the same time, after user u adopts the corresponding balancing strategy, the RRU and slice associated with the user will change, which will bring a certain amount of second balancing overhead. Based on the analysis of the cell-level load balancing strategy, the second balancing overhead It can be defined as:

[0181]

[0182] In the above formula (26), ρ j' Indicates that the equilibrium strategy M is adopted BS 、M NS-BS , or M New The load of the subsequent RRU j', i.e. β BS , β NS-BS , β New Take the equilibrium strategy M BS 、M NS-BS 、M New The unit price of expenses incurred, To select the equilibrium strategy M BS 、M NS-BS , or M New The previous connection indicator variable, To select the equilibrium strategy M BS 、M NS-BS , or M New The subsequent connection indicator variable M BS The above formula (26) means that if user u adopts the equilibrium strategy M BS Then, an inter-RRU handover is performed, for example, switching from RRU j to RRU j', where j' is a positive integer greater than or equal to 1 and less than or equal to J. RRU j and RRU j' are both within the coverage of slice n, that is, user u adopts the balancing strategy M BS The slices connected before and after have not changed, and the second balancing cost generated at this time is If user u adopts the equilibrium strategy M NS-BS After that, both the slice and the RRU are switched, for example, from RRU j to RRU j', and RRU j' is within the coverage of slice n' (slice n' at this time belongs to the original slice set), that is, user u adopts the balancing strategy M NS-BSThe slices connected before and after also change, and the second balanced cost generated at this time is If user u adopts the equilibrium strategy M New Then the corresponding slice n′ is deployed on the lightly loaded RRU. The newly deployed slice n′ does not belong to the original slice set. And RRU j' is within the coverage of slice n', M New That is, user u adopts the equilibrium strategy M New The slices and RRUs connected before and after have changed. The second balancing overhead generated at this time is

[0183] According to the above formulas (16) and (26), the utility function of user u can be obtained (which can be called the second user utility function or the second system utility) is:

[0184]

[0185] For cell-level load balancing, the system satisfaction and the second user utility function are also combined to optimize mobile load balancing. The optimization problem is expressed as:

[0186]

[0187] st(2)(6)

[0188]

[0189] In the above formula (28), x represents the equilibrium strategy M adopted BS 、M NS-BS 、M New ; Constraints (6) Indicates that the sum of the resource proportions allocated to each slice by each RRU cannot exceed the total amount of wireless resources of the RRU; constraint (29) indicates that the load of each RRU cannot exceed the overload threshold

[0190] In the disclosed embodiments, deep reinforcement learning can also be used to solve the problem of cell-level mobile load balancing. Therefore, a DQN-based cell-level mobile load balancing algorithm is proposed. By sensing the surrounding environment, RRUs and slices are selected for switching to achieve load balancing and improve system capacity. Similarly, a Markov decision process is formulated by defining states, actions, rewards, and next states. The agent, state, action, and reward functions are defined as follows:

[0191] 1) System State: s∈S represents the network state (S is the set of all states). At a certain moment, the current state can be expressed as:

[0192] s=[ρ1,...,ρ j ,...,ρ J ,E1,...,E j ,...,E J ] (30)

[0193] Among them, ρ j is the load of RRU j, calculated by the above formula (23); E j is the proportion of edge users in cell j. Here, edge users are classified into corresponding serving SBSs (Small Base Stations) and neighboring SBSs according to their downlink SINRs (Signal to Interference plus Noise Ratio).

[0194] 2) Action: The action taken is defined as the edge user's selection of the target RRU and RAN slice. At a certain moment, the action taken by the edge user to access slice n through RRU j is a = (j, n). j and n represent the selected target RRU and RAN slice, respectively (they can also be expressed as j' and n'. Here, j and n are used to indicate that the target RRU and RAN slice have been selected, but the actual switching to the selected target RRU j' and target slice n' has not yet been made. Instead, the switching will be made after the reward function is calculated later).

[0195] 3) Reward Function: At a certain decision moment, the user acquisition rate is affected by the equilibrium strategy. Therefore, at a certain decision moment, the second reward function r2 is defined as the joint optimization of system satisfaction and the second user utility function, where the second user utility function includes the service rate obtained after the user takes the corresponding action and the equilibrium cost (i.e., the second equilibrium cost) caused by taking the corresponding action. Otherwise, it is defined as a negative feedback, that is:

[0196]

[0197] 4) Next State: After receiving a reward for the selected action, the system enters the next state, where the load conditions, slices, and system satisfaction of each RRU are modified. The next state parameters are affected by the action taken. After the parameters are updated, they are stored in an experience pool and used to predict possible actions during training.

[0198] Table 2 Cell-level load balancing algorithm based on DQN

[0199]

[0200] After the cell-level load balancing is completed, the process may return to the above step S410 , ie, the system satisfaction monitoring module 310 continues to monitor the system satisfaction.

[0201] With the introduction of network slicing, traditional mobile load balancing mechanisms are no longer applicable. The mobile load balancing method provided in the embodiments of this disclosure considers the impact of network slicing on mobile load balancing. Building on the traditional mobile load balancing mechanism, a mobile load balancing method for 5G RAN slices is proposed. This method provides a solution for mobile load balancing in a network slicing architecture, addressing the significant challenges faced by RAN slice mobile load balancing. A mobile load balancing decision module for 5G RAN slices is proposed, including system satisfaction monitoring, slice-level load control achieved by adjusting the proportion of radio resources allocated to slices by RRUs, and cell-level load balancing based on user handover. This innovative approach addresses the mobile load balancing problem in the RAN slicing architecture from two perspectives: slice-level load control and cell-level load balancing. This approach effectively reduces the number of dissatisfied users, achieves higher system satisfaction, and reduces balancing overhead (including primary and secondary balancing overhead). A system satisfaction index is also defined to measure user and slice SLA satisfaction. Furthermore, a deep reinforcement learning algorithm is used to implement slice-level load control and cell-level load balancing, respectively. Network optimization is performed to ensure slice performance and user SLA requirements, ensuring the SLA contract rate for slices while minimizing balancing overhead and improving user experience. The mobile load balancing method provided in the embodiments of the present disclosure proposes several different types of mobile load balancing decisions for mobile load balancing under the 5G RAN slicing architecture.

[0202] Figure 6 A block diagram of a mobile load balancing device according to an embodiment of the present disclosure is schematically shown. The device provided by the embodiment of the present disclosure can be applied to a mobile network architecture based on network slicing, wherein the mobile network architecture includes N network slices, J remote radio frequency units (RRUs), and U users, where N, J, and U are all positive integers greater than or equal to 1.

[0203] like Figure 6 As shown, the mobile load balancing device 600 provided in the embodiment of the present disclosure may include a slice satisfaction obtaining unit 610, a system satisfaction obtaining unit 620 and a slice-level load control unit 630.

[0204] In the disclosed embodiment, the slice satisfaction obtaining unit 610 may be configured to obtain the satisfaction of each network slice. The system satisfaction obtaining unit 620 may be configured to obtain the system satisfaction based on the satisfaction of each network slice. The slice-level load control unit 630 may be configured to implement a slice-level load control strategy to adjust the proportion of radio resources allocated to each RRU in the network slice if the system satisfaction does not meet the requirements.

[0205] In an exemplary embodiment, the N network slices may include a network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N, and the U users may include a user u, where u is a positive integer greater than or equal to 1 and less than or equal to U. The slice satisfaction obtaining unit 610 may include: a achievable data rate obtaining unit, which may be used to obtain the achievable data rate of user u and the minimum service rate of user u; a user satisfaction obtaining unit, which may be used to obtain the satisfaction of user u with the service level agreement (SLA) requirement of network slice n based on the achievable data rate of user u and its minimum service data rate; a slice SLA requirement satisfying user determination unit, which may be used to determine user u whose satisfaction with the SLA requirement of network slice n is greater than 0.5 as a user satisfying the slice SLA requirement; a user number determination unit, which may be used to obtain the number of users satisfying the slice SLA requirement and the total number of users accessing the network through slice n; and a slice satisfaction calculation unit, which may be used to obtain the satisfaction of network slice n based on the number of users satisfying the slice SLA requirement and the total number of users accessing the network through slice n.

[0206] Among them, the system satisfaction obtaining unit 620 can be used to take the minimum value of the satisfaction of each network slice as the system satisfaction.

[0207] In an exemplary embodiment, the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J. The achievable data rate obtaining unit may include: a received signal-to-interference-and-noise ratio obtaining unit, configured to obtain a received signal-to-interference-and-noise ratio received by user u from RRU j; a spectrum efficiency calculating unit, configured to calculate a corresponding spectrum efficiency based on the received signal-to-interference-and-noise ratio received by user u from RRU j; a radio resource obtaining unit, configured to obtain corresponding radio resources allocated by RRU j to user u; and a achievable data rate calculating unit, configured to obtain a achievable data rate of user u based on the spectrum efficiency and the corresponding radio resources allocated by RRU j to user u.

[0208] In an exemplary embodiment, the N network slices include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N; the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J; and the U users include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U. The slice-level load control unit 630 may include: a network slice load obtaining unit, which may be used to obtain the load of RRU j and the load of network slice n on RRU j; a radio resource ratio reducing unit, which may be used to reduce the ratio of radio resources allocated to network slice n by RRU j when it is determined based on the load of RRU j that RRU j is not overloaded and the load of network slice n on RRU j is less than a utilization threshold; and a radio resource ratio increasing unit, which may be used to increase the ratio of radio resources allocated to network slice n by RRU j when it is determined based on the load of RRU j that RRU j is not overloaded and the load of network slice n on RRU j is greater than 1.

[0209] In an exemplary embodiment, the N network slices include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N, the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J, and the U users include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U; wherein the slice-level load control unit 630 may include: a network slice current state acquisition unit, which can be used to obtain the current state of network slice n, wherein the current state includes the satisfaction of network slice n, the proportion of wireless resources allocated to network slice n by RRU j, and the load of network slice n on RRU j; a load control action selection unit, which can be used to input the current state of network slice n into a deep reinforcement learning model to select a load control action for adjusting the proportion of wireless resources allocated to network slice n by RRU j based on the current state of network slice n; a network slice next state acquisition unit, which can be used to update the proportion of wireless resources allocated to network slice n by RRU j according to the selected load control action to obtain the next state of network slice n; and a first reward function calculation unit, which can be used to calculate a first reward function based on the next state of network slice n.

[0210] In an exemplary embodiment, the first reward function calculation unit may include: a system satisfaction updating unit, which can be used to obtain the system satisfaction after implementing the slice-level load control strategy based on the updated proportion of wireless resources allocated to the network slice n by RRU j; a benefit-overhead calculation unit, which can be used to obtain the service rate benefit and the first balanced overhead based on the updated proportion of wireless resources allocated to the network slice n by RRU j; a first user utility obtaining unit, which can be used to obtain the first user utility based on the service rate benefit and the first balanced overhead; a first reward function determination unit, which can be used to determine the first reward function based on the system satisfaction after implementing the slice-level load control strategy and the first user utility when the first constraint, the second constraint and the third constraint are met.

[0211] In an exemplary embodiment, the mobile load balancing device 600 may also include: a slice-level load control system satisfaction acquisition unit, which can be used to obtain the system satisfaction after implementing the slice-level load control strategy; a cell-level load balancing unit, which can be used to implement the cell-level load balancing strategy if the system satisfaction after implementing the slice-level load control strategy still does not meet the requirements, and the cell-level load balancing strategy includes a first cell-level load balancing strategy, a second cell-level load balancing strategy and a third cell-level load balancing strategy.

[0212] Among them, the first cell-level load balancing strategy can be user switching between RRUs covered by the same slice; the second cell-level load balancing strategy can be switching between different slices and RRUs at the same time; the third cell-level load balancing strategy can be deploying new slices in lightly loaded RRUs.

[0213] For other contents of the mobile load balancing device in the embodiment of the present disclosure, reference may be made to the above embodiment.

[0214] It should be noted that although several units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0215] Reference below Figure 7 , which shows a structural schematic diagram of an electronic device suitable for implementing the embodiments of the present application. Figure 7 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0216] Reference Figure 7The electronic device provided by the embodiment of the present disclosure may include: a processor 701, a communication interface 702, a memory 703 and a communication bus 704.

[0217] The processor 701 , the communication interface 702 and the memory 703 communicate with each other via the communication bus 704 .

[0218] Optionally, the communication interface 702 may be an interface of a communication module, such as an interface of a GSM (Global System for Mobile Communications) module. The processor 701 is configured to execute programs. The memory 703 is configured to store programs. The programs may include computer programs that include computer operating instructions. The programs may include game client programs.

[0219] The processor 701 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0220] The memory 703 may include a high-speed RAM (random access memory) memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0221] Among them, the program can be specifically used to: obtain the satisfaction of each network slice; obtain the system satisfaction based on the satisfaction of each network slice; if the system satisfaction does not meet the requirements, implement the slice-level load control strategy to adjust the proportion of wireless resources allocated to each RRU to the network slice.

[0222] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above-described embodiments.

[0223] It should be understood that any number of elements in the drawings of the present disclosure is for illustration only and not for limitation, and any naming is for distinction only and does not have any limiting meaning.

[0224] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0225] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A mobile load balancing method, characterized in that: The method is applied to a mobile network architecture based on network slicing, wherein the mobile network architecture includes N network slices, J remote radio unit (RRU) units, and U users, where N, J, and U are all positive integers greater than or equal to 1; the N network slices include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N; the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J; and the U users include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U; wherein the method includes: Obtain satisfaction of each network slice; Obtain system satisfaction based on the satisfaction of each network slice; If the system satisfaction does not meet the requirements, a slice-level load control strategy is implemented to adjust the proportion of radio resources allocated to each RRU in the network slice; The slice-level load control strategy is implemented to adjust the proportion of radio resources allocated to each RRU in the network slice, including: Obtain a current state of network slice n, where the current state includes a satisfaction level of network slice n, a proportion of radio resources allocated by RRU j to network slice n, and a load of network slice n on RRU j; Input the current state of network slice n into the deep reinforcement learning model to select a load control action to adjust the proportion of radio resources allocated by RRU j to network slice n based on the current state of network slice n; Update the proportion of radio resources allocated by RRU j to network slice n according to the selected load control action to obtain the next state of network slice n; Calculate a first reward function based on the next state of network slice n; The first reward function is calculated according to the next state of the network slice n, including: The system satisfaction after implementing the slice-level load control strategy is obtained based on the updated proportion of wireless resources allocated to network slice n by RRU j; Obtain service rate benefit and first balanced overhead according to the updated proportion of radio resources allocated to network slice n by RRU j; Obtaining a first user utility according to the service rate benefit and the first balanced overhead; When the first constraint, the second constraint, and the third constraint are satisfied, determining the first reward function according to the system satisfaction after implementing the slice-level load control strategy and the first user utility; The first constraint condition is that the sum of the proportions of radio resources allocated by each RRU to each network slice cannot exceed the total amount of radio resources of the RRU; The second constraint condition is that a user can only access one network slice through one RRU; The third constraint is that each network slice must meet the SLA requirements of all users associated with it.

2. The method according to claim 1, characterized in that The N network slices include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N; the U users include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U; wherein obtaining the satisfaction of each network slice includes: Obtain the achievable data rate of user u and the minimum service rate of user u; Obtain user u's satisfaction with the service level agreement (SLA) requirements of network slice n based on user u's achievable data rate and its minimum service data rate. User u whose satisfaction with the SLA requirements of network slice n is greater than 0.5 is determined to be a user who meets the SLA requirements of the slice; Obtain the number of users who meet the slice SLA requirements and the total number of users who access the network through slice n; Obtain the satisfaction of network slice n based on the number of users who meet the slice SLA requirements and the total number of users accessing the network through slice n. The system satisfaction is obtained based on the satisfaction of each network slice, including: The minimum value of the satisfaction of each network slice is taken as the system satisfaction.

3. The method according to claim 2, characterized in that The J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J. Obtaining the achievable data rate of user u includes: Obtain the received signal-to-interference-and-noise ratio (SINR) of user u received from RRU j; Calculate the corresponding spectrum efficiency based on the received signal-to-interference-and-noise ratio of user u to RRU j; Obtain the corresponding radio resources allocated by RRU j to user u; The achievable data rate of user u is obtained according to the spectrum efficiency and the corresponding wireless resources allocated by RRU j to user u.

4. The method according to claim 1, wherein Also includes: Obtain system satisfaction after implementing slice-level load control strategy; If the system satisfaction after implementing the slice-level load control strategy still does not meet the requirements, the cell-level load balancing strategy is implemented, and the cell-level load balancing strategy includes a first cell-level load balancing strategy, a second cell-level load balancing strategy, and a third cell-level load balancing strategy; Among them, the first cell-level load balancing strategy is user switching between RRUs covered by the same slice; the second cell-level load balancing strategy is switching between different slices and RRUs at the same time; the third cell-level load balancing strategy is to deploy new slices in lightly loaded RRUs.

5. A mobile load balancing device, characterized in that: The device is applied to a mobile network architecture based on network slicing, wherein the mobile network architecture includes N network slices, J remote radio unit RRUs, and U users, where N, J, and U are all positive integers greater than or equal to 1; the N network slices include network slice n, where n is a positive integer greater than or equal to 1 and less than or equal to N; the J RRUs include RRU j, where j is a positive integer greater than or equal to 1 and less than or equal to J; and the U users include user u, where u is a positive integer greater than or equal to 1 and less than or equal to U; wherein the device includes: A slice satisfaction obtaining unit, used to obtain the satisfaction of each network slice; A system satisfaction obtaining unit, configured to obtain system satisfaction based on the satisfaction of each network slice; A slice-level load control unit, configured to implement a slice-level load control strategy to adjust the proportion of radio resources allocated to each RRU to the network slice if the system satisfaction does not meet the requirements; The slice-level load control unit includes: a network slice current state obtaining unit, configured to obtain a current state of network slice n, wherein the current state includes a satisfaction level of network slice n, a proportion of radio resources allocated by RRU j to network slice n, and a load of network slice n on RRU j; a load control action selection unit, configured to input the current state of the network slice n into the deep reinforcement learning model, so as to select a load control action for adjusting the proportion of radio resources allocated by RRU j to the network slice n based on the current state of the network slice n; a network slice next state obtaining unit, configured to update the proportion of radio resources allocated by RRU j to network slice n according to the selected load control action, so as to obtain the next state of network slice n; A first reward function calculation unit, configured to calculate a first reward function according to a next state of the network slice n; The first reward function calculation unit includes: A system satisfaction updating unit, configured to obtain the system satisfaction after implementing the slice-level load control strategy according to the updated proportion of radio resources allocated by RRU j to network slice n; A revenue and overhead calculation unit, configured to obtain a service rate revenue and a first balanced overhead according to the updated proportion of radio resources allocated by RRU j to network slice n; A first user utility obtaining unit is configured to obtain a first user utility based on the service rate benefit and the first balanced overhead; a first reward function determining unit is configured to determine the first reward function based on the system satisfaction after implementing the slice-level load control strategy and the first user utility when the first constraint condition, the second constraint condition, and the third constraint condition are satisfied; The first constraint condition is that the sum of the proportions of radio resources allocated by each RRU to each network slice cannot exceed the total amount of radio resources of the RRU; The second constraint condition is that a user can only access one network slice through one RRU; The third constraint is that each network slice must meet the SLA requirements of all users associated with it.

6. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

7. An electronic device comprising: at least one processor; A storage device configured to store at least one program, which, when executed by the at least one processor, enables the at least one processor to implement the method according to any one of claims 1 to 4.

Images