A parameter optimization method and system for cell switching

Abstract

The present invention relates to the field of wireless communication technology, and provides a parameter optimization method and system for cell switching, the method comprising: constructing a cell switching model of the wireless communication network according to the system architecture of the wireless communication network; constructing a cell load standard deviation function according to the cell switching model of the wireless communication network; constructing an optimization problem with the goal of minimizing the cell load standard deviation function, taking several cell switching parameters as optimization variables, and taking the parameter ranges of several cell switching parameters as constraints; and using a preset multi-agent reinforcement learning method to iteratively solve the optimization problem. The present invention takes minimizing the cell load standard deviation function as the optimization goal, and uses a multi-agent reinforcement learning method to iteratively solve the optimization problem, which can simultaneously perform a global optimal solution to the load of each cell, and can achieve global balance of the load of each cell after algorithm iteration.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a parameter optimization method and system for cell switching. Background Art

[0002] Since the first generation of mobile communication technology (1G), in order to solve the problems of continuity and stability of the connection between users and base stations during high-speed movement, as well as load balancing between cells (i.e., base station coverage areas), cell switching technology was born, which allows users to continuously switch from one cell to another during movement. In the current 5G era, the characteristics of 5G wireless cellular network communication include dense deployment of base stations, large-scale user access, high-speed movement, uneven distribution in time and space, etc. The good user experience depends more on frequent switching between cells, so the 5G network has higher requirements for the accuracy and efficiency of cell switching.

[0003] In order to implement cell switching, it is generally necessary to configure corresponding cell switching parameters so that a cell switching request can be made when the configured cell switching parameters are met. In the prior art, cell switching parameters are generally configured in the following three ways:

[0004] (1) Manual parameter adjustment based on engineer experience: This method refers to the operator's engineers setting a set of fixed and safe parameters for base stations based on their experience. This method has no unified standard, and the effect is highly dependent on the level of engineers. It cannot be dynamically adjusted according to changes in user traffic, making it difficult to achieve global load balancing.

[0005] (2) Graph theory-based method: The relationship between base stations is modeled using a graph structure and optimized using graph theory knowledge. This method can consider the relative relationship between base stations and promote collaboration between base stations, but it is difficult to consider the impact of user traffic on the system and to achieve global load balancing.

[0006] (3) Adaptive learning methods: By collecting or simulating base station user interaction data (including traffic distribution, relative positions between base stations, etc.), a neural network or other model is used to fit a function that maps base station interaction data to base station parameters, and the base station is guided to dynamically adjust the corresponding parameters. This type of method is often adaptive. However, since this type of method is often based on a single-agent scenario, it is difficult to achieve collaboration between base stations to achieve global load balancing; in addition, since the trained model is only based on part of the static interaction data, there will be a long-term and severe performance degradation when migrating the model to other base stations (sometimes even worse than the manual parameter adjustment method based on engineer experience). Summary of the invention

[0007] Based on this, an object of the present invention is to provide a parameter optimization method and system for cell switching to solve at least one technical problem in the background technology.

[0008] A parameter optimization method for cell switching according to an embodiment of the present invention is applied to a wireless communication network including at least one base station, wherein each frequency and coverage area of ​​the base station corresponds to a cell, thereby forming multiple cells. The method comprises the following steps:

[0009] According to the system architecture of the wireless communication network, a cell switching model of the wireless communication network is constructed, wherein the cell switching model includes a plurality of cell switching parameters;

[0010] Constructing a cell load standard deviation function according to a cell handover model of the wireless communication network;

[0011] An optimization problem is constructed with minimizing the cell load standard deviation function as a goal, the plurality of cell switching parameters as optimization variables, and the parameter ranges of the plurality of cell switching parameters as constraints;

[0012] A preset multi-agent reinforcement learning method is used to iteratively solve the optimization problem to obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

[0013] In addition, the parameter optimization method for cell switching according to the above embodiment of the present invention may also have the following additional technical features:

[0014] Furthermore, the cell switching model at least includes any one or more of a coverage-based intra-frequency switching model, a coverage-based inter-frequency switching model, and a frequency priority-based inter-frequency switching model.

[0015] Furthermore, the coverage-based intra-frequency handover model is defined as:

[0016] When monitoring the user k From neighboring communities j Received power Out of current cell m Received power First cell switching threshold , and duration H 1, report the switching request, which is described by the formula:

[0017] ;

[0018] The coverage-based inter-frequency handover model is defined as:

[0019] When the current cell m ofRSRP Lower than the second cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula:

[0020] ;

[0021] When the current cell m of RSRP Higher than the third cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula:

[0022] ;

[0023] When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRP Better than the fourth cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula:

[0024] ;

[0025] The frequency priority-based inter-frequency switching model is defined as:

[0026] When the current cell m of RSRP Below the fifth cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula:

[0027] ;

[0028] When the current cell m of RSRP Higher than the sixth cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula:

[0029] ;

[0030] When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRP Better than the seventh cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula:

[0031] ;

[0032] In the formula, n To monitor the time slot, Fj For neighboring communities j The frequency, Fm For the current community m frequency.

[0033] Further, when the coverage-based intra-frequency handover model, the coverage-based inter-frequency handover model, and the frequency priority-based inter-frequency handover model exist simultaneously, the handover is performed according to the first-come-first-served principle.

[0034] Furthermore, the cell switching parameter includes a first cell switching threshold , Second cell switching threshold , the third cell switching threshold , the fourth cell switching threshold , Fifth cell switching threshold , the sixth cell switching threshold and the seventh cell switching threshold One or more of these;

[0035] Accordingly, the constraints include one or more of the following conditions:

[0036] .

[0037] Furthermore, the cell load standard deviation function is the negative of the absolute value of the difference between the real-time load of the current cell and the average load of the area, and the specific formula is:

[0038] ;

[0039] In the formula, For the current cell in each time slot t The standard deviation of the cell load, is the real-time load of the current cell in each time slot t, For the current cell in each time slot t The average load of the region.

[0040] Furthermore, according to the system architecture of the wireless communication network, the step of constructing a cell switching model of the wireless communication network includes:

[0041] According to the system architecture of the wireless communication network, a neighbor communication mechanism is constructed between cells so that each cell can communicate with each other in each time slot. t Only exchange their respective area average load with its neighbor cells ;

[0042] The average load of the region The initialization and update methods are:

[0043] ;

[0044] In the formula, Ni Indicates the community i Neighborhood j A collection of α is the coefficient.

[0045] Furthermore, real-time load The calculation formula is:

[0046] ;

[0047] in, Representative User k In each time slot t data needs, Representative User k In each time slot t The data received at is a set of binary variables, where:

[0048] ;

[0049] when Indicates time slot t Time User k By current cell m Serve, It means otherwise.

[0050] Furthermore, in the process of iteratively solving the optimization problem using the preset multi-agent reinforcement learning method, each cell is t The observation space is defined as the time slot td To time slot t+d The aggregation of observations over all time slots of .

[0051] A parameter optimization system for cell switching according to an embodiment of the present invention is applied to a wireless communication network including at least one base station, wherein each frequency and coverage area of ​​the base station corresponds to a cell, thereby forming multiple cells. The system includes:

[0052] A model building module, used to build a cell switching model of the wireless communication network according to the system architecture of the wireless communication network, wherein the cell switching model includes a plurality of cell switching parameters;

[0053] A function construction module, used to construct a cell load standard deviation function according to a cell switching model of the wireless communication network;

[0054] An optimization construction module, used to construct an optimization problem with minimizing the cell load standard deviation function as a goal, taking the several cell switching parameters as optimization variables, and taking the parameter ranges of the several cell switching parameters as constraints;

[0055] The iterative solution module is used to iteratively solve the optimization problem by using a preset multi-agent reinforcement learning method to obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

[0056] According to an embodiment of the present invention, a parameter optimization device for cell switching includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the parameter optimization method for cell switching as described above is implemented.

[0057] Compared with the existing technology: by constructing a cell switching model, and constructing a cell load standard deviation function based on the cell switching model, minimizing the cell load standard deviation function is taken as the optimization goal, and using a multi-agent reinforcement learning method to iteratively solve the optimization problem, it is possible to simultaneously perform a global optimal solution to the load of each cell (agent), and after algorithm iteration, the load of each cell can be globally balanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of the effect of cell switching to achieve load balancing provided by an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of a coverage-based same-frequency switching method provided in an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of an inter-frequency switching method based on coverage provided by an embodiment of the present invention;

[0061] Figure 4 A schematic diagram of an inter-frequency switching method based on frequency priority provided by an embodiment of the present invention;

[0062] Figure 5 A diagram illustrating the impact of cell switching parameters on system performance provided by an embodiment of the present invention;

[0063] Figure 6 is a flow chart of a parameter optimization method for cell switching in the first embodiment of the present invention;

[0064] Figure 7 A schematic diagram of a wireless communication scenario provided by an embodiment of the present invention;

[0065] Figure 8 A schematic diagram of the effect of neighbor communication and distributed average consensus provided by an embodiment of the present invention;

[0066] Fig. 9 A schematic diagram of the effect of the observation space provided by an embodiment of the present invention;

[0067] Fig.10 It is a structural schematic diagram of a parameter optimization system for cell switching in a third embodiment of the present invention;

[0068] Fig.11 It is a schematic diagram of the structure of a parameter optimization device for cell switching in the fourth embodiment of the present invention.

[0069] The following specific implementations will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0070] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0071] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0073] like Figure 1 As shown in the figure, it is of great value to use cell switching to achieve load balancing. In cellular networks, users have the characteristics of high-speed mobility, uneven spatial distribution, and different communication needs. A large number of users gather in hot spots, resulting in a shortage of communication resources and network congestion, while resources in nearby areas with low traffic are not fully utilized, and the system load is unbalanced, resulting in reduced throughput, network congestion, and substandard user service quality. Therefore, load balancing is of great value, which can improve resource utilization and support more users; save energy and reduce costs, increase benefits; enhance network performance, reduce service interruptions, and promote user fairness.

[0074] From the 1G era to the 5G era, major communication operators have been implementing the four-step switching process of "monitoring-reporting-decision-switching" when handling cell switching of user equipment. The formulation of this solution can be traced back to Bell Labs in 1970. Its original design intentions include:

[0075] (1) Maintaining communication continuity and stability: Cell switching allows users to continuously switch from one cell (base station coverage area) to another cell while moving (such as walking, driving, etc.), thereby ensuring that calls or data connections are not interrupted due to movement.

[0076] (2) Load regulation and network resource optimization management: Cell switching helps to rationally allocate and optimize network resources. When a user moves from a crowded cell to a less crowded cell, the system can use network resources more efficiently and reduce congestion.

[0077] (3) Improving network coverage and adapting to different environments: By deploying base stations in different geographical areas and implementing cell switching, mobile communication networks can cover a wider area, including cities, suburbs, and remote areas. In complex geographical and architectural environments (such as mountainous areas, tunnels, and high-rise buildings), cell switching ensures that users can maintain good communication connections while moving.

[0078] Specifically, cell switching within the same communication system can be divided into three categories, namely, intra-frequency switching based on coverage, inter-frequency switching based on coverage, and inter-frequency switching based on frequency priority. Each type of cell switching is composed of certain events among A1 to A5 events. The following will introduce in detail the A1 to A4 events and the three types of cell switching composed of them.

[0079] 1) Event A1: Triggered when the signal quality of the cell where the user equipment is located reaches or exceeds a preset threshold. Formula: Ms – Hyst > ThreshA1, and lasts for H1 time. Where Ms is the measured value of the current cell, and the measurement type is generally RSRP, RSRQ or SINR, with the unit of dBm. ThreshA1 is the threshold value of event A1, and the numerical range is [-140, -44], with the unit of dBm. Hyst is the amplitude hysteresis, which is used to reduce the ping-pong handover and misjudgment caused by wireless signal fluctuations. The numerical range is 0, 1, 2, 3, 4, 5, 6, and the default value is 1, with the unit of dBm. H1 is the time hysteresis, and event A1 will be triggered only after the inequality condition holds and lasts for this time. The numerical range is 0, 5, 10, 20, 30, 50, 100, 200, 300, 500, 750, 1280, 1920, etc., with the unit of ms; The trigger action of event A1 is that the user stops listening to the signal quality of neighboring cells. Among them, the cell handover parameter corresponding to event A1 is E1 = ThreshA1 + Hyst (many parameters maintain the default value in actual applications to simplify the solution difficulty).

[0080] Event A2: Triggered when the signal quality of the cell where the user equipment is located drops below a certain threshold. Formula: Ms + Hyst < ThreshA2, and lasts for H2 time. Where Ms is the measured value of the current cell, ThreshA2 is the threshold value of event A2, and the numerical range is [-140, -44], with the unit of dBm. Hyst is the amplitude hysteresis. H2 is the time hysteresis. If there is no special description for the same type of parameters, the settings are the same as above. The trigger action of event A2 is that the user starts listening to the signal quality of neighboring cells. Among them, the cell handover parameter corresponding to event A2 is E2 = ThreshA2 - Hyst.

[0081] A3 event: triggered when the signal quality of the adjacent cell is better than that of the main service cell to a certain extent. Formula: Mn+Ofn+Ocn-Hyst>Ms+Ofs+Ocs+Off, and lasts for H3 time. Ms and Mn are the measured values ​​of the current cell and the adjacent cell, Hyst: amplitude hysteresis, H3: time hysteresis. Ofn and Ofs are the frequency offsets of the adjacent cell and the current cell, which are used to adjust the attractiveness of cells with different frequencies. The optional parameter range is 0, 3, 6, 9, 12, 15, 18, the unit is dBm, and the default is 0. Ocn and Ocs are the offsets of the adjacent cell and the current cell, which are used to adjust the attractiveness of specific cells. The optional parameter range is 0, 3, 6, 9, 12, 15, 18, the unit is dBm, and the default is 0. Off is the switching offset, which can be understood as the threshold of A3, the range is (0,96), the unit is dBm. The triggering action of the A3 event is that the user reports the adjacent cell number to the current cell and initiates a switching request. Among them, the cell switching parameter corresponding to the A3 event is E3=Ofs+Ocs+Off+Hyst-Ofn-Ocn.

[0082] A4 event: triggered when the user equipment enters the coverage of a new cell and the signal quality of the cell reaches a certain threshold. Formula: Mn+Ofn+Ocn-Hyst>ThreshA4, and lasts for H4 time. Mn is the measurement value of the adjacent cell. Ofn is the frequency offset of the adjacent cell, and Ocn is the offset of the adjacent cell. Hyst is the amplitude hysteresis, and H4 is the time hysteresis. ThreshA4 is the threshold value of event A4, with a value range of [-140,-44], and a unit of dBm. The triggering action of the A4 event is that the user reports the adjacent cell number to the current cell and initiates a switching request. Among them, the cell switching parameter corresponding to the A4 event is E4=ThreshA4+Hyst-Ofn-Ocn.

[0083] Among them, Figure 2 As shown, the coverage-based same-frequency switching method is as follows:

[0084] Switching purpose: Based on coverage, maintaining continuity;

[0085] Problem solved: Devices maintain continuous connection between cells;

[0086] Switching direction: switching between the same frequencies;

[0087] Trigger: There is no need to adjust the frequency between the same frequencies, and the signal quality of the adjacent same-frequency cells can be obtained at any time;

[0088] Measurement: Continuous measurement;

[0089] Reporting: There are fast and slow decays, and the level fluctuates, so multiple reports are required to make a judgment;

[0090] Verdict: A3 event is met.

[0091] In addition, if Figure 3 As shown, the coverage-based inter-frequency switching method is as follows:

[0092] Switching purpose: Based on coverage, maintaining continuity;

[0093] Problem Solving: The coverage area of ​​the high-frequency cell is small. In order to maintain data continuity, switch to the low-frequency cell;

[0094] Switching direction: high frequency to low frequency;

[0095] Trigger: The monitoring of adjacent inter-frequency cells is triggered by the A2 event, and the monitoring of adjacent inter-frequency cells is turned off by the A1 event;

[0096] Measurement: After being triggered, the quality of the inter-frequency signals will be measured according to the priority.

[0097] Reporting: There are fast and slow decays, and the level fluctuates, so multiple reports are required to make a judgment;

[0098] Verdict: A4 event is met.

[0099] In addition, if Figure 4 As shown, the method of inter-frequency switching based on frequency priority is as follows:

[0100] Switching purpose: Adjust the load based on frequency priority;

[0101] Problem Solving: The capacity of low-frequency cells is low, so the load is artificially moved to high-frequency cells;

[0102] Switching direction: low frequency to high frequency;

[0103] Trigger: The monitoring of adjacent inter-frequency cells is triggered by the A2 event, and the monitoring of adjacent inter-frequency cells is turned off by the A1 event;

[0104] Measurement: After being triggered, the quality of the inter-frequency signals will be measured according to the priority.

[0105] Reporting: There are fast and slow decays, and the level fluctuates, so multiple reports are required to make a judgment;

[0106] Verdict: A4 event is met.

[0107] In the above three switching methods, each pair of adjacent cells must configure three independent sets of parameters to achieve the migration of user traffic. These three switching methods are configuration rules that engineers have gradually added in long-term practice in order to solve different problems. First come first served when the three switching methods coexist. Based on the above switching methods and their corresponding event combinations, users can smoothly and stably switch signals between cells; from a macro perspective, the offset switching parameters of adjacent cells control the logical boundaries of the cells, so that the edge users of overloaded cells can be handed over to adjacent cells to achieve load balancing, thereby improving network capacity.

[0108] like Figure 5 As shown in Figure 1, the value of the cell switching parameter has a significant impact on the performance of the entire system. Figure 5 (Left) A larger E3 causes the user to l 1 to l A competition area is generated in the distance interval of 2, and users will switch between AB repeatedly, resulting in a ping-pong effect; Figure 5 (middle) In the middle, E3 is small. When user P is far away from cell A, although the signal decays exponentially, it is still unable to switch to cell B, resulting in signal loss on the way. Figure 5 (Right) The interaction of multiple cells makes the problem more complicated, resulting in multiple contention areas. Therefore, it is of great value to optimize the cell switching parameters.

[0109] Currently, there are three main types of optimization for cell switching parameters:

[0110] (1) Manual parameter adjustment based on engineer experience: This method refers to the operator's engineers setting a set of fixed and safe parameters for base stations based on their experience. This method has no unified standard, and the effect is highly dependent on the level of engineers. It cannot be dynamically adjusted according to changes in user traffic, making it difficult to achieve global load balancing.

[0111] (2) Graph theory-based method: The relationship between base stations is modeled using a graph structure and optimized using graph theory knowledge. This method can consider the relative relationship between base stations and promote collaboration between base stations, but it is difficult to consider the impact of user traffic on the system and to achieve global load balancing.

[0112] (3) Adaptive learning methods: By collecting or simulating base station user interaction data (including traffic distribution, relative positions between base stations, etc.), a neural network or other model is used to fit a function that maps base station interaction data to base station parameters, and the base station is guided to dynamically adjust the corresponding parameters. This type of method is often adaptive. However, since this type of method is often based on a single-agent scenario, it is difficult to achieve collaboration between base stations to achieve global load balancing; in addition, since the trained model is only based on part of the static interaction data, there will be a long-term and severe performance degradation when migrating the model to other base stations (sometimes even worse than the manual parameter adjustment method based on engineer experience).

[0113] To this end, an embodiment of the present invention provides a parameter optimization method for cell switching, aiming to achieve global load balancing of each cell by proposing a new cell switching parameter optimization method, thereby greatly improving the cell switching optimization effect of the wireless communication network.

[0114] In order to more clearly illustrate a parameter optimization method for cell switching in an embodiment of the present invention, a specific embodiment will be given below.

[0115] Embodiment 1

[0116] See also Figure 6 , shown is a parameter optimization method for cell switching in the first embodiment of the present invention, which can be specifically implemented by software and / or hardware and applied to a wireless communication network including at least one base station, each frequency of the base station and its coverage area corresponds to a cell, thereby forming multiple cells, and the method specifically includes steps S01-step S04.

[0117] Step S01: constructing a cell switching model of the wireless communication network according to the system architecture of the wireless communication network, wherein the cell switching model includes a number of cell switching parameters.

[0118] Specifically, the cell switching model includes at least any one or more of a coverage-based intra-frequency switching model, a coverage-based inter-frequency switching model, and a frequency priority-based inter-frequency switching model.

[0119] The coverage-based intra-frequency switching model is defined as:

[0120] When monitoring the user k From neighboring areas j Received power Out of current cell m Received power First cell switching threshold , and duration H 1, report the switching request, which is described by the formula:

[0121] ;

[0122] The coverage-based inter-frequency handover model is defined as:

[0123] When the current cell m of RSRP Lower than the second cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula:

[0124] ;

[0125] When the current cell m of RSRP Higher than the third cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula:

[0126] ;

[0127] When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRP Better than the fourth cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula:

[0128] ;

[0129] The frequency priority-based inter-frequency switching model is defined as:

[0130] When the current cell m of RSRP Below the fifth cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula:

[0131] ;

[0132] When the current cell m of RSRP Higher than the sixth cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula:

[0133] ;

[0134] When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRPBetter than the seventh cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula:

[0135] ;

[0136] In the formula, n To monitor the time slot, Fj For neighboring communities j The frequency, Fm For the current community m frequency.

[0137] Specifically, in the process of constructing a cell switching model of a wireless communication network, it is necessary to construct a communication scenario, a cell switching process, etc. based on at least the traffic distribution of the wireless communication network, the relative positions between base stations, etc. The specific process of constructing a communication scenario is as follows:

[0138] In the embodiment of the present invention, the communication scenario is defined as consisting of at least one 5G base station (BS) and multiple ground user equipments (UEs), such as Figure 7 As shown in the figure, it is specifically shown as a communication scenario consisting of multiple 5G base stations (BS) and multiple ground user equipment (UE). The base station provides a downlink to the UE through orthogonal frequency division multiple access (OFDMA). The frequency band used by 5G base stations is relatively wide, covering the frequency band below 6 GHz (FR1) and the millimeter wave frequency band (FR2). Due to the large difference in the effective coverage range of different frequencies, a certain frequency of the base station and its coverage area are called a cell. All base stations form a total of M cells, and some cells from the same base station or adjacent base stations may overlap. The center position of cell m is , whose frequency is expressed as Fm , the bandwidth is Bm , xm,ym is the center coordinate of cell m. k In time slot t The data requirement is DK [ t ].

[0139] Adopt a service cycle T To allow the cell to transmit data to the user, and T Discretize into N time slots, and the duration of each time slot is dt=T / N .use Pm [ t ] indicates a community m In time slot t The transmission power of the user k In time slot t The position at time can be expressed as Q [t ],in Q [ t ]=[ xk [ t ], yk [ t ]], 1 ≤ k ≤ K , xk [ t ], yk [ t ] for users k In time slot t The position coordinates at time K is the number of users.

[0140] In each time slot, each user is served by a specific cell, which can be represented by a set of binary variables describe. Indicates time slot t Time User k By community m Serve, Therefore, the following constraints hold:

[0141] .

[0142] Consider an urban macro scenario [3GPP TR 38.901], which includes line-of-sight (LOS) and non-line-of-sight (NLOS) conditions, with cells m In time slot t Time to user k Line-of-sight path loss It can be expressed as:

[0143] ;

[0144] The unit is db, not line-of-sight path loss for:

[0145] ;

[0146] The unit is db, where Indicates the community m Center and users k The distance between Fm is the center frequency in GHz. The probability of LOS connection is expressed as , and the probability of NLOS connection is Therefore, the average path loss can be expressed as:

[0147] ;

[0148] and are the antenna gains of the transmitting and receiving ends respectively. is the fast fading factor, which obeys the Rayleigh distribution. t Time User k From the community m Received power It can be expressed as:

[0149] ;

[0150] The unit is dBm. Interference from neighboring cells on the same frequency is significant. Recording Cell m and community j Whether to use the same frequency. Connect to the cell m and users k Signal to Interference and Noise Ratio (SINR) It can be expressed as:

[0151] ;

[0152] Where σ2 represents the power of the additive white Gaussian noise (AWGN) at the receiving end.

[0153] Each cell allocates resources equally according to the users it serves. k In time slot t Data received at It can be expressed as:

[0154] ;

[0155] In the formula, Indicates time slot t Time Zone m The number of associated users, Indicates user k Occupies the bandwidth of cell m.

[0156] In addition, in the preferred embodiment of the present invention, when constructing the switching process, coverage-based intra-frequency switching, coverage-based inter-frequency switching, and frequency priority-based inter-frequency switching are considered at the same time, which can make the optimization result more in line with the cell switching process in the actual communication scenario. The details are as follows:

[0157] In each time slot, the cell uses power Broadcast reference signals (RS), which provide cell measurement, identification and synchronization for UEs. At the beginning of the cycle, each UE accesses a cell with the highest reference signal received power (RSRP). Then, in each time slot, the system implements user switching between cells based on the sequence of user monitoring-user reporting-cell decision-execution switching. For each cellm , configure three switching modes, namely, coverage-based intra-frequency switching, coverage-based inter-frequency switching, and frequency priority-based inter-frequency switching, which are used to handle the switching of users in the current cell to the adjacent cell. Among them:

[0158] 1) Coverage-based intra-frequency handover: The purpose of this handover method is to switch to an adjacent intra-frequency cell with better signal quality when the user moves to the edge of the current cell to ensure signal continuity. Since the cells are of the same frequency, users do not need to adjust the antenna system and can always monitor and obtain the RSRP of the adjacent cells. Specifically, the A3 event is used to configure coverage-based intra-frequency handover. For details, see the above definition of the coverage-based intra-frequency handover model.

[0159] 2) Coverage-based inter-frequency switching: Generally speaking, the coverage of low-frequency cells is larger than that of high-frequency cells. When a user moves to the edge of a high-frequency cell, it will use coverage-based inter-frequency switching to switch to an adjacent low-frequency cell with better signal quality to ensure signal continuity. This process can be divided into two steps: monitoring and reporting events. Since the monitoring of inter-frequency signals requires adjustment of the antenna system, and considering the energy consumption of the equipment, it is not possible to monitor all the time. Therefore, the A1 event is used to trigger the monitoring of adjacent inter-frequency cells, and the A2 event is used to turn off the monitoring of inter-frequency cells. For details, see the above-mentioned definition of the coverage-based inter-frequency switching model.

[0160] 3) Inter-frequency switching based on frequency priority: Low-frequency cells have a wider coverage area than high-frequency cells, but due to the narrow bandwidth, their throughput and rate are much lower than those of high-frequency cells. In order to ensure that low-frequency cells have enough capacity to serve users in edge areas, the system will migrate users in low-frequency cells to high-frequency cells as much as possible to achieve load balancing. Since the switching occurs between inter-frequency cells, the inter-frequency switching based on frequency priority is also divided into two steps: monitoring and reporting events, like the inter-frequency switching based on coverage, which are implemented using A1, A2 and A4 events respectively. For details, see the above definition of the inter-frequency switching model based on frequency priority.

[0161] In addition, it is set that when the same-frequency handover model based on coverage, the different-frequency handover model based on coverage, and the different-frequency handover model based on frequency priority exist at the same time, the handover is performed on a first-come, first-served basis. That is, when the above three handover modes coexist, the handover is performed on a first-come, first-served basis. m To the community j After the switch, .

[0162] Step S02: constructing a cell load standard deviation function according to the cell handover model of the wireless communication network.

[0163] The cell load standard deviation function is the negative of the absolute value of the difference between the real-time load of the current cell and the average load of the area, and the specific formula is:

[0164] ;

[0165] In the formula, For the current cell in each time slot t The standard deviation of the cell load, is the real-time load of the current cell in each time slot t, For the current cell in each time slot t The average load of the region.

[0166] Among them, real-time load It is defined as the time required to complete the request of the current service user, and its calculation formula is:

[0167] ;

[0168] in, Representative User k In each time slot t data needs, Representative User k In each time slot t The data received at is a set of binary variables, where:

[0169] ;

[0170] when Indicates time slot t Time User k By current cell m Serve, It means otherwise.

[0171] Step S03, constructing an optimization problem with the goal of minimizing the cell load standard deviation function, the plurality of cell switching parameters as optimization variables, and the parameter ranges of the plurality of cell switching parameters as constraints.

[0172] In the preferred embodiment of the present invention, the cell switching parameter E Including the first cell switching threshold , Second cell switching threshold , the third cell switching threshold , the fourth cell switching threshold , Fifth cell switching threshold , the sixth cell switching threshold and the seventh cell switching threshold ,Right now:

[0173] ;

[0174] Accordingly, constraints may include:

[0175]

[0176] In order to further improve the convergence speed, by further studying the feasible range of these parameters, in a more preferred embodiment, the constraints can be further constrained as follows:

[0177] .

[0178] In addition, in the embodiment of the present invention, the purpose is to optimize E To minimize the standard deviation of all cell loads, in order to achieve global load balance in each cell, the optimization problem (P1) can be expressed as:

[0179]

[0180] Step S04, using a preset multi-agent reinforcement learning method to iteratively solve the optimization problem, and obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

[0181] It should be noted that the above optimization problem involves complex nonlinear processes, and the coupling between cells and cells, and the coupling between same-frequency switching and different-frequency switching are relatively high. In order to reduce the load of a certain cell, it is necessary not only to optimize the parameters of the cell, but also to rely on the coordination of the parameters of the surrounding cells. Therefore, it is difficult to solve it with traditional optimization methods. Therefore, the embodiment of the present invention adopts multi-agent reinforcement learning (MARL for short) to solve it. MARL is an algorithm that studies the decision-making and learning of multiple agents in a shared environment. Unlike single-agent reinforcement learning, MARL involves multiple agents, which can cooperate or compete with each other, so its learning process is more complicated.

[0182] In MARL, each agent tries to maximize its cumulative reward by interacting with the environment. Since the environment includes not only static factors but also the dynamic behaviors of other agents, the uncertainty and complexity faced by each agent are greatly increased. The strategies of different agents will affect each other, resulting in the non-stationarity of the environment, that is, the state of the environment changes continuously as the strategy of the agent changes.

[0183] In addition, in the specific implementation, before using the multi-agent reinforcement learning method to solve the problem, the algorithm model needs to be defined as follows:

[0184] The cell switching problem consisting of multiple cells and multiple users is modeled as a distributed partially observable Markov decision process (Dec-POMDP), which can be expressed as :

[0185] S Represents the state space, including the number and distribution of cells in the entire system, the load of each cell, and the number and distribution of users.

[0186] represents the joint action space, where each agent selects an action from its own action space and executes it. Specifically, each agent needs to set its own 7 parameters (i.e. ) to set the value. Since these parameters are all integers, the action space is a discrete space.

[0187] R is the reward function (i.e., the standard deviation function of the cell load), specifically for the agent i Action The reward value is the negative of the absolute value of the difference between the cell and the average load of the region. That is, the penalty is imposed based on the difference between the cell and the average load.

[0188] is the joint observation space, specifically the agent i In time slot t The observation space, the information observed in each time slot includes the distribution of surrounding cells and the distribution of surrounding user traffic 。

[0189] is the discount rate, which is used to measure the importance of future earnings to the present.

[0190] In summary, the parameter optimization method for cell switching in the above-mentioned embodiment of the present invention constructs a cell switching model, and constructs a cell load standard deviation function based on the cell switching model, takes minimizing the cell load standard deviation function as the optimization goal, and adopts a multi-agent reinforcement learning method to iteratively solve the optimization problem. It can simultaneously perform a global optimal solution to the load of each cell (agent), and after algorithm iteration, the load of each cell can be globally balanced.

[0191] Embodiment 2

[0192] The second embodiment of the present invention also provides a parameter optimization method for cell switching, which can be implemented by software and / or hardware. The difference between the parameter optimization method for cell switching in this embodiment and the parameter optimization method for cell switching in the first embodiment is that:

[0193] According to the problem modeling, each cell is considered as an intelligent agent, and user traffic is considered as part of the environment. Since all cells have similar characteristics, goals, and working conditions, a parameter sharing architecture can be adopted, and the centralized training and distributed execution (CTDE) framework can be used to promote collaboration and accelerate training. In the scenario of multi-cell, multi-frequency, and multi-user switching, there are mainly two challenges:

[0194] 1) In large-scale distributed scenarios, a single cell cannot obtain global information in a timely manner, that is, for each cell, the environment is partially observable; and the optimization goal of the problem is to minimize the global load standard deviation, so it is necessary to build a reasonable information transmission framework and reasonably design rewards for cell actions to promote the strategy to update in the direction of the optimization goal and promote coordination between cells.

[0195] 2) Since the switching of a cell requires multiple links such as monitoring, reporting, decision-making, and switching, and each link takes a certain amount of time to complete, there is a time lag between the cell status and the parameter change. When a cell changes its own parameter value, it takes at least several time slots to observe the effect of the parameter change, so it is necessary to adjust the observation range from the time dimension to reduce the time lag error.

[0196] In order to solve the above challenge 1), in this embodiment, according to the system architecture of the wireless communication network, the step of constructing the cell switching model of the wireless communication network specifically includes:

[0197] According to the system architecture of the wireless communication network, a neighbor communication mechanism is constructed between cells so that each cell can communicate with each other in each time slot. t Only exchange their respective area average load with its neighbor cells ;

[0198] The average load of the region The initialization and update methods are:

[0199] ;

[0200] In the formula, Ni Indicates the community i Neighborhood j A collection of α is the coefficient.

[0201] That is, this embodiment designs a neighbor communication mechanism and introduces a distributed average consensus algorithm to obtain approximate global information under partially observable conditions. Among them, the goal of the distributed average consensus algorithm is to allow all nodes (or agents) in a network to gradually converge to the average value of a global quantity through multiple rounds of iterations and information exchange with neighboring nodes. For example, in a network, each node initially has a value, and the purpose of the algorithm is to make the final value of all nodes close to the average value of the initial values ​​of all nodes.

[0202] Specifically, each cell is made to t Only exchange their respective area average load with its neighbor cells , the initialization and update methods of this value are as above. The load of the cell changes continuously and smoothly over time. The convergence of this algorithm depends on the topological structure of the communication network. Since all cells can form a connected graph in the modeling of the above problem, this algorithm can ensure the average regional load of all cells. Approximately equal to the average global load The process can be visualized as Figure 8 .

[0203] In addition, in order to solve the above challenge 2), in the process of iteratively solving the optimization problem using the preset multi-agent reinforcement learning method, each cell in each time slot t The observation space is defined as the time slot td To time slot t+d The aggregation of observations of all time slots. Specifically, Fig. 9 As shown, this embodiment adopts the historical window method to slice the observation space in the time dimension and reconstruct the real state with the historical observation slices. i In time slot t The observation space It is designed to aggregate the observations of all time slots from time slot td to time slot t+d. In practice, it is preferred to set d to 5. That is, after an action is executed, the environment is allowed to run for d time slots to observe the results. Since the state before the action is executed also affects the short-term effect of the action, the observations from time slot td to time slot t+d are also incorporated. Specifically, the information observed in each time slot includes the distribution of surrounding cells and the distribution of surrounding user traffic, both of which are represented in matrix (pixel) form.

[0204] In addition, it is necessary to further explain that in the preferred embodiment, since the embodiment of the present invention adopts parameter sharing and CTDE strategy, the observation values ​​of different agents input into the model are quite different. In order to avoid overfitting, in the specific implementation, the PPO (Proximal Policy Optimization) algorithm can be preferentially used for iterative solution. Among them, PPO is one of the multi-agent reinforcement learning methods and belongs to the policy gradient method. PPO improves the performance of the agent by optimizing the policy function. Compared with the traditional policy gradient method, PPO introduces a "cutting" mechanism to limit the amplitude of policy changes in each update, thereby avoiding instability caused by excessive policy updates. Specifically, PPO maintains the stability and efficiency of policy updates by limiting the range of changes in the objective function during the optimization process. PPO has the advantages of easy implementation, high sample efficiency and stable training, and is widely used in various complex reinforcement learning tasks.

[0205] It should be noted that the above embodiments and their features can be arbitrarily combined without conflict, and the new technical solutions obtained by the combination still fall within the protection scope of the present invention.

[0206] Embodiment 3

[0207] Another aspect of the present invention is to provide a parameter optimization system for cell switching. Fig.10 , which is a parameter optimization system for cell switching in a third embodiment of the present invention, is applied to a wireless communication network including at least one base station, each frequency of the base station and its coverage area corresponds to a cell, thereby forming a plurality of cells, and the parameter optimization system for cell switching includes:

[0208] A model building module 11, configured to build a cell switching model of the wireless communication network according to the system architecture of the wireless communication network, wherein the cell switching model includes a plurality of cell switching parameters;

[0209] A function construction module 12, configured to construct a cell load standard deviation function according to a cell switching model of the wireless communication network;

[0210] An optimization construction module 13 is used to construct an optimization problem with minimizing the cell load standard deviation function as a goal, taking the several cell switching parameters as optimization variables, and taking the parameter ranges of the several cell switching parameters as constraints;

[0211] The iterative solution module 14 is used to adopt a preset multi-agent reinforcement learning method to iteratively solve the optimization problem to obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

[0212] The functions or operation steps implemented when the above modules and units are executed are generally the same as those in the above method embodiments, and will not be repeated here.

[0213] Embodiment 4

[0214] Another aspect of the present invention also provides a parameter optimization device for cell switching, see Fig.11 , shown is a parameter optimization device for cell switching in the fourth embodiment of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor, and when the processor 10 executes the computer program 30, the parameter optimization method for cell switching as described above is implemented.

[0215] Among them, the parameter optimization system for cell switching can specifically be a cloud server or an internal server of an enterprise, etc., and the processor 10 in some embodiments can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, which is used to run the program code stored in the memory 20 or process data, such as executing access restriction programs, etc.

[0216] Among them, the memory 20 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of the parameter optimization system for cell switching, such as the hard disk of the parameter optimization system for cell switching. In other embodiments, the memory 20 can also be an external storage device of the parameter optimization system for cell switching, such as a plug-in hard disk equipped on the parameter optimization system for cell switching, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 20 can also include both an internal storage unit and an external storage device of the parameter optimization system for cell switching. The memory 20 can not only be used to store application software and various types of data installed in the parameter optimization system for cell switching, but also can be used to temporarily store data that has been output or is to be output.

[0217] It should be pointed out that Fig.11 The structure shown does not constitute a limitation on the parameter optimization device for cell switching. In other embodiments, the parameter optimization device for cell switching may include fewer or more components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0218] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the parameter optimization method for cell switching as described above is implemented.

[0219] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0220] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0221] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0222] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0223] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A parameter optimization method for cell switching, characterized in that: Applied in a wireless communication network including at least one base station, each frequency of the base station and its coverage area corresponds to a cell, thereby forming multiple cells, the method comprises the following steps: According to the system architecture of the wireless communication network, a cell switching model of the wireless communication network is constructed, wherein the cell switching model includes a plurality of cell switching parameters; Constructing a cell load standard deviation function according to a cell handover model of the wireless communication network; An optimization problem is constructed with minimizing the cell load standard deviation function as a goal, the plurality of cell switching parameters as optimization variables, and the parameter ranges of the plurality of cell switching parameters as constraints; A preset multi-agent reinforcement learning method is used to iteratively solve the optimization problem to obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

2. The parameter optimization method for cell switching according to claim 1, characterized in that: The cell switching model includes at least any one or more of a coverage-based intra-frequency switching model, a coverage-based inter-frequency switching model, and a frequency priority-based inter-frequency switching model.

3. The parameter optimization method for cell switching according to claim 2, characterized in that: The coverage-based intra-frequency handover model is defined as: When monitoring the user k From neighboring areas j Received power Out of current cell m Received power First cell switching threshold , and duration H 1, report the switching request, which is described by the formula: ； The coverage-based inter-frequency handover model is defined as: When the current cell m of RSRP Lower than the second cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula: ； When the current cell m of RSRP Higher than the third cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula: ； When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRP Better than the fourth cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula: ； The frequency priority-based inter-frequency switching model is defined as: When the current cell m of RSRP Below the fifth cell switching threshold And continue When the time comes, start monitoring, which can be described by the formula: ； When the current cell m of RSRP Higher than the sixth cell switching threshold And continue When the time comes, close the monitoring, which can be described by the formula: ； When the user k After monitoring is turned on, if the adjacent cells with different frequencies j of RSRP Better than the seventh cell switching threshold And last for a period of time When , a switching request is reported, which is described by the formula: ； In the formula, n To monitor the time slot, Fj For neighboring communities j The frequency, Fm For the current community m frequency.

4. The parameter optimization method for cell switching according to claim 2 or 3, characterized in that: When the coverage-based intra-frequency handover model, the coverage-based inter-frequency handover model, and the frequency priority-based inter-frequency handover model exist at the same time, the handover is performed on a first-come, first-served basis.

5. The parameter optimization method for cell switching according to claim 4, characterized in that: The cell switching parameters include a first cell switching threshold , Second cell switching threshold , the third cell switching threshold , the fourth cell switching threshold , Fifth cell switching threshold , the sixth cell switching threshold and the seventh cell switching threshold One or more of these; Accordingly, the constraints include one or more of the following conditions: 。 6. The parameter optimization method for cell switching according to claim 1, characterized in that: The cell load standard deviation function is the negative of the absolute value of the difference between the real-time load of the current cell and the average load of the area. The specific formula is: ； In the formula, For the current cell in each time slot t The standard deviation of the cell load, is the real-time load of the current cell in each time slot t, For the current cell in each time slot t The average load of the region.

7. The parameter optimization method for cell switching according to claim 6, characterized in that: According to the system architecture of the wireless communication network, the step of constructing a cell switching model of the wireless communication network includes: According to the system architecture of the wireless communication network, a neighbor communication mechanism is constructed between cells so that each cell can communicate with each other in each time slot. t Only exchange their respective area average load with its neighbor cells ; The average load of the region The initialization and update methods are: ； In the formula, Ni Indicates the community i Neighborhood j A collection of α is the coefficient.

8. The parameter optimization method for cell switching according to claim 6, characterized in that: Real-time load The calculation formula is: ； in, Representative User k In each time slot t data needs, Representative User k In each time slot t The data received at is a set of binary variables, where: ； when Indicates time slot t Time User k By current cell m Serve, It means otherwise.

9. The parameter optimization method for cell switching according to claim 6, characterized in that: In the process of iteratively solving the optimization problem using the preset multi-agent reinforcement learning method, each cell is t The observation space is defined as the time slot td To time slot t+d The aggregation of observations over all time slots of .

10. A parameter optimization system for cell switching, characterized in that: Applied to a wireless communication network including at least one base station, each frequency of the base station and its coverage area corresponds to a cell, thereby forming multiple cells, the system includes: A model building module, used to build a cell switching model of the wireless communication network according to the system architecture of the wireless communication network, wherein the cell switching model includes a plurality of cell switching parameters; A function construction module, used to construct a cell load standard deviation function according to a cell switching model of the wireless communication network; An optimization construction module, used to construct an optimization problem with minimizing the cell load standard deviation function as a goal, taking the several cell switching parameters as optimization variables, and taking the parameter ranges of the several cell switching parameters as constraints; The iterative solution module is used to iteratively solve the optimization problem by using a preset multi-agent reinforcement learning method to obtain a number of optimized cell switching parameters to optimize the cell switching of the wireless communication network.

Images