Base station scheduling method and device, equipment, storage medium and product

Through multiple rounds of iteration and reinforcement learning, the problem of low base station scheduling accuracy is solved, efficient and accurate scheduling of base stations is achieved, and the accuracy of bandwidth allocation and operation and maintenance efficiency are improved.

CN119052934AActive Publication Date: 2024-11-29CHINA MOBILE GROUP DESIGN INST +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411004651.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-29
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

In the prior art, the accuracy of base station scheduling is poor, and it is impossible to effectively meet the bandwidth requirements of each cell to be optimized and the reasonable allocation of portable base stations.

Method used

By using multiple iteration methods, multiple agents of reinforcement learning are constructed, rewards are calculated based on bandwidth redundancy and transportation distance, and the base station scheduling scheme is gradually adjusted until the iteration termination condition is met, and the final base station scheduling scheme is output.

Benefits of technology

Improve the accuracy and efficiency of base station scheduling, reasonably allocate the cells to be optimized to the appropriate base station, save resources, reduce manual analysis time, and reduce maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052934A_ABST
    Figure CN119052934A_ABST
Patent Text Reader

Abstract

The invention discloses a base station scheduling method and device, equipment, a storage medium and a product, multiple rounds of iteration are performed on a base station scheduling scheme, and in each round of iteration, the base station transportation distance, the parking point base station surplus and the bandwidth redundancy of the base station scheduling scheme are calculated for adjusting the base station scheduling scheme and entering the next round of iteration. And outputting a final base station scheduling scheme until the iteration termination condition is met, thereby fully considering the bandwidth demand of each cell to be optimized, the performance of each portable base station and the distance between the portable base station and the cell to be optimized, enabling the cell to be optimized to be reasonably allocated to a proper base station, and improving the accuracy of base station allocation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a base station scheduling method, device, equipment, storage medium and product. Background Art

[0002] Portable base stations are a new type of small, mobile base stations that can be deployed in vehicles. They have the advantages of agile deployment, high mobility, self-powered and radio frequency modules, and the ability to quickly access existing communication networks, which can quickly restore signal coverage in wireless communication fault areas. Therefore, portable base stations can flexibly respond to various temporary support events, including emergency support (such as natural disasters, conference support), network optimization (such as temporary black spots, business surges), etc., and are one of the preferred network operation and maintenance solutions for operators.

[0003] However, in the process of analyzing the optimal equipment before equipment scheduling and transportation, expert experience analysis is mainly used, that is, offline statistics of the current operating status of the equipment and analysis of the geographical location and site performance data, and the dispatchable equipment is selected and assigned through expert experience. This method has obvious limitations and cannot accurately schedule base stations. Summary of the invention

[0004] Based on this, the present invention provides a base station scheduling method, device, equipment, storage medium and product to solve the defect of poor accuracy of manual base station scheduling in the prior art.

[0005] To achieve the above object, an embodiment of the present invention provides a base station scheduling method, including:

[0006] During the base station scheduling process, multiple rounds of iterations are performed on the base station scheduling plan;

[0007] In each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling plan, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan are calculated to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling plan is output; wherein the base station scheduling plan refers to a plan for scheduling portable base stations from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

[0008] As an improvement of the above scheme, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0009] Take a base station parking point as an agent and build multiple agents for reinforcement learning;

[0010] Perform multiple rounds of learning for all agents;

[0011] In each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually dispatch portable base stations from all the base station parking points for each cell to be optimized;

[0012] A round reward is calculated after each round to adjust the strategy of the agent; wherein the round reward includes a first reward and a second reward, the first reward is calculated based on bandwidth redundancy and base station transportation distance, the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after base station scheduling and the bandwidth requirement of the cell to be optimized, and the base station transportation distance refers to the sum of the transportation distances of all portable base stations scheduled by the cell to be optimized; the second reward is positively correlated with the minimum remaining number of base stations, and the minimum remaining number of base stations is the minimum value of the number of portable base stations remaining in all the base station parking points after the base station scheduling is completed;

[0013] After the reinforcement learning is completed, a final base station scheduling solution is outputted to be used for scheduling the portable base station from the base station parking point for the cell to be optimized.

[0014] As an improvement of the above solution, in each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually schedule portable base stations from all base station parking points for each cell to be optimized, including:

[0015] In a target step of a target round, all agents perform actions according to a strategy to dispatch portable base stations from all base station parking points for a target cell to be optimized; wherein the target round is any one of all rounds, the target step is any one of multiple steps of the target round, and the target cell to be optimized is a cell to be optimized corresponding to the target step;

[0016] Calculating a first reward for the target step, wherein the first reward for the target step is calculated based on the bandwidth redundancy of the target cell to be optimized and the base station transportation distance of the target cell to be optimized;

[0017] The strategy is adjusted according to the first reward, and the next step of the target step is entered until all the cells to be optimized complete base station scheduling, thus ending the target round.

[0018] As an improvement of the above solution, adjusting the strategy according to the first reward includes: adjusting the strategy according to the first reward using a Critic network.

[0019] As an improvement of the above scheme, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0020] Acquire initial information; the initial information includes all cells to be optimized and their initial bandwidth and bandwidth requirements, all base station parking points and the number of base stations stored therein, and the distance between the cells to be optimized and the base station parking points;

[0021] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0022] Constructing an objective function for the mathematical model, wherein the objective function is to minimize the sum of the transportation distances of the scheduled portable base stations, wherein the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0023] Constructing a quantity constraint condition and a bandwidth constraint condition for the mathematical model, wherein the quantity constraint condition is that the number of portable base stations scheduled in the base station parking point cannot exceed the number of portable base stations originally parked in the base station parking point, and the bandwidth constraint condition is that the bandwidth of the cell to be optimized after base station scheduling is greater than or equal to the bandwidth requirement of the cell to be optimized;

[0024] Under the quantity constraint and the bandwidth constraint, the objective function is solved to obtain a final base station scheduling solution, so as to schedule the portable base station at the base station parking point to the cell to be optimized according to the base station scheduling solution.

[0025] As an improvement of the above scheme, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0026] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0027] Constructing a first loss function, a second loss function and a third loss function for the mathematical model; wherein the first loss function is equal to the difference between the parking point base station storage quantity row vector and the parking point base station scheduling quantity row vector of the current base station scheduling scheme, and the parking point base station scheduling quantity row vector represents the number of portable base stations scheduled at each base station parking point; the second loss function is equal to the initial bandwidth row vector plus the bandwidth increment row vector of the current base station scheduling scheme minus the bandwidth demand row vector, and the bandwidth increment row vector represents the bandwidth increment of each cell to be optimized after the base station scheduling; the third loss function is equal to the inverse of the sum of the transportation distances of each scheduled portable base station in the current base station scheduling scheme, and the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0028] Iterate the base station scheduling scheme, and calculate the first loss function, the second loss function and the third loss function after each round of iteration;

[0029] The base station scheduling scheme of the iteration round that maximizes the total loss function is selected as the final base station scheduling scheme for base station scheduling, wherein the total loss function is positively correlated with the first loss function, the second loss function, and the third loss function, respectively.

[0030] To achieve the above object, an embodiment of the present invention further provides a base station scheduling device, including:

[0031] A scheme iteration module is used to perform multiple rounds of iterations on the base station scheduling scheme during the base station scheduling process;

[0032] The base station scheduling module is used to schedule the portable base station from the base station parking point to the cell to be optimized according to the current base station scheduling plan in each round of iteration, and calculate the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan, so as to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and output the final base station scheduling plan; wherein the base station scheduling plan refers to the plan for scheduling the portable base station from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

[0033] To achieve the above-mentioned purpose, an embodiment of the present invention also provides a base station scheduling device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the base station scheduling method described in any of the above embodiments.

[0034] To achieve the above-mentioned purpose, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the base station scheduling method as described in any of the above embodiments.

[0035] To achieve the above objectives, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the base station scheduling method as described in any of the above embodiments.

[0036] Compared with the prior art, the base station scheduling method, apparatus, device, storage medium and product disclosed in the embodiments of the present invention perform multiple rounds of iterations on the base station scheduling scheme during the base station scheduling process. In each round of iteration, the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output; wherein, the base station scheduling scheme refers to a scheme for scheduling portable base stations from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized, which fully considers the bandwidth requirements of each cell to be optimized, the performance of each portable base station and the distance between the portable base station and the cell to be optimized, so that the cell to be optimized can be reasonably allocated to a suitable base station, thereby improving the accuracy of base station allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flowchart of a base station scheduling method provided by an embodiment of the present invention;

[0039] Figure 2 It is a structural diagram of a base station scheduling device provided by an embodiment of the present invention;

[0040] Figure 3 It is a structural diagram of a base station scheduling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1 , Figure 1 : is a flow chart of a base station scheduling method provided by an embodiment of the present invention. The base station scheduling method comprises steps S11 to S12:

[0043] S11. During the base station scheduling process, the base station scheduling scheme is iterated multiple times;

[0044] S12. In each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling plan, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan are calculated to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling plan is output; wherein, the base station scheduling plan refers to a plan for scheduling portable base stations from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

[0045] Specifically, in a set area, multiple base station parking points are set up, including multiple cells. When the network of a cell needs to optimize wireless signals (such as a network failure), the cell can be used as a cell to be optimized, and vehicles with portable base stations deployed are dispatched from the base station parking points to the cell to be optimized to optimize the network of the cell. The method calculates the base station transportation distances of all portable base stations that are scheduled, the number of portable base stations that are not scheduled at each parking point (the remaining number of base stations at the parking point), and the difference between the bandwidth of each cell to be optimized after the base station scheduling and the bandwidth demand of the cell itself (the bandwidth redundancy) after the base station scheduling using different base station scheduling schemes. According to the base station transportation distance, the remaining number of base stations at the parking point, and the bandwidth redundancy corresponding to the base station scheduling scheme, the base station scheduling scheme is analyzed for performance, and the base station scheduling scheme is adjusted to obtain the best base station scheduling scheme for base station scheduling. It can be understood that after the base station scheduling is completed, the fewer vehicles scheduled in each base station parking point, the better, the larger the bandwidth of the cell to be optimized, the better, and the shorter the driving distance of all vehicles scheduled from the base station parking point to the cell to be optimized, the better, thereby achieving accurate scheduling of the base station, saving resources and increasing the bandwidth of the cell to be optimized.

[0046] Compared with the existing technology, the method significantly saves manual analysis time and improves the accuracy of base station scheduling, which is conducive to effectively solving pain points such as emergency communication guarantee, black spot rectification, and load optimization, and can significantly improve network satisfaction, with good social benefits. In addition, through research, China Mobile has deployed more than 5,000 portable base stations (communication cabinets) across the country. Considering the temporary nature of equipment deployment, online management and intelligent scheduling analysis of equipment based on automation can significantly reduce the workload of front-line maintenance personnel. Based on the calculation that a single device reduces the workload of maintenance personnel by 0.1 man-years / unit throughout the year and the labor cost is 50,000 yuan / man-year, it is estimated that the labor cost savings are about 25 million yuan, which has obvious economic benefits.

[0047] In one implementation, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0048] Acquire initial information; the initial information includes all cells to be optimized and their initial bandwidth and bandwidth requirements, all base station parking points and the number of base stations stored therein, and the distance between the cells to be optimized and the base station parking points;

[0049] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0050] Constructing an objective function for the mathematical model, wherein the objective function is to minimize the sum of the transportation distances of the scheduled portable base stations, wherein the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0051] Constructing a quantity constraint condition and a bandwidth constraint condition for the mathematical model, wherein the quantity constraint condition is that the number of portable base stations scheduled in the base station parking point cannot exceed the number of portable base stations originally parked in the base station parking point, and the bandwidth constraint condition is that the bandwidth of the cell to be optimized after base station scheduling is greater than or equal to the bandwidth requirement of the cell to be optimized;

[0052] Under the quantity constraint and the bandwidth constraint, the objective function is solved to obtain a final base station scheduling solution, so as to schedule the portable base station at the base station parking point to the cell to be optimized according to the base station scheduling solution.

[0053] Specifically, the base station scheduling scheme is determined by mathematical modeling, and the mathematical modeling process is as follows:

[0054] The m cells that need troubleshooting are denoted as A = {a1, a2, ..., a m} (i.e. the group of cells to be optimized). After encountering an emergency, the initial bandwidth of the jth cell is recorded as b i The required bandwidth to be achieved is denoted as q j , the initial bandwidth of m cells constitutes a row vector B = [b1, b2, ..., b m ], if there is no signal in the cell, the initial bandwidth is 0; the required bandwidth to be achieved constitutes a row vector Q = [q1, q2, ..., q m ].

[0055] At the same time, the n base station parking points that can be mobilized in the whole area are recorded as P = {p1, p2, ..., p n} (i.e., a base station parking point group), the number of portable base stations at n base station parking points is recorded as K = [k1, k2, ..., k n ] (i.e., the number of parking point base stations stored in the row vector); at the same time, the driving distance between the jth cell and the i-th parking point is recorded as d ij , the distance between the jth cell and all parking points constitutes a column vector matrix D j =[d 1j , d 2j,...,d nj ] T , the distances from each cell to different parking spots form a matrix D = [D1, D2, ..., D m ] (i.e., the distance matrix).

[0056] In order to enable emergency communications at the parking point to meet the bandwidth requirements of all cells, the portable base station is scheduled from P to A, and the following constraints need to be met:

[0057] (1) Vehicle number constraint, that is, the number of vehicles dispatched at each parking spot should be less than or equal to the number of vehicles at the parking spot; it is worth noting that the vehicle number constraint is a quantity constraint condition, one vehicle corresponds to one portable base station, and the number of portable base stations dispatched at a parking spot cannot exceed the number of portable base stations originally parked at the parking spot.

[0058] (2) Bandwidth demand constraint, that is, the actual bandwidth of each cell after scheduling should be higher than the required bandwidth;

[0059] The number of portable base stations scheduled from the i-th parking point to the j-th cell is recorded as c ij , all the portable base stations scheduled to all cells at all parking points are recorded as matrix M, expressed as:

[0060]

[0061] The present application proposal stipulates that for matrices X and Y with the same number of rows and columns, if each element in X is greater than or equal to the element at the corresponding position in Y, it is recorded as X≥Y. It is also stipulated that the bandwidth of any portable base station in the present application proposal is equal and is b, and the row vector matrix containing n 1s is recorded as I.

[0062] Therefore, according to requirement (1), the constraint can be expressed as:

[0063] M.I T ≤K

[0064] According to requirement (2), the constraint can be expressed as:

[0065] B+S·M≥Q

[0066] Wherein, S represents a row vector matrix containing n b (in this embodiment, it is assumed that the bandwidth of any portable base station is equal and is b). On the basis of satisfying two basic service constraints, considering that the speed of responding to emergencies needs to be as fast as possible, the total distance of each portable base station dispatched in this dispatch is calculated:

[0067]

[0068] With d xMinimization is the objective function. Under the quantity constraints and bandwidth constraints, the objective function is solved to obtain the final base station scheduling solution.

[0069] In one implementation, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0070] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0071] Constructing a first loss function, a second loss function and a third loss function for the mathematical model; wherein the first loss function is equal to the difference between the parking point base station storage quantity row vector and the parking point base station scheduling quantity row vector of the current base station scheduling scheme, and the parking point base station scheduling quantity row vector represents the number of portable base stations scheduled at each base station parking point; the second loss function is equal to the initial bandwidth row vector plus the bandwidth increment row vector of the current base station scheduling scheme minus the bandwidth demand row vector, and the bandwidth increment row vector represents the bandwidth increment of each cell to be optimized after the base station scheduling; the third loss function is equal to the inverse of the sum of the transportation distances of each scheduled portable base station in the current base station scheduling scheme, and the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0072] Iterate the base station scheduling scheme, and calculate the first loss function, the second loss function and the third loss function after each round of iteration;

[0073] The base station scheduling scheme of the iteration round that maximizes the total loss function is selected as the final base station scheduling scheme for base station scheduling, wherein the total loss function is positively correlated with the first loss function, the second loss function, and the third loss function, respectively.

[0074] Specifically, firstly, the basic information of the mathematical model is constructed (see the specific example of the above implementation mode for details, which will not be repeated here), and then a loss function is established for the mathematical model. The loss function construction process is as follows:

[0075] From the modeling process in the previous implementation method, it can be seen that the scheduling process needs to improve two constraint satisfaction problems and one optimization problem. These problems constitute multi-objective tasks, and each task can give a loss function based on mathematical modeling.

[0076] For the vehicle number constraint, the first loss function can be given:

[0077] L c (M) = KM·I T

[0078] Where L c (M) needs to be maximized and the minimum value needs to be greater than or equal to 0.

[0079] For the bandwidth requirement constraint, the second loss function can be given:

[0080] L b (M) = B + βI·MQ

[0081] Where β represents the bandwidth that each portable base station can provide; L b (M) needs to be maximized and the minimum value needs to be greater than or equal to 0.

[0082] For the optimal total distance, the third loss function can be given:

[0083] L d (M) = -d x

[0084] Where L d (M) needs to be maximized.

[0085] During the model training process, the model parameters are adjusted by combining the first loss function, the second loss function and the third loss function, and finally the optimal base station scheduling solution is obtained.

[0086] It is worth noting that by transforming the mathematical model of each sub-problem into a loss function, the key includes the following:

[0087] (1) Vehicle number constraint loss function (first loss function): Based on the characteristics of the mathematical model, the model is transformed into a function that can be compared with 0. This function gives a clear convergence direction for the vehicle number constraint.

[0088] (2) Bandwidth demand constraint loss function (second loss function): Based on the characteristics of the mathematical model, the model is transformed into a function that can be scaled with 0. This function gives a clear convergence direction for the bandwidth demand constraint.

[0089] (3) Distance target loss function (third loss function): Considering that there is no upper or lower limit on the distance target, the distance target is negated as the loss function of the distance target. The scheduling cost is the minimum, which represents the maximization of the function and gives a clear convergence direction for the distance target.

[0090] In one implementation, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0091] Take a base station parking point as an agent and build multiple agents for reinforcement learning;

[0092] Perform multiple rounds of learning for all agents;

[0093] In each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually dispatch portable base stations from all the base station parking points for each cell to be optimized;

[0094] A round reward is calculated after each round to adjust the strategy of the agent; wherein the round reward includes a first reward and a second reward, the first reward is calculated based on bandwidth redundancy and base station transportation distance, the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after base station scheduling and the bandwidth requirement of the cell to be optimized, and the base station transportation distance refers to the sum of the transportation distances of all portable base stations scheduled by the cell to be optimized; the second reward is positively correlated with the minimum remaining number of base stations, and the minimum remaining number of base stations is the minimum value of the number of portable base stations remaining in all the base station parking points after the base station scheduling is completed;

[0095] After the reinforcement learning is completed, a final base station scheduling solution is outputted to be used for scheduling the portable base station from the base station parking point for the cell to be optimized.

[0096] Specifically, from the modeling process of the above implementation, it can be seen that the matrix M (i.e., the base station scheduling plan) composed of the independent variables, each column and each row are not completely independent of each other, so it is difficult to simplify the solution process by discussing the entire matrix. Therefore, in this implementation, the calculation is divided into multiple agents, each agent represents a base station parking point, and each cell to be optimized is discussed step by step, which simplifies the calculation process by deconstructing the matrix and ensures the recursive solution characteristics of each row and column.

[0097] Furthermore, in each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually schedule a portable base station from all the base station parking points for each cell to be optimized, including:

[0098] In a target step of a target round, all agents perform actions according to a strategy to dispatch portable base stations from all base station parking points for a target cell to be optimized; wherein the target round is any one of all rounds, the target step is any one of multiple steps of the target round, and the target cell to be optimized is a cell to be optimized corresponding to the target step;

[0099] Calculating a first reward for the target step, wherein the first reward for the target step is calculated based on the bandwidth redundancy of the target cell to be optimized and the base station transportation distance of the target cell to be optimized;

[0100] The strategy is adjusted according to the first reward, and the next step of the target step is entered until all the cells to be optimized complete base station scheduling, thus ending the target round.

[0101] Furthermore, adjusting the strategy according to the first reward includes: adjusting the strategy according to the first reward using a Critic network (value function network).

[0102] Specifically, each column and row of the matrix M composed of independent variables are not completely independent of each other, so it is difficult to simplify the solution process by discussing the entire matrix. Therefore, this embodiment divides the calculation into multi-step decisions of multiple agents, and discusses each decision separately, which not only simplifies the calculation process by deconstructing the matrix, but also ensures the recursive solution characteristics of each row and column.

[0103] Among them, the matrix formed by all parking points for all cells to schedule portable base stations is:

[0104]

[0105] In the matrix M, c ij It represents the number of portable base stations that dispatch the i-th base station parking point to the j-th cell to be optimized. A row represents the number of base stations dispatched from one base station parking point to each cell to be optimized, and a column represents the number of base stations dispatched from each base station parking point to a cell to be optimized.

[0106] The training process of multi-agent reinforcement learning consists of multiple episodes, each of which contains several steps, and all agents will make a decision in each step. This application proposal stipulates that multiple agents complete the scheduling process of one of the cells to be optimized in the same step, that is, they are responsible for improving one column of the matrix M in the same step.

[0107] Specifically, the multi-agent proximal policy optimization algorithm (MAPPO) is used to implement the multi-agent reinforcement learning algorithm to solve the multi-objective optimization problem in the portable base station scheduling process. The process of multi-agent multi-objective reinforcement learning is as follows:

[0108] First, consider the multi-agent reinforcement learning process and split the above loss functions into reward functions for each agent. On the one hand, the unclear or difficult-to-calculate loss function can automatically consider the optimization of the goal through interaction with the environment; on the other hand, thanks to the characteristics of MAPPO, the reward function can be more intuitive, updated more frequently, and converge faster than sequential learning. Therefore:

[0109]

[0110] Where i represents a certain row, i is greater than or equal to 1 and less than or equal to m, j represents a certain column, j is greater than or equal to 1 and less than or equal to n, u = 1, 2, ..., m, v = 1, 2, ..., n. All The number of vehicles constrained R c .

[0111] Secondly, considering the multi-objective learning process, we propose to use r d Mainly, b For each step of auxiliary detection, R c For each episode of auxiliary detection, the following reward function is given:

[0112]

[0113] Based on the reward function, each time the reinforcement learning model can give the next scheduling strategy according to the current strategy, that is, predict the most likely result in the x+1 column based on the current result of filling in the xth column of the matrix M. Specifically, when the scheduling has not started, it will start making decisions from the preset strategy s0, and execute a random decision a0 to form the result of filling in the first column, and calculate the reward of decision a0. After that, the x+1th decision a x+1 According to the xth strategy s x and the xth decision ax Calculated and according to strategy s x , Decision a x Complete decision a x Post-formation strategies x+1 . And use MAPPO's Critic network to calculate the strategy s x , Decision a x The evaluation value Q(s x , a x ), which is used to guide the prediction of the next strategy in the Actor network of MAPPO. After each prediction, s x 、a x , the reward r calculated according to formula (1) x Stored in the dispatch experience pool E. It can be understood that here "s" represents the number of emergency communication vehicles that all communities have "obtained", which is accurate to obtaining a certain number of emergency communication vehicles from a certain parking point, which can be understood as the "dispatching plan" at the "current stage"; and "a" represents the number of emergency communication vehicles that a certain community will obtain from other parking points, which is accurate to obtaining a certain number of emergency communication vehicles from a certain parking point, which can be understood as a "decision" for "optimal adjustment" of the current dispatch plan.

[0114] When all agents complete the scheduling of all cells, they give additional feedback for all scheduling records stored in E in this scheduling according to the satisfaction of the current vehicle number constraint:

[0115] r′ x =r x +min{R c}

[0116] Among them, min{R c} represents R c If the vehicle number constraint is satisfied, that is, the number of vehicles dispatched from any base station parking point is less than the number of vehicles parked at the parking point, then min{R c}>0,r′ x It indicates that positive feedback is obtained from the vehicle number constraint; if it is not satisfied, negative feedback is obtained.

[0117] At the same time, each base station parking point is assigned a separate Actor network as each agent, and each agent will give a scheduling of a cell in one step. After an episode, the final reward R' is calculated, and the Critic network is used to conduct a centralized learning of all decisions, that is, to minimize the gap between the current action strategy and the next action strategy:

[0118] During the learning process, the critic network in the MAPPO model will sample from E and select all the schedules s in the sample. x Scheduling plan a x And reward r x Conduct centralized learning. The loss function of the Critic network during centralized learning can be expressed as

[0119]

[0120] Among them, L c (θ) indicates that the current strategy s will be used during learning x 、Current decision a x , next strategy x+1 and the next decision a x+1 ∈ is a constant, and the clip function is used to limit the gradient generated by learning to the range of constants 1-∈ and 1+∈, avoiding the huge impact of gradient explosion and gradient disappearance problems.

[0121] In this implementation, 1. About multi-agent reinforcement learning. Take each base station parking point as an agent, and give the scheduling arrangement of all cells through the distributed decision-making of each agent. On the one hand, through the theory of bounded rationality, each parking point can actively find a solution and determine the most likely solution through probability; on the other hand, try multiple scheduling solutions according to probability to prevent falling into the local optimum, and finally obtain the global optimal solution with a high probability. At the same time, a centralized training is carried out for the arrangement, and a clear judgment is given for each scheduling of each agent. After multiple iterative processes of judgment, Nash equilibrium is reached between each agent, and the overall scheduling plan reaches Pareto optimality. 2. About the reward function. According to the first loss function, the second loss function and the third loss function, fine-tuning is performed according to the characteristics of the multi-agent reinforcement learning loss function, and the scheduling problem is transformed into a multi-step decision problem. The decoupled sub-problems are further decomposed into a multi-step solution process. At the same time, the iterative process of the loss function is decomposed into each scheduling in the form of a reward function, and the next scheduling can be judged according to the current scheduling, which simplifies the design and optimization of the loss function in the iterative process. Finally, the normalized reward function can prevent the reward from increasing abruptly, which would have a devastating impact on the learning process. 3. About step feedback and episode feedback. Taking into account the coordination between multi-objective problems, according to business requirements, the constraints are placed in the step and episode respectively to play a feedback role. Among them, step feedback can provide auxiliary constraint checks for each iteration. If the constraint is not met, the constraint loss continues to be added to the distance loss; if the constraint is met, the constraint reward is added to the distance loss. Episode feedback can be checked after the scheduling is completed, and then a guiding inspection process is provided for an episode, enriching the inspection methods of various constraints.

[0122] Compared with the prior art, this implementation method studies the characteristics of the cell to be optimized and the portable base station, and proposes a multi-agent multi-objective scheduling method. According to the multiple practical problems that need to be considered in the actual scheduling process, the practical problems are transformed into constraints through mathematical modeling methods. At the same time, for the problems that are difficult to consider in large-scale scheduling scenarios, the optimization process is simplified in the form of "trial and error" through a multi-agent reinforcement learning process with centralized learning and distributed execution. Finally, multiple optimization objectives are constructed based on multiple constraints, and the multi-objective and multi-agent are integrated through the reward function Reward Function, so that the model can comprehensively consider the optimization process of multiple objectives and give scheduling results that meet the constraints.

[0123] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of a base station scheduling device provided by an embodiment of the present invention, the device comprising:

[0124] A scheme iteration module 21 is used to perform multiple rounds of iterations on the base station scheduling scheme during the base station scheduling process;

[0125] The base station scheduling module 22 is used to schedule the portable base station from the base station parking point to the cell to be optimized according to the current base station scheduling plan in each round of iteration, and calculate the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan, so as to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and output the final base station scheduling plan; wherein, the base station scheduling plan refers to the plan for scheduling the portable base station from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

[0126] In one implementation, the solution iteration module 21 is specifically configured to:

[0127] Take a base station parking point as an agent and build multiple agents for reinforcement learning;

[0128] Perform multiple rounds of learning for all agents;

[0129] The base station scheduling module 22 is specifically used to: in each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually schedule portable base stations from all the base station parking points for each cell to be optimized;

[0130] A round reward is calculated after each round to adjust the strategy of the agent; wherein the round reward includes a first reward and a second reward, the first reward is calculated based on bandwidth redundancy and base station transportation distance, the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after base station scheduling and the bandwidth requirement of the cell to be optimized, and the base station transportation distance refers to the sum of the transportation distances of all portable base stations scheduled by the cell to be optimized; the second reward is positively correlated with the minimum remaining number of base stations, and the minimum remaining number of base stations is the minimum value of the number of portable base stations remaining in all the base station parking points after the base station scheduling is completed;

[0131] After the reinforcement learning is completed, a final base station scheduling solution is outputted to be used for scheduling the portable base station from the base station parking point for the cell to be optimized.

[0132] In one embodiment, in each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually schedule a portable base station from all the base station parking points for each cell to be optimized, including:

[0133] In a target step of a target round, all agents perform actions according to a strategy to dispatch portable base stations from all base station parking points for a target cell to be optimized; wherein the target round is any one of all rounds, the target step is any one of multiple steps of the target round, and the target cell to be optimized is a cell to be optimized corresponding to the target step;

[0134] Calculating a first reward for the target step, wherein the first reward for the target step is calculated based on the bandwidth redundancy of the target cell to be optimized and the base station transportation distance of the target cell to be optimized;

[0135] The strategy is adjusted according to the first reward, and the next step of the target step is entered until all the cells to be optimized complete base station scheduling, thus ending the target round.

[0136] In one implementation, adjusting the strategy according to the first reward includes: using a Critic network to adjust the strategy according to the first reward.

[0137] In one implementation, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0138] Acquire initial information; the initial information includes all cells to be optimized and their initial bandwidth and bandwidth requirements, all base station parking points and the number of base stations stored therein, and the distance between the cells to be optimized and the base station parking points;

[0139] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0140] Constructing an objective function for the mathematical model, wherein the objective function is to minimize the sum of the transportation distances of the scheduled portable base stations, wherein the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0141] Constructing a quantity constraint condition and a bandwidth constraint condition for the mathematical model, wherein the quantity constraint condition is that the number of portable base stations scheduled in the base station parking point cannot exceed the number of portable base stations originally parked in the base station parking point, and the bandwidth constraint condition is that the bandwidth of the cell to be optimized after base station scheduling is greater than or equal to the bandwidth requirement of the cell to be optimized;

[0142] Under the quantity constraint and the bandwidth constraint, the objective function is solved to obtain a final base station scheduling solution, so as to schedule the portable base station at the base station parking point to the cell to be optimized according to the base station scheduling solution.

[0143] In one implementation, in the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including:

[0144] According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model;

[0145] Constructing a first loss function, a second loss function and a third loss function for the mathematical model; wherein the first loss function is equal to the difference between the parking point base station storage quantity row vector and the parking point base station scheduling quantity row vector of the current base station scheduling scheme, and the parking point base station scheduling quantity row vector represents the number of portable base stations scheduled at each base station parking point; the second loss function is equal to the initial bandwidth row vector plus the bandwidth increment row vector of the current base station scheduling scheme minus the bandwidth demand row vector, and the bandwidth increment row vector represents the bandwidth increment of each cell to be optimized after the base station scheduling; the third loss function is equal to the inverse of the sum of the transportation distances of each scheduled portable base station in the current base station scheduling scheme, and the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled;

[0146] Iterate the base station scheduling scheme, and calculate the first loss function, the second loss function and the third loss function after each round of iteration;

[0147] The base station scheduling scheme of the iteration round that maximizes the total loss function is selected as the final base station scheduling scheme for base station scheduling, wherein the total loss function is positively correlated with the first loss function, the second loss function, and the third loss function, respectively.

[0148] It is worth noting that the working principle of the device provided in the above embodiment can refer to the working process of the base station scheduling method provided in any of the above embodiments, which will not be described in detail here.

[0149] Compared with the prior art, the device provided by the embodiment of the present invention performs multiple rounds of iterations on the base station scheduling scheme during the base station scheduling process. In each round of iteration, the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output; wherein, the base station scheduling scheme refers to a scheme for scheduling portable base stations from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized, which fully considers the bandwidth requirements of each cell to be optimized, the performance of each portable base station and the distance between the portable base station and the cell to be optimized, so that the cell to be optimized can be reasonably allocated to a suitable base station, thereby improving the accuracy of base station allocation.

[0150] See also Figure 3 The embodiment of the present invention further provides a base station scheduling device, including a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, the steps in the above base station scheduling method embodiment are implemented, for example Figure 1 S11 to S12 in the above description; or, the processor 31 implements the functions of the modules in the above-mentioned device embodiments when executing the computer program.

[0151] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the base station scheduling device. For example, the computer program may be divided into multiple modules, each of which is used to execute the specific steps in the method described in any of the above embodiments.

[0152] The base station scheduling device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The base station scheduling device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art may understand that the base station scheduling device may also include input and output devices, network access devices, buses, etc.

[0153] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 31 is the control center of the base station dispatching device, and uses various interfaces and lines to connect various parts of the entire base station dispatching device.

[0154] The memory 32 can be used to store the computer program and / or module. The processor 31 implements various functions of the base station scheduling device by running or executing the computer program and / or module stored in the memory 32 and calling the data stored in the memory 32. The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0155] Wherein, if the module integrated in the base station scheduling device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor 31. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0156] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, and when the computer program / instruction is executed by a processor, the base station scheduling method as described in any of the above embodiments is implemented.

[0157] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A base station scheduling method, characterized in that: include: During the base station scheduling process, multiple rounds of iterations are performed on the base station scheduling plan; In each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling plan, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan are calculated to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling plan is output; wherein the base station scheduling plan refers to a plan for scheduling portable base stations from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

2. The base station scheduling method according to claim 1, characterized in that: In the base station scheduling process, the base station scheduling scheme is iterated for multiple rounds; in each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling scheme, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling scheme are calculated to adjust the base station scheduling scheme and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling scheme is output, including: Take a base station parking point as an agent and build multiple agents for reinforcement learning; Perform multiple rounds of learning for all agents; In each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually dispatch portable base stations from all the base station parking points for each cell to be optimized; A round reward is calculated after each round to adjust the strategy of the agent; wherein the round reward includes a first reward and a second reward, the first reward is calculated based on bandwidth redundancy and base station transportation distance, the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after base station scheduling and the bandwidth requirement of the cell to be optimized, and the base station transportation distance refers to the sum of the transportation distances of all portable base stations scheduled by the cell to be optimized; the second reward is positively correlated with the minimum remaining number of base stations, and the minimum remaining number of base stations is the minimum value of the number of portable base stations remaining in all the base station parking points after the base station scheduling is completed; After the reinforcement learning is completed, a final base station scheduling solution is outputted to be used for scheduling the portable base station from the base station parking point for the cell to be optimized.

3. The base station scheduling method according to claim 2, characterized in that: In each round, all agents gradually perform actions on each cell to be optimized according to the current strategy, so as to gradually schedule portable base stations from all base station parking points for each cell to be optimized, including: In a target step of a target round, all agents perform actions according to a strategy to dispatch portable base stations from all base station parking points for a target cell to be optimized; wherein the target round is any one of all rounds, the target step is any one of multiple steps of the target round, and the target cell to be optimized is a cell to be optimized corresponding to the target step; Calculating a first reward for the target step, wherein the first reward for the target step is calculated based on the bandwidth redundancy of the target cell to be optimized and the base station transportation distance of the target cell to be optimized; The strategy is adjusted according to the first reward, and the next step of the target step is entered until all the cells to be optimized complete base station scheduling, thus ending the target round.

4. The base station scheduling method according to claim 3, characterized in that: The adjusting the strategy according to the first reward includes: adjusting the strategy according to the first reward using a Critic network.

5. The base station scheduling method according to claim 1, characterized in that: In the base station scheduling process, the base station scheduling scheme is iterated multiple times; In each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling plan, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan are calculated to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling plan is output, including: Acquire initial information; the initial information includes all cells to be optimized and their initial bandwidth and bandwidth requirements, all base station parking points and the number of base stations stored therein, and the distance between the cells to be optimized and the base station parking points; According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model; Constructing an objective function for the mathematical model, wherein the objective function is to minimize the sum of the transportation distances of the scheduled portable base stations, wherein the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled; Constructing a quantity constraint condition and a bandwidth constraint condition for the mathematical model, wherein the quantity constraint condition is that the number of portable base stations scheduled in the base station parking point cannot exceed the number of portable base stations originally parked in the base station parking point, and the bandwidth constraint condition is that the bandwidth of the cell to be optimized after base station scheduling is greater than or equal to the bandwidth requirement of the cell to be optimized; Under the quantity constraint and the bandwidth constraint, the objective function is solved to obtain a final base station scheduling solution, so as to schedule the portable base station at the base station parking point to the cell to be optimized according to the base station scheduling solution.

6. The base station scheduling method according to claim 1, characterized in that: In the base station scheduling process, the base station scheduling scheme is iterated multiple times; In each round of iteration, the portable base station is scheduled from the base station parking point to the cell to be optimized according to the current base station scheduling plan, and the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan are calculated to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and the final base station scheduling plan is output, including: According to the initial information, a cell group to be optimized, an initial bandwidth row vector, a bandwidth requirement row vector, a base station parking point group, a parking point base station storage quantity row vector and a distance matrix for characterizing the distance between the cell to be optimized and the base station parking point are constructed as basic information of a mathematical model; Constructing a first loss function, a second loss function and a third loss function for the mathematical model; wherein the first loss function is equal to the difference between the parking point base station storage quantity row vector and the parking point base station scheduling quantity row vector of the current base station scheduling scheme, and the parking point base station scheduling quantity row vector represents the number of portable base stations scheduled at each base station parking point; the second loss function is equal to the initial bandwidth row vector plus the bandwidth increment row vector of the current base station scheduling scheme minus the bandwidth demand row vector, and the bandwidth increment row vector represents the bandwidth increment of each cell to be optimized after the base station scheduling; the third loss function is equal to the inverse of the sum of the transportation distances of each scheduled portable base station in the current base station scheduling scheme, and the transportation distance refers to the distance between the base station parking point where the portable base station is located and the cell to be optimized to which the portable base station is scheduled; Iterate the base station scheduling scheme, and calculate the first loss function, the second loss function and the third loss function after each round of iteration; The base station scheduling scheme of the iteration round that maximizes the total loss function is selected as the final base station scheduling scheme for base station scheduling, wherein the total loss function is positively correlated with the first loss function, the second loss function, and the third loss function, respectively.

7. A base station scheduling device, characterized in that: include: A scheme iteration module is used to perform multiple rounds of iterations on the base station scheduling scheme during the base station scheduling process; The base station scheduling module is used to schedule the portable base station from the base station parking point to the cell to be optimized according to the current base station scheduling plan in each round of iteration, and calculate the base station transportation distance, the remaining number of base stations at the parking point and the bandwidth redundancy of the base station scheduling plan, so as to adjust the base station scheduling plan and enter the next round of iteration until the iteration termination condition is met, and output the final base station scheduling plan; wherein the base station scheduling plan refers to the plan for scheduling the portable base station from each base station parking point to each cell to be optimized, and the bandwidth redundancy refers to the difference between the bandwidth of the cell to be optimized after the base station scheduling and the bandwidth requirement of the cell to be optimized.

8. A base station scheduling device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the base station scheduling method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the base station scheduling method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the base station scheduling method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • A network optimization method and device

    CN109862580A

  • Unmanned aerial vehicle communication base station, communication system and communication system establishment method

    CN110176955A

  • Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning

    CN111786713A

  • Base station planning method and device, electronic equipment and storage medium

    CN116017477A

  • Online car-hailing parking station site selection method, system and equipment based on intelligent agent

    CN117273360A