Communication base station emergency power generation operation scheduling method and system based on reinforcement learning

Abstract

The invention discloses a communication base station emergency power generation operation scheduling method and system based on reinforcement learning, and relates to the technical field of communication base station emergency power generation. Comprising the following steps of: firstly, acquiring transmission flow and topographic feature data of a base station, calculating signal receiving and transmitting power, and calculating real-time power demand by combining the power of a cooling system; immediately, obtaining an equipment data generation state space, including standby power supply reserve and equipment health state, including the electric energy reserve surplus of the standby power supply and the health state of the power generation equipment in the communication base station; the defined action space covers operation and maintenance options of the power generation equipment, and a reward function is designed through power generation cost, power balance and loss. And then, training the reinforcement learning model by using a Q-learning algorithm to form a scheduling scheme. Through an intelligent and automatic scheduling process, efficient and reliable emergency power management is realized, and a solid guarantee is provided for the stability and safety of a communication network.

Description

Technical Field

[0001] The present invention relates to the technical field of emergency power generation for communication base stations, and in particular to a method and system for scheduling emergency power generation operations for communication base stations based on reinforcement learning. Background Art

[0002] In modern communication networks, base stations are key nodes responsible for carrying a large amount of data transmission and user connections. However, when natural disasters, equipment failures, or other force majeure events cause power supply interruptions, the normal operation of communication base stations will be seriously affected. In this case, how to effectively carry out emergency power generation scheduling to ensure the continuity and reliability of communication services has become an important research topic.

[0003] Traditional emergency power management methods for communication base stations usually rely on preset operation manuals and manual decisions, with low response speed and adaptability, and are difficult to meet the needs of dynamically changing environments. This method is not only difficult to adjust the allocation of power resources in real time, resulting in resource waste, but may also cause power outages at critical moments due to human errors. In addition, the experience-based scheduling method cannot fully utilize the real-time data and environmental information of communication base stations for refined management, and lacks effective guidance for the optimal scheduling strategy in complex environments.

[0004] In emergency situations, the signal transmission of base stations may fluctuate due to changes in environmental factors, equipment conditions, etc., so real-time power demand analysis and reasonable power resource scheduling are required to maintain the effective operation of base stations. By comprehensively utilizing multi-dimensional information such as base station transmission flow data, terrain characteristics, equipment health status, and ambient temperature, the reinforcement learning-based method can dynamically adjust the start and stop of power generation equipment and power distribution strategies, thereby improving the efficiency and reliability of emergency power management.

[0005] In the prior art, publication number CN118839991A discloses a communication base station energy management system based on machine learning, including a maintenance optimization unit, a data processing unit, a model training and prediction unit, an energy allocation unit, an energy execution unit and a user interface unit, wherein the maintenance optimization unit, the data processing unit, the model training and prediction unit, the energy allocation unit, the energy execution unit and the user interface unit are connected in sequence, and the maintenance optimization unit is connected to the model training and prediction unit and the energy allocation unit, the user interface unit is connected to the energy allocation unit, and the energy execution unit is connected to the data processing unit. This solution improves the operational efficiency of the communication base station, reduces dependence on traditional energy, and thus promotes sustainable development of the environment. However, since the operating environment and conditions of the communication base station may change over time, such as equipment aging, changes in user needs, environmental factors, etc., the model needs to have sufficient adaptability and robustness to cope with these dynamic changes. If the model is not updated frequently enough or does not have adaptive capabilities, it may lead to poor performance under new or extreme conditions. As a result, the accuracy and effectiveness of the system are reduced.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a communication base station emergency power generation operation scheduling method and system based on reinforcement learning to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for dispatching emergency power generation operations of a communication base station based on reinforcement learning, the specific steps comprising:

[0010] Collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station;

[0011] Collecting equipment data in the communication base station to be monitored, and generating a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining amount of power reserve of the backup power supply and the health status of the power generation equipment in the communication base station;

[0012] defining an action space of a communication base station, wherein the action space of the communication base station includes starting and shutting down a specific power generation equipment, switching to a backup power source, performing equipment maintenance or overhaul, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, a power supply balance index, and equipment loss;

[0013] A reinforcement learning model is established based on the state space, action space and reward function of the communication base station. The reinforcement learning model is trained using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station.

[0014] Based on the obtained communication base station emergency power generation operation scheduling model, the real-time state space data of the communication base station to be monitored is input, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

[0015] Furthermore, the transmission flow data of the communication base station to be monitored in an emergency situation is collected, and the signal receiving power and transmitting power of the communication base station are calculated by combining the transmission flow data of the communication base station with the terrain feature data, wherein the specific formula for calculating the receiving power is:

[0016] P rec (t) = P0 + k u *D dow (t)+ΔP ter

[0017] Where P rec (t) is the received power of the communication base station at time t, P0 is the reference power, which means the power of the communication base station itself when the communication base station is not transmitting data, k u is the power coefficient required per unit flow, D dow (t) is the downlink data flow at time t, ΔP ter Indicates the terrain correction value;

[0018] The formula for calculating the transmit power is:

[0019] P tra (t) = P0 + k u *D up (t)+ΔP ter

[0020] Where P tra (t) is the transmission power of the communication base station at time t, D up (t) is the uplink data flow at time t;

[0021] The terrain correction value ΔP terThe terrain correction value ΔP is calculated based on the terrain feature data, which includes the average height of the terrain around the communication base station, the average height of the surrounding obstacles, and the distance of the signal transmission path. ter The specific formula for the calculation is:

[0022]

[0023] In the formula, H obs is the average height of obstacles, d path is the distance of the signal transmission path, H avg It represents the average height of the terrain around the communication base station, k1 and k2 are the weight coefficients of obstacle obstruction and terrain height obstruction respectively, where k1>k2 and both k1 and k2 are greater than 0.

[0024] Furthermore, the real-time power demand of the communication base station is calculated by combining the signal receiving power, the transmitting power and the power of the cooling system in the communication base station, wherein the formula based on which the real-time power demand of the communication base station is calculated is:

[0025] P total (t) = P rec (t)+P tra (t)+P coll (t)+P aux (t)

[0026] Where P total (t) is the real-time power demand of the communication base station at time t, P coll (t) is the power of the cooling system in the communication base station at time t, P aux (t) is the power requirement of auxiliary equipment in the communication base station at time t, specifically including the power required by routers, switches and monitoring equipment;

[0027] Where the power of the cooling system in the communication base station at time t is P coll (t) The calculation is based on the formula:

[0028] P coll (t) = P base (t)*{1+w1*(T S -T ref )}

[0029] Where P base (t) is the cooling power requirement at the reference temperature at time t, T S is the real-time ambient temperature, T ref is the reference temperature, w1 is the temperature sensitivity constant.

[0030] Furthermore, the data of the equipment in the communication base station to be monitored is collected, and the state space of the communication base station is generated based on the real-time power demand of the communication base station, the data of the equipment in the communication base station, and the average temperature of the environment in which the communication base station is located. The state space includes four parameters: the real-time power demand of the communication base station, the remaining power reserve of the backup power supply, the health status of the power generation equipment in the communication base station, and the average temperature of the environment in which the communication base station is located, which is specifically expressed as:

[0031] S t =[P total (t), E t , G it , T t ]

[0032] G t =[G 1t , G 2t , …, G it , …, G nt ]

[0033] In the formula, S t represents the state vector at time t, P total (t) is the real-time power demand of the communication base station at time t, E t is the remaining power reserve of the backup power supply at time t, G it and T t They are respectively the health status of the i-th power generation equipment in the communication base station at time t and the average temperature of the environment in which the communication base station is located at time t, where i is the index of the power generation equipment, and i∈[1,n], n represents the number of power generation equipment started at time t.

[0034] Furthermore, the health status G of the power generation equipment in the communication base station at time t it It is characterized by the actual output power and actual power generation efficiency of the power generation equipment, and the specific formula is:

[0035]

[0036] In the formula, Z i (t) is the total usage time of the i-th power generation equipment before time t, HI pi (t) is the power generation index of the i-th power generation equipment at time t, HI ei (t) is the power generation efficiency index of the i-th power generation equipment at time t; the formula for calculating the power generation index of the i-th power generation equipment is:

[0037]

[0038] Where P iout (t) is the actual output power of the i-th power generation equipment at time t, Piratde is the rated power of the i-th power generation equipment, where the actual output power of the power generation equipment P iout (t) The calculation is based on the formula:

[0039]

[0040] Where V iout (t) and I iout (t) are the voltage and current at the output end of the i-th power generation equipment at time t, is the power factor at the output;

[0041] Among them, the formula for calculating the power generation efficiency index of the i-th power generation equipment is:

[0042]

[0043] Where P input (t) The input power of the i-th generating equipment at time t, where the input power is calculated based on the formula:

[0044] P input (t) = RZ*XH i (t)*ηη

[0045] In the formula, RZ is the calorific value of fuel, XH i (t) is the fuel consumption rate of the i-th power generation equipment at time t, specifically the amount of fuel consumed per unit time, and ηη is the conversion efficiency.

[0046] Furthermore, a reward function is designed based on the power generation cost of the power generation equipment, the power supply balance index and the equipment loss, wherein the formula for calculating the reward function is:

[0047]

[0048] In the formula, R is the reward function, C i (t) is the power generation cost of the i-th power generation equipment at time t, B is the power supply balance index, L is the equipment loss, where ω1, ω2 and ω3 are the weight coefficients of the power generation cost, power supply balance index and equipment loss of the power generation equipment, respectively, where ω2≥ω1>ω3 and ω1, ω2 and ω3 are all greater than 0;

[0049] The formula for calculating equipment loss L is:

[0050]

[0051] In the formula, n is the number of power generation equipment started;

[0052] The specific formula for the power supply balance index B is:

[0053]

[0054] Where P total (t) is the real-time power demand of the communication base station at time t.

[0055] Furthermore, the reinforcement learning model is trained using the Q-learning algorithm through finite element analysis, and the specific formula is:

[0056]

[0057] In the formula, Q(S t , a t ) indicates state S t Next, perform action a t After that, the cumulative weighted reward, a t Indicates the action to be performed in the current state, S t is the state vector at time t, S t+1 To perform action a t After the state, R t+1 Indicates execution of action a t The immediate reward after is calculated by the reward function, α is the learning rate that controls convergence, and γ is the discount factor, where a t All of them are actions included in the action space of the communication base station;

[0058] According to the above formula, the Q value after performing different actions in different states is continuously iteratively updated based on the reward function, and the above process is repeated until convergence. The training of the reinforcement learning model is completed, and the communication base station emergency power generation operation scheduling model is obtained.

[0059] The present invention also provides a communication base station emergency power generation operation scheduling system based on reinforcement learning, and the communication base station emergency power generation operation scheduling system based on reinforcement learning is used to execute the above-mentioned communication base station emergency power generation operation scheduling method based on reinforcement learning, including:

[0060] The power demand analysis module is used to collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station;

[0061] A state space definition module is used to collect equipment data in the communication base station to be monitored, and generate a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining power reserve of the backup power supply and the health status of the power generation equipment in the communication base station;

[0062] A reward and action space definition module, used to define the action space of the communication base station, wherein the action space of the communication base station includes starting and shutting down specific power generation equipment, switching to backup power supply, performing equipment maintenance or repair, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, the power supply balance index, and the equipment loss;

[0063] The scheduling model training module is used to establish a reinforcement learning model based on the state space, action space and reward function of the communication base station, and train the reinforcement learning model using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station;

[0064] The power generation operation scheduling module is used to input the real-time state space data of the communication base station to be monitored based on the obtained communication base station emergency power generation operation scheduling model, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] This solution solves the problems of slow response, poor adaptability and resource waste in traditional emergency power generation dispatching methods by introducing advanced reinforcement learning algorithms, and significantly improves the response capability of communication base stations in emergencies. By collecting and analyzing the transmission flow data, terrain features, equipment data and other information of communication base stations, the solution can calculate the power demand of base stations in real time, thereby performing power dispatch and avoiding communication interruption caused by insufficient power. By defining clear state space and action space, and designing reward functions based on power generation cost, power supply balance and equipment loss, the reinforcement learning model can learn and optimize dispatching strategies in a variety of complex scenarios. The model can quickly output the optimal dispatching action in different environments and states, improving the speed and accuracy of emergency response. Especially in the case of limited resources, it can reasonably allocate the use of backup power and power generation equipment, effectively reducing energy consumption and equipment loss, thereby extending the service life and service time of the equipment. In addition, the Q-learning algorithm is used for model training, so that the solution can quickly converge to the optimal strategy under the conditions of limited data and computing resources. Therefore, the communication base station emergency power generation operation scheduling method based on reinforcement learning realizes efficient and reliable emergency power management through an intelligent and automated scheduling process, providing a solid guarantee for the stability and security of the communication network. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0068] Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0070] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0071] Example:

[0072] See also Figure 1 , the present invention provides a technical solution:

[0073] A method for dispatching emergency power generation operations of a communication base station based on reinforcement learning, the specific steps comprising:

[0074] Step 1: Collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station.

[0075] The emergency situation specifically refers to a power outage caused by a natural disaster or an emergency.

[0076] Collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, and calculate the signal receiving power and transmitting power of the communication base station. The specific formula for calculating the receiving power is:

[0077] P rec (t) = P0 + k u *D dow (t)+ΔP ter

[0078] Where P rec (t) is the received power of the communication base station at time t, P0 is the reference power, which means the power of the communication base station itself when the communication base station is not transmitting data, k u is the power coefficient required per unit flow, D dow (t) is the downlink data flow at time t. The downlink data flow refers to the data flow transmitted from the base station to the user equipment. ΔP ter Indicates the terrain correction value;

[0079] By analyzing the historical power consumption and traffic data of operators or devices, k can be estimated by regression analysis and other methods. u , 2G / 3G network k u It may be tens to hundreds of milliwatts per bit per second (mW / bit / s). These older network technologies are generally less efficient, especially in poor signal conditions. u Typically in the range of a few milliwatts to tens of milliwatts per bit per second (mW / bit / s). LTE networks are more efficient than earlier technologies, especially under better signal conditions. uIt may be further reduced to less than one milliwatt to several milliwatts per bit per second (mW / bit / s).

[0080] Among them, the reference power P0 represents the power consumption of the communication base station itself when there is no data to be transmitted. This is a basic power consumption level, which is related to the energy consumption of the base station in the idle state. Even if there is no user data traffic, the base station still needs to maintain basic operations, such as signal maintenance, control channel function operation, etc., so it will consume a certain amount of power. This reference power can be regarded as the minimum power required to maintain the basic operation of the base station.

[0081] The acquisition of the reference power P0 usually depends on the type, model, supplier, configuration and operating conditions of the base station. The technical documents or equipment specifications provided by the manufacturer usually list the power consumption characteristics of the base station, including the power consumption in the idle state.

[0082] The formula for calculating the transmit power is:

[0083] P tra (t) = P0 + k u *D up (t)+ΔP ter

[0084] Where P tra (t) is the transmission power of the communication base station at time t, D up (t) is the uplink data traffic at time t, where the uplink data traffic refers to the data traffic transmitted from the user equipment to the base station;

[0085] Among them, the uplink traffic data and downlink traffic data can be obtained by using the network monitoring tools and device logs of the communication base station.

[0086] The terrain correction value ΔP ter The terrain correction value ΔP is calculated based on the terrain feature data, which includes the average height of the terrain around the communication base station, the average height of the surrounding obstacles, and the distance of the signal transmission path. ter The specific formula for the calculation is:

[0087]

[0088] In the formula, H obs is the average height of obstacles, d path is the distance of the signal transmission path, H avg It represents the average height of the terrain around the communication base station, k1 and k2 are the weight coefficients of obstacle obstruction and terrain height obstruction respectively, where k1>k2 and both k1 and k2 are greater than 0.

[0089] In wireless signal propagation, obstacles are the main attenuation factor. Tall buildings, trees or other obstacles can significantly hinder signal propagation, so the average height of obstacles H is added to the formula. obs The logarithmic term of , to reflect the nonlinear effect of obstacles on the signal;

[0090] The average height of the terrain around the base station H avg The scattering and reflection paths that affect the signal, especially in uneven areas, usually increase with increasing distance, so Used to represent the squared effect of distance.

[0091] The height of obstacles has a significant effect on signal attenuation. obs Will increase ΔP ter The signal needs a larger compensation power, and the distance d of the signal transmission path path The square of H is used in the formula to emphasize the significant attenuation of signal strength at longer distances. avg The influence of the overall characteristics of the terrain on signal propagation is considered, especially in areas with complex terrain. Since obstacles have a greater and more direct impact on the signal, k1>k2 is set and both k1 and k2 are greater than 0.

[0092] Step 2: Collect equipment data in the communication base station to be monitored, and generate a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining energy reserve of the backup power supply and the health status of the power generation equipment in the communication base station.

[0093] The real-time power demand of the communication base station is calculated by combining the signal receiving power, the transmitting power and the power of the cooling system in the communication base station. The formula for the real-time power demand of the communication base station is:

[0094] P total (t) = P rec (t)+P tra (t)+P coll (t)+P aux (t)

[0095] Where P total (t) is the real-time power demand of the communication base station at time t, P coll (t) is the power of the cooling system in the communication base station at time t, P aux(t) is the power requirement of auxiliary equipment in the communication base station at time t, including the power required by routers, switches and monitoring equipment. The nominal power consumption of each device is obtained from the equipment specification sheet provided by the manufacturer. These documents usually list the typical power consumption range of the equipment. The nominal power consumption of all devices is summed up to obtain the total power requirement of auxiliary equipment of the base station under normal circumstances;

[0096] Where the power of the cooling system in the communication base station at time t is P coll (t) The calculation is based on the formula:

[0097] P coll (t) = P base (t)*{1+w1*(T S -T ref )}

[0098] Where P base (t) is the cooling power requirement at the reference temperature at time t, T S is the real-time ambient temperature, T ref is the reference temperature, w1 is the temperature sensitivity constant.

[0099] The reference temperature T ref Generally 25℃, P base (t) is the cooling power requirement at the reference temperature at time t. The rated power at the reference temperature is obtained from the technical specifications of the air-conditioning equipment or refrigeration system. The manufacturer may provide performance data of the equipment under different environmental conditions, including the power requirement under standard conditions. The general range is from a few kilowatts (kW) to tens of kilowatts.

[0100] Collect the equipment data in the communication base station to be monitored, and generate the state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station and the average temperature of the environment in which the communication base station is located. The state space includes four parameters: the real-time power demand of the communication base station, the remaining power reserve of the backup power supply, the health status of the power generation equipment in the communication base station and the average temperature of the environment in which the communication base station is located. It is specifically expressed as follows:

[0101] S t =[P total (t), E t , G t , T t ]

[0102] G t =[G 1t , G 2t , …, G it , …, G nt ]

[0103] In the formula, S trepresents the state vector at time t, P total (t) is the real-time power demand of the communication base station at time t, E t is the remaining power reserve of the backup power supply at time t, G t and T t They are respectively the health status of the i-th power generation equipment in the communication base station at time t and the average temperature of the environment in which the communication base station is located at time t, where i is the index of the power generation equipment, and i∈[1,n], n represents the number of power generation equipment started at time t.

[0104] The health status G of the power generation equipment in the communication base station at time t it It is characterized by the actual output power and actual power generation efficiency of the power generation equipment, and the specific formula is:

[0105]

[0106] In the formula, Z i (t) is the total usage time of the i-th power generation equipment before time t, HI pi (t) is the power generation index of the i-th power generation equipment at time t, HI ei (t) is the power generation efficiency index of the i-th power generation equipment at time t; record the time each time the equipment is started and shut down, and accumulate these times regularly to obtain the total usage time. Record the operating time during equipment maintenance or inspection, and the accumulated time can usually be found in the maintenance log.

[0107] The formula for calculating the power generation index of the i-th power generation equipment is:

[0108]

[0109] Where P iout (t) is the actual output power of the i-th power generation equipment at time t, P iratde is the rated power of the i-th generating equipment. The nameplate of the generating equipment usually contains the technical specifications of the equipment, including the rated power. This is the most direct and authoritative source. In addition to the nameplate, there may be other identification labels or plates on the equipment that list the rated power.

[0110] The actual output power of the power generation equipment is P iout The calculation is based on the formula:

[0111]

[0112] Where V iout (t) and I iout (t) are the voltage and current at the output end of the i-th power generation equipment at time t, To measure the power factor at the output, modern power generation equipment is usually equipped with a built-in power factor meter or display that can directly read the power factor.

[0113] Among them, the formula for calculating the power generation efficiency index of the i-th power generation equipment is:

[0114]

[0115] Where P input is the input power of the i-th power generation equipment at time t, where the formula for calculating the input power is:

[0116] P input (t) = RZ*XH i (t)*η

[0117] In the formula, RZ is the calorific value of fuel, XH i (t) is the fuel consumption rate of the i-th power generation equipment at time t, specifically the amount of fuel consumed per unit time, and η is the conversion efficiency. The fuel calorific value will be affected by the ambient humidity, resulting in a decrease in the fuel calorific value. The specific correction formula is:

[0118]

[0119] Where RZ0 is the initial calorific value of the fuel. Fuel suppliers usually provide technical specifications of the fuel, including its calorific value. This information can be found in the purchase contract or product manual. s is the real-time ambient humidity, HR ref Reference ambient humidity, generally 50%.

[0120] Step 3: Define the action space of the communication base station, which includes starting and shutting down specific power generation equipment, switching to backup power, performing equipment maintenance or overhaul, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, the power supply balance index, and the equipment loss.

[0121] The reward function is designed based on the power generation cost of the power generation equipment, the power supply balance index and the equipment loss. The formula for calculating the reward function is:

[0122]

[0123] In the formula, R is the reward function, C i(t) is the power generation cost of the i-th power generation equipment at time t, B is the power supply balance index, L is the equipment loss, where ω1, ω2 and ω3 are the weight coefficients of the power generation cost, power supply balance index and equipment loss of the power generation equipment, respectively, where ω2≥ω1>ω3 and ω1, ω2 and ω3 are all greater than 0, and the power generation cost C is calculated by the fuel consumption rate XH i (t) is calculated, and the specific formula is:

[0124] C i (t) = XH i (t)*P f

[0125] Where P f The unit price of fuel.

[0126] The formula for calculating equipment loss L is:

[0127]

[0128] In the formula, n is the number of power generation equipment started;

[0129] The specific formula for the power supply balance index B is:

[0130]

[0131] Where P total (t) is the real-time power demand of the communication base station at time t.

[0132] Through the exponential function e -C To measure the impact of power generation costs, lower costs will significantly increase the reward value. The use of exponential functions shows that the negative impact of cost increases on rewards is nonlinear, and a small increase in cost will lead to a significant reduction in rewards; the reward function emphasizes the importance of reducing costs. Cost is a key factor in the operation of power systems, and reducing costs can improve economic benefits. Therefore, the power generation cost C is inversely proportional to the reward function R.

[0133] The power supply balance index B highlights the stability and reliability of power supply. The balance of power supply directly affects the safety of the power grid. The logarithmic function ln(1+B) is used to represent the gain of power supply balance. The logarithmic function is suitable for describing the effect of diminishing returns. The initial small improvement has a greater impact on the reward, while the incremental benefits of subsequent improvements gradually decrease. Therefore, the power supply balance index B is proportional to the reward function R.

[0134] Loss is a factor that cannot be ignored in the operation of all equipment. 2, indicating that the negative impact of loss on reward is accelerated. That is, the weakening effect of increased loss on reward is aggravated, and the reward function is more sensitive to increased loss, which encourages reducing equipment loss to extend equipment life and improve efficiency. Therefore, equipment loss L is inversely proportional to the reward function R.

[0135] Step 4: A reinforcement learning model is established based on the state space, action space and reward function of the communication base station. The reinforcement learning model is trained using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station.

[0136] The reinforcement learning model is trained using the Q-learning algorithm through finite element analysis. The specific formula is:

[0137]

[0138] In the formula, Q(S t , a t ) indicates state S t Next, perform action a t After that, the cumulative weighted reward, a t Indicates the action to be performed in the current state, S t is the state vector at time t, S t+1 To perform action a t After the state, R t+1 Indicates execution of action a t The immediate reward after is calculated by the reward function, α is the learning rate that controls convergence, and γ is the discount factor, where a t All of them are actions included in the action space of the communication base station;

[0139] The specific training steps include: creating a Q table, the dimension of which is the state space multiplied by the action space, the Q values ​​in the initial table are all 0, setting learning parameters: determining the learning rate α and the discount factor γ, where the learning rate α ranges from (0, 1), the discount factor γ ranges from [0, 1), γ = 0 represents immediate reward, and γ tends to 1 represents future reward. Use the Q-learning algorithm for training: obtain the initial state S0 from the environment, and according to the above formula, continuously iteratively update the Q value after performing different actions in different states based on the reward function, repeat the above process until convergence, complete the training of the reinforcement learning model, and obtain the communication base station emergency power generation operation scheduling model. Since the Q-learning algorithm is a mature technical means, it will not be described here.

[0140] Step 5: Based on the obtained communication base station emergency power generation operation scheduling model, the real-time state space data of the communication base station to be monitored is input, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

[0141] Acquire the status information of the communication base station in real time, including power demand, battery power, weather conditions, current power generation equipment status, etc., to ensure data accuracy and timeliness, format the acquired real-time data into a state compatible with the model, find the Q value of all possible actions in the trained Q table according to the current state, select the action with the highest Q value, and execute the selected optimal action in the communication base station.

[0142] See also Figure 2 The present invention also provides a communication base station emergency power generation operation scheduling system based on reinforcement learning, and the communication base station emergency power generation operation scheduling system based on reinforcement learning is used to execute the above-mentioned communication base station emergency power generation operation scheduling method based on reinforcement learning, including:

[0143] The power demand analysis module is used to collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station;

[0144] A state space definition module is used to collect equipment data in the communication base station to be monitored, and generate a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining power reserve of the backup power supply and the health status of the power generation equipment in the communication base station;

[0145] A reward and action space definition module, used to define the action space of the communication base station, wherein the action space of the communication base station includes starting and shutting down specific power generation equipment, switching to backup power supply, performing equipment maintenance or repair, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, the power supply balance index, and the equipment loss;

[0146] The scheduling model training module is used to establish a reinforcement learning model based on the state space, action space and reward function of the communication base station, and train the reinforcement learning model using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station;

[0147] The power generation operation scheduling module is used to input the real-time state space data of the communication base station to be monitored based on the obtained communication base station emergency power generation operation scheduling model, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

[0148] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0149] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A communication base station emergency power generation operation scheduling method based on reinforcement learning, characterized in that: The specific steps include: Collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station; Collecting equipment data in the communication base station to be monitored, and generating a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining amount of power reserve of the backup power supply and the health status of the power generation equipment in the communication base station; defining an action space of a communication base station, wherein the action space of the communication base station includes starting and shutting down a specific power generation equipment, switching to a backup power source, performing equipment maintenance or overhaul, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, a power supply balance index, and equipment loss; A reinforcement learning model is established based on the state space, action space and reward function of the communication base station. The reinforcement learning model is trained using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station. Based on the obtained communication base station emergency power generation operation scheduling model, the real-time state space data of the communication base station to be monitored is input, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

2. According to the reinforcement learning-based communication base station emergency power generation operation scheduling method of claim 1, it is characterized by: Collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, and calculate the signal receiving power and transmitting power of the communication base station. The specific formula for calculating the receiving power is: P rec (t)=P0+k u *D dow (t)+ΔP ter Where P rec (t) is the received power of the communication base station at time t, P0 is the reference power, which means the power of the communication base station when it is not transmitting data, k u is the power coefficient required per unit flow, D dow (t) is the downlink data flow at time t, ΔP ter Indicates the terrain correction value; The formula for calculating the transmit power is: P tra (t)=P0+k u *D up (t)+ΔP ter Where P tra (t) is the transmission power of the communication base station at time t, D up (t) is the uplink data flow at time t; The terrain correction value ΔP ter The terrain correction value ΔP is calculated based on the terrain feature data, which includes the average height of the terrain around the communication base station, the average height of the surrounding obstacles, and the distance of the signal transmission path. ter The specific formula for the calculation is: In the formula, H obs is the average height of obstacles, d path is the distance of the signal transmission path, H avg It represents the average height of the terrain around the communication base station, k1 and k2 are the weight coefficients of obstacle obstruction and terrain height obstruction respectively, where k1>k2 and both k1 and k2 are greater than 0.

3. According to claim 2, a communication base station emergency power generation operation scheduling method based on reinforcement learning is characterized in that: The real-time power demand of the communication base station is calculated by combining the signal receiving power, the transmitting power and the power of the cooling system in the communication base station. The formula for the real-time power demand of the communication base station is: P total (t)=P rec (t)+P tra (t)+P coll (t)+P aux (t) Where P total (t) is the real-time power demand of the communication base station at time t, P coll (t) is the power of the cooling system in the communication base station at time t, P aux (t) is the power requirement of auxiliary equipment in the communication base station at time t, specifically including the power required by routers, switches and monitoring equipment; Where the power of the cooling system in the communication base station at time t is P coll (t) The calculation is based on the formula: P coll (t)=P base (t)*{1+w1*(T S -T ref )} Where P base (t) is the cooling power requirement at the reference temperature at time t, T S is the real-time ambient temperature, T ref is the reference temperature, w1 is the temperature sensitivity constant.

4. According to a method for dispatching emergency power generation operations of communication base stations based on reinforcement learning as described in claim 1, it is characterized by: Collect the equipment data in the communication base station to be monitored, and generate the state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station and the average temperature of the environment in which the communication base station is located. The state space includes four parameters: the real-time power demand of the communication base station, the remaining power reserve of the backup power supply, the health status of the power generation equipment in the communication base station and the average temperature of the environment in which the communication base station is located. It is specifically expressed as follows: S t =[P total (t),E t ,G t ,T t ] G t =[G 1t ,G 2t ,…,G it ,…,G nt ] In the formula, S t represents the state vector at time t, P total (t) is the real-time power demand of the communication base station at time t, E t is the remaining power reserve of the backup power supply at time t, G t and T t are respectively the health status of the i-th power generation equipment in the communication base station at time t and the average temperature of the environment in which the communication base station is located at time t, where i is the index of the power generation equipment, and i∈[1,n], n represents the number of power generation equipment started at time t.

5. According to claim 4, a communication base station emergency power generation operation scheduling method based on reinforcement learning is characterized in that: The health status G of the power generation equipment in the communication base station at time t it It is characterized by the actual output power and actual power generation efficiency of the power generation equipment, and the specific formula is: In the formula, Z i (t) is the total usage time of the i-th power generation equipment before time t, HI pi (t) is the power generation index of the i-th power generation equipment at time t, HI ei (t) is the power generation efficiency index of the i-th power generation equipment at time t; the formula for calculating the power generation index of the i-th power generation equipment at time t is: Where P iout (t) is the actual output power of the i-th power generation equipment at time t, P iratde is the rated power of the i-th power generation equipment, where the actual output power of the power generation equipment P iout (t) The calculation is based on the formula: Where V iout (t) and I iout (t) are the voltage and current at the output end of the i-th power generation equipment at time t, is the power factor at the output; Among them, the formula for calculating the power generation efficiency index of the i-th power generation equipment is: Where P input (t) is the input power of the i-th power generation equipment at time t, where the formula for calculating the input power is: P input (t)=RZ*XH i (t)*η In the formula, RZ is the calorific value of the fuel, XH(t) is the fuel consumption rate of the i-th power generation equipment at time t, specifically the amount of fuel consumed per unit time, and η is the conversion efficiency.

6. The method for dispatching emergency power generation operations of a communication base station based on reinforcement learning according to claim 5 is characterized in that: The reward function is designed based on the power generation cost of the power generation equipment, the power supply balance index and the equipment loss. The formula for calculating the reward function is: In the formula, R is the reward function, C i (t) is the power generation cost of the i-th power generation equipment at time t, B is the power supply balance index, L is the equipment loss, where ω1, ω2 and ω3 are the weight coefficients of the power generation cost, power supply balance index and equipment loss of the power generation equipment, respectively, where ω2≥ω1>ω3 and ω1, ω2 and ω3 are all greater than 0; The formula for calculating equipment loss L is: The specific formula for the power supply balance index B is: Where P total (t) is the real-time power demand of the communication base station at time t.

7. The method for dispatching emergency power generation operations of a communication base station based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning model is trained using the Q-learning algorithm through finite element analysis. The specific formula is: In the formula, Q(S t , a t ) indicates state S t Next, perform action a t After that, the cumulative weighted reward, a t Indicates the action to be performed in the current state, S t is the state vector at time t, S t+1 To perform action a t After the state, R t+1 Indicates execution of action a t The immediate reward after is calculated by the reward function, α is the learning rate that controls convergence, and γ is the discount factor, where a t All of them are actions included in the action space of the communication base station; According to the above formula, the Q value after performing different actions in different states is continuously iteratively updated based on the reward function, and the above process is repeated until convergence. The training of the reinforcement learning model is completed, and the communication base station emergency power generation operation scheduling model is obtained.

8. A communication base station emergency power generation operation dispatching system based on reinforcement learning, characterized in that: The communication base station emergency power generation operation scheduling system based on reinforcement learning is used to execute the communication base station emergency power generation operation scheduling method based on reinforcement learning according to any one of claims 1 to 7, including: The power demand analysis module is used to collect the transmission flow data of the communication base station to be monitored in an emergency situation, combine the transmission flow data of the communication base station with the terrain feature data, calculate the signal receiving power and transmitting power of the communication base station, and combine the signal receiving power, transmitting power and the power of the cooling system in the communication base station to calculate the real-time power demand of the communication base station; A state space definition module is used to collect equipment data in the communication base station to be monitored, and generate a state space of the communication base station based on the real-time power demand of the communication base station, the equipment data in the communication base station, and the average temperature of the environment in which the communication base station is located, wherein the equipment data includes the remaining power reserve of the backup power supply and the health status of the power generation equipment in the communication base station; A reward and action space definition module, used to define the action space of the communication base station, wherein the action space of the communication base station includes starting and shutting down specific power generation equipment, switching to backup power supply, performing equipment maintenance or repair, and adjusting the input power of the power generation equipment, and designing a reward function based on the power generation cost of the power generation equipment, the power supply balance index, and the equipment loss; The scheduling model training module is used to establish a reinforcement learning model based on the state space, action space and reward function of the communication base station, and train the reinforcement learning model using the Q-learning algorithm through finite element analysis to obtain the emergency power generation operation scheduling model of the communication base station; The power generation operation scheduling module is used to input the real-time state space data of the communication base station to be monitored based on the obtained communication base station emergency power generation operation scheduling model, and the model outputs the optimal action in the action space under the current state space for execution, wherein the optimal action in the action space is the specific action with the largest expected return in the action space under the current state space.

Images