Energy scheduling method and system for railway station

Abstract

The application provides an energy scheduling method and system for a railway station, based on a multi-source complementary power supply system constructed by the railway station, constructs environment parameters, electricity price state parameters, energy equipment state parameters and user behavior parameters as a state space, constructs multiple types of power generation equipment control parameters, energy storage equipment control parameters, power buying and selling control parameters and load equipment control parameters as an action space, selects control actions based on deep reinforcement learning (DRL), and realizes dynamic scheduling and management schemes for multiple power supply equipment, energy storage equipment and load equipment in the system. The cost saving amount, energy utilization efficiency, user satisfaction, equipment wear degree and environmental impact degree are introduced to calculate the reward value, so that the cost can be reduced and the efficiency can be improved in the control optimization process, the user satisfaction can be improved, the equipment wear and carbon emission can be reduced, and automatic, efficient and stable energy scheduling can be realized.

Description

Technical Field

[0001] This invention relates to the field of energy dispatching technology, and in particular to an energy dispatching method and system for railway stations. Background Technology

[0002] To meet the demands of cost reduction, efficiency improvement, and carbon emission reduction, railway stations are transitioning from a single-energy model entirely reliant on the power grid to a diversified energy model that integrates multi-source power supply equipment with the power grid, relying on the environment. This comprehensive decarbonization faces new requirements and challenges, making a high proportion of renewable energy generation an inevitable trend. Multi-source power supply equipment primarily utilizes renewable energy sources for power generation, such as photovoltaic panels, wind turbines, and combined heat and power (CHP) equipment. However, renewable energy power generation is highly dependent on the external natural environment, exhibiting instability, unpredictability, and uncontrollability, severely impacting the stable operation of railway stations. To ensure the stable operation of railway stations and other infrastructure, complex scheduling of multi-source power supply equipment, energy storage devices, load devices, and the external power network within the railway station's power system is required. Therefore, a scheduling scheme based on multi-energy complementarity is urgently needed. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an energy dispatching method and system for railway stations to eliminate or improve one or more defects in the prior art and solve the problem that the prior art cannot efficiently dispatch the multi-source power system of railway stations to ensure stable operation.

[0004] One aspect of the present invention provides an energy dispatching method for railway stations, the method being applicable to a multi-source complementary power system, the multi-source complementary power system comprising multiple grid-connected power generation devices, energy storage devices, and load devices, the method comprising the following steps:

[0005] A state space is constructed, comprising environmental parameters of the target railway station, electricity price parameters, energy equipment parameters, and user behavior parameters. The environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. The electricity price parameters include electricity purchase and sales prices. The energy equipment parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charge / discharge status of energy storage devices, the energy demand of load equipment, equipment health status parameters, and carbon emissions. The user behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters.

[0006] Construct an action space, which includes control parameters for multiple types of power generation equipment, energy storage equipment, electricity trading control parameters, and load equipment control parameters.

[0007] Based on deep reinforcement learning, a main network is constructed, comprising a main policy network and a main evaluation network. The main policy network takes the parameters of the state space at the current moment as input and outputs the control action selected in the action space. A reward value is calculated by combining cost savings, energy utilization efficiency, user satisfaction, equipment wear and tear, and environmental impact. A value function based on the selected action in the state is established according to the reward value. The main evaluation network fits the value function to obtain the main Q value. An experience replay buffer is constructed to store the data generated by each interaction for random sampling training to update the main network. A target network with the same structure as the main policy network and the main evaluation network is introduced. The mean square error of the target Q value obtained by fitting the value function to the main Q value and the main Q value is calculated as a loss function. The parameters of the main network are updated by minimizing the loss function, and the parameters of the main network are periodically copied to the target network.

[0008] The main network selects control actions to achieve energy scheduling of the target railway station and continuously updates and optimizes the main network.

[0009] In some embodiments, the external natural environment parameters include meteorological information, geological information, and time information of the area where the target railway station is located; the meteorological information includes light intensity, wind speed, temperature, and humidity; the time information includes date, season, and current time.

[0010] The power generation equipment includes: photovoltaic panels, wind turbines, and a combined heat and power (CHP) subsystem;

[0011] The device usage preference parameters are quantified by marking the load selected by the user, the frequency of use of the load device, and the duration of load operation;

[0012] The user satisfaction feedback parameters include user satisfaction scores fed back through a preset link, quantified adjustment suggestions, the usage frequency of the load device, and the recommendation index of the load device;

[0013] The power generation equipment control parameters include start-stop control parameters and operating power control parameters for the power generation equipment.

[0014] The energy storage device control parameters include scheduling parameters for multiple energy storage devices and charging / discharging power control parameters;

[0015] The power trading control parameters include the sales scheduling parameters for the power output of the power generation equipment and the power stored in the energy storage equipment, as well as the external power procurement control parameters.

[0016] The load device control parameters include start / stop control parameters and operating status control parameters for each load device;

[0017] The projected power generation is predicted using an ANN neural network.

[0018] In some embodiments, the main policy network and the main valuation network employ fully connected networks, convolutional neural networks, or recurrent neural networks, and introduce ReLU or sigmoid activation functions.

[0019] In some embodiments, the reward value is calculated by combining cost savings, energy efficiency, user satisfaction, equipment wear and tear, and environmental impact, including:

[0020] The procurement cost of the multi-source complementary power system is calculated using the following formula:

[0021] R cost_saving =-(Price) purchased ×Power purchased );

[0022] Among them, R cost_saving Price represents the procurement cost. purchased Indicates the purchase price of electricity, Power purchased Indicates the amount of electricity purchased;

[0023] The sales revenue of the multi-source complementary power system is calculated using the following formula:

[0024] R sales_gain =Price sold ×Power sold ;

[0025] Among them, R sales_gain Price represents the sales revenue. sold Indicates the sales price of electricity, Power sold Indicates the volume of electricity sold;

[0026] Obtain the energy utilization rate of the multi-source complementary power system at each moment, and calculate the change in energy utilization rate at the current moment. The calculation formula is as follows:

[0027] R efficiency =Efficiency current Efficiency previous ;

[0028] Among them, R efficiency This represents the change in energy efficiency. current Efficiency represents the current energy utilization rate. previous This indicates the energy utilization rate at the previous moment;

[0029] The health status of the multi-source complementary power system at each moment is obtained to characterize the wear and tear of the equipment. The change in the system health status at the current moment is calculated using the following formula:

[0030] R health =Health current -Health previous ;

[0031] Among them, R health Health represents the change in the system's health status. current This indicates the current health status of the multi-source complementary power system. previous This indicates the health status of the multi-source complementary power system at the previous moment; the health status is calibrated using the remaining service life or failure rate of each device.

[0032] The reward value is calculated using the following formula:

[0033] R t =ω1(R cost_saving +R sales_gain )+ω2·R efficiency +ω3·R satisfaction +ω4·R halth +ω5·R environmental ;

[0034] R environmental =-Emissions current ;

[0035] Among them, R t R represents the reward value at time t. satisfaction For the quantified satisfaction score, R environmental The environmental impact value is Emissions current ω1, ω2, ω3, ω4 and ω5 are the carbon emissions of the multi-source complementary power system.

[0036] In some embodiments, the master valuation network fits the value function to obtain the master Q value, expressed as:

[0037] Q(s,a,θ)=E[R t +γmax a′ Q(s′,a′,θ)∣s,a];

[0038] Where Q(s,a,θ) represents the principal Q value, E represents the expectation, and R tLet represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ represent the parameters of the main valuation network.

[0039] The target Q value obtained by fitting the value function to the target network is expressed as follows:

[0040] y = R t +γmax a′ Q target (s′,a′,θ′);

[0041] Where y represents the target Q value, R t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ′ represent the parameters of the target network.

[0042] In some embodiments, the expression for the loss function is:

[0043] L(θ) = E[(yQ(s,a,θ)) 2 ]

[0044] Where L(θ) represents the loss function, E represents the expectation, y represents the target Q value, and Q(s,a,θ) represents the principal Q value.

[0045] In some embodiments, the master strategy network and the master valuation network are pre-trained based on historical state space and historical action space constructed from historical data, and then transferred to the operation and continuous updating of the multi-source complementary power system.

[0046] On the other hand, the present invention also provides a multi-source complementary power system suitable for railway stations, the system comprising:

[0047] The multi-source power generation electronic system includes photovoltaic panels, wind turbines, and a combined heat and power subsystem, and the multi-source power generation electronic system is connected to the external power grid.

[0048] An energy storage subsystem is connected to the multi-source power generation system and the external power grid.

[0049] Multiple load devices are connected to the multi-source electronic system, the energy storage subsystem, and the external power grid;

[0050] The equipment management subsystem is used to execute the energy dispatching method for railway stations described above. Based on environmental parameters, electricity price status parameters, energy equipment status parameters, and user behavior parameters, it selects and executes control actions to achieve energy dispatching for the target railway station. The environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. The electricity price status parameters include electricity purchase and sales prices. The energy equipment status parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charge / discharge status of energy storage equipment, the energy demand of load equipment, equipment health status parameters, and carbon emissions. The user behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters.

[0051] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0052] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0053] The beneficial effects of the present invention are at least as follows:

[0054] The energy dispatching method and system for railway stations described in this invention are based on the multi-source complementary power supply system constructed in the railway station. Environmental parameters, electricity price status parameters, energy equipment status parameters, and user behavior parameters are constructed into a state space. Control parameters for various types of power generation equipment, energy storage equipment, electricity trading, and load equipment are constructed into an action space. Deep reinforcement learning (DRL) is used to select control actions, enabling dynamic dispatching and management of various power supply equipment, energy storage equipment, and load equipment in the system. Reward values ​​are calculated based on cost savings, energy utilization efficiency, user satisfaction, equipment wear and tear, and environmental impact to ensure cost reduction and efficiency improvement, increased user satisfaction, reduced equipment wear and carbon emissions, and automated, efficient, and stable energy dispatching during control optimization.

[0055] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0056] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0057] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0058] Figure 1 This is a schematic flowchart of an energy dispatching method for railway stations according to an embodiment of the present invention.

[0059] Figure 2 This is a graph showing the changes in photovoltaic output, wind power output, charging power, and discharge power during the execution of the energy dispatching method for railway stations according to another embodiment of the present invention.

[0060] Figure 3 This is a graph showing the change in battery energy ratio during the execution of the energy dispatching method for railway stations according to another embodiment of the present invention.

[0061] Figure 4 This is a graph showing the change in user power demand during the execution of the energy dispatching method for railway stations according to another embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0063] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0064] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0065] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0066] This invention applies Deep Reinforcement Learning (DRL) to the Multi-Energy Complementary System (MECS) of the Intelligent Building Energy Management System for Railway Stations. It mainly optimizes the system's operating strategy by learning the interaction between the learning environment and the agent, thereby achieving higher energy utilization efficiency, lower costs, and better reliability.

[0067] This invention utilizes Dynamic Dispatch Logic (DRL) to achieve intelligent scheduling of energy storage systems, ensuring the rational allocation and use of energy across different time periods. In multi-energy complementary systems, various energy sources (such as solar, wind, and natural gas) and equipment (such as energy storage devices and combined heat and power systems) exist. Dynamic scheduling and optimization of these energy sources and equipment improve the overall system performance. Adapting to changes in environmental conditions (such as weather variations) and user demands, manual or static optimization is applied. The DRL learns the patterns of environmental change and automatically adjusts its strategies to achieve dynamic optimization, which is then used to realize intelligent control of buildings or power grids. By training the DRL model, it can automatically adjust the operating status of various devices within a building based on real-time energy supply and demand, achieving both comfort requirements and energy savings.

[0068] One aspect of the present invention provides an energy dispatching method for railway stations, the method being applicable to multi-source complementary power systems, the multi-source complementary power systems comprising various grid-connected power generation devices, energy storage devices, and load devices, such as... Figure 1 As shown, the method includes the following steps S101 to S104:

[0069] Step S101: Construct the state space, which includes environmental parameters of the target railway station, electricity price parameters, energy equipment parameters, and user behavior parameters. Environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. Electricity price parameters include electricity purchase and sales prices. Energy equipment parameters include current and expected power generation of various types of power generation equipment, remaining power and charge / discharge status of energy storage equipment, energy demand of load equipment, equipment health status parameters, and carbon emissions. User behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters.

[0070] Step S102: Construct the action space, which includes control parameters for various types of power generation equipment, energy storage equipment, electricity trading, and load equipment.

[0071] Step S103: Based on deep reinforcement learning, a main network is constructed, comprising a main policy network and a main evaluation network. The main policy network takes the parameters of the current state space as input and outputs the control action selected in the action space. A reward value is calculated by combining cost savings, energy utilization efficiency, user satisfaction, equipment wear and tear, and environmental impact. A value function based on state-selected action is established according to the reward value. The main evaluation network fits the value function to obtain the main Q value. An experience replay buffer is constructed to store the data generated by each interaction for random sampling training to update the main network. A target network with the same structure as the main policy network and the main evaluation network is introduced. The mean square error of the target Q value and the main Q value obtained by fitting the value function of the target network is calculated as the loss function. The parameters of the main network are updated by minimizing the loss function, and the parameters of the main network are periodically copied to the target network.

[0072] Step S104: Execute the control actions selected by the main network to realize the energy scheduling of the target railway station, and continuously update and optimize the main network.

[0073] The methods described in steps S101-S104 target railway stations that utilize a multi-source complementary power system, employing various energy sources such as solar, wind, and geothermal energy to generate electricity for auxiliary power supply. However, electricity generated from natural resources suffers from instability. To ensure stable power supply, the power generation equipment at railway stations must be connected to the national power grid. Simultaneously, energy storage devices are deployed within the station, coordinating power generation, storage, power dispatch, and load control to achieve efficient, stable, environmentally friendly, and low-carbon control. Deep reinforcement learning is used to establish a mapping relationship between environmental states and overall system control parameters. Environmental modeling involves defining the state space, action space, and reward function of the environment, which are the foundation for the DRL algorithm's learning and optimization. Configuring an appropriate environmental model based on the application scenario can capture the dynamic characteristics of environmental changes and support agents learning effective strategies through interaction with the environment.

[0074] The state space S describes the set of variables representing the current state of the environment. Examples include current time, weather conditions, the status of various energy devices (such as the power generation of photovoltaic panels, the remaining power of energy storage systems), and electricity prices. The action space A represents all possible operations that the agent can take. Examples include charging and discharging operations of energy storage systems, starting / stopping renewable energy devices, and load management. The reward function R defines the reward or penalty for the agent taking a certain action. The design of the reward function should be based on the specific optimization objective, such as energy saving, cost reduction, and user satisfaction.

[0075] In step S101, based on the multi-source complementary power system of the railway station, a state space needs to be constructed simultaneously for multiple types of data, including the external environment affecting the power output of generating equipment and the energy consumption of loads. For example, sunlight affects the power efficiency of photovoltaic power plants, and temperature determines the power consumption of air conditioning in railway stations. Due to the instability of generating equipment, complex energy dispatching is required, such as storing energy and selling electricity to the power grid during periods of high power generation efficiency, and dispatching stored energy or purchasing electricity from the power grid during periods of low power generation efficiency. In the power dispatching process, electricity price state parameters affect the decisions on electricity sales and purchases. Energy equipment state parameters reflect the operating status of generating equipment, energy storage equipment, and load equipment, which is one of the foundations for dispatching decisions. User behavior parameters reflect the specific experience and needs of users during use, and are also one of the foundations for dispatching decisions.

[0076] In some embodiments, external natural environment parameters include meteorological information, geological information, and time information of the area where the target railway station is located; meteorological information includes light intensity, wind speed, temperature, and humidity; time information includes date, season, and current time. All parameters are labeled according to preset units of measurement; for example, light intensity is measured in watts per square meter (W / m²). 2 Wind speed is measured in meters per second (m / s), and temperature is measured in degrees Celsius (°C).

[0077] The power generation equipment includes: photovoltaic panels, wind turbines, and combined heat and power (CHP) subsystems.

[0078] The status of energy equipment includes the current or projected power generation of photovoltaic panels and wind turbines; the current operating status (on / off), power generation, and heat output of combined heat and power (CHP) subsystems; the current remaining power and charging / discharging status of energy storage devices; and the current energy demand of load equipment, such as the energy consumption of air conditioning and lighting equipment.

[0079] User behavior comprises two parts: user needs and user feedback. User needs typically include basic requirements for energy use, such as electricity demand and comfort requirements. User feedback is the user's reaction to the actual use of a product or service, which can be positive (e.g., satisfaction, appreciation) or negative (e.g., dissatisfaction, complaints). These parameters are quantified.

[0080] Quantification methods include, for example: 1) Satisfaction rating: User satisfaction rating of the product or service. 2) Specific comments: Detailed comments or suggestions from users on the product or service, selected from multiple preset items. 3) Usage frequency: How often users use the product or service. 4) Recommendation willingness: The degree to which users are willing to recommend the product or service to others.

[0081] Device usage preference parameters are quantified by marking the load selected by the user, the frequency of use of the load device, and the duration of load operation.

[0082] User satisfaction feedback parameters include user satisfaction scores provided through preset links, quantified adjustment suggestions, frequency of use of load devices, and recommendation index of load devices.

[0083] The projected power generation is predicted using an ANN neural network, specifically including the following steps S201 to S206:

[0084] Step S201: Data preparation and preprocessing. First, collect historical power generation equipment operation data, including inputs (such as environmental factors like weather conditions, temperature, and humidity) and outputs (i.e., power generation). Clean the data, handle missing and outlier values, and perform normalization or standardization to meet the training requirements of the neural network.

[0085] Step S202: Model configuration. Select a suitable neural network architecture for time series prediction, such as a multilayer perceptron (MLP). Configure the number of network layers and the number of neurons per layer. For example, you can set one input layer corresponding to the number of features, several hidden layers for feature extraction, and one output layer corresponding to the predicted power generation. Use ReLU or sigmoid as the activation function and select a suitable optimizer, such as Adam or SGD.

[0086] Step S203: Train the model by dividing the dataset into training and test sets. Use the training set data to train the neural network using the backpropagation algorithm, adjusting weights and biases to minimize prediction error. During training, cross-validation can be used to avoid overfitting, and early stopping can be used to monitor performance on the validation set; training stops once performance no longer improves.

[0087] Step S204: Model Evaluation and Tuning. Evaluate the model's performance on the test set, using metrics such as mean squared error (MSE) or mean absolute percentage error (MAPE) to measure prediction accuracy. Adjust the network structure, learning rate, and other hyperparameters based on the evaluation results, and repeat the training and evaluation process until satisfactory prediction performance is achieved.

[0088] Step S205: Deployment and Use. Deploy the trained model into the actual power generation prediction system. Collect new input data in real time and obtain the predicted power generation through network forward propagation. Make resource planning and scheduling decisions based on the prediction results.

[0089] Step S206: Continuously monitor and update the model, and retrain it regularly with newly collected data to adapt to environmental changes and new data patterns, ensuring that the prediction accuracy remains stable over time.

[0090] In step S102, the action space refers to the control actions and parameter settings for all controllable power generation equipment, energy storage equipment, and load equipment in the multi-source complementary power system. The action space defines all possible operations that an agent can perform, which are directly related to energy production, storage, consumption, and interaction with the external energy market. To achieve effective energy management and optimization, each dimension in the action space is processed in depth, and how they combine to achieve optimal overall system operation is considered.

[0091] Power generation equipment control parameters include start-up and shutdown control parameters and operating power control parameters. For renewable energy equipment, such as photovoltaic (PV) and wind power, output is significantly affected by natural conditions. Therefore, the equipment's control strategy should be able to adjust based on weather forecasts, real-time weather conditions, and other system status information to maximize energy efficiency. For example, adjusting the tilt angle of the photovoltaic panels to 30 degrees and the angle of the wind turbine blades to 45 degrees.

[0092] Energy storage device control parameters include scheduling parameters for multiple energy storage devices and charge / discharge power control parameters. Control of a Battery Energy Storage System (BESS) is crucial for smoothing supply-demand imbalances. Controlling charging and discharging power not only affects the system's immediate performance but also determines battery life and cost-effectiveness. Therefore, the agent must consider factors such as battery state (e.g., SOC), current electricity prices, and projected renewable energy output when making decisions. For example, instructing the energy storage system to charge at 2.5kW and discharge at 1.5kW.

[0093] Electricity trading control parameters include the sales scheduling parameters for the output of power generation equipment and the stored electricity in energy storage equipment, as well as external energy procurement control parameters. External energy procurement involves electricity trading with the grid. Electricity trading control can coordinate the interaction between the generated electricity and the grid, achieving cost reduction and efficiency improvement. When locally generated energy is insufficient, electricity can be purchased from the grid; conversely, when there is a surplus, the excess electricity can be sold back to the grid. This strategy helps reduce overall operating costs and helps balance grid load. Alternatively, to increase efficiency and reduce costs, electricity can be sold during high-price periods and purchased and stored during low-price periods. For example, 2.0 kWh of electricity can be purchased from the grid between 9 a.m. and 11 a.m., and 1.5 kWh of electricity can be sold back to the grid between 1 p.m. and 2 p.m.

[0094] Load equipment control parameters include start / stop control parameters and operating status control parameters for each load device. Load management involves controlling various electrical devices within a building to ensure efficient energy use while meeting user comfort requirements. This can be achieved through intelligent scheduling of device operating times, for example, operating high-energy-consuming equipment when electricity prices are low, or reducing grid power consumption when solar energy production is high. More specific examples include adjusting the air conditioner's set temperature to 22 degrees Celsius and setting the lighting brightness to 50%.

[0095] Furthermore, in the case of a combined heat and power (CHP) system, the simultaneous provision of electricity and heat is crucial for improving energy efficiency. The control strategy should consider the balance between electricity and heat demand, as well as any possible energy storage solutions. For example, the power output of the CHP system could be adjusted to 5.0 kW, and the heat supply adjusted to 3.0 kW.

[0096] In step S103, after the state space and action space are constructed in steps S101 and S102, a mapping between state and action is established based on deep reinforcement learning.

[0097] Deep reinforcement learning (DRL) is an artificial intelligence technique that combines deep learning and reinforcement learning. It solves decision-making problems in complex environments through an end-to-end perception and control system. In DRL, the agent interacts with the environment and learns through trial and error to maximize cumulative reward.

[0098] During the perception process, at each moment, the agent acquires high-dimensional observation data from the environment and uses deep learning methods (such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) to process this data and extract useful state feature representations.

[0099] During the decision-making process, based on the extracted state features, the intelligent system evaluates the value function of each possible action and can select the current optimal action to execute through a certain strategy (such as ε-greedy strategy, Softmax strategy, etc.).

[0100] During execution and feedback, the environment reacts to the agent's actions, generating new states and reward signals, and feeding this information back to the agent. Based on this feedback, the agent adjusts its strategy to better adapt to the environment.

[0101] The execution process includes: initialization, setting the agent's initial state, policy network, and value network (if a value function-based approach is used), as well as the relevant hyperparameters. Iterative loop: at each time step, the agent performs the following steps: perceives the environmental state, selects an action using the policy network, executes the action and observes the environmental feedback (including new states and rewards), stores the experience (state, action, reward, new state) in the experience replay pool, and samples a batch of experience from the experience replay pool to update the policy network and value network (if a value function-based approach is used). The above steps are repeated until a termination condition is met (e.g., reaching the maximum number of time steps, accumulating a threshold, etc.).

[0102] During training, the agent continuously optimizes its policy network to improve its ability to obtain higher rewards in future environments. This is typically achieved through optimization algorithms such as gradient descent, with the goal of minimizing the difference between the action output by the policy network and the optimal action (if using a policy gradient-based method), or maximizing the cumulative reward (if using a value function-based method). This invention employs the former.

[0103] Specifically, the main policy network and the main valuation network adopt fully connected networks, convolutional neural networks, or recurrent neural networks, and introduce ReLU or sigmoid activation functions.

[0104] The reward function R defines the immediate feedback an agent receives after taking a certain action. The design of the reward function should guide the agent to learn strategies that align with the expected goals. In a multi-energy complementary system, the reward function may include: cost savings, calculated based on energy procurement costs and sales revenue; energy efficiency, rewarding or penalizing based on energy utilization rates; user satisfaction, adjusting rewards based on user feedback; equipment health status, rewarding or penalizing based on equipment health status to encourage preventative maintenance; and environmental impact, calculating rewards based on carbon emissions to encourage low-carbon operations.

[0105] In some embodiments, a reward value is calculated by combining cost savings, energy efficiency, user satisfaction, equipment wear and tear, and environmental impact, including steps S301 to S305:

[0106] Step S301: Calculate the procurement cost of the multi-source complementary power system. The calculation formula is as follows:

[0107] R cost_saving =-(Price) purchased ×Power purchased );

[0108] Among them, R cost_saving Price represents the purchase cost. purchased Indicates the purchase price of electricity, Power purchasedThis indicates the amount of electricity purchased.

[0109] Step S302: Calculate the sales revenue of the multi-source complementary power system. The calculation formula is as follows:

[0110] R sales_gain =Price sold ×Power sold ;

[0111] Among them, R sales_gain Price represents sales revenue. sold Indicates the sales price of electricity, Power sold Indicates the volume of electricity sold;

[0112] Step S303: Obtain the energy utilization rate of the multi-source complementary power system at each moment, and calculate the change in energy utilization rate at the current moment. The calculation formula is:

[0113] R efficiency =Efficiency current Efficiency previous ;

[0114] Among them, R efficiency This represents the change in energy efficiency. current Efficiency represents the current energy utilization rate. previous This indicates the energy utilization rate at the previous moment;

[0115] Step S304: Obtain the health status of the multi-source complementary power system at each time point to characterize the degree of equipment wear, and calculate the change in system health status at the current time. The calculation formula is:

[0116] R health =Health current -Health previous ;

[0117] Among them, R health Health represents the change in the system's health status. current This indicates the current health status of the multi-source complementary power system. previous This indicates the health status of the multi-source complementary power system at the previous moment; the health status is calibrated using the remaining service life or failure rate of each device.

[0118] Step S305: Calculate the reward value, using the following formula:

[0119] R t =ω1(R cost_saving +R sales_gain )+ω2·R efficiency +ω3·Rsatisfaction +ω4·R halth +ω5·R environmental ;

[0120] R environmental =-Emissions current ;

[0121] Among them, R t R represents the reward value at time t. satisfaction For quantified satisfaction rating, R environmental Emissions are environmental impact values. current The carbon emissions of the multi-source complementary power system are represented by ω1, ω2, ω3, ω4 and ω5, which are weighting coefficients.

[0122] Steps S301 and S302 aim to control cost savings, which refers to reducing energy procurement costs through optimized energy management and trading strategies, and potentially increasing revenue through the sale of surplus energy. Based on these two calculations, reducing procurement costs results in a positive reward whenever the agent lowers the cost of energy purchased from the grid by adjusting energy storage systems, renewable energy devices, or external energy procurement strategies. Increasing sales revenue results in a positive reward if the agent can sell surplus energy to the grid or other consumers at a suitable price.

[0123] Step S303 involves controlling energy utilization efficiency, which reflects the degree of energy loss during production, storage, transmission, and use. High efficiency means less energy waste. The reward function is designed based on energy utilization efficiency to encourage agents to improve overall energy efficiency.

[0124] Step S304 controls the health status of the equipment, which directly affects the system's reliability and operating costs. The reward function can be designed based on the equipment's health status to encourage agents to take actions to extend equipment lifespan and prevent failures. Positive rewards are given whenever an agent takes action to improve the equipment's health status (such as regular maintenance or replacing worn parts).

[0125] In step S305, user satisfaction is further introduced. User satisfaction is a subjective indicator that depends on the user's perception of service quality. In a multi-energy complementary system, this may involve the user's satisfaction with energy availability and quality. The reward function can be adjusted based on user feedback to encourage the agent to provide a better service experience. Rewards are adjusted based on user feedback ratings, with positive feedback corresponding to positive rewards and negative feedback corresponding to negative rewards.

[0126] In some embodiments, the master valuation network fits a value function to obtain the master Q value, expressed as:

[0127] Q(s,a,θ)=E[R t +γmax a′ Q(s′,a′,θ)∣s,a];

[0128] Where Q(s,a,θ) represents the principal Q value, E represents the expectation, and R... t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ represent the parameters of the master evaluation network.

[0129] The target Q-value obtained by fitting the value function to the target network is expressed as:

[0130] y = R t +γmax a′ Q target (s′,a′,θ′);

[0131] Where y represents the target Q value, R t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that will obtain the maximum value function value in state s′, γ represent the decay factor, and θ′ represent the parameters of the target network.

[0132] In some embodiments, the optimization objective is to minimize the difference between the action output by the main policy network and the optimal action, and the expression for the loss function is:

[0133] L(θ) = E[(yQ(s,a,θ)) 2 ];

[0134] Where L(θ) represents the loss function, E represents the expectation, y represents the target Q value, and Q(s,a,θ) represents the principal Q value.

[0135] In some embodiments, the master strategy network and the master valuation network are pre-trained based on historical state space and historical action space constructed from historical data, and then transferred to the operation and continuous updating of a multi-source complementary power system.

[0136] On the other hand, the present invention also provides a multi-source complementary power system suitable for railway stations, the system comprising:

[0137] The multi-source power generation electronic system includes photovoltaic panels, wind turbines, and cogeneration subsystems, and is connected to the external power grid.

[0138] The energy storage subsystem is connected to the multi-source power generation system and the external power grid.

[0139] Multiple load devices connect to a multi-source power generation system, an energy storage subsystem, and an external power grid.

[0140] The equipment management subsystem is used to execute the energy dispatching method for railway stations described in steps S101 to S104 above. Based on environmental parameters, electricity price status parameters, energy equipment status parameters, and user behavior parameters, it selects and executes control actions to achieve energy dispatching for the target railway station. Environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. Electricity price status parameters include electricity purchase and sales prices. Energy equipment status parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charging / discharging status of energy storage equipment, the energy demand of load equipment, equipment health status parameters, and carbon emissions. User behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters.

[0141] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0142] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0143] The present invention will now be described with reference to a specific embodiment:

[0144] This embodiment provides an energy dispatching method for a railway station, implemented in a railway station with the following facilities: photovoltaic panels with a total installed capacity of 20kW; wind turbines with a total installed capacity of 10kW; a battery energy storage system (BESS) with a total capacity of 50kWh; and functional load equipment including lighting, air conditioning, monitoring, and turnstiles.

[0145] By combining state space, action space, reward function, and DQN algorithm formula, a complete deep reinforcement learning framework is constructed to optimize the operating strategy of multi-energy complementary systems. This framework can automatically learn and adjust the strategy to maximize energy utilization efficiency and minimize costs. As training progresses, the agent will gradually learn to take optimal actions in different states, thereby improving the overall system performance.

[0146] The following is a detailed description of the specific process:

[0147] 1. Data Collection

[0148] Collect various data required for the operation of multi-energy complementary systems to provide a foundation for environment construction. Data types include the following:

[0149] Historical electricity consumption data, including past electricity consumption, peak electricity consumption, and electricity consumption patterns.

[0150] Weather forecast data, including meteorological parameters such as sunlight intensity, wind speed, and temperature, is crucial for predicting the output of renewable energy.

[0151] Electricity price data, especially real-time electricity price information, is crucial for optimizing energy procurement strategies.

[0152] Equipment status data, including remaining power of the energy storage system, output power of photovoltaic panels, and output power of wind turbines.

[0153] User behavior data, such as users' electricity usage habits and preferences, can help optimize the user experience.

[0154] Data sources include sensor data (such as photovoltaic panels, energy storage systems, etc.), weather station data, real-time electricity price information provided by power companies, and user feedback data.

[0155] 2. Construct the state space, action space, and reward function.

[0156] A simulation environment is built based on the collected data to train and test the DRL model. The construction steps include:

[0157] Define a state space S, representing the state space of the environment, including weather conditions, time information, electricity prices, and equipment status. The state space includes environmental parameters of the target railway station, electricity price parameters, energy equipment status parameters, and user behavior parameters. Environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. Electricity price parameters include electricity purchase and sales prices. Energy equipment status parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charge / discharge status of energy storage devices, the energy demand of load equipment, equipment health status parameters, and carbon emissions. User behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters. Define the state space S = {s} t}, s t This represents the state vector at time t, which contains information in multiple dimensions, such as weather conditions, current time, real-time electricity price, output power of photovoltaic panels, output power of wind turbines, and remaining power of the energy storage system.

[0158] Action space A defines all possible actions that the agent can take, such as charging and discharging the energy storage system, adjusting the angle of the photovoltaic panels, etc. A = {a t}, here a tThis represents the action vector taken by the agent at time t, which may include the charging and discharging power of the energy storage system, adjusting the load (such as air conditioning temperature, lighting brightness, etc.).

[0159] The reward function R is defined as the immediate feedback received by the agent after taking a certain action, which is used to guide the agent to learn the optimal strategy.

[0160] In multi-energy complementary systems, it is common practice to define a reward function to evaluate the actions taken by an agent (e.g., a control system) under specific conditions. This reward function typically aims to maximize the overall benefits of the system while minimizing adverse effects.

[0161] The formula for calculating the procurement cost of a multi-source complementary power system is as follows:

[0162] R cost_saving =-(Price) purchased ×Power purchased );

[0163] Among them, R cost_saving Price represents the purchase cost. purchased Indicates the purchase price of electricity, Power purchased This indicates the amount of electricity purchased.

[0164] The formula for calculating the sales revenue of a multi-source complementary power system is as follows:

[0165] R sales_gain =Price sold ×Power sold ;

[0166] Among them, R sales_gain Price represents sales revenue. sold Indicates the sales price of electricity, Power sold Indicates the volume of electricity sold;

[0167] Obtain the energy utilization rate of the multi-source complementary power system at each moment, and calculate the change in energy utilization rate at the current moment. The calculation formula is as follows:

[0168] R efficiency =Efficiency current Efficiency previous ;

[0169] Among them, R efficiency This represents the change in energy efficiency. current Efficiency represents the current energy utilization rate. previous This indicates the energy utilization rate at the previous moment;

[0170] To obtain the health status of a multi-source complementary power system at various times to characterize equipment wear, the change in the system health status at the current time is calculated using the following formula:

[0171] R health =Health current -Health previous ;

[0172] Among them, R health Health represents the change in the system's health status. current This indicates the current health status of the multi-source complementary power system. previous This indicates the health status of the multi-source complementary power system at the previous moment; the health status is calibrated using the remaining service life or failure rate of each device.

[0173] The reward value is calculated using the following formula:

[0174] R t =ω1(R cost_saving +R sales_gain )+ω2·R efficiency +ω3·R satisfaction +ω4·R halth +ω5·R environmental ;

[0175] R environmental =-Emissions current ;

[0176] Among them, R t R represents the reward value at time t. satisfaction For quantified satisfaction rating, R environmental Emissions are environmental impact values. current The carbon emissions of the multi-source complementary power system are represented by ω1, ω2, ω3, ω4 and ω5, which are weighting coefficients.

[0177] 3. Network Design

[0178] Design deep neural networks for learning to approximate the Q-function or policy function.

[0179] Network architecture: Choose a suitable network architecture, such as a fully connected network, a convolutional neural network (CNN), or a recurrent neural network (RNN).

[0180] Input layer: Defined according to the state space, the input layer receives environmental state information.

[0181] Output layer: Define the output layer based on the action space, which defines the actions that the output agent can take.

[0182] Hidden layers: Design an appropriate number of hidden layers and nodes to capture the complex relationship between states and actions.

[0183] Activation function: Choose an appropriate activation function, such as ReLU, sigmoid, etc.

[0184] Specifically, DQN uses deep neural networks to approximate the Q function and introduces two techniques, an Experience Replay Buffer and a Target Network, to improve the stability and efficiency of learning.

[0185] Build an experience replay buffer

[0186] After each interaction, (s) t ,a t ,R t ,s t+1 Store it in the experience playback buffer D.

[0187] D←(s t ,a t ,R t ,s t+1 );

[0188] Then, a batch of samples (s,a,R,s′) is randomly selected to update the network.

[0189] The master value network fits the value function to obtain the master Q value, which is expressed as:

[0190] Q(s,a,θ)=E[R t +γmax a′ Q(s′,a′,θ)∣s,a];

[0191] Where Q(s,a,θ) represents the principal Q value, E represents the expectation, and R... t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ represent the parameters of the master evaluation network.

[0192] Construct the target network Q target The parameter θ target Regularly from the main network Q main The parameter θ main It was copied.

[0193] θ target ←θ main ;

[0194] The target network is used to calculate the target Q-value:

[0195] y = R t +γmax a′ Q target (s′,a′,θ′);

[0196] Where y represents the target Q value, R t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that will maximize the value function value in state s′, γ represent the decay factor, and θ′ represent the parameters of the target network.

[0197] Network updates use Mean Squared Error (MSE) as the loss function:

[0198] L(θ) = E[(yQ(s,a,θ)) 2 ];

[0199] Where L(θ) represents the loss function, E represents the expectation, y represents the target Q value, and Q(s,a,θ) represents the principal Q value.

[0200] Minimize the loss function using gradient descent:

[0201]

[0202] Where α is the learning rate.

[0203] 4. Training process

[0204] The DRL model learns the optimal policy through interaction with the environment. The training steps are as follows:

[0205] Initialization: Set the initial state S0 and initialize the deep neural network Q. main and Q target The parameters.

[0206] Interaction: At each time step t, the agent determines the current state S. t and the current policy π(a|s) t Select action a t .

[0207] Execution: Perform action a t Observe the new state s t+1 and reward R t .

[0208] Storage experience: will (s t ,a t ,R t ,s t+1 Store it in the experience playback buffer D.

[0209] Random sampling: Randomly select a batch of samples (s,a,R,s′) from the experience replay buffer D.

[0210] Calculate the target Q value: Use the target network to calculate the target Q value y.

[0211] Update the network: Update the parameters of the main network using gradient descent.

[0212] Copy parameters: Periodically copy the parameters of the master network to the target network.

[0213] 5. Evaluation and Adjustment

[0214] Evaluate the agent's performance and adjust strategies or network structure based on the evaluation results.

[0215] Set evaluation metrics based on specific optimization goals, such as cost savings, energy efficiency, and user satisfaction. Regularly test the agent's performance in a simulation environment and record key metrics. Adjust the reward function and network parameters based on the evaluation results to optimize agent performance.

[0216] 6. Deployment and Monitoring

[0217] The trained model is deployed to a real system for online decision-making, and its performance is continuously monitored.

[0218] Deployment Steps: Deploy the trained model to the actual operating environment, ensuring the model can interact with the real system. Based on the current environmental state, the agent selects the optimal action in real time to control the operation of the multi-energy complementary system. Continuously monitor the agent's actual performance and record key indicators such as cost and efficiency. Adjust model parameters periodically based on actual operating data to adapt to environmental changes.

[0219] Simulation experimental data:

[0220] Based on the previously constructed dataset, the performance of the system after adopting the Deep Reinforcement Learning (DRL) optimization strategy is evaluated. The state space is constructed as follows: weather conditions (sunny, cloudy, overcast, rainy), light intensity (watts / square meter), wind speed (meters / second), temperature (degrees Celsius), current time (hours), real-time electricity price (yuan / kWh), photovoltaic panel output power (kW), wind turbine output power (kW), remaining energy storage system capacity (kWh), user electricity demand (kW), and equipment health index (between 0 and 1). The action space is constructed as follows: energy storage system charging power (kW), energy storage system discharging power (kW), load adjustment (e.g., air conditioning temperature, lighting brightness), and equipment maintenance (whether maintenance is required).

[0221] like Figure 2 , 3As shown in Figure 4, the photovoltaic output power gradually increases during the day (08:00 to 16:00), reaches its maximum value, and then gradually decreases. At night (00:00 to 07:00), the photovoltaic output is zero. Wind power output is higher at night and in the early morning (00:00 to 07:00), reaching 2kW. During the day (08:00 to 16:00), wind power output is lower, remaining at 1kW. Charging occurs during periods of lower electricity prices (00:00 to 07:00), with a charging power of 2kW. Discharging occurs during periods of higher electricity prices (17:00 to 23:00), with the discharge power adjusted according to user demand. The battery charges at night, with the charge gradually increasing. During the day, the charge gradually decreases but remains at a high level. User demand remains constant throughout the day at 2kW.

[0222] By optimizing the strategy, the system can store electricity when electricity prices are low and use the stored electricity when prices are high, thereby reducing overall costs. Before optimization, the system directly purchased electricity when prices were high, without considering the use of energy storage. After optimization, the system charges when prices are low and discharges when prices are high, while simultaneously adjusting the load to reduce overall costs.

[0223] Through optimization, the system reduces power waste and improves energy efficiency. Before optimization, the system directly used all generated power, wasting any excess. After optimization, the system tries to store excess power and use it when needed.

[0224] Define the following metrics for user satisfaction: Power Supply Stability: Is the power supply stable and uninterrupted? Cost Efficiency: Are electricity costs effectively controlled? Indoor Comfort Level: Is the indoor temperature comfortable and the lighting appropriate? System Response Speed: How quickly does the system respond to changes in user needs?

[0225] Each metric is assigned a weight and given a scoring range (e.g., 0 to 1). The scores for each metric are calculated based on the results, and then the overall user satisfaction is calculated by combining the scores.

[0226] Before optimization: Power Supply Stability: The table shows stable power supply throughout the day, therefore a score of 0.95. Cost Efficiency: Based on electricity prices and the charging and discharging of the energy storage system, the user uses stored power instead of purchasing directly from the grid during periods of high electricity prices, indicating good cost control. The overall daily cost control is appropriate, with a score of 0.85. Indoor Comfort Level: Based on load adjustments (such as adjusting air conditioning temperature), the indoor temperature is moderate throughout the day, with a score of 0.95. The table shows that the air conditioning was adjusted at night, therefore a score of 0.95. System Response Speed: The system can respond promptly to changes in user power demand, with a score of 0.9. The system responds promptly, with a score of 0.9. The following calculations were performed:

[0227] User opinion 前 =(Supply Stability×ω1)+(Cost Efficiency×ω2)+(ComfortLevel×ω3)+(Response Speed×ω4)

[0228] Where ω1, ω2, ω3, and ω4 are the weights of each indicator, assumed to be 0.25 (i.e., equal weight), therefore:

[0229] User satisfaction 前 = (0.95 × 0.25) + (0.85 × 0.25) + (0.95 × 0.25) + (0.9 × 0.25) = 0.9125

[0230] The optimized system showed improved response speed and better control of electricity costs, while other indicators remained unchanged. Power Supply Stability: Unchanged before and after optimization, score: 0.95. Electricity Cost Efficiency: Better cost control after optimization, score: 0.90. Indoor Comfort Level: Unchanged before and after optimization, score: 0.95. System Response Speed: Faster response speed after optimization, score: 0.95. The following calculations were performed:

[0231] User satisfaction 后 =(Supply Stability×ω1)+(Cost Efficiency×ω2)+(ComfortLevel×ω3)+(Response Speed×ω4)

[0232] Where ω1, ω2, ω3, and ω4 are the weights of each indicator, assumed to be 0.25 (i.e., equal weight), therefore:

[0233] User satisfaction 后 = (0.95 × 0.25) + (0.90 × 0.25) + (0.95 × 0.25) + (0.95 × 0.25) = 0.9375

[0234] Before optimization, user satisfaction was 0.9125, and after optimization, it was 0.9375. The optimization improved overall user satisfaction, particularly in electricity cost control and system response speed.

[0235] Equipment wear and tear is calculated using defined metrics, including: Charge / Discharge Cycles: the number of times the energy storage system charges and discharges throughout the day; Operating Hours: the total operating time of the energy storage system throughout the day; Maintenance Frequency: the number of times the energy storage system undergoes maintenance throughout the day; and Load Adjustment Times: the number of times the load is adjusted throughout the day. Each metric is assigned a weight and given a scoring range (e.g., 0 to 1). The scores for each metric are then calculated based on actual conditions, and finally, the overall equipment wear and tear is calculated.

[0236] Before optimization: Charge / Discharge Cycles: According to the table data, there were 10 charge / discharge operations throughout the day (from t=0 to t=16, each charge / discharge operation counts as one operation). The number of charge / discharge operations throughout the day is relatively low, with a score of 0.3 (0 indicates no charge / discharge, 1 indicates frequent charge / discharge). Operating Hours: The operating hours throughout the day are relatively long, with a score of 0.7 (0 indicates no operation, 1 indicates operation throughout the day). According to the table data, the energy storage system performed charging operations during the day and discharging operations at night, therefore operating for most of the day. Maintenance Frequency: No maintenance was required throughout the day, with a score of 0 (0 indicates no maintenance, 1 indicates frequent maintenance). According to the table data, no maintenance operations were performed throughout the day. Load Adjustment Times: According to the table data, a total of 5 load adjustments were made throughout the day (from t=17 to t=23, each adjustment counts as one operation). The number of load adjustments throughout the day was relatively low, resulting in a score of 0.3 (0 indicates no adjustments, 1 indicates frequent adjustments). The following calculations were performed:

[0237] Equipment wear 前=(Charge / Di charge Cycles×ω1)+(Operating Hours×ω2)+(Maintenance Frequency×ω3)+(Load Adjustment Times×ω4)

[0238] Wherein, ω1, ω2, ω3, and ω4 are the weights of each indicator, and are assumed to be 0.25 (i.e., equal weights).

[0239] Equipment wear 前 = (0.3 × 0.25) + (0.7 × 0.25) + (0 × 0.25) + (0.3 × 0.25) = 0.325

[0240] After optimization, the number of charge / discharge cycles and load adjustment times decreased, while other metrics remained unchanged. Charge / Discharge Cycles: The number of charge / discharge cycles decreased after optimization, resulting in a score of 0.2. Operating Hours: The number of operating hours remained unchanged after optimization, resulting in a score of 0.7. Maintenance Frequency: The number of maintenance hours remained unchanged after optimization, resulting in a score of 0. Load Adjustment Times: Assuming the number of load adjustment times decreased after optimization, the score was 0.2. The following calculations were performed:

[0241] Equipment wear 后 =(Charge / Di charge Cycles×ω1')+(Operating Hours×ω2')+(Maintenance Frequency×ω3')+(Load Adjustment Times×ω4')

[0242] Wherein, ω1, ω2, ω3, and ω4 are the weights of each indicator, and are assumed to be 0.25 (i.e., equal weights).

[0243] Equipment wear 后 = (0.2 × 0.25) + (0.7 × 0.25) + (0 × 0.25) + (0.2 × 0.25) = 0.275

[0244] Before optimization, the wear coefficient of the equipment was 0.325, and after optimization, it was 0.275. The optimization significantly reduced the wear coefficient, particularly in terms of the number of charge / discharge cycles and load adjustment cycles. These results demonstrate that the optimization measures effectively reduced equipment wear and extended its service life.

[0245] Charging period: From 0:00 to 16:00, as wind power generation and solar energy storage are charging, and as the solar intensity is high, the PV output power increases, the energy storage system charges during this period, while ensuring that the load uses the excess power, until charging is basically completed at 4:00 in the afternoon.

[0246] Discharge period: from 5 pm to 0 am the next day. During this period, the wind is weak and the sunlight is reduced, the PV output power decreases, and the energy storage system begins to discharge to meet the electricity demand at night.

[0247] During the day when there is sufficient sunlight, the energy storage system performed multiple charging operations to ensure that the battery is fully charged.

[0248] After nighttime light intensity decreases, the energy storage system performs a discharge operation to meet the nighttime user demand.

[0249] During the daytime when electricity prices are low (approximately from 8 a.m. to 4 p.m.), the energy storage system primarily performs charging operations, reducing the need to purchase expensive grid electricity.

[0250] When electricity prices rise overnight (around 5 p.m. to 2 a.m. the next day), the energy storage system begins to discharge, reducing the cost of purchasing electricity at higher prices.

[0251] User demand remains constant throughout the day, and the energy storage system ensures the stability of power supply through reasonable scheduling, while also reducing users' electricity expenses.

[0252] Photovoltaic panels and wind turbines, as renewable energy power generation devices, make full use of solar energy during the day, thus improving the utilization rate of renewable energy.

[0253] By storing excess electricity through energy storage systems and releasing it during off-peak hours, reliance on traditional fossil fuel power generation is reduced.

[0254] Throughout the simulation, the equipment health index remained at a high level (0.95), indicating that the equipment was operating well and there were no obvious faults or performance degradation.

[0255] Simulation results show that no equipment maintenance is required within a given timeframe, which helps reduce maintenance costs and downtime.

[0256] These results demonstrate that the DRL framework has significant advantages in optimizing the operation strategy of multi-energy complementary systems, and helps to achieve efficient, economical and sustainable operation of the system.

[0257] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0258] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0259] In summary, the energy dispatching method and system for railway stations described in this invention, based on the multi-source complementary power supply system constructed in the railway station, constructs a state space using environmental parameters, electricity price status parameters, energy equipment status parameters, and user behavior parameters. It constructs an action space using control parameters for various types of power generation equipment, energy storage equipment, electricity trading, and load equipment. Based on deep reinforcement learning (DRL), it selects control actions to achieve dynamic dispatching and management of various power supply equipment, energy storage equipment, and load equipment in the system. It introduces cost savings, energy utilization efficiency, user satisfaction, equipment wear and tear, and environmental impact to calculate reward values, ensuring cost reduction and efficiency improvement, increased user satisfaction, reduced equipment wear and carbon emissions, and achieving automated, efficient, and stable energy dispatching during the control optimization process.

[0260] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0261] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0262] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0263] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An energy dispatching method for railway stations, characterized in that, The method is applicable to multi-source complementary power systems, which include various grid-connected power generation devices, energy storage devices, and load devices. The method includes the following steps: A state space is constructed, comprising environmental parameters of the target railway station, electricity price parameters, energy equipment parameters, and user behavior parameters. The environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. The electricity price parameters include electricity purchase and sales prices. The energy equipment parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charge / discharge status of energy storage devices, the energy demand of load equipment, equipment health status parameters, and carbon emissions. The user behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters. Construct an action space, which includes control parameters for multiple types of power generation equipment, energy storage equipment, electricity trading control parameters, and load equipment control parameters. Based on deep reinforcement learning, a main network is constructed, comprising a main policy network and a main evaluation network. The main policy network takes the parameters of the state space at the current moment as input and outputs the control action selected in the action space. A reward value is calculated by combining cost savings, energy utilization efficiency, user satisfaction, equipment wear and tear, and environmental impact. A value function based on the selected action in the state is established according to the reward value. The main evaluation network fits the value function to obtain the main Q value. An experience replay buffer is constructed to store the data generated by each interaction for random sampling training to update the main network. A target network with the same structure as the main policy network and the main evaluation network is introduced. The mean square error of the target Q value obtained by fitting the value function to the main Q value and the main Q value is calculated as a loss function. The parameters of the main network are updated by minimizing the loss function, and the parameters of the main network are periodically copied to the target network. The main network selects control actions to achieve energy scheduling of the target railway station and continuously updates and optimizes the main network.

2. The energy dispatching method for railway stations according to claim 1, characterized in that, The external natural environment parameters include meteorological information, geological information, and time information of the area where the target railway station is located; the meteorological information includes light intensity, wind speed, temperature, and humidity; the time information includes date, season, and current time. The power generation equipment includes: photovoltaic panels, wind turbines, and a combined heat and power (CHP) subsystem; The device usage preference parameters are quantified by marking the load selected by the user, the frequency of use of the load device, and the duration of load operation; The user satisfaction feedback parameters include user satisfaction scores fed back through a preset link, quantified adjustment suggestions, the usage frequency of the load device, and the recommendation index of the load device; The power generation equipment control parameters include start-stop control parameters and operating power control parameters for the power generation equipment. The energy storage device control parameters include scheduling parameters for multiple energy storage devices and charging / discharging power control parameters; The power trading control parameters include the sales scheduling parameters for the power output of the power generation equipment and the power stored in the energy storage equipment, as well as the external power procurement control parameters. The load device control parameters include start / stop control parameters and operating status control parameters for each load device; The projected power generation is predicted using an ANN neural network.

3. The energy dispatching method for railway stations according to claim 2, characterized in that, The main policy network and the main valuation network adopt fully connected networks, convolutional neural networks or recurrent neural networks, and introduce ReLU or sigmoid activation functions.

4. The energy dispatching method for railway stations according to claim 3, characterized in that, The reward value is calculated by combining cost savings, energy efficiency, user satisfaction, equipment wear and tear, and environmental impact, including: The procurement cost of the multi-source complementary power system is calculated using the following formula: R cost_saving =-(Price purchased ×Power purchased ); Among them, R cost_saving Price represents the procurement cost. purchased Indicates the purchase price of electricity, Power purchased Indicates the amount of electricity purchased; The sales revenue of the multi-source complementary power system is calculated using the following formula: R sales_gain =Price sold ×Power sold ; Among them, R sales_gain Price represents the sales revenue. sold Indicates the sales price of electricity, Power sold Indicates the volume of electricity sold; Obtain the energy utilization rate of the multi-source complementary power system at each moment, and calculate the change in energy utilization rate at the current moment. The calculation formula is as follows: R efficiency =Efficiency current -Efficiency previous ; Among them, R efficiency This represents the change in energy efficiency. current Efficiency represents the current energy utilization rate. previous This indicates the energy utilization rate at the previous moment; The health status of the multi-source complementary power system at each moment is obtained to characterize the wear and tear of the equipment. The change in the system health status at the current moment is calculated using the following formula: R health =Health current -Health previous ; Among them, R health Health represents the change in the system's health status. current This indicates the current health status of the multi-source complementary power system. previous This indicates the health status of the multi-source complementary power system at the previous moment; the health status is calibrated using the remaining service life or failure rate of each device. The reward value is calculated using the following formula: R t =ω1(R cost_savin g +R sales_gain )+ω2·R efficiency +ω3·R satisfacti on +ω4·R halth +ω5·R environmen tal ; R environmen tal =-Emissions current ; Among them, R t R represents the reward value at time t. satisfacti on For the quantified satisfaction score, R environmen tal Emissions are environmental impact values. current ω1, ω2, ω3, ω4 and ω5 are the carbon emissions of the multi-source complementary power system.

5. The energy dispatching method for railway stations according to claim 4, characterized in that, The master valuation network fits the value function to obtain the master Q value, which is expressed as: Q(s,a,θ)=E[R t +γmax a′ Q(s′,a′,θ)∣s,a]; Where Q(s,a,θ) represents the principal Q value, E represents the expectation, and R t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ represent the parameters of the main valuation network. The target Q value obtained by fitting the value function to the target network is expressed as follows: y=R t +γmax a′ Q target (s′,a′,θ′); Where y represents the target Q value, R t Let represent the reward value at time t, s represent the state at time t, a represent the action chosen at time t, s′ represent the state at time t+1, a′ represent the action that can obtain the maximum value function value in state s′, γ represent the decay factor, and θ′ represent the parameters of the target network.

6. The energy dispatching method for railway stations according to claim 5, characterized in that, The expression for the loss function is: L(θ)=E[(yQ(s,a,θ)) 2 ] Where L(θ) represents the loss function, E represents the expectation, y represents the target Q value, and Q(s,a,θ) represents the principal Q value.

7. The energy dispatching method for railway stations according to claim 6, characterized in that, The master strategy network and the master valuation network are pre-trained based on the historical state space and historical action space constructed from historical data, and then transferred to the operation and continuous updating of the multi-source complementary power system.

8. A multi-source complementary power system suitable for railway stations, characterized in that, The system includes: The multi-source power generation electronic system includes photovoltaic panels, wind turbines, and a combined heat and power subsystem, and the multi-source power generation electronic system is connected to the external power grid. An energy storage subsystem is connected to the multi-source power generation electronic system and the external power grid; Multiple load devices are connected to the multi-source electronic system, the energy storage subsystem, and the external power grid; The equipment management subsystem is used to execute the energy dispatching method for railway stations as described in any one of claims 1 to 7. Based on environmental parameters, electricity price status parameters, energy equipment status parameters, and user behavior parameters, it selects and executes control actions to achieve energy dispatching for the target railway station. The environmental parameters include external natural environment parameters and internal temperature and humidity parameters of the railway station. The electricity price status parameters include electricity purchase and sales prices. The energy equipment status parameters include the current and expected power generation of various types of power generation equipment, the remaining power and charging / discharging status of energy storage equipment, the energy demand of load equipment, equipment health status parameters, and carbon emissions. The user behavior parameters include electricity demand parameters, indoor temperature and humidity demand parameters, equipment usage preference parameters, cost sensitivity parameters, environmental preference parameters, and user satisfaction feedback parameters.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images