Power dispatching optimization method and system based on deep reinforcement learning

Abstract

The invention discloses a power dispatching optimization method and system based on deep reinforcement learning. The method comprises the following steps: modeling and representing the state of a power system by using a deep neural network model; defining an action space of power scheduling based on the state representation of the power system; constructing a reward function of a power dispatching strategy by taking economic cost, carbon emission and system stability as targets, training an intelligent agent by using a deep reinforcement learning algorithm, and optimizing the reward function based on the state and action space of a power system to obtain the power dispatching strategy; dynamically adjusting and optimizing a power dispatching strategy according to real-time data and environment change information through a self-adaptive adjustment mechanism; and testing and verifying the adjusted and optimized power dispatching strategy in a simulated power system environment, and applying the power dispatching strategy to power system dispatching after verification is passed. According to the scheme, the intelligent level of power dispatching is improved, and the optimal balance of economy, environmental protection and stability of a power system is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of power systems and automation thereof, and in particular relates to a power dispatch optimization method and system based on deep reinforcement learning. Background Art

[0002] With the development of the power industry and the widespread application of renewable energy, power dispatching faces more and more challenges, such as demand fluctuations, supply uncertainty, and increased environmental protection requirements. In traditional power systems, power dispatching mainly relies on manual experience and simple mathematical models to make decisions, which cannot cope with the increasingly complex power network and changing power demand. With the large-scale access of renewable energy and the rapid development of the power market, the uncertainty and complexity of power system operation have increased significantly, which has brought unprecedented challenges to power dispatching.

[0003] For example, the output of renewable energy such as wind and solar energy is affected by many factors such as weather conditions, and is highly volatile and unpredictable; at the same time, the peak-to-valley difference in electricity demand and the safe and stable operation requirements of the power grid also put forward higher requirements for power dispatching. In addition, the pressure of environmental protection makes reducing carbon emissions one of the important factors that must be considered in power dispatching.

[0004] With the application of artificial intelligence technology in power dispatching, deep learning technology has been introduced into power system state prediction and fault diagnosis due to its advantages in processing large-scale data and complex pattern recognition. Reinforcement learning technology is used to optimize the operation strategy and dispatch plan of the power system due to its adaptability and real-time performance in the decision-making process.

[0005] However, how to effectively combine deep learning with reinforcement learning so that it can not only accurately perceive and predict the state of the power system, but also make optimal dispatching decisions in a complex and changing environment is still an urgent problem to be solved in the current power dispatching field. Summary of the invention

[0006] In order to address the deficiencies in the prior art, the present invention provides a power dispatch optimization method and system based on deep reinforcement learning to address the limitations of traditional power dispatch methods in dealing with complex and changeable power system environments. The power system state is accurately represented through deep learning, and intelligent decision-making is performed through reinforcement learning to achieve automation and intelligence in power dispatching, thereby improving the economy, environmental protection and stability of the power system.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0008] The present invention first discloses a power dispatch optimization method based on deep reinforcement learning, which includes the following steps:

[0009] Step 1: Use a deep neural network model to model and represent the state of the power system;

[0010] Step 2: Based on the state representation of the power system, define an action space for power dispatching, where the action space includes all dispatching actions in the power system;

[0011] Step 3: construct a multi-objective reward function of the power dispatch strategy with economic cost, carbon emissions and system stability as the objectives, train the intelligent agent using a deep reinforcement learning algorithm, and optimize the multi-objective reward function based on the state and action space of the power system to obtain the power dispatch strategy;

[0012] Step 4: The intelligent agent dynamically adjusts and optimizes the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism;

[0013] Step 5: Test and verify the adjusted and optimized power dispatch strategy in a simulated power system environment, and apply the verified intelligent agent to power system dispatch.

[0014] The present invention further includes the following preferred embodiments:

[0015] The dispatching actions in the power system include the start and stop operations of the generator sets, the output adjustment operations, the charging and discharging operations of the energy storage devices and the implementation operations of the demand response.

[0016] The deep reinforcement learning algorithm includes a deep Q network DQN, a dual deep Q network DDPG, or a proportional policy optimization PPO algorithm.

[0017] The adaptive adjustment mechanism includes an online learning strategy, a policy gradient method, or an exploration and exploitation balance technique in reinforcement learning.

[0018] Testing and verifying the adjusted and optimized power dispatch strategy in a simulated power system environment, further including:

[0019] Simulate various power system operation scenarios, including extreme weather, surge in power demand, and equipment failure, to test and verify the power dispatch strategy.

[0020] The power system includes a microgrid, a large-scale power system, and a power system including renewable energy.

[0021] The present invention also discloses a power dispatch optimization system based on deep reinforcement learning using the aforementioned power dispatch optimization method based on deep reinforcement learning, comprising:

[0022] A state representation module, which is used to model and represent the state of the power system using a deep neural network model;

[0023] An action definition module, used for defining an action space of power dispatch based on the state representation of the power system, wherein the action space includes all dispatch actions in the power system;

[0024] A reward function construction module is used to construct a multi-objective reward function of a power dispatching strategy with economic cost, carbon emissions and system stability as objectives, train an intelligent agent using a deep reinforcement learning algorithm, and optimize the multi-objective reward function based on the state and action space of the power system to obtain a power dispatching strategy;

[0025] An adaptive adjustment module, used for the intelligent agent to dynamically adjust and optimize the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism;

[0026] The simulation test module is used to test and verify the adjusted and optimized power dispatching strategy in a simulated power system environment, and apply the verified intelligent agent to the power system dispatching.

[0027] Accordingly, the present application also discloses a terminal, including a processor and a storage medium;

[0028] The storage medium is used to store instructions;

[0029] The processor is used to operate according to the instructions to execute the steps of the aforementioned power dispatch optimization method based on deep reinforcement learning.

[0030] Correspondingly, the present application also discloses a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned power dispatch optimization method based on deep reinforcement learning.

[0031] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for optimizing power dispatching based on deep reinforcement learning, which combines deep learning and reinforcement learning by utilizing an efficient and intelligent power dispatching optimization method, and significantly improves the dispatching quality and operation efficiency of the power system. First, the deep learning model can accurately predict the future state of the power system and provide reliable data support for dispatching decisions. Secondly, the design of the multi-objective reward function enables power dispatching to achieve the best balance between economy, environmental protection and system stability. Furthermore, the adaptive adjustment mechanism gives the intelligent agent flexibility in the face of real-time data and environmental changes, ensuring the real-time and adaptability of the dispatching strategy. In addition, the application of the deep reinforcement learning algorithm greatly improves the ability to handle large-scale power system optimization problems, shortens the calculation time, and improves the accuracy of decision-making. Finally, by constructing a simulated test environment, the performance of the intelligent agent is fully verified before actual application, ensuring the reliability and practicality of the dispatching scheme. The present invention improves the economic and environmental performance of the power system, and also enhances the stability and security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the actual deployment and monitoring system architecture diagram in the present invention.

[0033] Figure 2 It is a flow chart representing the state of the electric power system in the present invention.

[0034] Figure 3 It is a flow chart of the adaptive adjustment mechanism in the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0036] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0037] In view of the shortcomings of the prior art, the present invention proposes a method and system for optimizing power dispatching based on deep reinforcement learning, which accurately represents the complex state of the power system by constructing a deep neural network, including key factors such as the state of generator sets, load demand, weather conditions, and the output of renewable energy. At the same time, a wide range of action spaces are defined, allowing the agent to make decisions in terms of starting and stopping generator sets, output adjustment, and operation of energy storage equipment. A multi-objective reward function is designed, which comprehensively considers key indicators such as cost minimization, carbon emission reduction, and system stability improvement to ensure the comprehensiveness and sustainability of the power dispatching scheme. By applying advanced deep reinforcement learning algorithms such as DQN, DDPG or PPO, the agent autonomously learns and optimizes the dispatching strategy in the interaction with the power system environment. In addition, the adaptive adjustment mechanism enables the agent to dynamically adjust its decision according to real-time data and environmental changes, significantly improving the flexibility and adaptability of power dispatching. The present invention not only improves the economy and environmental protection of the power system, but also enhances the stability and reliability of the system.

[0038] The power dispatch optimization method based on deep reinforcement learning disclosed in the present invention performs intelligent optimization on the dispatch of the power system by comprehensively utilizing the powerful data processing capability of deep learning and the decision-making capability of reinforcement learning to adapt to the complexity and uncertainty in the operation of the power system. Specifically, the method includes the following steps:

[0039] Step 1: Use a deep neural network model to model and represent the state of the power system.

[0040] The power system is a highly complex and dynamically changing network involving a large number of variables and uncertainties. First, deep learning technology is used to model and represent the state of the power system, including the design of the network structure, the preparation of training data, and the optimization of the training process. Figure 2 , by building a deep neural network, the network is able to process and analyze a large amount of historical and real-time data, including but not limited to the operating status of generators, grid load demand, weather forecast data, renewable energy output, etc. The network structure design considers convolutional neural networks (CNN) to handle spatial correlation, and recurrent neural networks (RNN) or long short-term memory networks (LSTM) to handle the correlation of time series data. Through training, the network can accurately predict the future state of the system and provide reliable information support for scheduling decisions.

[0041] Collect comprehensive real-time and historical data from the power system, which may include but are not limited to power generation, load demand, equipment status, weather conditions, market prices, etc. Ensure that the data covers all relevant power system components, such as generators, transmission lines, energy storage devices, etc. Clean the collected data to remove outliers and noise to ensure data quality. Normalize or standardize the data so that data of different magnitudes can be compared and processed at the same scale. To handle missing values, missing data points can be filled by interpolation, mean or median. Through feature engineering methods, extract features that are representative of the power system status from the raw data. Features may include time series analysis results, statistical data, equipment operating parameters, etc. Techniques such as correlation analysis, principal component analysis (PCA) or autoencoders can be used to select and extract the most useful features. Use the preprocessed teaching data set to train the deep learning model and optimize the model weights through the back propagation algorithm. Divide the data set into training, validation and test sets to ensure that the model has good generalization ability. During the training process, monitor the performance of the model and evaluate it using indicators such as accuracy, recall, F1 score, etc. The trained deep learning model can analyze the input data features and output the state representation of the power system. The state representation can be the prediction of future load demand, the assessment of equipment failure risk, the trend of market price changes, etc. The state representation output by the deep learning model is used as the input of power dispatch optimization to provide a decision-making basis for the intelligent agent. The intelligent agent selects actions based on the state representation and executes the power dispatch strategy to achieve the optimization goal.

[0042] The present invention uses deep learning technology, especially convolutional neural network (CNN) and recurrent neural network (RNN) or long short-term memory network (LSTM), to effectively model and represent the state of the power system and improve the accuracy of state prediction. To accurately represent and predict the state of the power system.

[0043] Step 2: Based on the state representation of the power system, define an action space for power dispatching, wherein the action space includes all dispatching actions in the power system.

[0044] Based on the state representation, the present invention defines an action space for power dispatching. The action space covers all possible dispatching operations, including the start and stop operations of generator sets, output adjustment operations, and the charging and discharging operations of energy storage devices and the implementation operations of demand response. Each action corresponds to a specific system state and may lead to different system changes and results.

[0045] Step 3: Construct a multi-objective reward function for the power dispatching strategy with economic cost, carbon emissions and system stability as the objectives, use a deep reinforcement learning algorithm to train the intelligent agent, and optimize the multi-objective reward function based on the state and action space of the power system to obtain the power dispatching strategy.

[0046] Electric power dispatching needs to consider multiple conflicting goals at the same time, such as cost minimization, carbon emission reduction and system stability improvement. In order to guide the learning process of the intelligent agent (dispatching system), the present invention designs a reward function that comprehensively considers multiple goals. This function not only focuses on the economy of power system operation, such as minimizing power generation costs, but also includes environmental factors, such as minimizing carbon emissions, and system stability, such as ensuring supply and demand balance and avoiding potential power shortages. The design and implementation of the reward function is the key to the present invention, which directly affects the learning direction and optimization goals of the intelligent agent. By constructing a multi-objective reward function and applying it to the deep reinforcement learning framework, effective balance and optimization of multiple goals are achieved. In practical applications, designing a multi-objective reward function usually requires considering how to convert different goals into a unified quantitative indicator and determining the weight of each goal for optimization in the reinforcement learning process.

[0047] Suppose we have three goals:

[0048] -C: Economic cost, hope to minimize

[0049] -E: Carbon emissions, hope to minimize

[0050] -S: System stability, hope to maximize

[0051] A simple target reward function R can be designed as a linear combination of these three objectives:

[0052]

[0053] Among them, w C , w E and w S is the weight corresponding to economic cost, carbon emission and system stability, which is used to balance the importance of these goals. max , E max , and S max They are the possible maximum values ​​of these three indicators under specific conditions, which are used for normalization so that the output of the reward function is within a reasonable range.

[0054] As the scale of power systems continues to expand, traditional optimization algorithms face problems such as high computational complexity and slow convergence when dealing with large-scale systems. The present invention uses a deep reinforcement learning algorithm to train the agent so that it can autonomously learn and optimize in the decision-making process of power system dispatch. The selected algorithms include deep Q-network (DQN), dual deep Q-network (DDPG), proportional policy optimization (PPO), etc., which effectively handle high-dimensional inputs and outputs, as well as continuous or discrete action spaces. Through interaction with the environment, the agent constantly tries different actions, learns according to the reward function, and gradually forms the optimal power dispatch strategy. In deep reinforcement learning, the agent learns the optimal strategy through interaction with the environment. This process usually involves the following key steps:

[0055] (1) Initialization: The parameters of the agent (such as the weights of the neural network) are initialized randomly or obtained through pre-training.

[0056] (2) Exploration and Exploitation: The agent needs to find a balance between exploration (trying new, unknown actions to discover better strategies) and exploitation (using the currently known best strategy to obtain rewards). This can be achieved using the ε-greedy strategy, where ε is the probability of exploration and 1-ε is the probability of exploitation.

[0057] (3) Action selection: At each step, the agent selects an action based on the current state. In the DQN (Deep Q-Network) algorithm, this is usually implemented through a Q network that takes the current state as input and outputs the Q value (i.e. expected reward) of each possible action.

[0058] (4) Environment interaction: The agent performs the selected action and then returns a new state, a reward, and a signal indicating whether the session is over.

[0059] (5) Experience replay: The agent stores the experience of this interaction (current state, action, reward, new state) in an experience replay buffer.

[0060] (6) Network update: In DQN, the agent periodically randomly draws a batch of experiences from the experience replay buffer and uses them to update the Q network. This is usually achieved by minimizing the following loss function: L(θ) = E[yQ(s'α;θ)) 2 ]

[0061] Among them, y is the target Q value, which can be calculated by the Bellman equation, Q(s'a; θ) is the current Q network's prediction for state-action, and θ is the network parameter.

[0062] (7) Target network: To stabilize the training process, DQN uses a target network whose parameters θ are copied from the main network at regular intervals. The target Q value y is calculated through this target network.

[0063] (8) Discount factor: When calculating Q-values, future rewards are discounted by a factor γ to reduce their influence, reflecting that near-term rewards are more important than far-term rewards.

[0064] (9) Repeated iteration: The agent repeats the above steps and gradually improves its strategy through continuous learning.

[0065] (10) Adaptive adjustment: During the training process, the agent may also adjust its learning rate, exploration rate and other hyperparameters based on the current learning progress and environmental feedback.

[0066] Combined with the PPO (Proximal Policy Optimization) algorithm. The process will be different. PPO is a policy gradient method that optimizes the policy through stochastic gradient ascent while ensuring the stability of the policy update through a reward function. The key steps of the PPO algorithm include:

[0067] s1. Collect data: The agent executes the current strategy and collects states, actions, probability ratios (i.e., the ratio of the probabilities of the new and old strategies), and cumulative rewards.

[0068] s2. Calculate PPO cut: For each state-action pair, calculate the PPO cut value, which is a value less than 1 used to limit the amplitude of the policy update.

[0069] s3. Policy update: Update the policy by stochastic gradient ascent using the PPO clipping value and probability ratio.

[0070] s4. Value function update: A value function is updated at the same time to estimate the state value under the current strategy.

[0071] s5. Repeat iteration: Repeat the above steps until the strategy converges.

[0072] The learning process of the agent requires tens of thousands of iterations, each step involving a lot of calculations and data processing. In addition, the learning process of the agent needs to be monitored and adjusted to ensure that it is progressing in the right direction.

[0073] Step 4: The intelligent agent dynamically adjusts and optimizes the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism.

[0074] The uncertainty and real-time changes in power system operation require the dispatch strategy to respond quickly and adapt to new situations. In order to enable the intelligent agent to adapt to the real-time changes in power system operation, the present invention introduces an adaptive adjustment mechanism, such as Figure 3 As shown in Figure 2. This mechanism allows the agent to dynamically adjust its scheduling strategy based on the latest system status and environmental feedback. This mechanism can be implemented based on online learning strategies, policy gradient methods, or exploration and exploitation balance techniques to ensure that the agent continuously optimizes its decisions in a changing environment.

[0075] The adaptive adjustment mechanism is used as a continuous process to continuously execute the above steps in a loop. The above mechanism of the present invention enables the intelligent agent to dynamically adjust its power dispatching strategy according to real-time data and environmental changes, which significantly improves the real-time and adaptability of power dispatching.

[0076] Step 5: Test and verify the adjusted and optimized power dispatch strategy in a simulated power system environment, and apply the verified intelligent agent to power system dispatch.

[0077] After the agent training is completed, the agent's performance is verified and tested by testing it in a simulated power system environment. The simulation test can simulate various power system operation scenarios, including extreme weather, surges in power demand, equipment failures, etc. Through these tests, the agent's dispatch strategy can be evaluated in different situations, and necessary adjustments and optimizations can be made. Ultimately, the fully verified agent will be applied to actual power system dispatch to achieve automation and intelligence in power dispatch.

[0078] Through the above process, the power dispatch optimization method of the present invention not only improves the operating efficiency and reliability of the power system, but also helps to achieve environmental protection goals. The present invention predicts the output of renewable energy through deep learning technology, and integrates the prediction results into the power dispatch optimization model, effectively solving the problem of renewable energy integration. The application capability of the method of the present invention in power systems of different sizes and types, including microgrids, large-scale power systems, and power systems containing a high proportion of renewable energy.

[0079] The above-mentioned power dispatch optimization method of the present invention is explained below in conjunction with two specific embodiments.

[0080] Embodiment 1:

[0081] Step A1: First, collect historical power system operation data, including power generation, load demand, equipment status, weather information, etc. Preprocess the data, including cleaning, normalization, and missing value processing, to ensure the quality of input data.

[0082] Step A2: Use convolutional neural network (CNN) to process spatial correlation data, including the distribution map of generator sets; use recurrent neural network (RNN) or long short-term memory network (LSTM) to process time series data, including load demand changes. The network structure parameters are set as follows: 3 convolutional layers are used, each convolutional layer is connected to a maximum pooling layer; RNN / LSTM uses 2 hidden layers, each containing 128 neurons; activation function ReLU is used for convolutional layers, and tanh is used for recurrent layers; loss function uses mean square error (MSE); optimization algorithm is Adam optimizer.

[0083] Step A3: Define the action space as the start and stop of the generator set, output adjustment and the charging and discharging strategy of the energy storage device. Design the reward function as:

[0084] Economics: Negative power generation costs;

[0085] Environmental protection: negative carbon emissions;

[0086] Stability: A positive score for the system's stability indicator.

[0087] In step A3, the action space defined includes the start and stop operation of the generator set, output adjustment, and the charging and discharging strategy of the energy storage device. In order to quantify the reward function, we can define a specific quantitative indicator for each goal.

[0088] 1. Economic efficiency: usually measured by power generation cost, which can be expressed as the sum of fuel cost and operation and maintenance cost of all generator sets. The quantitative formula can be: Where N is the number of generating units, c i，fuel is the material cost coefficient of the ith generator set, p i is the output of the generator set, c i，maint It is the operating and maintenance cost of the generator set.

[0089] 2. Environmental protection: measured by carbon emissions, which can be expressed as the sum of carbon emissions of all generators. The quantitative formula can be: Among them, e i is the carbon emission coefficient of the ith generator set.

[0090] 3. Stability: This can be measured based on indicators such as system load balance and frequency deviation. A simple way to quantify this is to use the absolute value of the difference between system supply and demand: Where T is the length of the time period considered, D(t) is the demand at time t, and P(t) is the total power generation at time t.

[0091] Combining the above three indicators, we can get a multi-objective reward function R: R = W C (-C)+wE ·(-E)+W S ·S

[0092] Among them, w C , w E and w S They are the weights of economy, environmental protection and stability, which are used to adjust the importance of different goals.

[0093] Step A4: Select the Deep Q Network (DQN) as the deep reinforcement learning algorithm and train it. The parameters of DQN are set as follows: experience replay pool size: 10000; learning rate: 0.001; discount factor: 0.95; exploration rate initial value: 1.0, decaying to 0.01.

[0094] Step A5: During the training process, an adaptive adjustment mechanism is introduced to dynamically adjust the exploration rate and learning rate according to environmental feedback to optimize the learning process.

[0095] Step A6: Test the performance of the agent in a simulated environment. The simulated environment includes different power system states, such as high load demand, extreme weather conditions, etc. Through simulation tests, the dispatch effect of the agent in various situations is evaluated, and the model parameters are adjusted according to the test results.

[0096] Step A7: Evaluate the performance of the agent using metrics such as cost savings, carbon emissions reduction, and system stability improvement. Based on the evaluation results, further optimize the parameters of the deep learning model and reinforcement learning algorithm.

[0097] Step A8: Apply the fully trained and tested agent to the actual power system dispatch. Monitor its operation and continuously optimize the model based on the actual operation data.

[0098] Through the above steps, the present invention realizes an efficient and intelligent power dispatching system, which automatically adjusts the power dispatching strategy according to real-time data and environmental changes, achieves cost savings of about 15%, reduces carbon emissions by about 20%, and significantly improves the stability of the system.

[0099] Embodiment 2:

[0100] Step B1: Collect historical operation data, including power generation, load demand, equipment status, weather information, etc. Extract key features through feature engineering, such as the availability of generators, peak and valley periods of load demand, expected output of renewable energy, etc. Normalize the extracted features so that they can be used as input for the deep learning model.

[0101] Step B2: Design a deep neural network with multiple convolutional layers and recurrent layers to process complex power system state data. The network parameters are set as follows: The convolutional layer uses a 3×3 convolution kernel with a step size of 1 and an activation function of ReLU. The recurrent layer uses LSTM units, and each recurrent layer contains 256 neurons. The fully connected layer is located at the end of the network to output the predicted state. The loss function uses categorical cross entropy, which is suitable for multi-classification problems. The Adam optimizer is used, and the learning rate is set to 0.0005. The model is trained using the back-propagation algorithm until convergence.

[0102] Step B3: Define the action space of power dispatch, including the start and stop of generator sets, output adjustment, charging and discharging of energy storage devices, etc. Design a policy network to generate the best action under given conditions. The policy network can be a deterministic policy or a random policy based on a value function.

[0103] Step B4: According to the goal of power system dispatch, design a reward function that comprehensively considers economy, environmental protection and stability. The specific form of the reward function can be:

[0104] Economical: Negative power generation cost, including fuel cost and operation and maintenance cost.

[0105] Environmental protection: Negative carbon emissions, calculated based on the carbon emission coefficient of the generator set.

[0106] Stability: A stability score based on system load balance and frequency deviation.

[0107] Step B5: PPO (Proximal Policy Optimization) is used as the deep reinforcement learning algorithm to train the policy network. The hyperparameters of the PPO algorithm are set as follows: learning rate: 0.001; discount factor: 0.98; KL divergence weight: 0.01; number of iterations: 10,000. During the training process, a simulated power system environment is used for policy evaluation and optimization.

[0108] Step B6: Implement an adaptive adjustment mechanism that allows the agent to adjust its strategy based on environmental feedback. Use an online learning method to enable the agent to continuously learn and adapt to changes in the power system.

[0109] Step B7: Test the agent in a simulation environment, simulating different operating scenarios, such as load fluctuations, equipment failures, etc. Evaluate the performance of the agent, using indicators such as cost savings rate, carbon emission reduction rate, system stability indicators, etc.

[0110] Step B8: Deploy the fully tested agent to the actual power system. Monitor the operation of the agent to ensure that it dispatches power as expected. Continuously optimize and adjust the agent's strategy based on actual operation data and feedback.

[0111] Through the above steps, the power dispatch optimization method of the present invention can achieve efficient power dispatch in practical applications, significantly improving the economy, environmental protection and stability of the power system. In actual cases, the method can achieve more than 20% cost savings, reduce 30% carbon emissions, and significantly improve the stability and reliability of the system.

[0112] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for optimizing power dispatching based on deep reinforcement learning, which combines deep learning and reinforcement learning by utilizing an efficient and intelligent power dispatching optimization method, and significantly improves the dispatching quality and operation efficiency of the power system. First, the deep learning model can accurately predict the future state of the power system and provide reliable data support for dispatching decisions. Secondly, the design of the multi-objective reward function enables power dispatching to achieve the best balance between economy, environmental protection and system stability. Furthermore, the adaptive adjustment mechanism gives the intelligent agent flexibility in the face of real-time data and environmental changes, ensuring the real-time and adaptability of the dispatching strategy. In addition, the application of the deep reinforcement learning algorithm greatly improves the ability to handle large-scale power system optimization problems, shortens the calculation time, and improves the accuracy of decision-making. Finally, by constructing a simulated test environment, the performance of the intelligent agent is fully verified before actual application, ensuring the reliability and practicality of the dispatching scheme. The present invention improves the economic and environmental performance of the power system, and also enhances the stability and security of the system.

[0113] The present invention may be a system, a method and / or a computer program product. The present invention also discloses a power dispatch optimization system based on deep reinforcement learning based on the aforementioned power dispatch optimization method based on deep reinforcement learning, comprising:

[0114] A state representation module, which is used to model and represent the state of the power system using a deep neural network model;

[0115] An action definition module, used for defining an action space of power dispatch based on the state representation of the power system, wherein the action space includes all dispatch actions in the power system;

[0116] A reward function construction module is used to construct a multi-objective reward function of a power dispatching strategy with economic cost, carbon emissions and system stability as objectives, train an intelligent agent using a deep reinforcement learning algorithm, and optimize the multi-objective reward function based on the state and action space of the power system to obtain a power dispatching strategy;

[0117] An adaptive adjustment module, used for the intelligent agent to dynamically adjust and optimize the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism;

[0118] The simulation test module is used to test and verify the adjusted and optimized power dispatching strategy in a simulated power system environment, and apply the verified intelligent agent to the power system dispatching.

[0119] Based on the spirit of the present invention, those skilled in the art can easily think that a computer program product can be obtained based on the aforementioned power dispatch optimization method based on deep reinforcement learning. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the aforementioned power dispatch optimization method based on deep reinforcement learning.

[0120] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0121] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0122] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A power dispatch optimization method based on deep reinforcement learning, characterized in that: The following steps are involved: Step 1: Use a deep neural network model to model and represent the state of the power system; Step 2: Based on the state representation of the power system, define an action space for power dispatching, where the action space includes all dispatching actions in the power system; Step 3: construct a multi-objective reward function of the power dispatch strategy with economic cost, carbon emissions and system stability as the objectives, train the intelligent agent using a deep reinforcement learning algorithm, and optimize the multi-objective reward function based on the state and action space of the power system to obtain the power dispatch strategy; Step 4: The intelligent agent dynamically adjusts and optimizes the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism; Step 5: Test and verify the adjusted and optimized power dispatch strategy in a simulated power system environment, and apply the verified intelligent agent to power system dispatch.

2. The power dispatch optimization method based on deep reinforcement learning according to claim 1, characterized in that: The dispatching actions in the power system include the start and stop operations of the generator sets, the output adjustment operations, the charging and discharging operations of the energy storage devices and the implementation operations of the demand response.

3. The power dispatch optimization method based on deep reinforcement learning according to claim 2 is characterized in that: The deep reinforcement learning algorithm includes a deep Q network DQN, a dual deep Q network DDPG, or a proportional policy optimization PPO algorithm.

4. The power dispatch optimization method based on deep reinforcement learning according to claim 3 is characterized in that: The adaptive adjustment mechanism includes an online learning strategy, a policy gradient method, or an exploration and exploitation balance technique in reinforcement learning.

5. The power dispatch optimization method based on deep reinforcement learning according to claim 4 is characterized in that: Testing and verifying the adjusted and optimized power dispatch strategy in a simulated power system environment, further including: Simulate various power system operation scenarios, including extreme weather, surge in power demand, and equipment failure, to test and verify the power dispatch strategy.

6. The power dispatch optimization method based on deep reinforcement learning according to claim 5 is characterized in that: The power system includes a microgrid, a large-scale power system, and a power system including renewable energy.

7. A power dispatch optimization system based on deep reinforcement learning, characterized in that: include: A state representation module, which is used to model and represent the state of the power system using a deep neural network model; An action definition module, used for defining an action space of power dispatch based on the state representation of the power system, wherein the action space includes all dispatch actions in the power system; A reward function construction module is used to construct a multi-objective reward function of a power dispatching strategy with economic cost, carbon emissions and system stability as objectives, train an intelligent agent using a deep reinforcement learning algorithm, and optimize the multi-objective reward function based on the state and action space of the power system to obtain a power dispatching strategy; An adaptive adjustment module, used for the intelligent agent to dynamically adjust and optimize the power dispatching strategy according to the real-time data of the power system and environmental change information through an adaptive adjustment mechanism; The simulation test module is used to test and verify the adjusted and optimized power dispatching strategy in a simulated power system environment, and apply the verified intelligent agent to the power system dispatching.

8. The power dispatch optimization system based on deep reinforcement learning according to claim 7 is characterized in that: The dispatching actions in the power system include the start and stop operations of the generator sets, the output adjustment operations, the charging and discharging operations of the energy storage devices and the implementation operations of the demand response.

9. The power dispatch optimization system based on deep reinforcement learning according to claim 8, characterized in that: The deep reinforcement learning algorithm includes a deep Q network DQN, a dual deep Q network DDPG, or a proportional policy optimization PPO algorithm.

10. The power dispatch optimization system based on deep reinforcement learning according to claim 9, characterized in that: The adaptive adjustment mechanism includes an online learning strategy, a policy gradient method, or an exploration and exploitation balance technique in reinforcement learning.

11. The power dispatch optimization system based on deep reinforcement learning according to claim 10, characterized in that: The simulation test module is further used for: Simulate various power system operation scenarios, including extreme weather, surge in power demand, and equipment failure, to test and verify the power dispatch strategy.

12. The power dispatch optimization system based on deep reinforcement learning according to claim 11, characterized in that: The power system includes a microgrid, a large-scale power system, and a power system including renewable energy.

13. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the power dispatch optimization method based on deep reinforcement learning according to any one of claims 1-6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the power dispatch optimization method based on deep reinforcement learning described in any one of claims 1 to 6 are implemented.

Images