Virtual power plant power generation plan making and economic dispatching method based on deep reinforcement learning

Through deep reinforcement learning combined with mixed integer planning, the problem that reinforcement learning algorithms in virtual power plant scheduling is solved, the feasibility of strict constraint execution and scheduling plan in actual systems is realized, and the stability of the power grid system is improved. efficiency.

CN119009968APending Publication Date: 2024-11-22GUANGZHOU JIAYUAN ELECTRIC POWER TECH
View PDF 0 Cites 7 Cited by

Patent Information

Application Number
CN202411062658.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing reinforcement learning algorithms are difficult to implement strict operational constraints, which limits their application in actual power grid systems, especially in scheduling of virtual power plants.

Method used

Using the virtual power plant power generation plan formulation and economic scheduling method based on deep reinforcement learning, by constructing constraints that meet actual operation, combining DQN algorithm and mixed integer planning, a reinforcement learning network is formed, and the constraints such as power balance, step-up and buck are strictly implemented. .

Benefits of technology

It strictly implements constraints in the action space in real-time operation, ensures the feasibility of the scheduling plan, and improves the stability and operation efficiency of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119009968A_ABST
    Figure CN119009968A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of power grids, and more specifically relates to a virtual power plant power generation plan making and economic dispatching method based on deep reinforcement learning, and the method comprises two stages of offline training and online execution. In offline training, exploration and utilization are balanced through an epsilon-greedy algorithm, and data utilization efficiency and stability are improved in combination with an experience playback mechanism and a target network. Meanwhile, by introducing the target network, the deviation in the training process is reduced, and the training stability is enhanced. In the online execution stage, the deep neural network is converted into a mixed integer programming (MIP) model, and strict execution of constraint conditions is achieved. In specific implementation, a commercial MIP solver is utilized to obtain an optimal action, so that a scheduling strategy is ensured to be more accurate and reasonable in actual execution. The method has an important application prospect in solving the scheduling problem of the power system, scheduling requirements under complex constraint conditions can be effectively met, and the stability and the operation efficiency of the system are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of power grid technology, and more specifically, to a method for formulating power generation plans and economic dispatching of virtual power plants based on deep reinforcement learning. Background Art

[0002] In the construction of new power systems today, the proportion of renewable energy generation represented by photovoltaics and wind power is increasing. Renewable energy sources (RES) are themselves susceptible to unstable factors such as weather conditions, with high volatility and difficulty in prediction, and their parameters are relatively less controllable than traditional power generation equipment. Their large-scale integration in new power systems increases the instability and scheduling difficulty of the system. This has given rise to the concept of "virtual power plants" to integrate traditionally geographically dispersed and diverse power resources. Existing scheduling schemes are mainly divided into model-based methods and model-free deep reinforcement learning algorithms. Model-based methods refer to machine learning methods that explicitly establish mathematical models for data classification and learn and predict through the models. With the increasing proportion of renewable power generation resources in virtual power plants and the increasing number of power equipment types, the data generated by virtual power plants is larger in dimension, which poses a great challenge to the construction of data models. The reinforcement learning method is based on the autonomous exploration and learning of the interactive data between virtual power plants and the external power environment, providing an inspiring solution to the scheduling optimization problem of virtual power plants.

[0003] However, current reinforcement learning algorithms have difficulty enforcing strict operating constraints (such as power balancing, boost or buck constraints), limiting their application in practical systems. Summary of the invention

[0004] The present invention provides a method for formulating power generation plans and economic dispatching of virtual power plants based on deep reinforcement learning, which aims to solve the problem that current reinforcement learning algorithms are difficult to execute strict operating constraints, limiting their application in practical systems.

[0005] The virtual power plant power generation plan formulation and economic dispatch method based on deep reinforcement learning includes the following steps:

[0006] Based on the operating power of the virtual power plant, the constraints that meet the actual operation of the virtual power plant are constructed;

[0007] For virtual power plants, the objective function is constructed under constraints to pursue the goal of minimizing the operating cost of the virtual power plant in the entire time span T during which it participates in the electricity market;

[0008] The real-time scheduling problem of virtual power plants is formulated as a Markov decision process, and the state space, action space, state transition probability function, reward function and discount factor are defined.

[0009] Fit the real reward function based on the DQN algorithm to learn the expected rewards of different action strategies taken in different states;

[0010] The DQN algorithm is combined with mixed integer programming to form a reinforcement learning network, and mixed integer programming is performed based on the trained network parameters in the reinforcement learning network to obtain the best action strategy;

[0011] The method of combining the DQN algorithm with mixed integer programming is to express the deep neural network DNN of the DQN algorithm as a mixed integer linear formula.

[0012] Preferably, the constraints include:

[0013] Virtual power plant power balance constraints:

[0014]

[0015] Where: P pv,i,t , P mt,i,t , P ess,i,t are the power generation of the ith photovoltaic, thermal power and energy storage units at time t; P maket,t is the power purchased from the external power grid at time t; P load,i,t is the power consumption of the i-th load point at time t; N PV Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load;

[0016] Interaction power constraints with external grid:

[0017]

[0018] Where: Indicates the lower limit of the interaction power between the virtual power plant and the external power plant; Indicates the upper limit of the power of interaction between the virtual power plant and the external power grid;

[0019] Output constraints of thermal power generating units:

[0020]

[0021] Where: is the lower limit of the output of the i-th thermal power generating unit; is the upper limit of the output of the i-th thermal power generating unit;

[0022] Thermal power generator set climbing constraints:

[0023]

[0024] Where: is the extreme value of the climbing rate of the i-th thermal power generating unit; P mt,i,t-1 represents the power generation of the i-th thermal power unit at time t-1;

[0025] Energy storage device charging and discharging power constraints:

[0026]

[0027] Where: represents the lower limit of the charging and discharging power of the i-th energy storage device, is the upper limit of the charging and discharging power of the i-th energy storage device;

[0028] Energy storage device state of charge SOC constraints:

[0029]

[0030] Where: represents the lower limit of the charge state of the i-th energy storage device; represents the upper limit of the state of charge of the i-th energy storage device; η soc,i represents the power of charging and discharging of the i-th energy storage; E so,i represents the capacity of the i-th energy storage device; S soc,i,t represents the charge state of the i-th energy storage device at time t; Δt is the decision time interval.

[0031] Load power limit:

[0032]

[0033] Where: P c-load,i,t represents the power consumption of the i-th traditional load at time t; P a-load,i,t represents the power consumption of the i-th adjustable load at time t; is the minimum energy consumption power of the ith adjustable load in the actual operating state; It is the maximum energy consumption of the i-th adjustable load at time t in the actual operating state.

[0034] Preferably, the objective function is as follows:

[0035]

[0036] Where: F is the total operating cost of the virtual power plant; f mt (t) represents the cost of distributed thermal power generators; f ess (t) represents the cost of energy storage system; f maket (t) represents the transaction cost between the virtual power plant and the grid;

[0037]

[0038] Where: a mt represents the secondary parameter of operating cost; b mt represents the linear parameter; c mt Represents a constant parameter; N mt Indicates the number of thermal power units; P mt,i,t represents the power generation of the i-th thermal power generator at time t;

[0039] Energy storage system cost:

[0040]

[0041] Where: N ess Indicates the number of energy storage units; P ess,i,t represents the charging and discharging power of the i-th energy storage unit at time t; c ess The cost coefficient representing the charging and discharging of the energy storage unit;

[0042] Transaction costs between the virtual power plant system and the power grid:

[0043]

[0044] Where: c maket,t is the price of electricity purchased from the external power grid at time t; β is a positive selling discount coefficient less than 1; P maket,t is the power purchased from the external power grid at time t; P maket,t When P > 0, it means that the virtual power plant purchases electricity from the external power grid; maket,t When <0, it means that the virtual power plant sells electricity to the external power grid.

[0045] Preferably, the specific steps of expressing the real-time scheduling problem of the virtual power plant as a Markov decision process are as follows:

[0046] Define the state space:

[0047] S t = {P pv,i,t ,P mt,i,t ,P load,i,t ,S soc,i,t};

[0048] Where: S t represents the state space; P pv,i,t represents the power generation of the ith photovoltaic unit at time t; P mt,i,t represents the power generation of the i-th thermal power unit at time t; P load,i,t is the power consumption of the i-th load point at time t; S soc,i,t represents the charge state of the i-th energy storage device at time t;

[0049] Define the action space:

[0050] a t = {P mt,i,t ,P ess,i,t};

[0051] Where: a t represents a set of action variables; P ess,i,t represents the power generation of the i-th energy storage at time t;

[0052] State transition probability function:

[0053] P=P{S t =s t ′|S t =s t ,A=a t};

[0054] Reward function:

[0055] r t = -σ1[f mt (t)+f ess (t)+f maket (t)]-σ2ΔP t ;

[0056] Where: r t Represents the cost loss during operation; ΔP t is the power imbalance at time step t; f mt (t) represents the cost of distributed thermal power generators; f ess (t) represents the cost of energy storage system; f maket (t) represents the transaction cost between the virtual power plant and the grid; σ1 and σ2 are the reward control coefficients used to control the magnitude and trade-off between minimizing the operating cost and the penalty generated in the case of power imbalance;

[0057]

[0058] Where: P maket,t is the power purchased from the external power grid at time t; N pv Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load;

[0059] Discount factor: using the action value function Q π (S, a) represents the expected reward of executing action a based on strategy π in state S; a discount factor γ with a value range of [0, 1] is introduced.

[0060] Preferably, the real reward function is fitted based on the DQN algorithm to learn the expected rewards generated by different action strategies taken under different states. The specific steps are as follows:

[0061] Define Q π (S t ,A t ) is the action value function, which is used to estimate t Take action t And then follow the expectation of strategy π, and get the action value function based on Bellman expectation formula:

[0062]

[0063] Where: S t represents the state space; r(s t ,a t ) represents the state space s t , the action space is a t The cost loss when γ represents the discount factor;

[0064] The optimal value function Q of the Q function in the DQN algorithm * for:

[0065] Q * (s t ,a t )=E π [r t +γmaxQ π (s t+1 ,a t+1 )];

[0066] Where: Q * (s t ,a t ) is the state space s t , the action space is a t The optimal value function when Q π (s t+1 ,a t+1 ) is used to estimate the t+1 Take action t+1 And then find the expectation of strategy π; E π represents the expected reward function of strategy π.

[0067] During the training process, the loss is calculated based on the training results through the Q function. The DNN network of the DQN algorithm continuously updates the network parameters through back propagation and performs iterative optimization:

[0068]

[0069] Where: Q * The approximation function of ; α represents the learning rate, which is between 0 and 1;

[0070] In S t The strategy to obtain the maximum Q value is recorded as:

[0071]

[0072] In the formula, is the action space, which is a set of actions a; For state s t Take the Q function estimate of action a.

[0073] The loss function of the DQN algorithm is based on the Bellman expectation formula, which uses the mean square error to minimize the gap between the actual Q value and the target Q value:

[0074]

[0075] In the formula, Q θ is the neural network of the target strategy, s t,i and a t,i are the state and action of sample i at time t respectively.

[0076] Preferably, the DQN algorithm introduces an experience replay mechanism, which stores the data of each interaction with the environment in an experience replay buffer; then the data stored in the experience replay buffer is randomly extracted to update the Q function. θ value.

[0077] Preferably, the DQN algorithm includes a target network and a behavior network, wherein the structures of the target network and the behavior network are the same, and the parameters of the behavior network are updated five times, and the parameters of the target network are updated once;

[0078] The target network is used to generate the target of Q value; the behavior network directly learns the Q function that changes in each training and is responsible for predicting the action value in a given state.

[0079] Preferably, the DNN model in the DQN algorithm is converted into a mixed integer programming, and the specific steps are as follows:

[0080] Assume that DNN consists of K+1 layers, including from layer 0 to layer k; layer 0 is the input layer of DNN, and layer k is the output layer; in each layer, there is U k units; the jth unit in the kth layer uses u j,k Indicates; x k is regarded as the output vector of the kth layer, then u j,k The output (j=1,2,…,U k) is the output of the kth layer; when the 0th layer is the input layer, it represents the jth input value of the DNN; for all layers with k ≥ 1, the unit u defined by the DNN j,k The output vector x k It is expressed as:

[0081] x k =h(W k-1 x k-1 +b k-1 );

[0082] Where: W k-1 and b k-1 represents the weight and bias parameter matrix constituting the DNN parameter set θ, θ = {W, b}, h represents the activation function; the activation function is the ReLU activation function, which sets all negative input values ​​to zero and keeps the positive values ​​constant;

[0083] A binary activation variable z is used in the DNN to model the ReLU activation function. Specifically, the big M Formulation method is used to model the ReLU activation function. The binary variable z of the big M Formulation method is used to convert the logical or constraint condition into a linear constraint condition.

[0084] The optimization goal of DNN is to minimize the loss function of the neural network, which is expressed as the summation by layer, that is, the MIP formula is as follows:

[0085]

[0086] Where: Weight deviation is a fixed constant parameter of the neural network, and the loss function of the neural network and The coefficients are also fixed constant parameters. k The value before applying the activation function ReLU to the k-th layer neuron, and are the upper and lower limits of x and s respectively.

[0087] The beneficial effects of the present invention include:

[0088] The present invention improves the Deep Q-network (DQN) and expresses it as a Mixed Integer Programming (MIP) problem, thereby strictly enforcing constraints such as power balance restrictions and boost / lower voltage restrictions in the action space, ensuring that the defined scheduling plan is feasible in real-time operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0090] Figure 1 A schematic diagram of the system structure of a virtual power plant provided in an embodiment of the present invention.

[0091] Figure 2 A flow chart of the MIP-DQN training process provided by an embodiment of the present invention.

[0092] Figure 3 A structural diagram of a deep Q neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0094] See also Figure 1 As shown, the best embodiment of the present invention is further described;

[0095] See also Figure 1 As shown in the figure, the virtual power plant planning and economic dispatch method based on deep learning is Figure 1 The input and output variables of the virtual power plant include two dimensions: the electric energy flow of various physical entity power energy equipment, and the information flow composed of various communication networks; in the virtual fishing ground scheduling research of the present invention, the energy flow during the operation of the virtual power plant is mathematically modeled, mainly focusing on the types of various component equipment in the virtual power plant and the power operation conditions.

[0096] See also Figure 1 As shown in the figure, the virtual power plant includes various equipment such as distributed generators, photovoltaic power generation systems, energy storage systems, and many distributed energy sources such as power loads. The division of the virtual power plant wiring in the figure is not based on the physical distance restrictions between the members of each group, but is divided according to whether the same virtual power plant system is used for scheduling; in order to effectively manage and control the virtual power plant, a central controller is deployed to communicate with the local controllers of each power generation resource in a two-way manner to collect information and issue instructions; the central controller is responsible for dynamically monitoring the power generation and consumption of the virtual power plant, and formulating scheduling plans in real time to control the operation of each member in the virtual power plant.

[0097] by Figure 1Taking the components of the virtual power plant in as an example, from the perspective of operating power, the virtual power plant meets the following constraints during operation:

[0098] Virtual power plant power balance constraints:

[0099] The most important constraint in a virtual power plant is that the output power of all components is strictly equal to the total output power of the virtual power plant to avoid an output power vacuum;

[0100]

[0101] Where: P pv,i,t , P mt,i,t , P ess,i,t are the power generation of the ith photovoltaic, thermal power and energy storage units at time t; P maket,t is the power purchased from the external power grid at time t; P load,i,t is the power consumption of the i-th load point at time t; N PV Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load;

[0102] Interaction power constraints with external grid:

[0103] As a whole, the virtual power plant usually has a maximum input and output power limit with the external infinite power grid, which is expressed as follows:

[0104]

[0105] Where: Indicates the lower limit of the interaction power between the virtual power plant and the external power plant; Indicates the upper limit of the power of interaction between the virtual power plant and the external power grid;

[0106] Output constraints of thermal power generating units:

[0107] The operating range and output upper limit of each thermal generator. The lower limit of output power is specified here to allow the generator to operate under its normal working conditions and not deviate from the actual operating state due to excessive pursuit of optimization goals;

[0108]

[0109] Where: is the lower limit of the output of the i-th thermal power generating unit; is the upper limit of the output of the i-th thermal power generating unit;

[0110] Thermal power generator set climbing constraints:

[0111] When the generator set adjusts its output power and responds to changes in grid demand, its rate of change is limited by the technical characteristics and safety operation limitations of the unit. This constraint is an important consideration to ensure the stable operation of the generator set:

[0112]

[0113] Where: is the extreme value of the climbing rate of the i-th thermal power generating unit; P mt,i,t-1 represents the power generation of the i-th thermal power unit at time t-1;

[0114] Energy storage device charging and discharging power constraints:

[0115] Energy storage devices, such as the commonly used lithium iron phosphate batteries, also have their technical limitations in terms of charging and discharging power;

[0116]

[0117] Where: represents the lower limit of the charging and discharging power of the i-th energy storage device, is the upper limit of the charging and discharging power of the i-th energy storage device;

[0118] Energy storage device state of charge SOC constraints:

[0119] Based on the life and operating status of the energy storage device, combined with its own electrochemical structure characteristics, the upper and lower limits of its storage capacity percentage (i.e., battery state of charge, State of Charge, SOC) are specified. SOC is a unit of measurement used to represent the percentage of the current power stored in the energy storage device relative to its total capacity;

[0120]

[0121] Where: represents the lower limit of the charge state of the i-th energy storage device; represents the upper limit of the state of charge of the i-th energy storage device; η soc,i represents the power of charging and discharging of the i-th energy storage; E so,i represents the capacity of the i-th energy storage device; S soc,i,t represents the charge state of the i-th energy storage device at time t; Δt is the decision time interval.

[0122] It should be noted that in actual operating conditions, the initial SOC value of the battery is random. Therefore, it is appropriate to use a random value in the algorithm to represent the initial SOC value of the energy storage device within a specified SOC range, without having to set a fixed period for the SOC of the energy storage device to return to its initial value in subsequent scheduling.

[0123] Load power limit:

[0124] The loads in a virtual power plant include traditional loads and adjustable loads represented by electric vehicle batteries; the load absorption power is equal to the combined absorption power of traditional loads and adjustable loads; the traditional load power consumption is regarded as a constant, and the adjustable load can be regulated by the virtual power plant control center as a dependent variable; however, the actual power upper and lower limits are still subject to the constraints of physical laws:

[0125]

[0126] Where: P c-load,i,t represents the power consumption of the i-th traditional load at time t; P a-load,i,t represents the power consumption of the i-th adjustable load at time t; is the minimum energy consumption power of the ith adjustable load in the actual operating state; It is the maximum energy consumption of the i-th adjustable load at time t in the actual operating state.

[0127] The virtual power plant optimization scheduling model is a practical mathematical model for maximizing the optimization target. In this embodiment, a method for minimizing the operating cost under the power energy constraints of each component is constructed, that is, the minimum operating cost under strict compliance with the constraints;

[0128] Assuming that the electricity price for each hour is known in advance, the objective function of virtual power plant scheduling optimization is defined, and the optimization goal is to minimize the operating cost within the time span T when the virtual power plant participates in the electricity market, that is, to achieve the following objective function:

[0129]

[0130] Where: F is the total operating cost of the virtual power plant; it is equal to the cost of the distributed thermal power generator f mt (t), energy storage system cost f ess (t) and the cost of purchasing and selling electricity to the power system f maket (t) and.

[0131] According to the empirical formula of thermal power generator output and thermal power generator operating cost, we get:

[0132]

[0133] Where: a mt is the operating cost quadratic parameter, b mt is the linear parameter, c mt is a constant parameter, N mt is the number of thermal power units, P mt,i,t is the power generation of the i-th thermal power generator at time t.

[0134] Energy storage system cost:

[0135]

[0136] Where: N ess Indicates the number of energy storage units; P ess,i,t represents the charging and discharging power of the i-th energy storage unit at time t; c ess The cost coefficient representing the charging and discharging of the energy storage unit;

[0137] Transaction costs between the virtual power plant system and the power grid:

[0138]

[0139] Where: c maket,t is the price of electricity purchased from the external power grid at time t; β is a positive selling discount coefficient less than 1; P maket,t is the power purchased from the external power grid at time t; P maket,t When P > 0, it means that the virtual power plant purchases electricity from the external power grid; maket,t When <0, it means that the virtual power plant sells electricity to the external power grid.

[0140] Based on the above-mentioned objective function and all constraints of the virtual power plant, scientific scheduling is used to minimize the operating costs of the virtual power plant while taking into account system stability and economic benefits; the scheduling model combines market mechanisms, taking into account prices, demand response and energy trading to support the competitiveness of virtual power plants in the electricity market.

[0141] In solving the virtual power plant scheduling problem, this embodiment simulates the virtual power plant central controller as a reinforcement learning "intelligent agent", which can obtain the operating information of each discrete power resource in the virtual power plant and issue instructions to control the operating status of each member; the scheduling intelligence issued by the central controller to each member is the action of the intelligent agent; and the environment is determined by the actions of the virtual power plant itself and the dynamic electricity price of the external power grid; therefore, in this embodiment, the Markov decision process is used to describe the interaction process between the intelligent agent and the external environment; by considering a representative virtual power plant composed of photovoltaics, loads, thermal power generating units and battery energy storage systems; wherein the photovoltaic power generation uses the average value of historical data as a known parameter that changes over time, then the goal of the virtual power plant is to find an optimal strategy for the real-time scheduling of thermal power generating units and energy storage equipment systems in the virtual power plant according to the load power consumption, so as to minimize the daily operating costs of the virtual power plant.

[0142] The Markov decision process consists of a quintuple (S, A, P, R, γ), which includes a state space S, an action space A, a state transition probability function P, a reward function R, and a discount factor γ.

[0143] Define the state space:

[0144] The state space S is the set of all environmental state information that the agent can observe, including the charge state of the controllable energy storage unit, the output power of the thermal generator, the power output of the photovoltaic power generation system, and the current load level. This set represents the state of the interactive environment and provides a basis for the agent to evaluate and make decisions. At time t, the state space S t Defined as:

[0145] S t = {P pv,i,t ,P mt,i,t ,P load,i,t ,S soc,i,t};

[0146] Where: S t represents the state space; P pv,i,t represents the power generation of the ith photovoltaic unit at time t; P mt,i,t represents the power generation of the i-th thermal power unit at time t; P load,i,t is the power consumption of the i-th load point at time t; S soc,i,t represents the charge state of the i-th energy storage device at time t;

[0147] Define the action space:

[0148] The action space A is a series of action variables a taken by the agent t The virtual power plant central controller is an intelligent agent, and the decision variable P mt,i,t ,P ess,i,t is the action taken by the agent, so the action space A at time t is defined as:

[0149] a t = {P mt,i,t ,P ess,i,t};

[0150] Where: a t represents a set of action variables; P ess,i,t represents the power generation of the i-th energy storage at time t;

[0151] State transition probability function:

[0152] The state transition probability function refers to the environment in a state s t Next, perform action a t Then switch to another state s t The probability of ′, the conditional probability is expressed as follows:

[0153] P=P{S t =s t ′|S t=s t ,A=a t};

[0154] In model-based algorithms, probabilities are measured using prior probability distributions or samples from static data. In contrast, reinforcement learning algorithms are model-free approaches that learn probability distributions from data obtained through interaction.

[0155] Reward function:

[0156] The reward function R represents the expected reward that the agent can obtain when transferring to a certain state S=s; in this embodiment, the reward function R is intended to guide the agent to choose the action that minimizes the total cost, while also taking into account the constraints of executing power balance. Therefore, at time t, it is defined as:

[0157] r t = -σ1[f mt (t)+f ess (t)+f maket (t)]-σ2ΔP t ;

[0158] Where: r t Represents the cost loss during operation; ΔP t is the power imbalance at time step t; f mt (t) represents the cost of distributed thermal power generators; f ess (t) represents the cost of energy storage system; f maket (t) represents the transaction cost between the virtual power plant and the grid; σ1 and σ2 are the reward control coefficients used to control the magnitude and trade-off between minimizing the operating cost and the penalty generated in the case of power imbalance;

[0159]

[0160] Where: P maket,t is the power purchased from the external power grid at time t; N pv Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load;

[0161] Discount factor: Decision-making tasks often involve multiple rounds of interactions, that is, sequential decision-making, which emphasizes the correlation between each decision. Therefore, the reward signal r of the decision in the current round t The maximum solution may not be the long-term optimal solution, and the reward for each action cannot be used independently to judge the quality of the action strategy.

[0162] Therefore, in actual tasks, we often use return to evaluate the value of an action. The return function comprehensively considers the short-term and long-term benefits brought by the action. We use the action value function Q π (s,a) represents the expected return of executing action a in state s based on strategy π. Therefore, a discount factor γ with a value range of [0,1) is introduced. The discount factor is introduced because long-term benefits have a certain degree of uncertainty. Sometimes we prefer to get some rewards as soon as possible, so we need to discount the long-term benefits. γ close to 1 focuses more on long-term cumulative rewards, and γ close to 0 considers short-term rewards more.

[0163] Deep Q-Network (DQN) is a reinforcement learning algorithm first proposed by the Google DeepMind team in 2013 that uses deep learning technology. The DQN algorithm originated from the Q-learning algorithm and is a reinforcement learning method based on deep learning that makes effective decisions in a continuous state space. The DQN algorithm was the first to realize the application of deep neural networks in the field of reinforcement learning, and played a key leading role in the rapid development of reinforcement learning algorithms. The application of deep neural networks has greatly improved the accuracy of reinforcement learning, and also marked that the application field of reinforcement learning has been greatly expanded; the original Q-learning algorithm continuously updates the value of the Q table to infinitely approach the true reward function under all (s, a) state pairs.

[0164] However, in this embodiment, in the virtual power plant scheduling, which is a non-exhaustive state continuous space, the method of updating the Q table is not applicable. Therefore, in this embodiment, a method of using a deep neural network (DNN) to find the Q function is used, that is, the DQN method is used to fit the real reward function, so as to learn the expected rewards generated by different action strategies taken under different states. At this time, the Q table is replaced by a more powerful tool, the deep neural network.

[0165] First, define Q π (S t ,A t ) is the action value function, which is used to estimate t Take action t And thereafter follow the expectations of strategy π.

[0166] From the Bellman expectation formula we can know that:

[0167]

[0168] Where: S tRepresents the state space, whose elements are states s t ; r(s t ,a t ) indicates state s t 、Action a t The cost loss when γ represents the discount factor;

[0169] Assume there is one or more optimal strategies π * , that is, the agent can obtain the maximum expected return by taking this strategy from the initial state, then we call this strategy the optimal strategy, denoted by π * . Then we know that the optimal value function Q of the Q function * for:

[0170] Q * (s t ,a t )=E π [r t +γmaxQ π (s t+1 ,a t+1 )];

[0171] Where: Q * (s t ,a t ) is the state space s t , the action space is a t The optimal value function when Q π (s t+1 ,a t+1 ) is used to estimate the t+1 Take action t+1 And then find the expectation of strategy π; E π represents the expected reward function of strategy π.

[0172] During the training process, the loss is calculated based on the training results through the Q function. The DNN network of the DQN algorithm continuously updates the network parameters through back propagation and performs iterative optimization. The incremental change of the Q function is written as:

[0173]

[0174] Where: Q * The approximation function of ; α represents the learning rate, which is between 0 and 1;

[0175] In S t The strategy to obtain the maximum Q value is recorded as:

[0176]

[0177] In the formula, is the action space, which is a set of actions a; For state s t Take the Q function estimate of action a.

[0178] The loss function of the DQN algorithm is based on the Bellman expectation formula, which uses the mean square error to minimize the gap between the actual Q value and the target Q value:

[0179]

[0180] Where: Q θ is the neural network of the target strategy, s t,i and a t,i are the state and action of sample i at time t respectively.

[0181] If the deep neural network parameter is θ, it can be written as In this case, the iterative process shown by the incremental change of the Q function can be regarded as a regression problem, whose goal is to estimate the parameters θ of the deep neural network by stochastic gradient ascent. The specific update method is r t +γmaxQ θtarget This calculates the optimal strategy; using the objective Q function Q θtarget To update the current Q function Q θ .

[0182] In the context of virtual power plant scheduling, the DQN algorithm is applied to optimize the scheduling strategy to reduce operating costs and improve energy efficiency. By establishing a Markov decision process (MDP) model, the scheduling problem of the virtual power plant is formalized as an optimization problem, which includes elements such as state space, action space, reward function, and state transition probability. This model not only considers the operation of power generation and energy storage equipment, but also the dynamic changes of the external power grid.

[0183] The DQN algorithm introduces an experience replay mechanism, which stores the data of each interaction with the environment (state, action, reward, and subsequent state) in a structure called an experience pool. This mechanism allows samples to be randomly drawn from the experience pool during training, which aims to break the correlation between samples, thereby improving the efficiency of data use and enhancing the generalization performance of the model. Specifically, in the offline training phase of deep neural network parameters, DQN sets up a replay buffer (Buffer) r to collect training data transition tuples (s t ,a t ,r t ,s t+1 ) and store them. These tuples will then be randomly selected to update the Q function Q θ. Since the execution of an action will result in a state transition and be accompanied by a reward, each execution of an action will generate a quadruple data, which is then added to the experience pool. There is an intrinsic correlation between these quadruple data because the state transition is coherent. If continuous quadruple data are directly selected as training sets, it may cause model overfitting because these training samples are not independent. To solve this problem, the strategy is to randomly select a small batch of quadruple data from the experience pool as a training batch. This method ensures the independence and identical distribution of training samples, and the sample size of each batch is small, which can speed up the training process, avoid direct correlation between samples, and improve data utilization efficiency.

[0184] During the DNN training process, after each training, Q, which is the training target, θ This increases the deviation caused by frequent changes in the target during training and may cause Q θ In order to improve the stability of training and reduce the deviation during training, we use two neural networks in the DQN algorithm:

[0185] In DQN, the target network has the same structure as the main behavior network, but the frequency of parameter updates is low. When programming, the target network is updated once every five times the behavior network updates its parameters. It is used to generate the target of the Q value, making the target relatively stable during the training process, avoiding divergence or instability caused by frequent updates of the target value.

[0186] The other neural network is the main network, the Behavior Network, which directly learns the Q function that changes with each training and is responsible for predicting the action value in a given state. The parameters of the Behavior Network are frequently updated to adapt to new training results. By using the target network to calculate the Q value target, the Behavior Network can have a stable reference when updating its parameters.

[0187] The pseudo code of the neural network parameter training program is as follows:

[0188]

[0189]

[0190] See also Figure 2 As shown in the figure, during the DQN parameter training process, the dual Q network improves the learning stability and performance through the cooperation of two Q networks in the DQN algorithm. First, initialize the two networks: the main Q network is used to select and update the action strategy, and the target Q network is used to calculate the target Q value. The experience replay buffer stores (s t ,a t ,rt ,s t+1 ) in the form of empirical samples. In the strategy update phase, a batch of empirical samples are randomly sampled from the buffer, the main Q network is used to predict the Q value of the current state, and then the main Q network selects the best action for the next state, and the target Q network calculates the Q value Q of the corresponding action. target The target Q value is composed of the current reward and Q target Then the error between the predicted Q value of the main Q network and the target Q value is calculated through the loss function, and the parameters of the main Q network are optimized.

[0191] The target Q network soft-updates its weights at a certain frequency and obtains the latest weights from the main Q network. In the policy application, the main Q network selects actions based on the current state and the ε-greedy strategy to achieve a balance between exploration and exploitation. The training process continuously updates the experience replay buffer and repeatedly samples and updates the policy process. By calculating the target Q value separately from the main Q network and the target Q network, the dual Q network reduces the estimation bias of the Q value and improves the stability of learning.

[0192] Unlike other deep reinforcement learning algorithms, after training we use the action value function Q θ , instead of the approximate strategy π ω In addition, it is important to note that the power balance constraint is only enforced by adding a penalty in the reward function. Therefore, at the end of training, this equality constraint is often not strictly satisfied. To ensure that the constraint is strictly enforced when deployed or executed online, the constraint enforcement method will be described in detail in the subsequent online execution.

[0193] After the training program converges, when implementing action control, the action value function Q with fixed parameters θ obtained through the training neural network can be used. θ To adjust different energy resources in the virtual power plant. Therefore, we need to adjust θ The optimal strategy π is obtained by inverting the function extreme value * , so that

[0194]

[0195] Since this function Q θ Represents a deep neural network (DNN). In order to derive the optimal strategy π *, the above DNN model can be converted into a mixed integer programming (MIP) formula using the latest progress in deep neural network research. This paper adopts the method of linearizing the Relu function of the excitation layer in the deep neural network to achieve strict execution of the virtual power plant constraints in the scheduling process. The following will be explained in detail: In this way, the action value function can perform economic optimal control operations between various energy resources, so that it can meet specific optimization goals and constraints at the same time, ensuring that the scheduling strategy is more accurate and reasonable in actual execution.

[0196] by Figure 3 Take the deep neural network shown in the figure as an example, each intermediate hidden layer has an activation layer; assume that the deep neural network DNN consists of K+1 layers, including from layer 0 to layer K. Layer 0 is the input layer of DNN, and layer K is its output layer. In each layer, there is U k units, the jth unit in the kth layer uses u j,k Indicates. k is regarded as the output vector of the kth layer, then u j,k The output (j=1,2,…,U k ) is the output of the kth layer. When the 0th layer is used as the input layer, it represents the jth input value of the DNN. For all layers with k ≥ 1, the deep neural network defines the unit u j,k The output vector x k It is expressed as:

[0197] x k =h(W k-1 x k-1 +b k-1 );

[0198] Where W k-1 and b k-1 It is the weight and bias parameter matrix that constitutes the neural network parameter set θ, that is, θ = {W, b}, and h is the activation function.

[0199] We use the ReLU (Rectified Linear Unit) function as the activation function. Its function characteristics are: all negative input values ​​are set to zero and positive values ​​are kept unchanged, that is, f(x) = max(0,x). In this example, it is described as: for vector x k have:

[0200] x k =ReLU(W k-1 x k-1 +b k-1 );

[0201] By definition, a DNN with a fixed parameter θ in a neural network graph can model the ReLU function by using a binary activation variable z (i.e. z can only take the value of 0 or 1, and the binary variable takes the value of 0 or 1 to indicate whether the neuron is activated). The ReLU function is expressed as a linear condition, and the "big M Formulation" is used to be considered a valid MIP problem; usually the big M method includes two parts, big M and binary variable techniques, and is often used to convert logical calculations into standard linear programming. M is set to a very large constant. Sometimes some problems only require the introduction of binary variables, and do not require big M, but it is still called the big M method according to convention.

[0202] The process of converting logical expressions into linear constraint equations using the Big M method can be referred to in the following example:

[0203] Assuming that a power plant is in operation, the power generation capacity is required to be between 400 and 500 kW, that is:

[0204] P = 0 or 400 ≤ P ≤ 500;

[0205] We can declare a binary variable z, then the above problem becomes the following problem:

[0206] z*400≤P≤z*500;

[0207] Assuming that the power plant consumes t tons of fuel, k is a constant, and c is the coefficient of fuel conversion to power, the functional relationship is:

[0208]

[0209] We can use the binary variable z to convert the logical or constraint into a linear constraint as follows:

[0210]

[0211] In the formula: B is a number greater than zero but extremely small, M is a very large constant, thus realizing t>0→z=1, and the constant term k can also be replaced by a variable.

[0212] Therefore, the optimization goal of DNN is to minimize the loss function of the neural network, which is expressed as the summation by layer. The MIP formula can be expressed as:

[0213]

[0214] The weight deviation is a fixed constant parameter of the neural network, and the loss function of the neural network and The coefficients are also fixed constant parameters. k The value before applying the activation function ReLU to the k-th layer neuron, and are the upper and lower limits of x and s respectively.

[0215] The MIP model models the parameter adjustment and activation state of the neural network as a mixed integer linear programming problem with binary variables, ensuring the effectiveness of ReLU activation and the integrity of the network through constraints and objective functions.

[0216] According to the definition and examples of mixed integer programming listed above, it is not difficult to see that if some or all variables in a linear programming problem are integer variables, the mathematical programming problem belongs to a mixed integer programming MIP problem. Integer constraints allow the MIP model to capture the discrete nature of certain decisions. For example, a variable z (i.e., a binary variable) with a value restricted to 0 or 1 can be used to decide whether to activate a neuron.

[0217] The principle of solving mixed integer programming problems usually starts with solving its relaxation problem, that is, omitting the integer restrictions on variables and finding an optimal solution. If the result just meets all the overall restrictions, this relaxed solution is the optimal solution to the original MIP; if not, the branch and bound method is used to discuss the corresponding integer variables, rounding the two ends of the decimal solution, and comparing the respective objective function values ​​for screening, forming the so-called search tree. After completing the screening of the entire search tree, the optimal solution of the MIP problem can be obtained. Other solution strategies, such as node selection strategies, determine which nodes should be further explored, and heuristic strategies are used to reduce the search space and improve search efficiency; cutting plane technology is used to enhance the compactness of the model by adding additional linear constraints to help better describe the feasible domain; parameter adjustment dynamically optimizes algorithm parameters according to the characteristics of the problem to improve the overall solution efficiency.

[0218] In actual operation, we usually use mature commercial processors to solve linear programming problems. With the help of highly optimized commercial solvers such as Gurobi and CPLEX, we can find feasible solutions or optimal solutions more quickly.

[0219] In this embodiment, the python code calls the gurobipy library when implementing the MIP solution operation, and uses a mature algorithm to quickly solve the deep neural network solution, thereby improving the code simplicity and operation efficiency.

[0220] The optimal action obtained by solving this MIP will strictly enforce all operational constraints in the action space. This problem can be expressed as:

[0221]

[0222] The online execution process of the MIP-DQN algorithm is as follows:

[0223]

[0224]

[0225] In this embodiment, the improved MIP-DQN deep reinforcement learning algorithm is used to solve the problem of strict operating constraints in virtual power plant scheduling. The algorithm is divided into two stages: offline training and online execution. In offline training, the ε-greedy algorithm balances exploration and utilization, combined with the experience replay mechanism and target network innovation, to improve data utilization efficiency and stability. At the same time, the introduction of the target network reduces the deviation in the training process and enhances the stability of the training. In the online execution stage, strict execution of constraints is achieved by converting the deep neural network into a mixed integer programming (MIP) model. In the specific implementation, a commercial MIP solver, such as gurobi, is used to solve the MIP model and obtain the optimal action, thereby ensuring that the scheduling strategy is more accurate and reasonable in actual execution. This method has important application prospects in solving power system scheduling problems, can effectively respond to scheduling requirements under complex constraints, and improves the stability and operation efficiency of the system.

[0226] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A virtual power plant power generation plan formulation and economic dispatch method based on deep reinforcement learning, characterized in that: The following steps are involved: Based on the operating power of the virtual power plant, the constraints that meet the actual operation of the virtual power plant are constructed; For virtual power plants, the objective function is constructed under constraints to pursue the goal of minimizing the operating cost of the virtual power plant in the entire time span T during which it participates in the electricity market; The real-time scheduling problem of virtual power plants is formulated as a Markov decision process, and the state space, action space, state transition probability function, reward function and discount factor are defined. Fit the real reward function based on the DQN algorithm to learn the expected rewards of different action strategies taken in different states; The DQN algorithm is combined with mixed integer programming to form a reinforcement learning network, and mixed integer programming is performed based on the trained network parameters in the reinforcement learning network to obtain the best action strategy; The method of combining the DQN algorithm with mixed integer programming is to express the deep neural network DNN of the DQN algorithm as a mixed integer linear formula.

2. The method for virtual power plant power generation planning and economic dispatching based on deep reinforcement learning according to claim 1 is characterized in that: The constraints include: Virtual power plant power balance constraints: Where: P pv,i,t , P mt,i,t , P ess,i,t are the power generation of the ith photovoltaic, thermal power and energy storage units at time t; P maket,t is the power purchased from the external power grid at time t; P load,i,t is the power consumption of the i-th load point at time t; N PV Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load; Interaction power constraints with external power grid: Where: Indicates the lower limit of the interaction power between the virtual power plant and the external power plant; Indicates the upper limit of the power of interaction between the virtual power plant and the external power grid; Output constraints of thermal power generating units: Where: is the lower limit of the output of the i-th thermal power generating unit; is the upper limit of the output of the i-th thermal power generating unit; 1 Thermal power generator set climbing constraints: Where: is the extreme value of the climbing rate of the i-th thermal power generating unit; P mt,i,t-1 represents the power generation of the i-th thermal power unit at time t-1; Energy storage device charging and discharging power constraints: Where: represents the lower limit of the charging and discharging power of the i-th energy storage device, is the upper limit of the charging and discharging power of the i-th energy storage device; Energy storage device state of charge SOC constraint: Where: represents the lower limit of the charge state of the i-th energy storage device; represents the upper limit of the state of charge of the i-th energy storage device; η soc,i represents the power of charging and discharging of the i-th energy storage; E so,i represents the capacity of the i-th energy storage device; S soc,i,t represents the charge state of the i-th energy storage device at time t; Δt is the decision time interval; Load power limit: Where: P c-load,i,t represents the power consumption of the i-th traditional load at time t; P a-load,i,t represents the power consumption of the i-th adjustable load at time t; is the minimum energy consumption power of the ith adjustable load in the actual operating state; It is the maximum energy consumption of the i-th adjustable load at time t in the actual operating state.

3. The virtual power plant planning and economic dispatching method based on deep reinforcement learning according to claim 1 is characterized in that: The objective function is as follows: Where: F is the total operating cost of the virtual power plant; f mt (t) represents the cost of distributed thermal power generators; f ess (t) represents the cost of energy storage system; f maket (t) represents the transaction cost between the virtual power plant and the grid; Where: a mt represents the secondary parameter of operating cost; b mt represents the linear parameter; c mt Represents a constant parameter; N mt Indicates the number of thermal power units; P mt,i,t represents the power generation of the i-th thermal power generator at time t; Energy storage system cost: Where: N ess Indicates the number of energy storage units; P ess,i,t represents the charging and discharging power of the i-th energy storage unit at time t; C ess The cost coefficient representing the charging and discharging of the energy storage unit; Transaction costs between the virtual power plant system and the power grid: Where: c maket,t is the price of electricity purchased from the external power grid at time t; β is a positive coefficient of selling discount less than 1; P maket,t is the power purchased from the external power grid at time t; P maket,t >0 means the virtual power plant purchases electricity from the external power grid; P maket,t When <0, it means that the virtual power plant sells electricity to the external power grid.

4. The method for virtual power plant power generation planning and economic dispatching based on deep reinforcement learning according to claim 1 is characterized in that: The specific steps of formulating the real-time dispatch problem of virtual power plants as a Markov decision process are as follows: Define the state space: S t ={P pv,i,t ,P mt,i,t ,P load,i,t ,S soc,i,t }; Where: S t represents the state space; P pv,i,t represents the power generation of the ith photovoltaic unit at time t; P mt,i,t represents the power generation of the i-th thermal power unit at time t; P load,i,t is the power consumption of the i-th load point at time t; S soc,i,t represents the charge state of the i-th energy storage device at time t; Define the action space: a t ={P mt,i,t ,P ess,i,t }; Where: a t represents a set of action variables; P ess,i,t represents the power generation of the i-th energy storage at time t; State transition probability function: P=P{S t =s t ′| S t =s t ,A=a t }; Reward function: r t =-σ1[f mt (t)+f ess (t)+f maket (t)]-σ2ΔP t ; Where: r t Represents the cost loss during operation; ΔP t is the power imbalance at time step t; f mt (t) represents the cost of distributed thermal power generators; f ess (t) represents the cost of energy storage system; f maket (t) represents the transaction cost between the virtual power plant and the grid; σ1 and σ2 are the reward control coefficients used to control the magnitude and trade-off between minimizing the operating cost and the penalty generated in the case of power imbalance; Where: P maket,t is the power purchased from the external power grid at time t; N pv Indicates the number of photovoltaics; N mt Indicates the number of generators; N ess Indicates the amount of energy storage; N load Indicates the quantity of load; Discount factor: Using the action value function Q π σ, a) represents the expected return of executing action a based on strategy π in state S; a discount factor γ with a value range of [0, 1] is introduced.

5. The method for virtual power plant planning and economic dispatching based on deep reinforcement learning according to claim 1 is characterized in that: The DQN algorithm is used to fit the real reward function and learn the expected rewards of different action strategies taken in different states. The specific steps are as follows: Define Q π (S t , A t ) is the action value function, which is used to estimate t Take action t And then follow the expectation of strategy π, and get the action value function based on Bellman expectation formula: Where: S t Represents the state space, whose elements are states s t ; r(s t , a t ) indicates state s t 、Action a t The cost loss when γ represents the discount factor; The optimal value function Q* of the Q function in the DQN algorithm is: Where: Q * (s t , a t ) is the state space s t , the action space is a t The optimal value function when Q π (s t+1 , a t+1 ) is used to estimate the t+1 Take action t+1 And then find the expectation of strategy π; E π represents the expected reward function of strategy π; During the training process, the loss is calculated based on the training results through the Q function. The DNN network of the DQN algorithm continuously updates the network parameters through back propagation and performs iterative optimization: Where: is the approximation function of Q*; α represents the learning rate, which is between 0 and 1; In S t The strategy π*(S t ) is recorded as: In the formula, is the action space, which is a set of actions a; For state s t Use the Q function estimate of action a; The loss function of the DQN algorithm is based on the Bellman expectation formula, which uses the mean square error to minimize the gap between the actual Q value and the target Q value: Where: Q θ is the neural network of the target strategy, s t , i and a t , i are the state and action of sampling i at time t respectively.

6. The method for virtual power plant power generation planning and economic dispatching based on deep reinforcement learning according to claim 5 is characterized in that: The DQN algorithm introduces an experience replay mechanism, which stores the data of each interaction with the environment in an experience replay buffer; the data stored in the experience replay buffer is then randomly extracted to update the Q function θ value.

7. The method for virtual power plant power generation planning and economic dispatching based on deep reinforcement learning according to claim 5 is characterized in that: The DQN algorithm includes a target network and a behavior network, wherein the structures of the target network and the behavior network are the same, and the parameters of the behavior network are updated five times, and the parameters of the target network are updated once; The target network is used to generate the target of Q value; the behavior network directly learns the Q function that changes in each training and is responsible for predicting the action value in a given state.

8. The method for virtual power plant power generation planning and economic dispatching based on deep reinforcement learning according to claim 5 is characterized in that: The DNN model in the DQN algorithm is converted into a mixed integer program. The specific steps are as follows: Assume that DNN consists of K+1 layers, including from layer 0 to layer k; layer 0 is the input layer of DNN, and layer k is the output layer; in each layer, there is U k units; the jth unit in the kth layer uses u j,k Indicates; x k is regarded as the output vector of the kth layer, then u j,k The output (j=1, 2, ..., U k ) is the output of the kth layer; when the 0th layer is the input layer, it represents the jth input value of the DNN; for all layers with k ≥ 1, the unit u defined by the DNN j,k The output vector x k It is expressed as: x k =h(W k-1 x k-1 +b k-1 ); Where: W k-1 and b k-1 represents the weight and bias parameter matrix constituting the DNN parameter set θ, θ={W, b}, h represents the activation function; the activation function is the ReLU activation function, which sets all negative input values ​​to zero and keeps the positive values ​​constant; A binary activation variable z is used in the DNN to model the ReLU activation function. Specifically, the big M Formulation method is used to model the ReLU activation function. The binary variable z of the big M Formulation method is used to convert the logical or constraint condition into a linear constraint condition. The optimization goal of DNN is to minimize the loss function of the neural network, which is expressed as the summation by layer, that is, the MIP formula is as follows: Where: Weight deviation is a fixed constant parameter of the neural network, and the loss function of the neural network and The coefficient is also a fixed constant parameter; s k The value before applying the activation function ReLU to the k-th layer neuron, and are the upper and lower limits of x and s respectively.

Citation Information

Cited By

  • Virtual power plant multi-stage coordinated regulation and control method and system

    CN119275940A

  • Service network scheduling method based on deep reinforcement learning

    CN119342102A

  • Virtual power plant scheduling method and related equipment

    CN119558621A

  • Field operation energy scheduling system based on Markov chain structure

    CN119651595A

  • Integer programming automatic modeling method based on large language model

    CN119721249A