Micro-grid multi-objective optimization scheduling method based on improved DQN network

Abstract

The invention discloses a micro-grid multi-objective optimization scheduling method based on an improved DQN network. The method comprises the following steps of obtaining historical data and performing preprocessing; inputting the preprocessed data into a conditional generative adversarial network CGANs, generating power generation data and load power consumption data of each power generation unit in the microgrid, and forming a plurality of operation scenes; reducing the generated scene by adopting a K-means + + clustering method, and extracting a typical scene; optimizing the learning rate of the CGANs and the number of network layers of a generator and a discriminator by using a crown porcupine optimization algorithm CPO, and optimizing the initial centroid selection of K-means + +; establishing a micro-grid model; taking minimization of the total operation cost of the micro-grid system as a target function, and setting system operation constraint conditions; carrying out optimized scheduling by adopting an improved DQN network; inputting typical scene data into the improved DQN network, and outputting an optimal scheduling strategy; inputting the strategy into the micro-grid model for verification, and feeding back a verification result to the DQN network to optimize the strategy; according to the invention, the overall performance and reliability of the micro-grid system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-objective optimization scheduling of microgrids, and in particular to a multi-objective optimization scheduling method for microgrids based on an improved DQN network. Background Art

[0002] Microgrids are a type of power grid that integrates distributed energy resources such as wind and solar power with energy storage systems, enhancing grid flexibility and reliability. However, the randomness and volatility of renewable energy sources pose challenges to microgrid scheduling. Traditional optimization methods, such as mixed-integer linear programming and heuristic algorithms, are widely used in microgrid scheduling but can struggle to make real-time decisions in dynamic and uncertain environments.

[0003] Deep reinforcement learning (DRL), particularly deep Q-networks (DQNs), has attracted attention for its ability to handle complex decision-making problems in dynamic environments. DQNs combine deep learning with reinforcement learning, continuously improving decision-making strategies through interaction with the environment. They have shown great potential in fields such as gaming, robotic control, and autonomous driving. In microgrid scheduling, DQNs can dynamically adjust scheduling plans based on real-time data, enhancing system responsiveness and adaptability.

[0004] However, the traditional DQN algorithm overestimates the true value, leading to unstable performance during training. To address this issue, various improvements have been proposed, such as Dueling DQN, which decomposes the Q-value function into a state-value function and an advantage function, and Double DQN, which uses independent networks to estimate the target Q-value.

[0005] Furthermore, microgrid scheduling involves multiple objectives, such as reducing operating costs, increasing renewable energy utilization, and ensuring grid stability. Multi-objective optimization algorithms are used to balance these objectives, but they may not be effective in dynamic and uncertain microgrid environments. Summary of the Invention

[0006] Objective: This invention provides a multi-objective optimization scheduling method for microgrids based on an improved DQN network. The improved DQN network can better handle the dynamics and uncertainties of microgrids, providing a more stable and efficient scheduling strategy. This method aims to simultaneously optimize multiple objectives and improve the overall performance and reliability of the microgrid system.

[0007] Technical solution: The present invention provides a multi-objective optimization scheduling method for microgrids based on an improved DQN network, comprising the following steps:

[0008] (1) Obtain historical data including power generation data of photovoltaic panels, wind turbines, micro gas turbines, and lithium battery energy storage systems, weather data, and load power consumption data, and interpolate and normalize missing values ​​and zero values;

[0009] (2) The preprocessed data is input into the conditional generative adversarial network (CGANs) to generate power generation data and load power consumption data of each power generation unit in the microgrid, forming various operation scenarios;

[0010] (3) Use K-means++ clustering method to reduce the generated scenes and extract typical scenes;

[0011] (4) Use the crown porcupine optimization algorithm (CPO) to optimize the learning rate of CGANs, the number of network layers of the generator and discriminator, and optimize the initial centroid selection of K-means++;

[0012] (5) Establish a microgrid model that includes wind power, photovoltaics, micro gas turbines, battery energy storage systems, hydrogen storage devices, and electric vehicle clusters;

[0013] (6) Taking the minimization of the total cost of microgrid system operation as the objective function, set the system operation constraints;

[0014] (7) Use the improved DQN network for optimized scheduling: input typical scenario data into the improved DQN network and output the optimal scheduling strategy; input the strategy into the microgrid model for verification, and feed the verification results back to the DQN network to optimize the strategy.

[0015] Furthermore, the interpolation process in step (1) adopts the mean method:

[0016]

[0017] Where x q is the current missing value; x q-1 with x q+1 They are the upper and lower valid values ​​adjacent to the missing value respectively;

[0018] Normalize the padded data and the calculation formula is as follows:

[0019]

[0020] In the formula, x' is the normalized value; x is the original data value; x min is the minimum value in the data set; x max is the maximum value in the data set.

[0021] Furthermore, the objective function of CGANs in step (2) is a minimax game:

[0022]

[0023] Where V(D,G) is a combination of expected values, representing the sum of the loss functions of the generator and the discriminator.

[0024] Furthermore, in step (3), the operation steps of K-means++ include: randomly selecting the first cluster center; calculating the minimum distance from the data point to the existing center; selecting the next center according to probability; iteratively updating the center until convergence, and selecting the data point closest to the center in each cluster as a typical scenario.

[0025] Furthermore, in step (4), the optimization process of the CPO algorithm includes: initialization to generate candidate solutions; cyclic population reduction CPR: Dynamically adjust the population size; where T is a variable that determines the number of cycles, t is the current function evaluation, and T max The maximum number of function evaluations, % represents the remainder operator, N min It is the minimum number of individuals in a newly generated population; exploration phase: visual strategy and sound strategy are used to update the position; development phase: odor strategy and physical attack strategy are used to update the position; position update rule: exploration or development strategy is selected according to a random threshold.

[0026] Furthermore, in step (5), the component mathematical models of the microgrid model include:

[0027]

[0028] Where, P WT and P N are the output power and rated power of the wind turbine (KW), ν, ν in 、ν out and ν N are the actual wind speed, rated wind speed, cut-in wind speed and cut-out wind speed (m / s), A, B and C are the operating characteristic parameters of the wind turbine;

[0029] Photovoltaic power generation model:

[0030]

[0031] Where, I t is the actual light radiation intensity at time t; I stc is the rated light radiation intensity; P PV,t is the photovoltaic output power at time t; P stc is the photovoltaic rated power; T t is the actual temperature of the photovoltaic panel at time t; T env,t is the ambient temperature at time t; T stc is the rated temperature of the photovoltaic panel; k is the power temperature coefficient of the photovoltaic panel;

[0032] Microturbine Model:

[0033]

[0034] Where CMT,t is the total cost of the micro gas turbine; P MT,t is the output power of the micro gas turbine; η MT,t is the operating efficiency of the micro gas turbine; K MT is the operation and maintenance cost coefficient of the micro gas turbine; L is the natural gas price; E LHV is the lower calorific value of natural gas;

[0035] Energy storage system model:

[0036] Charging model:

[0037]

[0038] Discharge model:

[0039]

[0040] Where, SOC ESS (t+1) and SOC ESS (t) is the state of charge of the energy storage battery at time t+1 and t; P ESS,ch (t) and P ESS,dch (t) are the charge and discharge power of the energy storage battery at time t; η ESS is the charging and discharging efficiency of the energy storage system at time t; E ESS is the total capacity of the energy storage battery; Δt is the rate of change of the battery state of charge;

[0041] Electric vehicle charging mathematical model:

[0042]

[0043] 0≤C n,EV (t)≤1

[0044] Where N is the number of electric vehicles; P EV (t) is the total charging power of n EVs at time t; P EVC (t) is the charging power of a single EV at time t; C n,EV (t)) is the charging state of the nth EV at time t; SOC EV (t) is the state of charge of a single EV battery at time t; SOC EV (t+1) is the state of charge of a single EV battery at the next moment; l represents the EV charging efficiency; EVR represents the battery capacity of a single EV;

[0045] Hydrogen storage system model:

[0046] Electrolyzer model:

[0047]

[0048] In the formula, H2O represents water; H2 and O2 distribution represent hydrogen and oxygen; and are the output electric power and input hydrogen power of the electrolyzer at time t; η et is the hydrogen production efficiency of the electrolyzer; and are the upper and lower limits of the input power of the electrolyzer at time t; and are the upper and lower limits of the climbing power of the electrolyzer at time t respectively; is the input hydrogen power of the electrolyzer at time t+1;

[0049] Hydrogen storage tank model:

[0050]

[0051] Where S H,t With S H,t+1 are the hydrogen storage capacity of the hydrogen storage tank at time t and time t+1 respectively; and are the input and output hydrogen power of the hydrogen storage tank at time t respectively; and are the hydrogen storage and release efficiencies of the hydrogen storage tank respectively; and are the minimum and maximum capacities of hydrogen storage tanks respectively; and are the minimum and maximum hydrogen storage power of the hydrogen storage tank respectively; and are the minimum and maximum hydrogen discharge power of the hydrogen storage tank respectively; x H,t It is a 0-1 variable that represents the storage and release status of the hydrogen storage tank.

[0052] Furthermore, the objective function of step (6) is:

[0053] minC=C om +C grid +C car

[0054] Where, C, C om 、C grid 、C car They are the total operating cost of the microgrid system, fuel cost for equipment operation and maintenance, system electricity purchase cost, and carbon treatment cost;

[0055]

[0056] C BESS,t =C om,BESS P ESS,ch +C om,BESS P ESS,dch

[0057] Where C BESS,t is the total cost of energy storage battery; C WT,t is the total cost of wind power generation; C PV,t Total cost of wind power generation; C om,BESS is the operation and maintenance cost coefficient of the energy storage battery; T is the scheduling period; P ESS,ch 、P ESS,dch are the charging and discharging power of the energy storage device respectively;

[0058]

[0059] Where, and are the purchase price and sales price of electricity of the microgrid respectively; and are the purchased power and sold power of the microgrid and the external grid respectively;

[0060]

[0061] Where, is the unit carbon treatment cost; is the total carbon emissions;

[0062] Constraints:

[0063] Electric power balance constraints:

[0064] P L (t) = P PV,t +P WT,t +P MT,t +P grid,t +P BESS,t

[0065] Where, P L (t) is the electric load power; P PV,t is the photovoltaic power generation power; P WT,t P is the wind turbine power generation; MT,t P is the power generated by the gas turbine; grid,t The transmission power of the main network; P BESS,t charging and discharging power for energy storage devices;

[0066] Equipment climbing constraints:

[0067]

[0068] Where, P x,t is the climbing power of device x at time t, P x,t-1 Then it is the climbing power at the previous moment; and are the uphill and downhill power limits of device x respectively; Δt is the duration of the scheduling period;

[0069] Grid transmission power constraints:

[0070]

[0071] Where, and They are the upper and lower limits of the transmission power of the tie line, respectively. A positive value indicates power purchase, while a negative value indicates power sale.

[0072] Energy storage device constraints:

[0073]

[0074] Where, E t is the stored energy of the energy storage device during period t; γ char / dis is the maximum charge and discharge power of the energy storage device; W is the rated capacity of the energy storage device; is the minimum and maximum charge of the energy storage device; N day is the maximum number of cycles per day; S bat,t is the state variable of the energy storage device; U bat,t is the charge and discharge status of the energy storage device in time period t, 1 represents charging and 0 represents discharging;

[0075] Carbon emission constraints:

[0076]

[0077] Where, is the carbon emissions of equipment x during period t; is the maximum allowable carbon emission of equipment x during period t; is the total carbon emissions of equipment x during the scheduling period; is the maximum allowable total carbon emissions of equipment x during the scheduling period.

[0078] Furthermore, in step (7), the improvements to the DQN network include: Q value update improvements:

[0079] Q target =r+γQ(s′,argmaxQ(s′,a′;θ);θ - )

[0080] Where Q target is the Q of the improved DQN target value;

[0081] Prioritized Experience Replay (PER): by priority P(i) = |δ i |+ε PER

[0082] δ i=r′+γmaxQ(s′,a′)-Q(s,a)

[0083] Where, δ i is the timing difference error; ε PER is a small positive number; r′ is the reward value at the next moment;

[0084] Activation function improvement: adopt T-ReLU function.

[0085] An electronic device according to the present invention includes a processor and a memory, wherein the memory stores a computer program, and the processor implements any one of the methods when executing the computer program.

[0086] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements any of the methods described above.

[0087] Beneficial effects: Compared with the existing technology, the present invention has the following significant advantages: the present invention uses CGANs to generate power generation data and load power consumption data of each output part in the microgrid, that is, scenario generation, and adopts K-means++ clustering method to reduce the large number of generated scenarios to obtain typical scenarios, which can fully consider various possible situations and uncertainty factors of microgrid operation, provide more comprehensive and representative data support for subsequent optimization scheduling, and effectively improve the accuracy and reliability of optimization scheduling results. The present invention uses the crown porcupine optimization algorithm CPO to optimize the learning rate of CGANs, the number of network layers of the generator and discriminator, and optimizes the selection of the initial centroid of K-means++, which can enable CGANs to generate high-quality scenario data more efficiently. The K-means++ clustering method can also reduce scenarios more accurately, thereby further improving the efficiency and quality of the entire data processing link, ensuring that the data input into the microgrid model is more typical and practical. The present invention establishes a microgrid model that covers wind power, photovoltaics, micro gas turbines, and battery energy storage systems, and takes the minimization of the total operating cost of the microgrid system as the objective function, establishes system operating constraints, and can comprehensively and accurately simulate the actual operating conditions of the microgrid, fully consider the operating characteristics and cost factors of each component, make the optimization scheduling plan more targeted and operational, and effectively reduce the operating cost of the microgrid. The present invention improves the playback mechanism and activation function of the traditional DQN network experience pool, adopts the improved DQN network as the microgrid multi-objective optimization scheduling method, inputs the obtained typical scenario data into the microgrid model, and obtains the optimal strategy through interactive optimization scheduling between the model and the improved DQN network. The optimal decision is input into the microgrid model for verification and feedback to the improved DQN network, realizing closed-loop control and continuous improvement of the optimized scheduling, which can effectively ensure the stable operation of the microgrid under different working conditions. Compared with traditional methods, it has superior stability and economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 is a microgrid scheduling flow chart of the present invention;

[0089] Figure 2 This is a schematic diagram of the conditional generative adversarial network of the present invention;

[0090] Figure 3 This is the DQN network structure diagram of the present invention;

[0091] Figure 4 This is the improved DQN network structure diagram of the present invention;

[0092] Figure 5 This is a schematic diagram of the microgrid optimization scheduling principle of the present invention. DETAILED DESCRIPTION

[0093] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0094] like Figure 1 As shown, the embodiment of the present invention provides a microgrid multi-objective optimization scheduling method and system based on an improved DQN network, which specifically includes the following steps:

[0095] Step 1: Obtain and process historical data, including power generation data and weather data for photovoltaic panels, wind turbines, micro gas turbines, and lithium battery energy storage systems, as well as load power consumption data. Interpolate and normalize missing and zero values.

[0096] The missing values ​​or zero values ​​in the data are filled using the mean method. The formula is as follows:

[0097]

[0098] Where x q is the current missing value; x q-1 with x q+1 They are the upper and lower valid values ​​adjacent to the missing value respectively;

[0099] Normalize the padded data and the calculation formula is as follows:

[0100]

[0101] In the formula, x' is the normalized value; x is the original data value; x min is the minimum value in the data set; x max is the maximum value in the data set;

[0102] Step 2: Input the processed data into the Conditional Generative Adversarial Network (CGANs) to generate a large amount of power generation data and load power consumption data for each output part in the microgrid, i.e. scenario generation. Figure 2As shown, CGANs mainly contain two parts of generator and discriminator: the target of the generator is to minimize the discrimination probability of the generated samples by the discriminator, that is, the generated samples are expected to deceive the discriminator as much as possible, so that the discriminator thinks that these samples are real; the target of the discriminator is to maximize the discrimination ability of the real samples and the generated samples, that is, the real samples and the generated samples are expected to be identified as accurately as possible:

[0103] Generator:

[0104]

[0105] In the formula, G represents the generator; The target of the generator is to minimize its loss function; is an expected value; z is a noise vector sampled from the prior distribution p z (z); y is conditional information sampled from the data distribution p data (y); -logD(G(z|y)) represents the loss function of the generator; G(z|y) is a sample generated by the generator, and the input is a noise vector z and conditional information y; D(G(z|y)) is the discrimination probability of the generated sample by the discriminator, that is, the probability that the discriminator thinks that the sample is real data;

[0106] Discriminator:

[0107]

[0108] In the formula, D represents the discriminator; The target of the discriminator is to maximize its loss function; x represents a real sample sampled from the data distribution p data (x); logD(x|y) represents the logarithmic loss function of the discrimination probability of the real sample by the discriminator; D(x|y) is the discrimination probability of the real sample by the discriminator; 1-D(G(z|y)) represents the probability that the discriminator thinks that the generated sample is false data;

[0109] The overall objective function of the conditional generative adversarial network CGANs is the game of the generator G and the discriminator D, which can be represented as a minimax game problem, and the formula is as follows:

[0110]

[0111] In the formula, V(D, G) is a combination of expected values, representing the sum of the loss functions of the generator and the discriminator;

[0112] Step 3: For a large number of generated scenes, the K-means++ clustering method is used to reduce a large number of scenes to obtain typical scenes for analysis and processing, and the running steps are as follows:

[0113] Select a data point c from the original data as the first cluster center, i.e. the initial centroid;

[0114] For each unselected data point x, calculate its minimum distance to the selected cluster centers:

[0115]

[0116] where c is the selected cluster center; C is the set of cluster centers; D(x) is the minimum distance of the unselected data point x to the selected cluster centers; ||x-c|| is the Euclidean distance between the data point x and the cluster center c;

[0117] Calculate the probability of each data point being selected as the next cluster center:

[0118]

[0119] where P(x) is the probability of the data point being selected as the next cluster center; X is the set of data points;

[0120] Repeat steps 4.2 and 4.3 to assign data points to the nearest cluster center and continuously update the cluster centers until the cluster centers no longer change, for each cluster, select the data point closest to the cluster center as the representative of the cluster, i.e. a typical scene;

[0121] Step 4: The network layers of the generator and discriminator of CGANs and the selection of the initial centroid of K-means++ are optimized using the CPO algorithm to make the data generated by CGANs and the clustering effect of K-means++ better. The optimization process is as follows:

[0122] Initialization:

[0123]

[0124] where N' is the population size, is the ith candidate solution in the search space, and are the upper and lower bounds of the search range, is a random number between 0 and 1.

[0125] The cyclic population reduction technique (CPR) not only accelerates convergence but also maintains population diversity. This strategy simulates the idea that not all CPs activate defense mechanisms, but only those that are threatened. Therefore, in this strategy, some CPs are taken from the population during the optimization process to accelerate convergence and reintroduced into the population to increase diversity and avoid falling into local minima; this cycle is based on a loop variable T, which determines the number of times the process is performed during the optimization process. The mathematical model of cyclic population size reduction is as follows:

[0126]

[0127] where T is the variable that determines the number of loops, t is the current function evaluation, and T max The maximum number of function evaluations, % represents the remainder or modulo operator, N min is the minimum number of individuals in the newly generated population, so the population size cannot be less than N min .

[0128] Exploration phase: According to the defensive behavior of CPs, when predators are far away, CPs have two defense strategies: visual strategy and acoustic strategy. These strategies involve surveying different areas and engaging in global exploration search. Visual and acoustic strategies allow CPs to scare predators away from them.

[0129] a) First Defense Strategy

[0130] When the CP becomes aware of the predator, it begins to lift and fan its feathers to give the predator a deeper impression. The predator has two options, move towards it or move away from it. In the first option, as the predator moves towards the CP, the distance between the predator and the CP decreases. This option encourages exploration of the area between the predator and the CP to speed up convergence. In contrast, in the second option, the distance between the predator and the CP is maximized because the predator chooses to move away. This option encourages exploration of distant areas to identify unvisited areas that may involve the desired solution. These options are mathematically simulated using a normal distribution to generate random values. If these random values ​​are less than 1 or greater than -1, moving closer to the CP is encouraged. Otherwise, the predator will move away from the CP. This behavior is mathematically simulated as follows:

[0131]

[0132] in, is the position of the i-th CP at time t+1; is the position of the i-th CP at time t; is the evaluation function The best solution for is the vector generated between the i-th CP and a randomly selected CP from the population; τ1 is a random number based on normal distribution, and τ2 is a random value in [0,1]. The mathematical formula is as follows:

[0133]

[0134] Here, r is a random number in [0,1].

[0135] b) Second Defense Strategy

[0136] In this strategy, the porcupine uses sound to create noise and intimidate the predator. When the predator approaches the porcupine, the porcupine's voice becomes louder. The mathematical formula that models this behavior is:

[0137]

[0138] Where, is a random vector between 0 and 1; r1 and r2 are random integers in [1, N]; τ3 is a random value in [0, 1]; and are the positions of two randomly selected CPs; Indicates the location of the predator;

[0139] Exploitation Phase: When a predator approaches, the CP has two defense strategies: scent and physical attack. These strategies focus on localized exploration and exploration, targeting promising areas. The CPO exploits the search space and approaches the optimal solution based on scent and physical attack strategies.

[0140] a) Third Defense Strategy

[0141] In this strategy, the CP secretes a foul odor that spreads in the surrounding area to prevent predators from approaching it. The mathematical formula is as follows:

[0142]

[0143] in, The position of the randomly selected CP is; r3 is a random number in [1, N]; δ is a parameter that controls the search direction; γ t It is a defense factor; is the odor diffusion factor, as follows:

[0144] b) The fourth defense strategy

[0145] The final strategy is a physical attack. When a predator gets very close, the CP will attack it with short, thick feathers. In a physical attack, the two objects merge violently, representing a one-dimensional inelastic collision. The mathematical formula for this physical attack is as follows:

[0146]

[0147] Where, represents the best solution obtained by CP; α is the convergence rate factor; τ4 and τ5 are random values ​​in [0,1]; is the average force affecting the CP of the i-th predator at time t;

[0148] CP Location Update:

[0149]

[0150] Among them, τ6~τ 10 is a randomly generated value between 0 and 1. In each generation, the objective function to be minimized or maximized is used to evaluate To achieve the desired result; Tf is a predetermined constant value between 0 and 1;

[0151] After the above steps, the optimal number of network layers of the CGANs generator and discriminator and the initial centroid of K-means++ are output;

[0152] Step 5: The microgrid model mainly covers wind turbines, photovoltaic generators, micro gas turbines, lithium battery energy storage systems, hydrogen storage devices, loads and electric vehicle clusters.

[0153] Wind power generation model:

[0154]

[0155] Where, P WT and P N are the output power and rated power of the wind turbine (KW), ν, ν in 、ν out and ν N are the actual wind speed, rated wind speed, cut-in wind speed and cut-out wind speed (m / s), A, B and C are the operating characteristic parameters of the wind turbine;

[0156] Photovoltaic power generation model:

[0157]

[0158]

[0159] Where, I t is the actual light radiation intensity at time t; I stc is the rated light radiation intensity; P PV,t is the photovoltaic output power at time t; P stc is the photovoltaic rated power; T tis the actual temperature of the photovoltaic panel at time t; T env,t is the ambient temperature at time t; T stc is the rated temperature of the photovoltaic panel; k is the power temperature coefficient of the photovoltaic panel;

[0160] Microturbine Model:

[0161]

[0162] Where C MT,t is the total cost of the micro gas turbine; P MT,t is the output power of the micro gas turbine; η MT,t is the operating efficiency of the micro gas turbine; K MT is the operation and maintenance cost coefficient of the micro gas turbine; L is the natural gas price; E LHV is the lower calorific value of natural gas;

[0163] Energy storage system model:

[0164] Charging model:

[0165]

[0166] Discharge model:

[0167]

[0168] Where, SOC ESS (t+1) and SOC ESS (t) is the state of charge of the energy storage battery at time t+1 and t; P ESS,ch (t) and P ESS,dch (t) are the charge and discharge power of the energy storage battery at time t; η ESS is the charging and discharging efficiency of the energy storage system at time t; E ESS is the total capacity of the energy storage battery; Δt is the rate of change of the battery state of charge;

[0169] Electric vehicle charging mathematical model:

[0170]

[0171] 0≤C n,EV (t)≤1

[0172] Where N is the number of electric vehicles; P EV (t) is the total charging power of n EVs at time t; P EVC (t) is the charging power of a single EV at time t; C n,EV (t)) is the charging state of the nth EV at time t; SOC EV (t) is the state of charge of a single EV battery at time t; SOCEV (t+1) is the state of charge of a single EV battery at the next moment; l represents the EV charging efficiency; EVR represents the battery capacity of a single EV;

[0173] Hydrogen storage system model:

[0174] Electrolyzer model:

[0175]

[0176] In the formula, H2O represents water; H2 and O2 distribution represent hydrogen and oxygen; and are the output electric power and input hydrogen power of the electrolyzer at time t; η et is the hydrogen production efficiency of the electrolyzer; and are the upper and lower limits of the input power of the electrolyzer at time t; and are the upper and lower limits of the climbing power of the electrolyzer at time t respectively; is the input hydrogen power of the electrolyzer at time t+1;

[0177] Hydrogen storage tank model:

[0178]

[0179] Where S H,t With S H,t+1 are the hydrogen storage capacity of the hydrogen storage tank at time t and time t+1 respectively; and are the input and output hydrogen power of the hydrogen storage tank at time t respectively; and are the hydrogen storage and release efficiencies of the hydrogen storage tank respectively; and are the minimum and maximum capacities of hydrogen storage tanks respectively; and are the minimum and maximum hydrogen storage power of the hydrogen storage tank respectively; and are the minimum and maximum hydrogen discharge power of the hydrogen storage tank respectively; x H,t is a 0-1 variable that characterizes the storage and release status of the hydrogen storage tank;

[0180] Input typical scenario data into the microgrid model to determine the status of each part in the microgrid;

[0181] Step 6: Establish the objective function and constraints:

[0182] Taking the lowest total operating cost of the microgrid system as the objective function, the formula is as follows:

[0183] minC=C om +Cgrid +C car

[0184] Where, C, C om 、C grid 、C car They are the total operating cost of the microgrid system, fuel cost for equipment operation and maintenance, system electricity purchase cost, and carbon treatment cost;

[0185]

[0186] C BESS,t =C om,BESS P ESS,ch +C om,BESS P ESS,dch

[0187] Where C BESS,t is the total cost of energy storage battery; C WT,t is the total cost of wind power generation; C PV,t Total cost of wind power generation; C om,BESS is the operation and maintenance cost coefficient of the energy storage battery; T is the scheduling period; P ESS,ch 、P ESS,dch are the charging and discharging power of the energy storage device respectively;

[0188]

[0189] Where, and are the purchase price and sales price of electricity of the microgrid respectively; and are the purchased power and sold power of the microgrid and the external grid respectively;

[0190]

[0191] Where, is the unit carbon treatment cost; is the total carbon emissions;

[0192] Constraints:

[0193] Electric power balance constraints:

[0194] P L (t) = P PV,t +P WT,t +P MT,t +P grid,t +P BESS,t

[0195] Where, P L (t) is the electric load power; P PV,t is the photovoltaic power generation power; P WT,t P is the wind turbine power generation;MT,t P is the power generated by the gas turbine; grid,t The transmission power of the main network; P BESS,t charging and discharging power for energy storage devices;

[0196] Equipment climbing constraints:

[0197]

[0198] Where, P x,t is the climbing power of device x at time t, P x,t-1 Then it is the climbing power at the previous moment; and are the uphill and downhill power limits of device x respectively; Δt is the duration of the scheduling period;

[0199] Grid transmission power constraints:

[0200]

[0201] Where, and They are the upper and lower limits of the transmission power of the tie line, respectively. A positive value indicates power purchase, while a negative value indicates power sale.

[0202] Energy storage device constraints:

[0203]

[0204] Where, E t is the stored energy of the energy storage device during period t; γ char / dis is the maximum charge and discharge power of the energy storage device; W is the rated capacity of the energy storage device; is the minimum and maximum charge of the energy storage device; N day is the maximum number of cycles per day; S bat,t is the state variable of the energy storage device; U bat,t is the charge and discharge status of the energy storage device in time period t, 1 represents charging and 0 represents discharging;

[0205] Carbon emission constraints:

[0206]

[0207] Where, is the carbon emissions of equipment x during period t; is the maximum allowable carbon emission of equipment x during period t; is the total carbon emissions of equipment x during the scheduling period; is the maximum allowable total carbon emissions of equipment x during the scheduling period;

[0208] Step 7: The DQN architecture consists of a current value network, a target value network, an error function, a replay storage unit, etc. Figure 3 As shown in Figure 1, it uses DNN to estimate the action value function, while the experience replay mechanism and target value network are used to solve the instability and non-convergence problems when DNN approximates the action value function. The DQN algorithm defines the loss function as the variance between the target value and the predicted value, and the calculation formula is:

[0209] L=E[(r+γmaxQ(s′,a′,θ - )-Q(s,a,θ)) 2 ]

[0210]

[0211] Where r is the reward value; s′ is the next state of the environment; a′ is the possible action to be taken next; θ is the network parameter; α is the learning rate; γ is the discount factor; L is the loss function; E is the variance; Q is the action value function; represents the gradient;

[0212] The optimal strategy can be expressed as:

[0213] π * (s)=argmax a Q(s,a;θ)

[0214] Where, π * is the optimal strategy; s and a are the current environment state and action respectively;

[0215] Improvements to DQN include Figure 4 As shown in the figure, the traditional DQN adopts a greedy strategy when updating the Q value or its related parameters. By selecting the action Q value with the maximum expected reward in the next state s' to feedback the previous agent action a, the estimated Q value will be larger than its true value. Therefore, the Q value update method is improved and the calculation formula is:

[0216] Q target =r+γQ(s′,argmaxQ(s′,a′;θ);θ - )

[0217] Where Q target is the Q of the improved DQN target value;

[0218] The replay storage unit in the traditional DQN network uses an experience replay mechanism, which breaks the correlation between samples. Therefore, Prioritized Experience Replay (PER) is used to improve the experience replay mechanism. When extracting samples from the experience pool, the most valuable historical experience is prioritized. The formula is as follows:

[0219]

[0220] Where, P i is the priority of the i-th experience, specifically:

[0221] P(i)=|δ i |+ε PER

[0222] δ i =r′+γmaxQ(s′,a′)-Q(s,a)

[0223] Where, δ i is the timing difference error; ε PER is a small positive number; r′ is the reward value at the next moment;

[0224] While the PReLU activation function activates all eigenvalues ​​of the negative half-axis, some noise will also be activated accordingly, which has a certain impact on the final convergence effect. Therefore, the tanh function is used to optimize the ReLU function to obtain the T-ReLU function to improve the activation function of DQN. The expression of the T-ReLU function is:

[0225]

[0226] Where x is the activation function input; is the slope of the curve; υ is inversely proportional to the value of the activation function in the non-saturated region of the negative axis;

[0227] The data input into the microgrid model is transmitted to the improved DQN network, which is used as the controller of the microgrid to optimize and control multiple objectives of the microgrid according to the load information. The optimal scheduling strategy is given and input into the microgrid mathematical model for verification, such as Figure 5 shown.

Claims

1. A multi-objective optimization scheduling method for microgrids based on an improved DQN network, characterized in that: The following steps are involved: (1) Obtain historical data including power generation data of photovoltaic panels, wind turbines, micro gas turbines, and lithium battery energy storage systems, weather data, and load power consumption data, and interpolate and normalize missing values ​​and zero values; (2) The preprocessed data is input into the conditional generative adversarial network (CGANs) to generate power generation data and load power consumption data of each power generation unit in the microgrid, forming various operation scenarios; (3) Use K-means++ clustering method to reduce the generated scenes and extract typical scenes; (4) Use the crown porcupine optimization algorithm (CPO) to optimize the learning rate of CGANs, the number of network layers of the generator and discriminator, and optimize the initial centroid selection of K-means++; (5) Establish a microgrid model that includes wind power, photovoltaics, micro gas turbines, battery energy storage systems, hydrogen storage devices, and electric vehicle clusters; (6) Taking the minimization of the total operating cost of the microgrid system as the objective function, set the system operation constraints; (7) Use the improved DQN network for optimized scheduling: input typical scenario data into the improved DQN network and output the optimal scheduling strategy; input the strategy into the microgrid model for verification, and feed the verification results back to the DQN network to optimize the strategy.

2. According to claim 1, a multi-objective optimization scheduling method for microgrids based on an improved DQN network is characterized in that: The interpolation process in step (1) uses the mean method: Where x q is the current missing value; x q-1 with x q+1 They are the upper and lower valid values ​​adjacent to the missing value respectively; Normalize the padded data and the calculation formula is as follows: In the formula, x' is the normalized value; x is the original data value; x min is the minimum value in the data set; x max is the maximum value in the data set.

3. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: The objective function of CGANs in step (2) is a minimax game: Where V(D,G) is a combination of expected values, representing the sum of the loss functions of the generator and the discriminator.

4. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: In step (3), the operation steps of K-means++ include: randomly selecting the first cluster center; calculating the minimum distance from the data point to the existing center; selecting the next center according to probability; iteratively updating the center until convergence, and selecting the data point closest to the center in each cluster as a typical scenario.

5. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: In step (4), the optimization process of the CPO algorithm includes: initialization to generate candidate solutions; cyclic population reduction CPR: Dynamically adjust the population size; where T is a variable that determines the number of cycles, t is the current function evaluation, and T max The maximum number of function evaluations, % represents the remainder operator, N min It is the minimum number of individuals in a newly generated population; exploration phase: visual strategy and sound strategy are used to update the position; development phase: odor strategy and physical attack strategy are used to update the position; position update rule: exploration or development strategy is selected according to a random threshold.

6. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: In step (5), the component mathematical models of the microgrid model include: Where, P WT and P N are the output power and rated power of the wind turbine (KW), ν, ν in 、ν out and ν N are the actual wind speed, rated wind speed, cut-in wind speed and cut-out wind speed (m / s), A, B and C are the operating characteristic parameters of the wind turbine; Photovoltaic power generation model: Where, I t is the actual light radiation intensity at time t; I stc is the rated light radiation intensity; P PV,t is the photovoltaic output power at time t; P stc is the photovoltaic rated power; T t is the actual temperature of the photovoltaic panel at time t; T env,t is the ambient temperature at time t; T stc is the rated temperature of the photovoltaic panel; k is the power temperature coefficient of the photovoltaic panel; Microturbine Model: Where C MT,t is the total cost of the micro gas turbine; P MT,t is the output power of the micro gas turbine; η MT,t is the operating efficiency of the micro gas turbine; K MT is the operation and maintenance cost coefficient of the micro gas turbine; L is the natural gas price; E LHV is the lower calorific value of natural gas; Energy storage system model: Charging model: Discharge model: Where, SOC ESS (t+1) and SOC ESS (t) is the state of charge of the energy storage battery at time t+1 and t; P ESS,ch (t) and P ESS,dch (t) are the charge and discharge power of the energy storage battery at time t; η ESS is the charging and discharging efficiency of the energy storage system at time t; E ESS is the total capacity of the energy storage battery; Δt is the rate of change of the battery state of charge; Electric vehicle charging mathematical model: 0≤C n,EV (t)≤1 Where N is the number of electric vehicles; P EV (t) is the total charging power of n EVs at time t; P EVC (t) is the charging power of a single EV at time t; C n,EV (t)) is the charging state of the nth EV at time t; SOC EV (t) is the state of charge of a single EV battery at time t; SOC EV (t+1) is the state of charge of a single EV battery at the next moment; l represents the EV charging efficiency; EVR represents the battery capacity of a single EV; Hydrogen storage system model: Electrolyzer model: In the formula, H2O represents water; H2 and O2 distribution represent hydrogen and oxygen; and are the output electric power and input hydrogen power of the electrolyzer at time t; η et is the hydrogen production efficiency of the electrolyzer; and are the upper and lower limits of the input power of the electrolyzer at time t; and are the upper and lower limits of the climbing power of the electrolyzer at time t respectively; is the input hydrogen power of the electrolyzer at time t+1; Hydrogen storage tank model: Where S H,t With S H,t+1 are the hydrogen storage capacity of the hydrogen storage tank at time t and time t+1 respectively; and are the input and output hydrogen power of the hydrogen storage tank at time t respectively; and are the hydrogen storage and release efficiencies of the hydrogen storage tank respectively; and are the minimum and maximum capacities of hydrogen storage tanks respectively; and are the minimum and maximum hydrogen storage power of the hydrogen storage tank respectively; and are the minimum and maximum hydrogen discharge power of the hydrogen storage tank respectively; x H,t It is a 0-1 variable that represents the storage and release status of the hydrogen storage tank.

7. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: The objective function of step (6) is: minC=C om +C grid +C car Where, C, C om 、C grid 、C car They are the total operating cost of the microgrid system, fuel cost for equipment operation and maintenance, system electricity purchase cost, and carbon treatment cost; C BESS,t =C om,BESS P ESS,ch +C om,BESS P ESS,dch Where C BESS,t is the total cost of energy storage battery; C WT,t is the total cost of wind power generation; C PV,t Total cost of wind power generation; C om,BESS is the operation and maintenance cost coefficient of the energy storage battery; T is the scheduling period; P ESS,ch 、P ESS,dch are the charging and discharging power of the energy storage device respectively; Where, and are the purchase price and sales price of electricity of the microgrid respectively; and are the purchased power and sold power of the microgrid and the external grid respectively; Where, is the unit carbon treatment cost; is the total carbon emissions; Constraints: Electric power balance constraints: P L (t)=P PV,t +P WT,t +P MT,t +P grid,t +P BESS,t Where, P L (t) is the electric load power; P PV,t is the photovoltaic power generation power; P WT,t P is the wind turbine power generation; MT,t P is the power generated by the gas turbine; grid,t The transmission power of the main network; P BESS,t charging and discharging power for energy storage devices; Equipment climbing constraints: Where, P x,t is the climbing power of device x at time t, P x,t-1 is the climbing power at the previous moment; and are the uphill and downhill power limits of device x respectively; Δt is the duration of the scheduling period; Grid transmission power constraints: Where, and They are the upper and lower limits of the transmission power of the tie line, respectively. A positive value indicates power purchase, while a negative value indicates power sale. Energy storage device constraints: Where, E t is the stored energy of the energy storage device during period t; γ char / dis is the maximum charge and discharge power of the energy storage device; W is the rated capacity of the energy storage device; is the minimum and maximum charge of the energy storage device; N day is the maximum number of cycles per day; S bat,t is the state variable of the energy storage device; U bat,t is the charge and discharge status of the energy storage device in time period t, 1 represents charging and 0 represents discharging; Carbon emission constraints: Where, is the carbon emissions of equipment x during period t; is the maximum allowable carbon emission of equipment x during period t; is the total carbon emissions of equipment x during the scheduling period; is the maximum allowable total carbon emissions of equipment x during the scheduling period.

8. The microgrid multi-objective optimization scheduling method based on the improved DQN network according to claim 1 is characterized in that: In step (7), the improvements to the DQN network include: Q value update improvements: Q target n+γQ(s′,argmaxQ(s′,a′sθ)kθ - ) Where Q target is the Q of the improved DQN target value; Priority Experience Replay (PER): By priority P(i)=|δ i |+e PER d i =r′+γmaxQ(s′,a′)-Q(s,a) Where, δ i is the timing difference error; ε PER is a small positive number; r′ is the reward value at the next moment; Activation function improvement: adopt T-ReLU function.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images