Multi-mode switching strategy for isolated island microgrid considering economic regulation based on double Q learning

Abstract

The present invention discloses a multi-mode switching strategy for an isolated microgrid based on double Q learning and considering economic regulation. The strategy includes two design schemes: 1. A hybrid control strategy based on double Q learning is designed to solve the problem of selecting the optimal switching instructions for electrical equipment in microgrids for different external environments; 2. An economic control method based on a consensus algorithm is proposed. The method adopts a consensus algorithm to solve the optimization problem of the marginal cost of microgrid power generation and pollution emissions. The present invention has certain guiding significance for ensuring the stable operation of microgrids, improving the economic performance of microgrids, and reducing pollution emissions of microgrids.

Description

Technical Field

[0001] The invention relates to a design method for a multi-mode switching strategy of an island microgrid considering economic regulation based on double Q learning, and belongs to the field of smart grid control. Background Art

[0002] One of the main tasks of the AC microgrid system is to make full use of renewable energy to meet load demand when the external environment changes. In order to achieve this goal, an effective control mechanism needs to be designed. However, due to the constant changes in the external environment, continuous control signals (for example, the control signals required for the normal operation of each distributed energy source) and discrete control signals (for example, the signals required for the mode switching of each distributed energy source) will exist simultaneously in the system, so the AC microgrid system is usually considered to be a hybrid system. The system operation characteristics are as follows:

[0003] (1) Faced with different external conditions, the AC microgrid system needs to frequently start and stop various electrical equipment, which makes the controlled system should have the characteristics of multi-mode switching.

[0004] (2) Due to the uncertainty of wind turbines and photovoltaics, energy storage units and diesel generators need to adjust the charging and discharging methods under different circumstances. In this case, the energy storage unit and diesel generator should have multi-mode switching characteristics.

[0005] (3) When various devices are running continuously, switching instructions are generated discretely. Since discrete signals and continuous signals appear simultaneously in the AC microgrid system, the operation process of the system has hybrid characteristics.

[0006] At present, there are many studies on mode switching strategies, and these existing hybrid control strategies only consider operational stability without considering operational economy. At the same time, most of the switching instructions in these control strategies need to be calculated manually, which will consume a lot of human resources. While considering operational stability, the economic efficiency of operation should also be considered. In AC microgrid systems, electricity generated by renewable energy sources (such as wind turbines and photovoltaic generators) can usually help users reduce power generation costs, but due to the randomness generated by renewable energy, the system cannot completely eliminate the uncertainty of supply. Therefore, it is very important to design corresponding supporting technologies such as energy storage units and diesel generators. However, using diesel generators to generate electricity will increase power generation costs and pollutant emissions.

[0007] It is precisely for the above reasons that a control strategy that is both feasible and economical should be designed so that the AC microgrid can respond to the constant changes in the external environment of the system in a timely manner while considering the system economy and make the most reasonable mode switching instructions, thereby achieving the effect of stable operation of the AC microgrid system. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide an island microgrid multi-mode switching strategy based on dual Q learning and considering economic regulation. The control strategy is both feasible and economical, so that the AC microgrid can respond to the continuous changes in the external environment of the system in a timely manner while considering the system economy, and make the most reasonable mode switching instructions, thereby achieving the effect of stable operation of the AC microgrid system.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0010] A multi-mode switching strategy for an island microgrid considering economic regulation based on double Q learning is characterized by: designing a hybrid control structure, which is a cyber-physical system, including an information layer and a physical layer;

[0011] The information layer includes DS fusion part, dual-Q learning part and consistency control part; the DS fusion part is used to generate a signal indicating the current operating status and provide a suitable switching strategy; the dual-Q learning part is used to independently search for a suitable control strategy for different operating conditions; the consistency control part is used to reduce the power generation cost and pollutant emissions of diesel generators;

[0012] The physical layer includes various energy generation and storage units, sensors and inverter control parts that provide electric energy for user loads; among them, various energy generation and storage units include wind turbines, photovoltaic power generation, energy storage units and diesel generators, which are used to provide electric energy to maintain the normal operation of loads; sensors are used to transmit voltage data of various energy generation and storage units in the physical layer to form voltage criteria; inverter control parts are used to maintain the normal operation of various energy generation and storage units;

[0013] The operation modes of wind turbines, diesel generators, photovoltaic power generation, energy storage units and user loads in the hybrid control structure are defined. The sensors transmit the voltage data of the bus nodes at the physical layer to the information layer. The information layer generates the voltage criterion L based on the transmitted voltage data and the DS fusion part to evaluate the overall stability of the system. At the same time, it provides the initial state value for the dual-Q learning. Based on the dual-Q learning algorithm, the multi-modal switching strategy is obtained, and the generated modal switching signal is sent to each distributed energy generation and energy storage unit to complete the distributed energy mode switching. At the same time, the diesel generator is economically regulated, and the economic control strategy is designed to reduce the power generation cost and pollutant emissions.

[0014] A further improvement of the technical solution of the present invention is that the operation modes of the wind turbine, diesel generator, photovoltaic power generation, energy storage unit and user load are defined as follows:

[0015] Operation mode of wind turbine: When the wind speed is greater than the minimum allowable wind speed and less than the rated wind speed, the wind turbine operates in the maximum power point MPPT mode; when the wind speed is greater than the rated wind speed and less than the maximum allowable wind speed, the wind turbine operates in the constant power mode;

[0016] How photovoltaic power generation works: Photovoltaic power generation is a way of using solar energy to generate electricity. Multiple photovoltaic cells are combined into a photovoltaic array to provide energy. When the light intensity is greater than the threshold, the photovoltaic array will work in the maximum power point MPPT mode, otherwise it will shut down;

[0017] Operation mode of the energy storage unit: the energy storage unit can not only be charged but also discharged; the charging and discharging process of the energy storage unit is a process of mutual conversion between electrical energy and chemical energy; during the charging process, electrical energy is converted into chemical energy and stored, and during the discharging process, the conversion is reversed;

[0018] Operation mode of diesel generator: In microgrid, diesel generator is used as auxiliary energy. When wind turbine, photovoltaic power generation and energy storage unit cannot meet the user load demand, diesel generator provides additional energy. Diesel generator has two operation modes: working mode and standby mode.

[0019] Operation mode of user load: Two types of loads are considered: fixed load and non-critical load. The microgrid must ensure the safe power supply of the critical load. When the power supply is far less than the load demand, the non-critical load plays a role in maintaining power balance by reducing the load.

[0020] The further improvement of the technical solution of the present invention is: the step of generating the voltage criterion L based on the DS fusion part: first, all the measured bus node voltage deviations will form an identification framework U, which represents an uncertain element The overall set; then the evidence set divided by DS fusion theory is used to judge the voltage identification framework to obtain the credibility function of each individual. The credibility function is a belief function set according to the research ideas. Through the DS combination principle, the information is fused and the individuals whose voltage deviation does not meet the requirements are eliminated; finally, the information fusion rule is used to integrate multiple judgment results into a new voltage judgment index, which can be used as a comprehensive index for evaluating the overall voltage stability of the microgrid.

[0021] A further improvement of the technical solution of the present invention is that: a method for obtaining a multi-modal switching strategy based on a dual-Q learning algorithm is as follows:

[0022] Step 1: The dual Q learning algorithm uses the experience playback mechanism to store the collected data in real time in the form of storage units to obtain sample storage. Q will be updated according to formula (1):

[0023]

[0024] In formula (1), is the Q value under the dual Q learning experience playback mechanism; Q target is the target Q value; R t+1 is the reward function; γ is the learning factor, usually 0.01; S t+1 is the state value, a is the action value;

[0025] In addition, Q and Q- have two different sets of parameters: θ and θ - , used to select the action corresponding to the maximum value of Q; used to evaluate the Q value corresponding to the optimal action, θ, θ - Two sets of parameters are used to complete action selection and strategy evaluation respectively, reducing the risk of overestimation of Q value; in each iteration, the actual network Q will use the learning coefficient γ to learn the target network Q-; the error between Q- and actual Q is within a very small range, reducing the difficulty of finding the optimal solution;

[0026] Step 2: Take the randomly selected samples as the input of the current network and get the current network Q. The loss function will be obtained by the target network with a delay of C steps. θ converts the loss function L into i (θ i ) is minimized to calculate:

[0027]

[0028] In formula (2), S is the state before updating, a is the action before updating; S′ is the state after updating, a′ is the action after updating; θ i are the parameters of the current network in the i-th iteration; is the parameter of the target network in the i-th iteration; θ i Real-time updates; Update after each C iteration; if the loss function L i (θ i ) is different from θ i , then the gradient descent method in step 3 can be used to update the parameters;

[0029] Step 3: According to formula (3), update using the gradient descent method, update once every C steps, and get the target Q:

[0030]

[0031] In formula (3), Descend the gradient of the loss function; is the decreasing gradient of Q under action a in state S; by formula (3), we can get θ i and The update formula is:

[0032]

[0033] where θ i+1 and θ i+c are the network parameters for the i+1th iteration and the i+cth iteration respectively, α is the learning rate for adjusting and updating the Q value, and takes the value [0,1]. According to formulas (3) and (4), Q can be defined as:

[0034]

[0035] Step 4: It is found that the action set and state set will be sent to the two QL processes, and through the cooperation of the two networks, the corresponding action instructions can be effectively obtained according to the current external environment and the operation status of the microgrid.

[0036] The further improvement of the technical solution of the present invention is that: two objective optimization models of minimum power generation cost and reduction of pollutant emissions are proposed for economic regulation of diesel generators. The model is solved by a consistency algorithm. The specific design process is as follows:

[0037] The power generation cost and pollutant emission cost are described as quadratic functions of power, namely:

[0038]

[0039] In formula (6), Q1 is the power generation cost function; Q2 is the pollutant emission cost function; P DG,i is the diesel power generation; α i , β i , γ i are the coefficients of the quadratic term, linear term and constant term of power generation cost respectively; α′ i , β′ i , γ′ i They are respectively the coefficients of the quadratic term, linear term and constant term of pollutant emission cost.

[0040] When achieving the two goals of minimum power generation cost and reduced pollutant emissions, the following equality constraints should also be established;

[0041]

[0042] In formula (7), P pv is the power generation of each photovoltaic generator; P WT,i is the power generation of each wind turbine; P ESU,i is the output of each energy storage battery; P DG,i is the power generation of each diesel generator; P Load,i For the load amount.

[0043] After determining the weight coefficients λ and 1-λ of the two objective functions, the consensus algorithm is used to solve the objective function shown in formula (8);

[0044]

[0045]

[0046] Adding the Lagrange multiplier l to equation (8) yields:

[0047]

[0048] After calculating the partial derivative of formula (8), we get:

[0049]

[0050] To make formula (10) equal to zero, the following conditions must be met:

[0051]

[0052]

[0053] Among them [2α i λ+2α′ i (1-λ)]P i +λβ i +(1-λ)β i ′ is called the marginal cost of the i-th diesel generator considering pollutant emissions, and the i-th DG is recorded as l i ; The condition for obtaining the optimal value is that the marginal cost of the diesel generator must be consistent. Therefore, a controller such as equation (12) is used to complete the task:

[0054]

[0055] According to the above formula, the final marginal cost is adjusted to l L , then the output power of the i-th diesel generator is Each diesel generator has a power limit, the minimum value P DG.i.min , the maximum value P DG.i.max , then the actual output of each diesel generator is

[0056]

[0057] The design of the economic control strategy is completed, and the marginal cost is adjusted to achieve consistency through the consistency controller.

[0058] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0059] (1) The present invention proposes a multi-mode switching strategy based on a dual-Q learning algorithm, which first uses a neural network to generate a value function in the DQL process. Then, the DQL algorithm is used to automatically generate operating instructions for electrical equipment, so that multi-mode switching can independently handle various operating conditions.

[0060] (2) The present invention designs a control strategy based on a consistency algorithm, taking pollutant emissions into consideration, and adjusts the output power of the diesel generator by adjusting the marginal cost of the microgrid, thereby reducing the power generation cost and pollutant emissions of the microgrid under various operating conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a hybrid control structure diagram of the present invention;

[0062] Figure 2 It is a simplified double Q learning structure diagram of the present invention;

[0063] Figure 3 This is the specific process of the dual Q learning operation of the present invention. DETAILED DESCRIPTION

[0064] The present invention proposes a multi-mode switching strategy for an island microgrid based on double Q learning and considering economic regulation. It is a feasible and economical control strategy, which enables the AC microgrid to respond to the continuous changes in the external environment of the system in a timely manner while considering the economy of the system, and make the most reasonable mode switching instructions, thereby achieving the effect of stable operation of the AC microgrid system.

[0065] The process generally includes the following:

[0066] (1) Design a hybrid control structure diagram, which is a cyber-physical system consisting of an information layer and a physical layer. The components, work tasks and workflow of the information layer and the physical layer of the cyber-physical system designed by the present invention are generally introduced.

[0067] (2) Design a switching method that implements various energy mode switching and takes into account the power generation cost and pollution emissions of diesel generators. Define the operation mode of electrical equipment and system voltage criteria in the system of the present invention. Use DS fusion technology to evaluate the overall stability of the system through voltage data and obtain the L index; based on the analysis, design corresponding switching instructions to maintain the stable operation of the microgrid under different conditions.

[0068] (3) Design a multimodal switching strategy based on the dual Q learning algorithm: First, the dual Q learning uses the experience replay mechanism to store the collected data in real time in the form of storage units. Then, the algorithm uses randomly selected samples as the input of the current network and obtains the current network Q. The loss function will be obtained by the target network with a delay of C steps. Finally, it is updated by the gradient descent method, once every C steps, to obtain the target Q. It is found that the action set and state set will be sent to the two QL processes (i.e., the current network and the target network). Moreover, through the cooperation of the two networks, the corresponding action instructions can be effectively obtained according to the current external environment and the operation of the microgrid.

[0069] (4) Designing economic control strategies: Since renewable energy and energy storage devices have low power generation costs and almost no pollution emissions, the optimization focus of the present invention is diesel generators. The economic efficiency proposed by the present invention actually includes two aspects: 1. Under the premise of ensuring the balance between energy supply and demand, the diesel generator has the lowest power generation cost; 2. While optimizing the power generation cost, the emission of pollutants should also be minimized.

[0070] The present invention is further described in detail below in conjunction with embodiments:

[0071] Build as Figure 1 The hybrid control structure diagram shown in the figure. This structure is a cyber-physical system, which consists of an information layer and a physical layer. The information layer mainly includes the following parts: 1. DS fusion part, which is used to generate a signal indicating the current operating status and provide a suitable switching strategy; 2. Dual Q learning part, which is used to independently search for suitable control strategies for different operating conditions; 3. Consistency control part, which is used to reduce the power generation cost and pollutant emissions of diesel generators. The physical layer includes the following parts: 1. Each energy generation and energy storage unit, which is used to provide energy to maintain the normal operation of the load; 2. Sensors, which are used to transmit the voltage data of each energy in the physical layer to form a voltage criterion; 3. Inverter control part, which is used to maintain the normal operation of the energy.

[0072] Inverter is a common power electronic device in the power grid that converts direct current into alternating current. The power grid system is complex and has many units. As an important node in power grid transmission, inverter has also become the focus of power grid-related research. Inverter control strategy has also become an important research topic. The present invention introduces the traditional control principle of inverter, droop control. The so-called droop control is to select a frequency primary droop characteristic curve similar to that of a traditional generator as the control method of the micro-source, that is, to obtain stable frequency and voltage through P / f droop control and Q / V droop control respectively. This control method controls the active power and reactive power output by the micro-source in the microgrid respectively, without the need for communication and coordination between units, achieving the goals of plug-and-play and peer control of micro-sources, ensuring the power balance and frequency unification in the microgrid under the isolated island, and has the characteristics of simplicity and reliability.

[0073] exist Figure 1 In the paper, the overall operation process is described as follows: 1. The voltage data of the bus node at the physical layer is transmitted to the network layer; 2. The voltage criterion is generated based on the transmission data, and the initial state value is provided for the dual-Q learning through the DS fusion part. Subsequently, the dual-Q learning sends the generated mode switching signal to each distributed energy generation and storage unit to complete the operation mode switching; 3. The diesel generator is economically regulated to reduce the power generation cost and pollutant emissions.

[0074] Design and implement various energy mode switching and consider the switching method of diesel generator power generation cost and pollution emission. First, define the operation mode of electrical equipment in the system of the present invention. The operation mode of electrical equipment in the system defined in the present invention is roughly as follows:

[0075] (1) Operation mode of wind turbine: When the wind speed is less than the rated wind speed but greater than the minimum allowable wind speed, the wind turbine will operate in the maximum power point (MPPT) mode; when the wind speed is greater than the rated wind speed but less than the maximum allowable wind speed, the WT will operate in the constant power mode.

[0076] (2) Operation mode of photovoltaic power generation: Photovoltaic power generation is a way of using solar energy to generate electricity. We usually combine multiple photovoltaic cells into a photovoltaic array to provide energy. When the light intensity is greater than the threshold, the photovoltaic array will operate in maximum power point (MPPT) mode, otherwise it will shut down.

[0077] (3) Operation mode of energy storage device unit: Generally speaking, energy storage devices can not only be charged but also discharged. The charging and discharging process of energy storage devices is the process of mutual conversion between electrical energy and chemical energy. During the charging process, electrical energy is converted into chemical energy and stored. During the discharging process, the conversion is reversed.

[0078] (4) Operation mode of diesel generator: In microgrids, diesel generators can be used as auxiliary energy. When renewable energy and energy storage devices cannot meet load demand, diesel generators are needed to provide additional energy. Diesel generators have two operation modes: working mode and standby mode.

[0079] Operation Mode of Load In the present invention, two types of loads are considered: fixed loads (e.g., critical loads) and non-critical loads (e.g., interruptible loads). The microgrid must ensure safe power supply to critical loads. When the power supply is far less than the load demand, non-critical loads play a role in maintaining power balance by reducing load.

[0080] Then, the voltage criterion L is generated based on DS fusion: DS fusion technology is used to evaluate the overall stability of the system through voltage data. The main process is as follows: First, all measured bus node voltage deviations will form an identification framework U, which represents an uncertain element The overall set. Then, the voltage identification framework is judged by the evidence set divided by DS fusion theory, and the credibility function of each individual is obtained. The credibility function is a belief function set according to the research idea. Through the DS combination principle, the information is fused and the individuals whose voltage deviation does not meet the regulations are eliminated. Finally, the information fusion rule is used to integrate multiple judgment results into a new voltage judgment index, which can be used as a comprehensive index for evaluating the overall voltage stability of the microgrid.

[0081] Design a multi-modal switching strategy based on the dual Q learning algorithm: The simplified structure of the dual Q learning algorithm is as follows: Figure 2 As shown in the figure, on the one hand, dual Q learning can use the experience playback mechanism to store data from the external environment as samples in the form of memory units. In addition, the learning algorithm will randomly select samples as the input of the current network, which will interfere with the correlation between samples. On the other hand, when we calculate the target function, the target network Q- will be selected to replace the current network Q. After C steps of iterative calculation, the target network of Q- will also be updated, that is, Q in the current network will be copied to Q- in the target network. In this way, dual Q learning can reduce the correlation between the current network Q and the target network Q-. In addition, Q and Q- and have two different sets of parameters: θ and θ - , used to select the action corresponding to the maximum value of Q; used to evaluate the Q value corresponding to the optimal action, θ, θ - Using two sets of parameters to complete action selection and strategy evaluation respectively reduces the risk of overestimation of Q value. Finally, Q will be updated as follows:

[0082]

[0083] In formula (1), is the Q value under the dual Q learning experience playback mechanism; Q target is the target Q value; R t+1 is the return function (reward function); γ is the learning factor, usually 0.01; S t+1 is the state value, and a is the action value. In addition, Q and Q- have two different sets of parameters: θ and θ - , used to select the action corresponding to the maximum value of Q; used to evaluate the Q value corresponding to the optimal action, θ, θ -Two sets of parameters are used to complete action selection and strategy evaluation respectively, reducing the risk of overestimation of Q value; in each iteration, the actual network Q will use the learning coefficient γ to learn the target network Q-; the error between Q- and the actual Q is in a very small range, reducing the difficulty of finding the optimal solution. Among them, θ is calculated by minimizing the loss function, as shown in formula (2):

[0084]

[0085] Among them, S is the state before updating, a is the action before updating; S′ is the state after updating, a′ is the action after updating; θ i are the parameters of the current network in the i-th iteration; is the parameter of the target network in the i-th iteration; θ i Real-time updates; Update after each C iteration; if the loss function L i (θ i ) is different from θ i , then the gradient descent method in step 3 can be used to update the parameters; if the loss function is different from θ i ,So

[0086]

[0087] in, Descend the gradient of the loss function; is the decreasing gradient of Q under action a in state S.

[0088] From formula (3), we can get θ i and The update formula

[0089]

[0090] where θ i+1 and θ i+c are the network parameters for the i+1th iteration and the i+cth iteration respectively. According to formulas (3) and (4), Q can be defined as:

[0091]

[0092] To summarize the above process: First, dual Q learning uses the experience replay mechanism to store the collected data in real time in the form of storage units. Then, the algorithm uses randomly selected samples as the input of the current network and obtains the current network Q. The loss function will be obtained by the target network with a delay of C steps. Finally, it is updated by the gradient descent method, once every C steps, to obtain the target Q. It is found that the action set and state set will be sent to two QL processes (i.e., the current network and the target network). Moreover, through the cooperation of the two networks, the corresponding action instructions can be effectively obtained according to the current external environment and the operation of the microgrid.

[0093] As auxiliary power supply equipment in microgrids, renewable energy and energy storage equipment can effectively adjust the fluctuations in microgrids when diesel generators are insufficiently powered. They have the characteristics of low power generation cost and low pollution emissions, and are not the main objects of optimization in this invention. In view of the high power generation cost and high pollution emissions of diesel generators, this paper proposes the following two-objective optimization model, which can be solved using a consistency algorithm. The specific design process is as follows.

[0094] In this study, under the premise that the microgrid can meet its own load demand, two optimization objectives are proposed: 1. Minimum power generation cost; 2. Reduce pollutant emissions. Power generation cost and pollutant emissions can be described as a quadratic function of power, that is,

[0095]

[0096] Among them, Q1 is the power generation cost function; Q2 is the pollutant emission cost function; P DG,i is the diesel power generation; α i , β i , γ i are the coefficients of the quadratic term, linear term and constant term of power generation cost respectively; α′ i , β′ i , γ′ i They are respectively the coefficients of the quadratic term, linear term and constant term of pollutant emission cost.

[0097] Additionally, when achieving these two goals, ensure that the following equality constraints are established:

[0098]

[0099] Among them, P pv is the power generation of each photovoltaic generator; P WT,i is the power generation of each wind turbine; P ESU,i is the output of each energy storage battery; P DG,i is the power generation of each diesel generator; P Load,i For the load amount.

[0100] Determine the weight coefficients λ and 1 After -λ, the consensus algorithm should be used to solve the objective function shown in formula (8):

[0101]

[0102]

[0103] Adding the Lagrange multiplier l to equation (8) yields:

[0104]

[0105] After calculating the partial derivative of formula (8), we get:

[0106]

[0107] Let (10) be equal to zero, and we can find that the following conditions need to be met:

[0108]

[0109] Among them [2α i λ+2α′ i (1-λ)]P i +λβ i +(1-λ)β′ i The marginal cost of pollutant emissions of the i-th diesel generator is called l i The condition for obtaining the optimal value is that the marginal cost of the diesel generator must be consistent. Therefore, a controller such as equation (12) can be used to complete the task.

[0110]

[0111] According to the above formula, the final marginal cost is adjusted to l L . Then the output power of the i-th diesel generator can be obtained as Each diesel generator has a power limit (minimum value P DG.i.min , the maximum value P DG.i.max ), then the actual output of each diesel generator is

[0112]

[0113] The design of the economic control strategy has been completed, and the marginal cost can be adjusted to achieve consistency through the consistency controller.

[0114] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-mode switching strategy for island microgrids considering economic regulation based on double Q learning, characterized by: Design a hybrid control structure, which is a cyber-physical system, including an information layer and a physical layer; The information layer includes DS fusion part, dual-Q learning part and consistency control part; the DS fusion part is used to generate a signal indicating the current operating status and provide a suitable switching strategy; the dual-Q learning part is used to independently search for a suitable control strategy for different operating conditions; the consistency control part is used to reduce the power generation cost and pollutant emissions of diesel generators; The physical layer includes various energy generation and storage units, sensors and inverter control parts that provide electric energy for user loads; among them, various energy generation and storage units include wind turbines, photovoltaic power generation, energy storage units and diesel generators, which are used to provide electric energy to maintain the normal operation of loads; sensors are used to transmit voltage data of various energy generation and storage units in the physical layer to form voltage criteria; inverter control parts are used to maintain the normal operation of various energy generation and storage units; Define the operation mode of wind turbines, diesel generators, photovoltaic power generation, energy storage units and user loads in the hybrid control structure. The sensor transmits the voltage data of the bus node at the physical layer to the information layer. The information layer generates the voltage criterion L based on the transmitted voltage data and the DS fusion part to evaluate the overall stability of the system. At the same time, it provides the initial state value for the dual-Q learning. Based on the dual-Q learning algorithm, the multi-mode switching strategy is obtained, and the generated mode switching signal is sent to each distributed energy generation and energy storage unit to complete the distributed energy mode switching. At the same time, the diesel generator is economically regulated, and the economic control strategy is designed to reduce the power generation cost and pollutant emissions. The steps of generating the voltage criterion L based on the DS fusion part are as follows: First, all the measured bus node voltage deviations will form an identification framework U, which represents an uncertain element U i (j) The overall set of voltage identification frameworks is then judged using the evidence set divided by the DS fusion theory to obtain the credibility function of each individual. The information is fused through the DS combination principle to eliminate individuals whose voltage deviations do not meet the requirements. Finally, the information fusion rule is used to integrate multiple judgment results into a new voltage judgment index, which can be used as a comprehensive index for evaluating the overall voltage stability of the microgrid. The method for obtaining a multi-modal switching strategy based on the double Q learning algorithm is as follows: Step 1: The dual Q learning algorithm uses the experience playback mechanism to store the collected data in real time in the form of storage units to obtain sample storage. Q will be updated according to formula (1): In formula (1), is the Q value under the dual Q learning experience playback mechanism; Q target is the target Q value; R t+1 is the reward function; γ is the learning factor, usually 0.01; S t+1 is the state value, a is the action value; Step 2: Take the randomly selected samples as the input of the current network and get the current network Q. The loss function will be obtained by the target network with a delay of C steps. θ converts the loss function L into i (θ i ) is minimized to calculate: In formula (2), S is the state before updating, a is the action before updating; S′ is the state after updating, a′ is the action after updating; θ i are the parameters of the current network in the i-th iteration; is the parameter of the target network in the i-th iteration; θ i Real-time updates; Update after each C iteration; if the loss function L i (θ i ) is different from θ i , then the gradient descent method in step 3 can be used to update the parameters; Step 3: According to formula (3), update using the gradient descent method, update once every C steps, and get the target Q: In formula (3), Descend the gradient of the loss function; is the decreasing gradient of Q under action a in state S; by formula (3), we can get θ i and The update formula is: where θ i+1 and θ i+c are the network parameters for the i+1th iteration and the i+cth iteration respectively, α is the learning rate for adjusting and updating the Q value, and takes the value [0,1]. According to formulas (3) and (4), Q can be defined as: Step 4: It is found that the action set and state set will be sent to the two QL processes, and through the cooperation of the two networks, the corresponding action instructions can be effectively obtained according to the current external environment and the operation status of the microgrid.

2. The multi-mode switching strategy for island microgrids considering economic regulation based on dual Q learning according to claim 1 is characterized in that: The operation modes of wind turbines, diesel generators, photovoltaic power generation, energy storage units and user loads are defined as follows: Operation mode of wind turbine: When the wind speed is greater than the minimum allowable wind speed and less than the rated wind speed, the wind turbine operates in the maximum power point MPPT mode; when the wind speed is greater than the rated wind speed and less than the maximum allowable wind speed, the wind turbine operates in the constant power mode; How photovoltaic power generation works: Photovoltaic power generation is a way of using solar energy to generate electricity. Multiple photovoltaic cells are combined into a photovoltaic array to provide energy. When the light intensity is greater than the threshold, the photovoltaic array will work in the maximum power point MPPT mode, otherwise it will shut down; Operation mode of the energy storage unit: the energy storage unit can not only be charged but also discharged; the charging and discharging process of the energy storage unit is a process of mutual conversion between electrical energy and chemical energy; during the charging process, electrical energy is converted into chemical energy and stored, and during the discharging process, the conversion is reversed; Operation mode of diesel generator: In microgrid, diesel generator is used as auxiliary energy. When wind turbine, photovoltaic power generation and energy storage unit cannot meet the user load demand, diesel generator provides additional energy. Diesel generator has two operation modes: working mode and standby mode. Operation mode of user load: Two types of loads are considered: fixed load and non-critical load. The microgrid must ensure the safe power supply of the critical load. When the power supply is far less than the load demand, the non-critical load plays a role in maintaining power balance by reducing the load.

3. The multi-mode switching strategy for island microgrids considering economic regulation based on dual Q learning according to claim 1 is characterized by: The economic regulation of diesel generators proposes two objective optimization models: minimum power generation cost and reduction of pollutant emissions. The model is solved using a consistency algorithm. The specific design process is as follows: The power generation cost and pollutant emission cost are described as quadratic functions of power, namely: In formula (6), Q1 is the power generation cost function; Q2 is the pollutant emission cost function; P DG,i is the diesel power generation; α i , β i , γ i are the coefficients of the quadratic term, linear term and constant term of power generation cost respectively; α i ′、β i ′、γ i ′ are the coefficients of the quadratic term, linear term and constant term of pollutant emission cost respectively; When achieving the two goals of minimum power generation cost and reduced pollutant emissions, the following equality constraints should also be established; In formula (7), P pv is the power generation of each photovoltaic generator; P WT,i is the power generation of each wind turbine; P ESU,i is the output of each energy storage battery; P DG,i is the power generation of each diesel generator; P Load,i is the load; After determining the weight coefficients λ and 1-λ of the two objective functions, the consensus algorithm is used to solve the objective function shown in formula (8); Adding the Lagrange multiplier l to equation (8) yields: After calculating the partial derivative of formula (8), we get: To make formula (10) equal to zero, the following conditions must be met: Among them [2α i λ+2α′ i (1-λ)]P i +λβ i +(1-λ)β′ i The marginal cost of pollutant emissions of the i-th diesel generator is called l i ; The condition for obtaining the optimal value is that the marginal cost of the diesel generator must be consistent. Therefore, a controller such as formula (12) is used to complete the task: According to the above formula, the final marginal cost is adjusted to l L , then the output power of the i-th diesel generator is Each diesel generator has a power limit, the minimum value P DG.i.min , the maximum value P DG.i.max , then the actual output of each diesel generator is The design of the economic control strategy is completed, and the marginal cost is adjusted to achieve consistency through the consistency controller.

Images