Micro-grid control parameter setting method and device and electronic equipment

Abstract

The invention provides a micro-grid control parameter setting method and device, and electronic equipment. The method comprises the following steps: acquiring equipment parameters of a micro-grid; constructing a simulation model of the micro-grid based on the equipment parameters; obtaining a small interference stability evaluation model; constructing a Markov decision model for controlling parameter setting, wherein the Markov decision model comprises an energy storage agent, a photovoltaic agent and an integrated coordination agent; determining an energy storage target control parameter value based on the simulation model, the small-interference stability evaluation model and the energy storage intelligent agent; determining a photovoltaic target control parameter value of the photovoltaic agent based on the energy storage target control parameter value, the simulation model, the small interference stability evaluation model and the photovoltaic agent; and correcting the energy storage target control parameter value and the photovoltaic target control parameter value based on the energy storage agent, the photovoltaic agent and the integrated coordination agent. According to the technical scheme, the control parameter setting efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of micro-grid, and in particular to a micro-grid control parameter setting method and device and an electronic device. BACKGROUND

[0002] A micro-grid is a small power generation and distribution system integrating distributed power sources, energy storage devices, photovoltaic devices, etc., which is designed to operate in parallel with an external large power grid or independently (i.e., in island mode). The island micro-grid is the main development trend at present.

[0003] However, there are still some problems in the control parameter setting of the micro-grid, especially the island micro-grid. SUMMARY

[0004] To solve the above problems, the present application provides a micro-grid control parameter setting method and device and an electronic device.

[0005] In a first aspect, the present application provides a micro-grid control parameter setting method, including: obtaining device parameters of a micro-grid; constructing a simulation model of the micro-grid based on the device parameters; obtaining a small disturbance stability evaluation model; constructing a Markov decision model for control parameter setting, the Markov decision model including an energy storage agent, a photovoltaic agent, and an integrated coordination agent; determining an energy storage target control parameter value based on the simulation model, the small disturbance stability evaluation model, and the energy storage agent, wherein the energy storage target control parameter value includes parameter values of at least one control parameter related to an energy storage system; determining a photovoltaic target control parameter value of the photovoltaic agent based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model, and the photovoltaic agent, wherein the photovoltaic target control parameter value includes parameter values of at least one control parameter related to a photovoltaic system; and correcting the energy storage target control parameter value and the photovoltaic target control parameter value based on the energy storage agent, the photovoltaic agent, and the integrated coordination agent.

[0006] In combination with the first aspect, the small disturbance stability evaluation model is obtained by the following steps: randomly assigning values to the device parameters, the control parameters related to the energy storage system, the control parameters related to the photovoltaic system, the control parameters related to the generator, and the filter parameters, and simulating and performing small disturbance stability analysis by using the simulation model to generate a plurality of training samples; and training an initial neural network model based on the training samples to obtain the small disturbance stability evaluation model.

[0007] With reference to the first aspect, the energy storage agent includes a first energy storage agent and a second energy storage agent, and the energy storage target control parameter value includes a first energy storage target control parameter value and a second energy storage target control parameter value; the energy storage target control parameter value is determined based on the simulation model, the small disturbance stability evaluation model and the energy storage agent, including: the first energy storage target control parameter value is determined based on the simulation model, the small disturbance stability evaluation model and the first energy storage agent, wherein the first energy storage target control parameter value includes a parameter value of a DC-DC converter control parameter of the energy storage system; the second energy storage target control parameter value is determined based on the first energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the second energy storage agent, wherein the second energy storage target control parameter value includes a parameter value of a DC-AC converter control parameter of the energy storage system.

[0008] In combination with the first aspect, determining the first energy storage target control parameter value based on the simulation model, the small signal stability evaluation model and the first energy storage agent includes: determining an initial state of the first energy storage agent based on the simulation model; determining an action of the first energy storage agent based on the initial state of the first energy storage agent, and sending the action to the simulation model by using the first energy storage agent; determining a reward corresponding to the action of the first energy storage agent and a next state of the first energy storage agent based on the action of the first energy storage agent by using the simulation model and the small signal stability evaluation model; if the reward corresponding to the action of the first energy storage agent does not satisfy a first convergence condition, updating the action of the first energy storage agent based on the next state of the first energy storage agent by using the first energy storage agent, and updating the reward of the action of the first energy storage agent based on the updated action of the first energy storage agent by using the simulation model and the small signal stability evaluation model until the reward corresponding to the action of the first energy storage agent satisfies the first convergence condition; determining the first energy storage target control parameter value based on the action of the first energy storage agent corresponding to the reward satisfying the first convergence condition; and / or, determining a second energy storage target control parameter value based on the first energy storage target control parameter value, the simulation model, the small signal stability evaluation model and a second energy storage agent includes: taking the first energy storage target control parameter value as a constraint condition; determining an initial state of the second energy storage agent based on the simulation model; determining an action of the second energy storage agent based on the initial state of the second energy storage agent, and sending the action to the simulation model by using the second energy storage agent; determining a reward corresponding to the action of the second energy storage agent and a next state of the second energy storage agent based on the action of the second energy storage agent by using the simulation model and the small signal stability evaluation model; if the reward corresponding to the action of the second energy storage agent does not satisfy a second convergence condition, updating the action of the second energy storage agent based on the next state of the second energy storage agent by using the second energy storage agent, and updating the reward of the action of the second energy storage agent based on the updated action of the second energy storage agent by using the simulation model and the small signal stability evaluation model until the reward corresponding to the action of the second energy storage agent satisfies the second convergence condition; determining the second energy storage target control parameter value based on the action of the second energy storage agent corresponding to the reward satisfying the second convergence condition.

[0009] With reference to the first aspect, the photovoltaic agent comprises a first photovoltaic agent and a second photovoltaic agent, and the photovoltaic target control parameter value comprises a first photovoltaic target control parameter value and a second photovoltaic target control parameter value; the photovoltaic target control parameter value of the photovoltaic agent is determined based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the photovoltaic agent, comprising: the first photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the first photovoltaic agent, wherein the first photovoltaic target control parameter value comprises a parameter value of a DC-DC converter control parameter of the photovoltaic system; the second photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the first photovoltaic target control parameter value, the simulation model, the small disturbance stability evaluation model and the second photovoltaic agent, wherein the second photovoltaic target control parameter value comprises a parameter value of a DC-AC converter control parameter of the photovoltaic system.

[0010] In combination with the first aspect, the first photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the first photovoltaic agent, including: taking the energy storage target control parameter value as a constraint condition; determining an initial state of the first photovoltaic agent based on the simulation model; determining an action of the first photovoltaic agent based on the initial state of the first photovoltaic agent, and sending the action to the simulation model by using the first photovoltaic agent; determining a reward corresponding to the action of the first photovoltaic agent and a next state of the first photovoltaic agent based on the action of the first photovoltaic agent by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the first photovoltaic agent does not satisfy a third convergence condition, updating the action of the first photovoltaic agent based on the next state of the first photovoltaic agent by using the first photovoltaic agent, and updating the reward of the action of the first photovoltaic agent based on the updated action of the first photovoltaic agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the first photovoltaic agent satisfies the third convergence condition; determining the first photovoltaic target control parameter value based on the action of the first photovoltaic agent corresponding to the reward satisfying the third convergence condition; and / or, the second photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the first photovoltaic target control parameter value, the simulation model, the small disturbance stability evaluation model and the second photovoltaic agent, including: taking the energy storage target control parameter value and the first photovoltaic target control parameter value as constraint conditions; determining an initial state of the second photovoltaic agent based on the simulation model; determining an action of the second photovoltaic agent based on the initial state of the second photovoltaic agent, and sending the action to the simulation model by using the second photovoltaic agent; determining a reward corresponding to the action of the second photovoltaic agent and a next state of the second photovoltaic agent based on the action of the second photovoltaic agent by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the second photovoltaic agent does not satisfy a fourth convergence condition, updating the action of the second photovoltaic agent based on the next state of the second photovoltaic agent by using the second photovoltaic agent, and updating the reward of the action of the second photovoltaic agent based on the updated action of the second photovoltaic agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the second photovoltaic agent satisfies the fourth convergence condition; determining the second photovoltaic target control parameter value based on the action of the second photovoltaic agent corresponding to the reward satisfying the fourth convergence condition.

[0011] In combination with the first aspect, the energy storage target control parameter value and the photovoltaic target control parameter value are corrected based on the energy storage agent, the photovoltaic agent and the integrated coordination agent, including: correcting the energy storage target control parameter value and the photovoltaic target control parameter value by using a multi-agent deep deterministic policy gradient algorithm based on the energy storage agent, the photovoltaic agent and the integrated coordination agent.

[0012] In combination with the first aspect, before the Markov decision model of the control parameter setting is constructed, the method further includes: determining an evaluation index representing a control parameter setting effect, the evaluation index including at least one of an electric energy quality index, a voltage tracking characteristic index, a power tracking characteristic index, a dominant oscillation mode damping index representing system stability, and an action response index.

[0013] In the second aspect, the embodiments of the present application further provide a micro-grid control parameter setting device, including an acquisition module, a construction module and a determination module; the acquisition module is configured to acquire device parameters of the micro-grid; the construction module is configured to construct a simulation model of the micro-grid based on the device parameters; the acquisition module is further configured to acquire a small disturbance stability evaluation model; the construction module is further configured to construct a Markov decision model of the control parameter setting, the Markov decision model including an energy storage agent, a photovoltaic agent and an integrated coordination agent; the determination module is configured to determine an energy storage target control parameter value based on the simulation model, the small disturbance stability evaluation model and the energy storage agent, wherein the energy storage target control parameter value includes a parameter value of at least one control parameter related to the energy storage system; the determination module is further configured to determine a photovoltaic target control parameter value of the photovoltaic agent based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the photovoltaic agent, wherein the photovoltaic target control parameter value includes a parameter value of at least one control parameter related to the photovoltaic system; and the determination module is further configured to correct the energy storage target control parameter value and the photovoltaic target control parameter value based on the energy storage agent, the photovoltaic agent and the integrated coordination agent.

[0014] In the third aspect, the embodiments of the present application further provide an electronic device, including: a processor; a memory connected with the processor, the memory being configured to store a computer program, the computer program being configured to be executed by the processor to implement the micro-grid control parameter setting method described above.

[0015] In the fourth aspect, the embodiments of the present application provide a storage medium, the storage medium storing a computer program, the computer program being configured to be executed by a processor to implement the micro-grid control parameter setting method described above.

[0016] In the fifth aspect, the embodiments of the present application provide a computer program product, including computer program instructions, the computer program instructions being configured to be executed by a processor to implement the micro-grid control parameter setting method described above.

[0017] Through the above technical solution, the process of adjusting the control parameter is abstracted as a controlled Markov decision process, and the micro-grid control parameter optimization adjustment strategy is determined through multi-agent deep reinforcement learning, thereby improving the control parameter setting efficiency and accuracy and providing important support for the safe and stable operation of the micro-grid. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1is a flowchart of a micro-grid control parameter setting method provided by an embodiment of the present application.

[0019] Figure 2 is a schematic diagram of a multi-agent network architecture provided by an embodiment of the present application.

[0020] Figure 3 is a flowchart of determining an energy storage target control parameter value provided by an embodiment of the present application.

[0021] Figure 4 is a flowchart of determining a photovoltaic target control parameter value provided by an embodiment of the present application.

[0022] Figure 5 is a structural block diagram of a micro-grid control parameter setting device provided by an embodiment of the present application.

[0023] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0025] An island micro-grid (for example, a mine-type island micro-grid) contains many inverters and has a complex control strategy. There are many control parameters and coupling effects between the control parameters, and it is difficult to grasp the regulation law between the parameters, which increases the complexity of the micro-grid system control parameter setting. The control parameters are affected by the micro-grid operating conditions, environmental conditions, and changes in line impedance. Inappropriate control parameters not only bring power quality problems to the micro-grid, but also challenge the safe and stable operation of the micro-grid. Reasonable control parameters can not only ensure system stability but also meet the requirements of power quality. However, the existing control parameter setting usually relies on manual experience and uses a trial-and-error method for adjustment, which affects the efficiency of control parameter setting.

[0026] Based on this, the embodiment of the present application provides a micro-grid control parameter setting method, a Markov decision model for control parameter setting is constructed, the Markov decision model includes an energy storage agent, a photovoltaic agent and an integrated coordination agent; the energy storage target control parameter value is determined by using the energy storage agent; the photovoltaic target control parameter value is determined by using the photovoltaic agent, and the energy storage target control parameter value is used as a constraint condition of the photovoltaic agent; and the energy storage target control parameter value and the photovoltaic target control parameter value are corrected based on the energy storage agent, the photovoltaic agent and the integrated coordination agent. In the embodiment of the present application, the process of adjusting the control parameter is abstracted as a controlled Markov decision process, and the micro-grid control parameter optimization adjustment strategy is determined through multi-agent deep reinforcement learning, thereby improving the control parameter setting efficiency and accuracy and providing important support for the safe and stable operation of the micro-grid. The control parameter setting refers to adjusting the control parameter to make the micro-grid system achieve the expected performance indicators, such as stability, dynamic response speed, power quality, etc.

[0027] Figure 1 is a flowchart of the micro-grid control parameter setting method provided by an embodiment of the present application. As shown in Figure 1 the micro-grid control parameter setting method in the embodiment includes the following steps.

[0028] Step S110, obtaining the device parameters of the micro-grid.

[0029] Optionally, the device parameters include the maximum power, working voltage, working current and configuration number of the photovoltaic component (i.e. solar cell panel), the single cell voltage and capacity of the energy storage system, the rated voltage and rated capacity of the energy storage system, the motor load power and number, motor daily output statistical data, etc. It can be understood that the above-mentioned device parameters are only examples, and the micro-grid can also include other device parameters, for example, the rated capacity of the transformer, the rated power and output characteristic curve of each distributed power source (such as a fan, a fuel cell, etc.).

[0030] The device parameters of the micro-grid can be obtained from the real-time operation data or historical operation data of the micro-grid monitoring system database, or can be obtained by reading the device parameter configuration file, for example, by reading the micro-grid design drawing.

[0031] Step S120, constructing a simulation model of the micro-grid based on the device parameters.

[0032] Optionally, the simulation model comprises an electromagnetic transient simulation model. Illustratively, a simulation model of the microgrid is built according to the device parameters by using electromagnetic transient simulation software (for example, PSCAD / EMTDC, MATLAB / Simulink, DIgSILENT PowerFactory, etc.). The simulation model should accurately reflect the dynamic and static characteristics of each main component in the microgrid, for example, the power generation characteristics of the photovoltaic system, the charge and discharge characteristics of the energy storage system, the control model of the power conversion system (PCS), etc. Through the simulation model, the operating state of the microgrid under different operating conditions can be simulated, and a virtual test environment is provided for subsequent control parameter tuning.

[0033] In step S130, a small disturbance stability evaluation model is obtained.

[0034] Small disturbance stability analysis is used to evaluate the ability of the microgrid to return to the original equilibrium state or a new equilibrium state after being subjected to a small disturbance. The model is usually based on the state equation of the microgrid, and the small disturbance stability of the system is determined by calculating the eigenvalues of the system. If the real part of the eigenvalue is negative, the system is small disturbance stable; if there is an eigenvalue with a positive real part, the system is small disturbance unstable; if there is an eigenvalue with a zero real part, the system is in a critical stable state. However, since the small disturbance stability analysis process contains nonlinear differential algebraic equations, the calculation complexity is high and the running time is long, which makes it difficult to meet the requirements of microgrid safe and stable operation and control. Therefore, in the embodiments of the present application, a small disturbance stability evaluation model based on deep learning is obtained by training a neural network, which can improve the calculation efficiency, meet the requirements of microgrid safe and stable operation and control, and has good engineering practicability.

[0035] In some embodiments, the small-signal stability evaluation model is trained by the following steps: randomly assigning values to the device parameters, the control parameters related to the energy storage system, the control parameters related to the photovoltaic system, the control parameters related to the generator, and the filter parameters, and performing simulation and small-signal stability analysis using the simulation model to generate a plurality of training samples; based on the training samples, training an initial neural network model to obtain the small-signal stability evaluation model. Exemplarily, the device parameters include photovoltaic power values and grid-connected point voltages, energy storage power values and grid-connected point voltages, motor load power values and voltages, etc.; the control parameters related to the energy storage system include the control parameters of each inverter; the control parameters related to the photovoltaic system include the control parameters of each inverter; the control parameters related to the generator include the control parameters of each inverter; the filter parameters include the inductance values and capacitance values of LC filters and LCL filters, etc. The training samples are divided into a training set and a test set. The inputs of the neural network model include the active power and reactive power of each photovoltaic, the active power and reactive power of each energy storage, the active power and reactive power of each motor load, the voltage of each photovoltaic, the voltage of each energy storage, the voltage of each motor load, the control parameters of each photovoltaic inverter, the control parameters of each energy storage inverter, and the filter parameters; the output of the neural network model includes the dominant oscillation mode damping of each sample. The initial neural network model is trained by the training set to obtain the small-signal stability evaluation model after training; and the small-signal stability evaluation model after training is verified by the test set to ensure that the evaluation accuracy meets the preset requirements. If the test results do not meet the requirements, the model can be improved by increasing the number of training samples, adjusting the structure of the neural network model (such as increasing the number of hidden layers or the number of neurons), optimizing the activation function or learning rate, etc., until the performance of the model meets the standards.

[0036] In the embodiments of the present application, the initial neural network model is trained by a large number of training samples, so that the small-signal stability evaluation model after training can quickly and accurately evaluate the small-signal stability of the microgrid according to the input parameter combination, avoiding the complex eigenvalue calculation process in traditional small-signal stability analysis, and significantly improving the evaluation efficiency, providing an efficient stability judgment basis for subsequent parameter optimization of the intelligent agent.

[0037] Optionally, the method further includes determining an evaluation index representing the setting effect of the control parameters. The evaluation index is used to determine the reward in the Markov decision process. Exemplarily, the evaluation index includes at least one of an electric energy quality index, a voltage tracking characteristic index, a power tracking characteristic index, a dominant oscillation mode damping index representing system stability, and an action response index.

[0038] Specifically, (1) the electric energy quality index includes a total harmonic distortion rate THD u of the voltage waveform and a total harmonic distortion rate THD i of the current waveform.

[0039]

[0040] wherein U1 represents the fundamental voltage effective value, I1 represents the fundamental current effective value, U2, U3, K, U n represent the harmonic voltage effective value, I2, I3, K, I n represent the harmonic current effective value.

[0041] (2) Voltage tracking characteristic index f(U i )

[0042]

[0043] wherein T represents the running time, U i (t) represents the output voltage of the i-th inverter at time t, U i,ref (t) represents the reference voltage of the i-th inverter at time t.

[0044] (3) Power tracking characteristic index f(P i )

[0045]

[0046] wherein T represents the running time, P i (t) represents the output power of the i-th inverter at time t, and P s (t) represents the reference power of the i-th inverter at time t.

[0047] (4) Dominant oscillation mode damping index λ representing the stability of the system, which can be determined by a small disturbance stability evaluation model.

[0048] (5) Action response index includes adjustment time index t s representing response speed and overshoot suppression index δ 2 .

[0049] Step S140, a Markov decision model of control parameter setting is constructed.

[0050] Markov decision model (MDP) is a mathematical model for describing sequential decision-making in uncertain environment, which is composed of state space, action space, reward function and state transition probability. In the embodiments of the present application, the microgrid control parameter setting problem is modeled as MDP, and the setting of control parameters is realized through the collaborative decision of multiple agents.

[0051] Firstly, the micro-grid system is partitioned and the agent is mapped. In the island micro-grid (for example, mine island micro-grid), mainly containing energy storage and photovoltaic devices, the energy storage agent (Agent-ESS), photovoltaic agent (Agent-PV) and integrated coordination agent (Agent-Coord) are designed. That is, the Markov decision model includes energy storage agent, photovoltaic agent and integrated coordination agent.

[0052] Optionally, the energy storage agent (Agent-ESS) is responsible for the parameter adjustment of the energy storage DC-DC double loop (voltage loop PI parameter, current loop PI parameter) and DC-AC (VSG network type control parameter, such as virtual inertia J, damping coefficient D, droop coefficient m / n, and voltage and current double closed loop PI parameter).

[0053] The photovoltaic agent (Agent-PV) is responsible for the parameter adjustment of the photovoltaic DC-DC (MPPT control parameter, such as the step of perturbation and observation method, PI regulator parameter) and DC-AC double loop (voltage loop PI, current loop PI).

[0054] The integrated coordination agent (Agent-Coord) is mainly based on the results of the first two stages, and the integrated device parameters are optimized and adjusted across devices to enhance the anti-disturbance ability of the micro-grid.

[0055] In the embodiment of the application, different agents are processed in stages. In the first stage, the control parameters of the energy storage system are optimized and adjusted, and the energy storage system is used as the main power supply. In the second stage, the control parameters of the photovoltaic system are optimized and adjusted. In the third stage, the control parameters of the integrated system are optimized and adjusted. According to different stages, the state space, action space and reward are set.

[0056] Exemplarily, the Markov decision model of the control parameter setting can be expressed as a tuple (N, S1, S2, K, S N , A1, A2, K, A N , R1, R2, K, R N , γ)

[0057] Firstly, the state space is set. The state space S1, S2, K, S N is used to represent the state of each agent affected by the action of each converter control parameter. Therefore, the state space of the energy storage system, the state space of the photovoltaic system and the state space of the integrated system are set respectively.

[0058] 1) Energy storage system state space setting

[0059] Since the energy storage DC-DC adopts voltage and current double-loop control, and the DC-AC adopts VSG network type control, the energy storage agent is divided into a first energy storage agent (for example, an energy storage DC-DC agent) and a second energy storage agent (for example, an energy storage DC-AC agent), wherein the first energy storage agent includes a state space S ess,DC-DC , and the second energy storage agent includes a state space S ess,DC-AC .

[0060] S ess,DC-DC ={U_dc_ess,I_dc_ess}

[0061] Wherein U_dc_ess and I_dc_ess represent the output DC voltage and current of the bidirectional DC / DC converter of the energy storage system, respectively.

[0062] S ess,DC-AC ={U L _ess_ac,I L _ess_ac,U PCC _ess_ac,I PCC _ess_ac,f}

[0063] Wherein U L _ess_ac and I L _ess_ac represent the filter voltage and current values on the AC side of the converter of the energy storage system, U PCC _ess_ac and I PCC _ess_ac represent the voltage and current values at the grid connection point of the energy storage system, and f represents the frequency value of the energy storage system.

[0064] 2) Photovoltaic system state space setting

[0065] Since the photovoltaic inverter DC-DC adopts maximum power tracking control, and the DC-AC adopts voltage and current double-loop control, the photovoltaic agent includes a first photovoltaic agent (for example, a photovoltaic DC-DC agent) and a second photovoltaic agent (for example, a photovoltaic DC-AC agent), the first photovoltaic agent includes a state space S PV,DC-DC , and the second photovoltaic agent includes a state space S PV,DC-AC .

[0066] S PV,DC-DC ={U_PV_array,I_PV_array,U_dc_PV,I_dc_PV,ΔP MPPT ,Δλ PV}

[0067] Wherein, U_PV_array and I_PV_array represent photovoltaic array DC voltage and current respectively, U_dc_PV and I_dc_PV represent photovoltaic system DC / DC converter output DC voltage and current respectively, ΔP MPPT represents the maximum power tracking error, Δλ PV represents the perturbation observation method maximum power tracking perturbation step.

[0068] S PV,DC-AC ={U L _PV_ac,I L _PV_ac,U PCC _PV_ac,I PCC _PV_ac,f}

[0069] Wherein, U L _PV_ac and I L _PV_ac represent photovoltaic system converter AC side filter voltage and current values respectively, U PCC _PV_ac and I PCC _PV_ac represent photovoltaic system grid-connected point voltage and current values respectively, and f represents the photovoltaic system frequency value.

[0070] 3) Integrated system level state space setting

[0071] The integrated system introduces complex disturbance scenarios (such as sudden change of light, sudden start of motor, and low SOC of energy storage), and adjusts the anti-disturbance ability of the island microgrid (for example, a mine type island microgrid) through coordinated optimization of integrated control parameters. The integrated coordination agent includes state space S system .

[0072] S system ={S ess,DC-DC ,S ess,DC-AC ,S PV,DC-DC ,S PV,DC-AC ,ΔP PV_ess ,ΔU_pcc,Δf,ΔP motor}

[0073] Wherein, ΔP PV_ess represents the power difference between the energy storage system and the photovoltaic system, ΔU_pcc represents the grid-connected point voltage deviation, Δf represents the system frequency deviation, and ΔP motor represents the motor load impact, S ess,DC-DC and S ess,DC-AC represent the state space of the energy storage DC-DC agent and the state space of the energy storage DC-AC agent respectively, S PV,DC-DC and S PV,DC-AC represent the state space of the photovoltaic DC-DC agent and the state space of the photovoltaic DC-AC agent respectively.

[0074] Secondly, the action space setting is performed, and the action spaces A1, A2, K, A N is used to represent the scalar set controlled by each agent in the control parameter optimization adjustment task, therefore, the energy storage system action space, the photovoltaic system action space and the integrated system action space are set respectively.

[0075] 1) Energy storage system action space setting

[0076] The energy storage agent includes a first energy storage agent (for example, an energy storage DC-DC agent) and a second energy storage agent (for example, an energy storage DC-AC agent), wherein the first energy storage agent includes an action space A ess,DC-DC , and the second energy storage agent includes an action space A ess,DC-AC .

[0077] A ess,DC-DC ={k p_ess_DC_v1 ,k i_ess_DC_v1 ,k p_ess_DC_c1 ,k i_ess_DC_c1}

[0078] Wherein, k p_ess_DC_v1 and k i_ess_DC_v1 represent the PI control parameters of the energy storage DC-DC converter voltage loop, k p_ess_DC_c1 and k i_ess_DC_c1 represent the PI control parameters of the energy storage DC-DC converter current loop.

[0079]

[0080] Wherein, D and J represent the VSG network type control virtual inertia and damping coefficients, m and n represent the active and reactive droop control coefficients, k p_ess_AC_v11 ,k i_ess_AC_v11 ,k p_ess_AC_v12 ,k i_ess_AC_v12 represent the voltage double-loop PI control parameters, k p_ess_AC_c11 ,k i_ess_AC_c11 ,k p_ess_AC_c12 ,k i_ess_AC_c12 represent the current double-loop PI control parameters.

[0081] 2) Photovoltaic system action space setting

[0082] The photovoltaic agent includes a first photovoltaic agent and a second photovoltaic agent, the first photovoltaic agent includes an action space A PV,DC-DC , and the second photovoltaic agent includes an action space A PV,DC-AC .

[0083] A PV,DC-DC ={Δλ PV}

[0084] wherein Δλ PV denotes the disturbance observation method disturbance step.

[0085] A PV,DC-AC = {k p_PV_DC_v11 ,k i_PV_DC_v11 ,k p_PV_AC_v12 ,k i_PV_AC_v12 ,k p_PV_AC_c12 ,k i_PV_AC_c12}

[0086] wherein k p_PV_DC_v11 and k i_PV_DC_v11 denote the PI control parameters of the photovoltaic DC voltage outer loop, k p_PV_AC_v12 and k i_PV_AC_v12 denote the PI control parameters of the voltage inner loop, k p_PV_AC_c12 and k i_PV_AC_c12 denote the PI control parameters of the current inner loop.

[0087] 3) Integrated system level action space setting

[0088] The integrated agent includes an action space A system Due to the introduction of complex disturbance scenarios (such as sudden change of light, sudden start of motor, and low SOC of energy storage) by the integrated system, the action space of the integrated agent is the modification amount of the control parameters of each converter, which enhances the island microgrid (for example, a mine type island microgrid) through coordinated optimization adjustment of the integrated control parameters.

[0089]

[0090] wherein Δk p_ess_DC_v1 and Δk i_ess_DC_v1 denote the PI control parameter modification amount of the energy storage DC-DC converter voltage loop, Δk p_ess_DC_c1 and Δk i_ess_DC_c1 denote the PI control parameter modification amount of the energy storage DC-DC converter current loop, Δk p_ess_AC_v11 , Δk i_ess_AC_v11 , Δk p_ess_AC_v12 , Δk i_ess_AC_v12 denote the voltage double-loop PI control parameter modification amount, Δk p_ess_AC_c11 , Δk i_ess_AC_c11 , Δk p_ess_AC_c12 , Δk i_ess_AC_c12 denote the current double-loop PI control parameter modification amount, Δk p_PV_DC_v11 and Δk i_PV_DC_v11 denote the PI control parameter modification amount of the photovoltaic DC voltage outer loop, Δk p_PV_AC_v12 and Δk i_PV_AC_v12 denote the PI control parameter modification amount of the voltage inner loop, Δk p_PV_AC_c12 and Δk i_PV_AC_c12PI control parameter correction amount of the current inner loop.

[0091] Finally, the reward setting is performed, and rewards R1, R2, K, R N The reward is used to represent the action of each intelligent agent after the control parameter of each converter is changed, and therefore the energy storage system reward, photovoltaic system reward, and integrated system reward are set respectively.

[0092] 1) Energy storage system control parameter setting reward setting

[0093] The energy storage system control parameter setting is divided into two sub-tasks. First, the DC-DC converter control parameter of the energy storage system is optimized and adjusted, and on this basis, the DC-AC converter control parameter of the energy storage system is optimized and adjusted. Therefore, the reward also needs to be set separately. The energy storage intelligent agent includes a first energy storage intelligent agent and a second energy storage intelligent agent. The first energy storage intelligent agent includes a reward r ess,DC-DC , and the second energy storage intelligent agent includes a reward r ess,DC-AC .

[0094] ① The reward r ess,DC-DC of the first energy storage intelligent agent is set

[0095] DC voltage following reward:

[0096] r ess,DC-DC,1 = - |U_dc_ess(t) - U i,ref (t)|

[0097] DC voltage dynamic response reward:

[0098] r ess,DC-DC,2 = -t s,ess,DC-DC

[0099] DC voltage overshoot suppression reward:

[0100] r ess,DC-DC,3 = -δ 2 ess,DC-DC

[0101] ② The reward r ess,DC-AC of the second energy storage intelligent agent is set

[0102] AC voltage following reward:

[0103] r ess,DC-AC,1 = - |U PCC _ess_ac(t) - U PCC,ref (t)|

[0104] Frequency following reward:

[0105] r ess,DC-AC,2 = - |f(t) - f ref (t)|

[0106] Power-following reward:

[0107] r ess,DC-AC,3 = -|P PCC _ess_ac(t) - P PCC,ref (t)|

[0108] Power quality reward:

[0109] r ess,DC-AC,4 = -|THD u | -|THD i |

[0110] AC voltage dynamic response reward:

[0111] r ess,DC-AC,5 = -t s,ess,DC-AC

[0112] AC voltage overshoot suppression reward:

[0113] r ess,DC-AC,6 = -δ 2 ess,DC-AC

[0114] 2) Photovoltaic system control parameter setting reward setting

[0115] The photovoltaic system control parameter setting is divided into two sub-tasks, first, the photovoltaic system DC-DC converter control parameter is optimized and adjusted, and on this basis, the photovoltaic system DC-AC converter control parameter is optimized and adjusted, therefore, the reward also needs to be set separately. It is divided into sub-task 1 photovoltaic system DC-DC converter control parameter adjustment reward and sub-task 2 photovoltaic system DC-AC converter control parameter optimization adjustment reward. The photovoltaic intelligent agent includes a first photovoltaic intelligent agent and a second photovoltaic intelligent agent, the first photovoltaic intelligent agent includes a reward r PV,DC-DC , and the second photovoltaic intelligent agent includes a reward r PV,DC-AC .

[0116] ③ The reward r PV,DC-DC of the first photovoltaic intelligent agent is set

[0117] DC voltage following reward:

[0118] r PV,DC-DC,1 = -|U_dc_PV(t) - U PV,ref (t)|

[0119] DC voltage dynamic response reward:

[0120] r ess,DC-DC,2 = -t s,PV,DC-DC

[0121] DC voltage overshoot suppression reward:

[0122] r PV,DC-DC,3 = - δ 2 PV,DC-DC

[0123] r PV,DC-AC Reward setting

[0124] AC voltage follow-up reward:

[0125] r PV,DC-AC,1 = - |U PCC _PV_ac(t) - U PV,PCC,ref (t)|

[0126] Frequency follow-up reward:

[0127] r PV,DC-AC,2 = - |f(t) - f ref (t)|

[0128] Power follow-up reward:

[0129] r PV,DC-AC,3 = - |P PCC _PV_ac(t) - P PV,PCC,ref (t)|

[0130] Power quality reward:

[0131] r PV,DC-AC,4 = - |THD PV,u | - |THD PV,i |

[0132] AC voltage dynamic response reward:

[0133] r PV,DC-AC,5 = - t s,PV,DC-AC

[0134] AC voltage overshoot suppression reward:

[0135] r PV,DC-AC,6 = - δ 2 PV,DC-AC

[0136] 3) Integrated system control parameter collaborative optimization reward setting

[0137] This part is mainly based on the results of the previous two stages, introduces complex disturbance scenarios (such as sudden change of light, sudden start of motor, low SOC of energy storage), and optimizes the integrated device parameters across devices. Through the coordinated optimization adjustment of integrated control parameters, the anti-disturbance ability of the island microgrid (for example, mine-type island microgrid) is enhanced. The integrated coordination agent includes reward r system .

[0138] Common bus voltage following reward:

[0139] r system,1 = -|U PCC _PV_ac(t) - U PV,PCC,ref (t)|

[0140] Frequency following reward:

[0141] r system,2 = -|f(t) - f ref (t)|

[0142] Common bus voltage quality reward:

[0143] r system,4 = -|THD system,u | - |THD system,i |

[0144] Common bus voltage dynamic response reward:

[0145] r system,5 = -t s,system

[0146] Common bus voltage overshoot suppression reward:

[0147] r system,6 = -δ 2 system

[0148] System stability reward:

[0149] r system,7 = ε

[0150] After the Markov decision model is constructed, a multi-agent deep reinforcement learning algorithm is used to solve the Markov decision model of the control parameter setting. In the embodiment of the present application, based on the multi-agent deep reinforcement learning framework of phased adjustment, a "hierarchical agent network architecture" is designed, as shown in Figure 2 Fig. 1, first, the energy storage agent is used to determine the parameter value of the control parameter related to the energy storage system, then the photovoltaic agent is used to determine the parameter value of the control parameter related to the photovoltaic system, and the parameter value of the control parameter related to the energy storage system is used as a constraint condition, and finally, based on the results of the previous two stages, the integrated coordination agent is used to perform cross-device collaborative optimization on the integrated device parameters.

[0151] In step S150, the energy storage target control parameter value is determined based on the simulation model, the small disturbance stability evaluation model and the energy storage agent.

[0152] Optionally, the energy storage target control parameter refers to a parameter value of at least one control parameter related to the energy storage system. In the embodiment of the present application, the energy storage system control parameter setting is divided into two sub-tasks. First, the DC-DC converter control parameter of the energy storage system is optimized and adjusted, and on this basis, the DC-AC converter control parameter of the energy storage system is optimized and adjusted. Deep Deterministic Policy Gradient (DDPG) is used to optimize and adjust the two sub-tasks. Two sub-tasks use independent DDPG agents. Illustratively, the energy storage agent includes a first energy storage agent and a second energy storage agent, and the energy storage target control parameter value includes a first energy storage target control parameter value and a second energy storage target control parameter value. The first energy storage agent is used to determine the first energy storage target control parameter value, and the second energy storage agent is used to determine the second energy storage target control parameter value. Subsequently, the first energy storage target control parameter value and the second energy storage target control parameter value are combined Figure 3 Details are not described here.

[0153] In step S160, the photovoltaic target control parameter value of the photovoltaic agent is determined based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the photovoltaic agent.

[0154] Optionally, the photovoltaic target control parameter value includes a parameter value of at least one control parameter related to the photovoltaic system. In the embodiment of the present application, the energy storage target control parameter value is used as a constraint condition of the photovoltaic agent, that is, the energy storage target control parameter value is used as a fixed input. The photovoltaic system control parameter setting is divided into two sub-tasks. First, the DC-DC converter control parameter of the photovoltaic system is optimized and adjusted, and on this basis, the DC-AC converter control parameter of the photovoltaic system is optimized and adjusted. Deep Deterministic Policy Gradient (DDPG) is used to optimize and adjust the two sub-tasks. Two sub-tasks use independent DDPG agents. Illustratively, the photovoltaic agent includes a first photovoltaic agent and a second photovoltaic agent, and the photovoltaic target control parameter value includes a first photovoltaic target control parameter value and a second photovoltaic target control parameter value. The first photovoltaic agent is used to determine the first photovoltaic target control parameter value, and the second photovoltaic agent is used to determine the second photovoltaic target control parameter value. Subsequently, the first photovoltaic target control parameter value and the second photovoltaic target control parameter value are combined Figure 4 Details are not described here.

[0155] In step S170, the energy storage target control parameter value and the photovoltaic target control parameter value are corrected based on the energy storage agent, the photovoltaic agent and the integrated coordination agent.

[0156] In the embodiments of the present application, based on the previous stage results, complex disturbance scenarios (such as sudden change of light, sudden start of motor, and low SOC of energy storage, etc.) are introduced to perform cross-device collaborative optimization on the integrated device and parameters. In the embodiments of the present application, as shown in Figure 2 Based on the energy storage agent, the photovoltaic agent and the integrated coordination agent, the Multi-Agent Deep Deterministic Policy Gradient (MA-DDPG) is used to correct the energy storage target control parameter value and the photovoltaic target control parameter value.

[0157] In the embodiments of the present application, the process of adjusting the control parameters is abstracted as a controlled Markov decision process, and the microgrid control parameter optimization adjustment strategy is determined through multi-agent deep reinforcement learning, which improves the control parameter setting efficiency and accuracy and provides important support for the safe and stable operation of the microgrid. Moreover, the method is easy to implement and convenient to arrange, and can be further extended to the real-time optimization adjustment scene of the control parameters of the power station containing a high proportion of renewable energy. In addition, in the embodiments of the present application, the indexes such as power quality, voltage deviation, frequency deviation and damping are considered when setting the reward, so as to facilitate the agent to adjust the control parameters that meet the system stability and dynamic performance, provide important support for the safe and stable operation of the microgrid, and further provide a technical basis for the operation and control of the island microgrid containing multiple types of impact loads.

[0158] Optionally, the method further comprises optimizing and adjusting the control parameters of the energy storage converter and the photovoltaic converter based on the above-mentioned determined adjustment strategy (i.e. the energy storage target control parameter value and the photovoltaic target control parameter value).

[0159] Figure 3 is a flowchart of determining the energy storage target control parameter value provided by an embodiment of the present application. As shown in Figure 3 The method comprises the following steps.

[0160] Step S310, determining the first energy storage target control parameter value based on the simulation model, the small disturbance stability evaluation model and the first energy storage agent.

[0161] Optionally, the first energy storage target control parameter value includes the parameter value of the DC-DC converter control parameter of the energy storage system, for example, the PI control parameters of the DC-DC converter voltage loop and the PI control parameters of the current loop. Optionally, the first energy storage agent includes a DDPG agent, which contains a complete Actor-Critic network and an experience replay mechanism. The number of network layers and the number of neurons can be designed according to the state of the task, the dimension of the action space, and the physical characteristics (such as time scale and control frequency).

[0162] Specifically, as shown inFigure 2 As shown, based on the simulation model, an initial state of the first energy storage agent is determined; based on the initial state of the first energy storage agent, an action of the first energy storage agent is determined by using the first energy storage agent and is sent to the simulation model; based on the action of the first energy storage agent, a reward corresponding to the action of the first energy storage agent and a next state of the first energy storage agent are determined by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the first energy storage agent does not satisfy a first convergence condition, the action of the first energy storage agent is updated based on the next state of the first energy storage agent by using the first energy storage agent, and the reward of the action of the first energy storage agent is updated based on the updated action of the first energy storage agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the first energy storage agent satisfies the first convergence condition; and the first energy storage target control parameter value is determined based on the action of the first energy storage agent corresponding to the reward satisfying the first convergence condition. The first convergence condition is a preset threshold value, which can be set by a user according to actual requirements. Since the action space of the first energy storage agent is represented by the first energy storage target control parameter, the action of the first energy storage agent corresponding to the reward satisfying the first convergence condition is the first energy storage target control parameter value.

[0163] In step S320, the second energy storage target control parameter value is determined based on the first energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the second energy storage agent.

[0164] Optionally, the second energy storage target control parameter value includes a parameter value of a DC-AC converter control parameter of the energy storage system, for example, a VSG network type control virtual inertia and damping coefficient, an active and reactive power droop control coefficient, a voltage double-loop PI control parameter and a current double-loop PI control parameter. Optionally, the second energy storage agent includes a DDPG agent, which includes a complete Actor-Critic network and an experience replay mechanism. The number of network layers and the number of neurons can be designed according to the state of the task, the dimension of the action space, the physical characteristics (such as time scale and control frequency) and the like.

[0165] Specifically, as shown in FIG. 3, the method includes the following steps. Figure 2As shown, the first energy storage target control parameter value is taken as a constraint condition (i.e., the first energy storage target control parameter value is a fixed value); based on the simulation model, the initial state of the second energy storage agent is determined; based on the initial state of the second energy storage agent, the action of the second energy storage agent is determined by using the second energy storage agent and is sent to the simulation model; based on the action of the second energy storage agent, the reward corresponding to the action of the second energy storage agent and the next state of the second energy storage agent are determined by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the second energy storage agent does not satisfy the second convergence condition, the action of the second energy storage agent is updated based on the next state of the second energy storage agent by using the second energy storage agent, and the reward of the action of the second energy storage agent is updated based on the updated action of the second energy storage agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the second energy storage agent satisfies the second convergence condition; and the second energy storage target control parameter value is determined based on the action of the second energy storage agent corresponding to the reward satisfying the second convergence condition. The second convergence condition is a preset threshold value, which can be set by a user according to actual requirements. Since the action space of the second energy storage agent is represented by the second energy storage target control parameter, the action of the second energy storage agent corresponding to the reward satisfying the second convergence condition is the second energy storage target control parameter value.

[0166] In the embodiment of the application, the first energy storage target control parameter value is taken as a constraint condition, that is, the DC-DC converter control parameter optimization result is taken as an environmental constraint condition for DC-AC converter control parameter optimization, forming an optimization path of "first base and then upper layer". Through the staged DDPG optimization, efficient setting of the DC-DC converter control parameter and the DC-AC converter control parameter of the energy storage system can be realized, and the stability and dynamic response performance are taken into account.

[0167] Figure 4 is a flowchart of a method for determining a photovoltaic target control parameter value provided by an embodiment of the application. As shown in Figure 4 , the method comprises the following steps.

[0168] In step S410, the first photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the first photovoltaic agent.

[0169] Optionally, the first photovoltaic target control parameter value includes a parameter value of a DC-DC converter control parameter of a photovoltaic system, for example, a perturb and observe method perturbation step. Optionally, the first photovoltaic agent includes a DDPG agent, which includes a complete Actor-Critic network and an experience replay mechanism. The number of network layers and the number of neurons can be designed according to the state of the task, the dimension of the action space, physical characteristics (such as time scale, control frequency), etc.

[0170] Specifically, the energy storage target control parameter value is taken as a constraint condition; an initial state of the first photovoltaic agent is determined based on the simulation model; an action of the first photovoltaic agent is determined based on the initial state of the first photovoltaic agent, and is sent to the simulation model by using the first photovoltaic agent; a reward corresponding to the action of the first photovoltaic agent and a next state of the first photovoltaic agent are determined based on the action of the first photovoltaic agent by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the first photovoltaic agent does not satisfy a third convergence condition, the action of the first photovoltaic agent is updated based on the next state of the first photovoltaic agent by using the first photovoltaic agent, and the reward of the action of the first photovoltaic agent is updated based on the updated action of the first photovoltaic agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the first photovoltaic agent satisfies the third convergence condition; and the first photovoltaic target control parameter value is determined based on the action of the first photovoltaic agent corresponding to the reward satisfying the third convergence condition. The third convergence condition is a preset threshold value, which can be set by a user according to actual requirements. Since the action space of the first photovoltaic agent is represented by the first photovoltaic target control parameter, the action of the first photovoltaic agent corresponding to the reward satisfying the third convergence condition is the first photovoltaic target control parameter value.

[0171] In step S420, the second photovoltaic target control parameter value is determined based on the energy storage target control parameter value, the first photovoltaic target control parameter value, the simulation model, the small disturbance stability evaluation model and the second photovoltaic agent.

[0172] Optionally, the second photovoltaic target control parameter value includes a parameter value of a DC-AC converter control parameter of the photovoltaic system, for example, a PI control parameter of a photovoltaic direct-current voltage outer loop, a PI control parameter of a voltage inner loop and a PI control parameter of a current inner loop. The second photovoltaic agent includes a DDPG agent, which contains a complete Actor-Critic network and an experience replay mechanism. The number of network layers and the number of neurons can be designed according to the state of the task, the dimension of the action space, physical characteristics (such as time scale and control frequency) and the like.

[0173] Specifically, the energy storage target control parameter value and the first photovoltaic target control parameter value are taken as constraint conditions; based on the simulation model, an initial state of the second photovoltaic agent is determined; based on the initial state of the second photovoltaic agent, an action of the second photovoltaic agent is determined by using the second photovoltaic agent and is sent to the simulation model; based on the action of the second photovoltaic agent, a reward corresponding to the action of the second photovoltaic agent and a next state of the second photovoltaic agent are determined by using the simulation model and the small disturbance stability evaluation model; if the reward corresponding to the action of the second photovoltaic agent does not satisfy a fourth convergence condition, the action of the second photovoltaic agent is updated based on the next state of the second photovoltaic agent by using the second photovoltaic agent, and the reward of the action of the second photovoltaic agent is updated based on the updated action of the second photovoltaic agent by using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the second photovoltaic agent satisfies the fourth convergence condition; and the second photovoltaic target control parameter value is determined based on the action of the second photovoltaic agent corresponding to the reward satisfying the fourth convergence condition. The fourth convergence condition is a preset threshold value, which can be set by a user according to actual requirements. Since the action space of the second photovoltaic agent is represented by the second photovoltaic target control parameter, the action of the second photovoltaic agent corresponding to the reward satisfying the fourth convergence condition is the second photovoltaic target control parameter value.

[0174] In the embodiment of the present application, the first photovoltaic target control parameter value is taken as a constraint condition, that is, the DC-DC converter control parameter optimization result is taken as an environmental constraint condition for DC-AC converter control parameter optimization, forming an optimization path of "first base and then upper layer". Through the staged DDPG optimization, efficient setting of the photovoltaic system DC-DC converter control parameter and the DC-AC converter control parameter can be realized, taking into account the stability and dynamic response performance.

[0175] Figure 5 is a structural block diagram of a microgrid control parameter setting device provided by an embodiment of the present application. As shown in Figure 5 , the microgrid control parameter setting device 500 includes an acquisition module 510, a construction module 520, and a determination module 530.

[0176] Optionally, the acquisition module 510 is configured to acquire device parameters of the microgrid.

[0177] The construction module 520 is configured to construct a simulation model of the microgrid based on the device parameters.

[0178] The acquisition module 510 is further configured to acquire a small disturbance stability evaluation model.

[0179] The construction module 520 is further configured to construct a Markov decision model for control parameter setting, the Markov decision model including an energy storage agent, a photovoltaic agent, and an integrated coordination agent.

[0180] The determining module 530 is configured to determine a storage target control parameter value based on the simulation model, the small disturbance stability evaluation model and the storage agent, where the storage target control parameter value comprises a parameter value of at least one control parameter related to the storage system.

[0181] The determining module 530 is further configured to determine a photovoltaic target control parameter value of the photovoltaic agent based on the storage target control parameter value, the simulation model, the small disturbance stability evaluation model and the photovoltaic agent, where the photovoltaic target control parameter value comprises a parameter value of at least one control parameter related to the photovoltaic system.

[0182] The determining module 530 is further configured to correct the storage target control parameter value and the photovoltaic target control parameter value based on the storage agent, the photovoltaic agent and the integrated coordination agent.

[0183] The micro-grid control parameter setting device provided by the embodiments of the present application has similar specific working principles and benefits to the micro-grid control parameter setting method provided by the embodiments of the present application, and thus will not be described here.

[0184] Hereinafter, an electronic device according to an embodiment of the present application will be described with reference to Figure 6 FIG. 1. Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0185] As shown in Figure 6 , the electronic device 600 includes one or more processors 601 and a memory 602.

[0186] The processor 601 can be a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities, and can control other components in the electronic device 600 to perform desired functions.

[0187] The memory 602 can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache, and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 601 can run the program instructions to implement the micro-grid control parameter setting method of the embodiments of the present application described above and / or other desired functions. Various contents such as device parameters including the micro-grid, simulation models, small disturbance stability evaluation models, and the like can also be stored in the computer readable storage medium.

[0188] In one example, the electronic device 600 can further include an input device 603 and an output device 604, which are interconnected to each other through a bus system and / or other forms of connection mechanisms (not shown).

[0189] The input device 603 can include, for example, a keyboard, a mouse, and the like.

[0190] The output device 604 can output various information to the outside, including the energy storage target control parameter value, the photovoltaic target control parameter value, and the like. The output device 604 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0191] Of course, in order to simplify, Figure 6 In the figure, only some of the components in the electronic device 600 related to the present application are shown, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 600 can further include any other appropriate components according to specific application cases.

[0192] In addition to the above-mentioned methods and devices, the embodiments of the present application can also be computer program products, which include computer program instructions that make the processor execute the steps of the microgrid control parameter setting method according to various embodiments of the present application described above in the specification when the processor runs.

[0193] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including object-oriented programming languages such as Java, C++, and the like, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0194] In addition, the embodiments of the present application can also be a computer readable storage medium, which stores computer program instructions, and the computer program instructions make the processor execute the steps of the microgrid control parameter setting method according to various embodiments of the present application described above in the specification when the processor runs.

[0195] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0196] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0197] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0198] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0199] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0200] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to forms disclosed herein. Although several example aspects and embodiments have been discussed, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

[0201] The specific implementation described above does not constitute a limitation of the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for tuning microgrid control parameters, characterized in that, include: Obtain the equipment parameters of the microgrid; Based on the device parameters, a simulation model of the microgrid is constructed; Obtain a small-disturbance stability assessment model; A Markov decision model for control parameter tuning is constructed, wherein the Markov decision model includes an energy storage agent, a photovoltaic agent, and an integrated coordination agent; Based on the simulation model, the small disturbance stability assessment model, and the energy storage agent, the target control parameter values ​​for energy storage are determined, wherein the target control parameter values ​​for energy storage include the parameter values ​​of at least one control parameter related to the energy storage system. Based on the energy storage target control parameter values, the simulation model, the small disturbance stability evaluation model, and the photovoltaic intelligent agent, the photovoltaic target control parameter values ​​of the photovoltaic intelligent agent are determined, wherein the photovoltaic target control parameter values ​​include the parameter values ​​of at least one control parameter related to the photovoltaic system; Based on the energy storage intelligent agent, the photovoltaic intelligent agent, and the integrated coordination intelligent agent, the energy storage target control parameter value and the photovoltaic target control parameter value are corrected.

2. The method according to claim 1, characterized in that, The small-disturbance stability evaluation model is trained through the following steps: Randomly assign values ​​to the equipment parameters, the control parameters related to the energy storage system, the control parameters related to the photovoltaic system, the control parameters related to the generator, and the filter parameters, and use the simulation model to perform simulation and small-disturbance stability analysis to generate multiple training samples; Based on the training samples, an initial neural network model is trained to obtain the small-disturbance stability evaluation model.

3. The method according to claim 1, characterized in that, The energy storage intelligent agent includes a first energy storage intelligent agent and a second energy storage intelligent agent, and the energy storage target control parameter value includes a first energy storage target control parameter value and a second energy storage target control parameter value; The determination of target control parameter values ​​for energy storage based on the simulation model, the small-disturbance stability assessment model, and the energy storage agent includes: Based on the simulation model, the small disturbance stability evaluation model, and the first energy storage agent, the first energy storage target control parameter value is determined, wherein the first energy storage target control parameter value includes the parameter value of the DC-DC converter control parameter of the energy storage system; Based on the first energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model, and the second energy storage agent, the second energy storage target control parameter value is determined, wherein the second energy storage target control parameter value includes the parameter value of the DC-AC converter control parameter of the energy storage system.

4. The method according to claim 3, characterized in that, The determination of the first energy storage target control parameter value based on the simulation model, the small disturbance stability assessment model, and the first energy storage agent includes: Based on the simulation model, the initial state of the first energy storage agent is determined; Based on the initial state of the first energy storage agent, the action of the first energy storage agent is determined using the first energy storage agent and sent to the simulation model; Based on the actions of the first energy storage agent, the reward corresponding to the actions of the first energy storage agent and the next state of the first energy storage agent are determined using the simulation model and the small disturbance stability evaluation model. If the reward corresponding to the action of the first energy storage agent does not meet the first convergence condition, then based on the next state of the first energy storage agent, the action of the first energy storage agent is updated using the first energy storage agent, and based on the updated action of the first energy storage agent, the reward of the action of the first energy storage agent is updated using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the first energy storage agent meets the first convergence condition. Based on the action of the first energy storage agent that satisfies the first convergence condition, the value of the first energy storage target control parameter is determined. And / or, determining the second energy storage target control parameter value based on the first energy storage target control parameter value, the simulation model, the small disturbance stability assessment model, and the second energy storage agent includes: The first energy storage target control parameter value is used as a constraint condition; Based on the simulation model, the initial state of the second energy storage agent is determined; Based on the initial state of the second energy storage agent, the action of the second energy storage agent is determined and sent to the simulation model. Based on the actions of the second energy storage agent, the reward corresponding to the actions of the second energy storage agent and the next state of the second energy storage agent are determined using the simulation model and the small disturbance stability evaluation model. If the reward corresponding to the action of the second energy storage agent does not meet the second convergence condition, then based on the next state of the second energy storage agent, the action of the second energy storage agent is updated using the second energy storage agent, and based on the updated action of the second energy storage agent, the reward of the action of the second energy storage agent is updated using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the second energy storage agent meets the second convergence condition. The second energy storage target control parameter value is determined based on the action of the second energy storage agent that satisfies the second convergence condition.

5. The method according to claim 1, characterized in that, The photovoltaic intelligent agent includes a first photovoltaic intelligent agent and a second photovoltaic intelligent agent, and the photovoltaic target control parameter value includes a first photovoltaic target control parameter value and a second photovoltaic target control parameter value; The process of determining the photovoltaic target control parameter value of the photovoltaic intelligent agent based on the energy storage target control parameter value, the simulation model, the small disturbance stability assessment model, and the photovoltaic intelligent agent includes: Based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model, and the first photovoltaic intelligent agent, the first photovoltaic target control parameter value is determined, wherein the first photovoltaic target control parameter value includes the parameter value of the DC-DC converter control parameter of the photovoltaic system; Based on the energy storage target control parameter value, the first photovoltaic target control parameter value, the simulation model, the small disturbance stability evaluation model, and the second photovoltaic agent, the second photovoltaic target control parameter value is determined, wherein the second photovoltaic target control parameter value includes the parameter value of the DC-AC converter control parameter of the photovoltaic system.

6. The method according to claim 5, characterized in that, The process of determining the first photovoltaic target control parameter value based on the energy storage target control parameter value, the simulation model, the small disturbance stability assessment model, and the first photovoltaic intelligent agent includes: The target control parameter value for energy storage is used as a constraint condition; Based on the simulation model, the initial state of the first photovoltaic smart agent is determined; Based on the initial state of the first photovoltaic cell, the action of the first photovoltaic intelligent agent is determined using the first photovoltaic intelligent agent and sent to the simulation model; Based on the actions of the first photovoltaic agent, the reward corresponding to the actions of the first photovoltaic agent and the next state of the first photovoltaic agent are determined using the simulation model and the small disturbance stability evaluation model. If the reward corresponding to the action of the first photovoltaic agent does not meet the third convergence condition, then based on the next state of the first photovoltaic agent, the action of the first photovoltaic agent is updated using the first photovoltaic agent, and based on the updated action of the first photovoltaic agent, the reward of the action of the first photovoltaic agent is updated using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the first photovoltaic agent meets the third convergence condition. Based on the action of the first photovoltaic agent that satisfies the third convergence condition, the value of the first photovoltaic target control parameter is determined. And / or, determining the second photovoltaic target control parameter value based on the energy storage target control parameter value, the first photovoltaic target control parameter value, the simulation model, the small disturbance stability assessment model, and the second photovoltaic agent includes: The energy storage target control parameter value and the first photovoltaic target control parameter value are used as constraints. Based on the simulation model, the initial state of the second photovoltaic smart agent is determined; Based on the initial state of the second photovoltaic agent, the action of the second photovoltaic agent is determined and sent to the simulation model. Based on the actions of the second photovoltaic agent, the reward corresponding to the actions of the second photovoltaic agent and the next state of the second photovoltaic agent are determined using the simulation model and the small disturbance stability evaluation model. If the reward corresponding to the action of the second photovoltaic agent does not meet the fourth convergence condition, then based on the next state of the second photovoltaic agent, the action of the second photovoltaic agent is updated using the second photovoltaic agent, and based on the updated action of the second photovoltaic agent, the reward of the action of the second photovoltaic agent is updated using the simulation model and the small disturbance stability evaluation model, until the reward corresponding to the action of the second photovoltaic agent meets the fourth convergence condition. The second photovoltaic target control parameter value is determined based on the action of the second photovoltaic agent that satisfies the fourth convergence condition.

7. The method according to claim 1, characterized in that, The step of correcting the target control parameter values ​​of the energy storage agent, the photovoltaic agent, and the integrated coordination agent includes: Based on the energy storage agent, the photovoltaic agent, and the integrated coordination agent, the energy storage target control parameter values ​​and the photovoltaic target control parameter values ​​are corrected using a multi-agent deep deterministic policy gradient algorithm.

8. The method according to claim 1, characterized in that, Before constructing the Markov decision model with control parameter tuning, the method further includes: The evaluation index for characterizing the effect of control parameter tuning is determined, and the evaluation index includes at least one of the following: power quality index, voltage tracking characteristic index, power tracking characteristic index, dominant oscillation mode damping index characterizing system stability, and action response index.

9. A microgrid control parameter tuning device, characterized in that, include: The acquisition module is used to acquire equipment parameters of the microgrid. A construction module is used to construct a simulation model of the microgrid based on the device parameters; The acquisition module is also used to acquire a small-disturbance stability evaluation model; The building module is also used to build a Markov decision model for control parameter tuning, wherein the Markov decision model includes an energy storage agent, a photovoltaic agent, and an integrated coordination agent; The determination module is used to determine the target control parameter values ​​of energy storage based on the simulation model, the small disturbance stability evaluation model and the energy storage agent, wherein the target control parameter values ​​of energy storage include the parameter values ​​of at least one control parameter related to the energy storage system; The determining module is further configured to determine the photovoltaic target control parameter value of the photovoltaic intelligent agent based on the energy storage target control parameter value, the simulation model, the small disturbance stability evaluation model, and the photovoltaic intelligent agent, wherein the photovoltaic target control parameter value includes the parameter value of at least one control parameter related to the photovoltaic system; The determining module is further configured to, based on the energy storage intelligent agent, the photovoltaic intelligent agent, and the integrated coordination intelligent agent, correct the energy storage target control parameter value and the photovoltaic target control parameter value.

10. An electronic device, characterized in that, include: processor; A memory connected to the processor, the memory being used to store a computer program, which, when executed by the processor, implements the microgrid control parameter tuning method as described in any one of claims 1 to 8.

Images