DQN-based ship micro-grid reactive power distribution control method for virtual impedance optimization

Abstract

The invention provides a DQN-based ship micro-grid reactive power distribution control strategy for virtual impedance optimization. The strategy comprises the following steps: establishing a main circuit model of a main circuit system of a double-micro-source ship micro-grid; according to the ship micro-grid main circuit model system, constructing a droop controller based on DQN; and establishing a reactive power distribution precision index. According to the virtual impedance gain changing method based on the DQN, the intelligent agent can output better virtual impedance gain through training iteration. Aiming at the problem of reactive power distribution of each micro-source of the micro-grid, a novel droop control strategy of variable virtual impedance gain is introduced into traditional adaptive virtual impedance control, and the reactive power distribution precision of the micro-sources is improved.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a DQN-based ship microgrid reactive power distribution control method for virtual impedance optimization. Background Art

[0002] The International Maritime Organization (IMO) has increasingly stringent requirements for ship carbon emissions. Green ships have become an inevitable trend in the development of future ships. Ship microgrids provide an effective way for the large-scale application of new energy at sea. Droop control has the characteristics of not needing to obtain other inverter information and can achieve the advantage of plug-and-play, making it a very shining control method in ship microgrid control. Virtual impedance is a kind of virtual impedance that uses the impedance relationship between voltage and current to add a virtual impedance to droop control and reshape the output characteristics of the converter. However, droop control is highly dependent on virtual impedance. The application of virtual impedance technology will directly affect the performance of droop control, especially for system power distribution. At present, the scientific community combines droop control with intelligent control. By changing the virtual impedance change, many new methods of combining droop control and intelligent control have been proposed. However, the research of some scholars is still limited to adopting a fixed virtual impedance gain. Although the virtual impedance change has changed, the gain remains unchanged, and the power distribution accuracy still has room for further improvement.

[0003] Existing papers use fixed gain to adjust virtual impedance, which lacks flexibility in practical applications. Fixed gain cannot be adjusted in real time according to changes in actual working conditions, resulting in insufficient system response when load changes or system state changes. In addition, due to the fixed gain, the reactive power distribution accuracy of the system is limited, and optimal power distribution cannot be achieved.

[0004] Adaptively adjusted virtual impedance gain, by introducing DQN, the system can adaptively adjust the virtual impedance gain according to the real-time operating status, enhance system stability, and optimize the system reactive power distribution. Through the learning and optimization of DQN, more accurate reactive power distribution of each distributed micro-source (DG) can be achieved, improving the overall performance of the system.

[0005] References:

[0006] 1.X.Zhaoxia,Z.Tianli,L.Huaimin,J.M.Guerrero,C.-L.Su,and J.C.V′asquez,“Coordinated control of a hybrid-electric-ferry shipboard microgrid,”IEEETransactions on Transportation Electrification,vol.5,no.3,pp.828–839,2019.

[0007] 2.M.H.Rehmani,M.Reisslein,A.Rachedi,M.Erol-Kantarci,and M.Radenkovic,“Integrating renewable energy resources into the smart grid:Recentdevelopments in information and communication technologies,”IEEE Transactionson Industrial Informatics,vol.14,no.7,pp.2814–2825,2018.

[0008] 3.J.Kim,J.M.Guerrero,P.Rodriguez,R.Teodorescu,and K.Nam,“Modeadaptive droop control with virtual output impedances for an inverter-basedflexible ac microgrid,”IEEE Transactions on power electronics,vol.26,no.3,pp.689–701,2010.

[0009] 4.S.Ullah,L.Khan,I.Sami,and J.-S.Ro,“Voltage / frequency regulationwith optimal load dispatch in microgrids using smc based distributedcooperative control,”IEEE Access,vol.10,pp.64 873–64 889,2022.

[0010] 5.T.Chen,O.Abdel-Rahim,F.Peng,and H.Wang,“An improved finite controlset-mpc-based power distribution control strategy for islanded acmicrogrids,”IEEE Access,vol.8,pp.52 676–52 686,2020.

[0011] 6.J.Chen,D.Yue,C.Dou,L.Chen,S.Weng,and Y.Li,“A virtual compleximpedance based droop method for parallel-connected inverters in low-voltage ac microgrids,”IEEE Transactions on Industrial Informatics,vol.17,no.3,pp.1763–1773,2020.

[0012] 7.N.Mohammed,A.Lashab,M.Ciobotaru,and J.M.Guerrero,

[0013] “Accurate reactive power distribution strategy for droop-basedislanded ac microgrids,”IEEE Transactions on Industrial Electronics,vol.70,no.3,pp.2696–2707,2022.

[0014] 8.H.Mahmood,D.Michaelson,and J.Jiang,“Accurate reactive powerdistribution in an islanded microgrid using adaptive virtual impedances,”IEEETransactions on Power Electronics,vol.30,no.3,pp.1605–1617,2014.

[0015] 9.E.Jiang,J.Zhao,Z.Shi,Y.Mi,S.Lin,and S.Muyeen,“Intelligent virtualimpedance based control to enhance the stability of islanded microgrid,”Journal of Electrical Engineering&Technology,vol.18,no.5,pp.3971–3984,2023.

[0016] 10.J.Duarte,M.Velasco,P.Mart ι,A.Camacho,J.Miret,and C.Alfaro,“Decoupled simultaneous complex power distribution and voltage regulationinislanded ac microgrids,”IEEE Transactions on Industrial Electronics,vol.70,no.4,pp.3888–3898,2022.

[0017] 11.X.Sun,Y.Hao,Q.Wu,X.Guo,and B.Wang,“A multifunctional and wirelessdroop control for distributed energy storage units in islanded ac microgridapplications,”IEEE Transactions on Power Electronics,vol.32,no.1,pp.736–751,2016.

[0018] 12.Z.Jin,L.Meng,J.M.Guerrero,and R.Han,“Hierarchical control designfor a shipboard power system with dc distribution and energy storage aboardfuture more-electric ships,”IEEE Transactions on Industrial Informatics,vol.14,no.2,pp.703–714,2017.

[0019] 13.Q.Liu, T.Caldognetto, and S.Buso, “Review and comparison of grid-tiedinverter controllers in microgrids,” IEEE Transactions on Power Electronics, vol.35, no.7, pp.7624–7639, 2019.

[0020] 14.Vijay, N.Parth, S.Doolla, and MCChandorkar, "An adaptive virtualimpedance control for improving power distribution among inverters inislanded ac microgrids," IEEE Transactions on Smart Grid, vol.12, no.4, pp.2991–3003, 2021. Summary of the invention

[0021] According to the technical problems mentioned in the above background technology, a DQN-based ship microgrid reactive power distribution control method for virtual impedance optimization is provided. The present invention first models the ship microgrid system to simulate the real ship microgrid system environment; according to the environment, the reactive power distribution is obtained therefrom. An intelligent agent observation and reward mechanism is established, and the DQN agent (Agent) observes the power deviation data in real time, and according to the power deviation, the agent is rewarded or punished according to preset rules to optimize its output virtual impedance gain value. The system is iteratively optimized, and the agent continuously learns and optimizes its decision-making rules to improve system performance through iteration.

[0022] The technical means adopted by the present invention are as follows:

[0023] An adaptive reactive power allocation strategy for ship microgrid based on DQN includes the following steps:

[0024] Step 1: Establish the main circuit model of the dual micro-source ship microgrid main circuit system;

[0025] Step 2: According to the ship microgrid main circuit model system, a DQN-based droop controller is constructed;

[0026] Step 3: Establishment of reactive power distribution accuracy index.

[0027] Furthermore, the step 1 comprises the following steps:

[0028] Step 11: Select the corresponding micro-source inverter circuit and filter according to the micro-source capacity and application;

[0029] Step 12: Design the filter parameters so that the filter satisfies the constraint conditions.

[0030] Furthermore, the constraint condition is:

[0031]

[0032] Where, f0 represents the fundamental frequency of the inverter bridge output voltage; f c Indicates the cutoff frequency of the LC filter; f s Indicates the switching frequency of the IGBT; L indicates the inductance of the LC filter; U inv max Indicates the maximum value of the fundamental phase voltage output by the inverter bridge; Indicates the maximum value of the LC filter inductor current; U o max Indicates the maximum value of the output voltage of the micro-source inverter; U o Represents the output voltage of the micro-source inverter; V dc Indicates the DC bus voltage; S n represents the rated capacity of the micro-source inverter; ω0 represents the angular frequency of the output voltage of the micro-source inverter; C represents the capacitance value of the LC filter.

[0033] Furthermore, the step 2 comprises the following steps:

[0034] Step 21: Real-time monitoring of the output voltage phase E of the micro-source inverter i , load current phasor i oi ;

[0035] Step 22: Calculate the actual power P i With reference power P ref Compare and get the power deviation, which is used as the observation value of the intelligent agent and participates in the training of the intelligent agent;

[0036] Step 23: Initialize and use a neural network with randomly initialized weights to approximate the Q-value function Q(s, a; θ), where s represents the power deviation output by the micro-source inverter; a represents the virtual impedance gain; θ represents the weight of the neural network; then initialize a target network Q′(s, a; θ′) for stable training, where θ′ represents the weight of the target network; finally initialize the experience replay pool, which contains past states, actions, rewards, and new states; Step 24: Establish an intelligent agent observation and reward mechanism, and the reward function is:

[0037] r=μ∫max(0,min(1,γ(αΔQ+β)))+λ;

[0038] The DQN agent observes the power deviation data in real time and trains according to this reward function to obtain a larger reward. The agent rewards or punishes according to preset rules to optimize the decision-making process.

[0039] Step 24, terminating the training, and the training process continues until a predetermined number of training steps is reached and a specific performance indicator is achieved;

[0040] Step 25: After training is completed, the DQN agent uses the learned Q-value function to select the optimal action in each state in order to obtain the highest cumulative reward, thereby obtaining the optimal virtual impedance gain, which is combined with the virtual impedance to obtain the optimal virtual impedance size;

[0041] Step 26: Use the obtained optimal virtual impedance to correct the droop control strategy of the traditional fixed virtual impedance to obtain the updated reference voltage and current values ​​of the micro-source grid-connected inverter, and perform voltage and current dual closed-loop control on it. The dual closed-loop controller dynamically adjusts the voltage and current in real time through the feedback loop, and considers the dynamic characteristics and response time of the system to design the feedback loop.

[0042] Furthermore, the distribution accuracy standard is:

[0043]

[0044] Among them, Q i represents the reactive power measured at the output of unit i, Q i * represents the expected reactive power share of unit i.

[0045] Furthermore, in step 21, the actual output power undertaken by each micro-source inverter is calculated by the following formula:

[0046] P i =E i ·i oi ·cosδ Q i =E i ·i oi sinδ;

[0047] Among them, P i Indicates the active power output by the micro-source inverter; Q i represents the reactive power output by the micro-source inverter; δ represents the angle between the output voltage phasor of the micro-source inverter and the voltage phasor at both ends of the load.

[0048] Furthermore, the power deviation is:

[0049] ΔQ=|Q i -Q ref |.

[0050] Furthermore, in step 24, the specific performance indicator includes: the cumulative reward reaches a certain threshold or the model converges.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] The present invention provides a virtual impedance gain change method based on DQN. After training iterations, the intelligent agent can output a better virtual impedance gain. Aiming at the problem of reactive power distribution of each micro-source in the microgrid, the present invention introduces a new droop control strategy of variable virtual impedance gain in the traditional adaptive virtual impedance control, thereby improving the reactive power distribution accuracy of the micro-source. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 The figure is a flow chart of the method for balancing the power of each DG in a ship microgrid based on DQN of the present invention.

[0055] Figure 2 This is a schematic diagram of the actual ship microgrid system environment of the present invention.

[0056] Figure 3 Schematic diagram of the control strategy of the present invention.

[0057] Figure 4 This is a diagram showing the mechanism of realizing power sharing according to the control strategy of the present invention.

[0058] Figure 5 Schematic diagram of offline training of the intelligent agent of the present invention.

[0059] Figure 6 It is a schematic diagram of reactive power sharing of a ship microgrid under ordinary adaptive virtual impedance control of the present invention.

[0060] Figure 7 This is a schematic diagram of reactive power sharing of a ship microgrid under virtual impedance control in combination with DQN of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0062] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0063] The power sharing method of each DG in the ship microgrid based on DQN is as follows: Figure 1 As shown in the figure, the specific process is as follows: First, the ship microgrid system is modeled to simulate the real ship microgrid system environment, such as Figure 2 As shown, the purpose is to obtain the reactive power sharing situation according to the environment. Establish an intelligent agent observation and reward mechanism. The DQN agent observes the power deviation data in real time, and according to the power deviation, the agent rewards or punishes according to the preset rules to optimize its output virtual impedance gain value. The system is iteratively optimized. The agent continuously learns and optimizes its decision rules to improve system performance through iteration.

[0064] The present invention includes a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization, comprising the following steps:

[0065] Step 1: Establish the main circuit model of the dual micro-source ship microgrid main circuit system. It includes the following steps:

[0066] Step 11: Select appropriate micro-source inverter circuit and filter according to the micro-source capacity and its application;

[0067] Step 12: Design the filter parameters so that they meet the following constraints:

[0068]

[0069] Where, f0 represents the fundamental frequency of the inverter bridge output voltage; f c Indicates the cutoff frequency of the LC filter; f s Indicates the switching frequency of the IGBT; L indicates the inductance of the LC filter; U inv max Indicates the maximum value of the fundamental phase voltage output by the inverter bridge; Indicates the maximum value of the LC filter inductor current; U o max Indicates the maximum value of the output voltage of the micro-source inverter; U o Represents the output voltage of the micro-source inverter; V dc Indicates the DC bus voltage; S n represents the rated capacity of the micro-source inverter; ω0 represents the angular frequency of the output voltage of the micro-source inverter; C represents the capacitance value of the LC filter.

[0070] As a preferred embodiment, in this application, two distributed micro-sources with the same capacity are selected, and the LC filter is used to filter the three-phase half-bridge inverter circuit. Specifically, the effective value of the output phase voltage of the micro-source inverter is U0=220V, the frequency f0=50Hz, and the DC micro-source output voltage V dc =800V, the switching frequency of the IGBT is 10kHz, the rated capacity of the micro-source inverter is 30kVA, the filter inductor L is 3mH, the filter capacitor C is 27μF, according to The cutoff frequency of the LC filter is 559.21 Hz, which satisfies the above formula. Based on this, the main circuit model system of the ship microgrid is constructed.

[0071] Step 2: According to the ship microgrid main circuit model system, a DQN-based droop controller is constructed. The implementation steps are as follows:

[0072] Step 21: Monitor the output voltage and output current of the micro-source inverter in real time, calculate the actual output power of each micro-source inverter, compare the calculated actual power with the reference power, and obtain the power deviation. On actual ships, it is monitored by the ship's power management system (Power Management System, PMS). However, this simulation is performed directly in the simulink simulation software using voltage transformers and current transformers for direct measurement. In step 21, the actual output power of each micro-source inverter is calculated by the following formula:

[0073] P i =E i ·i oi ·cosδ Q i =E i ·i oi sinδ;

[0074] Among them, P iIndicates the active power output by the micro-source inverter; Q i represents the reactive power output by the micro-source inverter; δ represents the angle between the output voltage phasor of the micro-source inverter and the voltage phasor at both ends of the load.

[0075] Step 22: Calculate the actual power P i With reference power P ref Compare and get the power deviation, which is used as the observation value of the intelligent agent and participates in the training of the intelligent agent;

[0076] Step 23: Initialize and use a neural network with randomly initialized weights to approximate the Q-value function Q(s, a; θ), where s represents the power deviation output by the micro-source inverter; a represents the virtual impedance gain; θ represents the weight of the neural network; then initialize a target network Q′(s, a; θ′) for stable training, where θ′ represents the weight of the target network; finally initialize the experience replay pool, which contains past states, actions, rewards, and new states; the specific control strategy is as follows Figure 3 shown.

[0077] Step 24: Establish the agent observation and reward mechanism, the reward function is:

[0078] r=μ∫max(0,min(1,γ(αΔQ+β)))+λ;

[0079] The DQN agent observes the power deviation data in real time and trains according to this reward function to obtain a larger reward. The agent rewards or punishes according to the preset rules to optimize the decision-making process. The training process continues until the predetermined number of training steps is reached and the specific performance indicators are achieved.

[0080] like Figure 4 As shown in Figure 1, when the impedance of the two micro-source lines is inconsistent, the power distribution of each micro-source is shown in A and B. After adding the virtual impedance, the system corrects the bus voltage and corrects a new virtual bus voltage U′ pcc And droop relationship. So the two curves A and B representing the power distribution of the two micro-sources are: curve A becomes curve C, curve B becomes curve D; the reactive deviation between the two becomes ΔQ′1. The power deviation is:

[0081] ΔQ=|Q i -Q ref |.

[0082] Step 25: After training is completed, the DQN agent uses the learned Q-value function to select the optimal action in each state in order to obtain the highest cumulative reward, thereby obtaining the optimal virtual impedance gain, which is combined with the virtual impedance to obtain the optimal virtual impedance size.

[0083] Step 26: Use the obtained optimal virtual impedance to correct the droop control strategy of the traditional fixed virtual impedance to obtain the updated reference voltage and current values ​​of the micro-source grid-connected inverter, and perform voltage and current dual closed-loop control on it. The dual closed-loop controller dynamically adjusts the voltage and current in real time through the feedback loop, and considers the dynamic characteristics and response time of the system to design the feedback loop.

[0084] Figure 5 The figure shows the training status of the intelligent agent. It can be seen from the figure that after the random process ends, the reward converges and the intelligent agent completes the training.

[0085] Step 3: The reactive power distribution accuracy index is established according to the reactive power distribution accuracy standard in the IEEE trans paper. Where Q i is the reactive power measured at the output of unit i, Q i * is the expected reactive power share of unit i.

[0086] like Figure 6 The figure shows the reactive power sharing of the ship microgrid under the common adaptive virtual impedance control. Figure 7 The figure shows the reactive power sharing of the ship microgrid under the virtual impedance control combined with DQN. According to the above formula, the reactive power distribution accuracy of the system is increased from 3.79% of the traditional adaptive virtual impedance to 2.87% after combining the DQN algorithm.

[0087] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0088] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A DQN-based ship microgrid adaptive reactive power allocation strategy, characterized in that: The following steps are involved: Step 1: Establish the main circuit model of the dual micro-source ship microgrid main circuit system; Step 2: According to the ship microgrid main circuit model system, a DQN-based droop controller is constructed; Step 3: Establishment of reactive power distribution accuracy index.

2. According to claim 1, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: The step 1 comprises the following steps: Step 11: Select the corresponding micro-source inverter circuit and filter according to the micro-source capacity and application; Step 12: Design the filter parameters so that the filter satisfies the constraint conditions.

3. According to claim 2, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: The constraints are: Where, f0 represents the fundamental frequency of the inverter bridge output voltage; f c Indicates the cutoff frequency of the LC filter; f s Indicates the switching frequency of the IGBT; L indicates the inductance of the LC filter; U invmax Indicates the maximum value of the fundamental phase voltage output by the inverter bridge; Indicates the maximum value of the LC filter inductor current; U omax Indicates the maximum value of the output voltage of the micro-source inverter; U o Represents the output voltage of the micro-source inverter; V dc Indicates the DC bus voltage; S n represents the rated capacity of the micro-source inverter; ω0 represents the angular frequency of the output voltage of the micro-source inverter; C represents the capacitance value of the LC filter.

4. According to claim 1, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: The step 2 comprises the following steps: Step 21: Real-time monitoring of the output voltage phase E of the micro-source inverter i , load current phasor i oi ; Step 22: Calculate the actual power P i With reference power P ref Compare and get the power deviation, which is used as the observation value of the intelligent agent and participates in the training of the intelligent agent; Step 23: Initialize and use a neural network with randomly initialized weights to approximate the Q-value function Q(s, a; θ), where s represents the power deviation output by the micro-source inverter; a represents the virtual impedance gain; θ represents the weight of the neural network; then initialize a target network Q′(s, a; θ′) for stable training, where θ′ represents the weight of the target network; finally initialize the experience replay pool, which contains past states, actions, rewards, and new states; Step 24: Establish the agent observation and reward mechanism, the reward function is: r=μ∫max(0,min(1,γ(αΔQ+β)))+λ; The DQN agent observes the power deviation data in real time and trains according to this reward function to obtain a larger reward. The agent rewards or punishes according to the preset rules to optimize the decision-making process. The training process continues until the predetermined number of training steps is reached and the specific performance indicators are achieved. Step 25: After training is completed, the DQN agent uses the learned Q-value function to select the optimal action in each state in order to obtain the highest cumulative reward, thereby obtaining the optimal virtual impedance gain, which is combined with the virtual impedance to obtain the optimal virtual impedance size; Step 26: Use the obtained optimal virtual impedance to correct the droop control strategy of the traditional fixed virtual impedance to obtain the updated reference voltage and current values ​​of the micro-source grid-connected inverter, and perform voltage and current dual closed-loop control on it. The dual closed-loop controller dynamically adjusts the voltage and current in real time through the feedback loop, and considers the dynamic characteristics and response time of the system to design the feedback loop.

5. According to claim 1, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: The distribution accuracy standard is: Among them, Q i represents the reactive power measured at the output of unit i, Q i * represents the expected reactive power share of unit i.

6. According to claim 1, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: In step 21, the actual output power of each micro-source inverter is calculated by the following formula: P i =E i ·i oi ·cosδ Q i =E i ·i oi ·sinδ; Among them, P i Indicates the active power output by the micro-source inverter; Q i It represents the reactive power output by the micro-source inverter; δ represents the angle between the output voltage phasor of the micro-source inverter and the voltage phasor at both ends of the load.

7. According to claim 1, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: The power deviation is: ΔQ=|Q i -Q ref |。 8. According to claim 4, a DQN-based ship microgrid reactive power distribution control strategy for virtual impedance optimization is characterized in that: In step 24, the specific performance indicator includes: the cumulative reward reaches a certain threshold or the model converges.

Images