Island microgrid distributed frequency control method and device

Abstract

The invention provides an island micro-grid distributed frequency control method and device, and the method comprises the steps: constructing a decentralized control architecture, enabling each power generation unit to serve as an intelligent agent with independent decision-making capability, and enabling each intelligent agent to generate a control action based on a local frequency deviation and an operation state; dividing each agent into different roles, and performing offline strategy training of introducing entropy enhancement constraint on each agent based on a role training mechanism to obtain a role strategy of each agent; carrying out value function decomposition and entropy regularization collaborative optimization on the role strategy to generate a distributed control strategy of each agent; and in the online operation process of the island microgrid, based on a distributed control strategy, each agent autonomously executes a power regulation instruction according to the local frequency deviation. According to the technical scheme, the frequency control stability, the strategy adaptability and the online response capability under the island micro-grid disturbance working condition are improved, and full-link distributed autonomous frequency control from strategy learning to real-time control is realized.

Description

Technical Field

[0001] This invention belongs to the field of power system dispatch automation technology, specifically relating to a distributed frequency control method and device for islanded microgrids. Background Technology

[0002] Against the backdrop of the global energy transition, islanded microgrids are becoming increasingly important as a key carrier for the efficient utilization and reliable power supply of distributed energy resources. Islanded microgrids can operate independently, providing stable power to remote areas and emergency scenarios. However, their frequency control currently faces many challenging issues, and traditional control methods are no longer sufficient to meet the operational requirements under complex conditions.

[0003] In isolated microgrids, frequency stability control is a core element in ensuring the safe and efficient operation of the system, but the technical challenges it faces are becoming increasingly prominent with the increasing penetration rate of renewable energy. Traditional centralized frequency control architectures are difficult to adapt to complex scenarios due to inherent defects: on the one hand, this architecture relies heavily on real-time communication networks to transmit data from each node to the central controller. Communication delays can cause control commands to be out of sync with actual load fluctuations, resulting in repeated frequency oscillations around the target value, and in extreme cases, even exceeding the safety threshold. On the other hand, as the decision-making hub, if the central node fails due to hardware failure, software crashes, or external attacks, all distributed power sources will lose their coordination basis and fall into a state of disordered operation. Diesel generators may blindly overclock, and energy storage units may exhaust their capacity due to charging and discharging conflicts, ultimately leading to system frequency collapse. In addition, hierarchical coordination problems between the controller and the power distributor often cause power overshoot. When there are deviations in the response speed and parameter matching between the two, "command overshoot" is likely to occur, causing the frequency to rise rapidly into the overshoot range, exacerbating equipment losses and potentially triggering malfunctions of protection devices.

[0004] While existing distributed control schemes attempt to circumvent the drawbacks of centralized systems, they still have significant limitations: some schemes rely on frequent communication between nodes, making them susceptible to communication barriers; most methods lack the ability to effectively handle high-dimensional decision spaces. In hybrid wind-solar-storage microgrids, due to the strong fluctuations in renewable energy output, traditional reinforcement learning algorithms often get stuck in local optima, exhibiting low sample efficiency and fragile convergence, making it difficult to cover optimal decision paths in complex scenarios. Furthermore, the "curse of dimensionality" in large-scale microgrids makes it difficult to solve the global value function. Existing value decomposition methods fail to fully integrate exploration strategy optimization, resulting in insufficient adaptability to sudden disturbances, frequency deviations often exceeding safe limits, and persistently high regulation costs.

[0005] Therefore, there is an urgent need for a new control method that combines communication robustness, decision adaptability, and cost-effectiveness to achieve accurate and stable frequency control of isolated microgrids. Summary of the Invention

[0006] This invention provides a distributed frequency control method for islanded microgrids. The solution proposes a distributed frequency control system and method for islanded microgrids based on role-based entropy enhancement. A decentralized neural network distributed control topology is designed, integrating a distributed decomposition multi-role multi-agent deep deterministic policy gradient algorithm with an offline maximum entropy deep reinforcement learning algorithm. An optimal policy library is generated through offline training, enabling online autonomous decision-making to balance power allocation and economy.

[0007] In a first aspect, the present invention provides a distributed frequency control method for an islanded microgrid, the method comprising the following steps: S1. Construct a decentralized multi-agent control architecture based on multiple distributed power units, treating each distributed power unit as an agent with independent decision-making capabilities. S2. Divide each agent into roles with different behavioral characteristics, and train each agent's role policy offline based on the role-based training mechanism to obtain the role-based policy of each agent based on the global optimum. S3. Perform value function decomposition on the role-making strategy to obtain the role-making strategy after value function decomposition. S4. Perform entropy regularization collaborative optimization on the role-based strategy after value function decomposition to generate distributed control strategies for each agent. S5. During the online operation of the islanded microgrid, based on the distributed control strategy of each agent, each agent generates its autonomous power regulation based on the local frequency deviation and the historical output status of each distributed power unit.

[0008] By adopting the above scheme, the distributed frequency control method for isolated microgrids of the present invention provides a new frequency control paradigm for isolated microgrids with robust communication, fast response, and optimal economy through cross-border integration of "bionic architecture + multi-agent reinforcement learning". It is the core technical support for the safe and efficient operation of microgrids in the era of high proportion of new energy.

[0009] In some embodiments of the present invention, in step S5, the autonomous power adjustment amount of each intelligent agent based on the local frequency deviation and the historical output state of each power unit is generated in the following way: Each agent acquires its local frequency deviation signal and the historical output status of each distributed power unit as current agent operating data, and uses the current agent operating data as the input vector of its distributed control strategy. The distributed control strategy outputs the autonomous power adjustment amount based on the input vector.

[0010] In some embodiments of the present invention, step S2, specifically includes the following steps for offline role-based strategy training for each agent based on the role-based training mechanism: The intelligent agent is assigned different roles: explorer, demonstrator, and leader; among which... The explorer generates explorer action samples based on random actions using a greedy strategy. The demonstrator generates demonstrator action samples based on the principle of a controller or distributor with preset performance and using a Gaussian noise detection strategy. The leader generates leader action samples based on policy fusion and using the OU noise detection strategy; Based on the Explorer action samples, Demonstrator action samples, and Leader action samples, a role strategy library containing action samples of multiple roles is constructed. The entropy of the role-based policy of each agent is calculated based on the policy corresponding to the sample in the role policy library.

[0011] In some embodiments of the present invention, in step S3, the value function decomposition is implemented using the following formula: The objective of the globally optimal policy is decomposed into the sum of the local value functions of each agent. , in, R total This is the globally optimal strategy. V i ( s ) is the first i The local value function of each agent.

[0012] In some embodiments of the present invention, step S4, which performs entropy regularization collaborative optimization on the role-based strategy after value function decomposition, is implemented using the following formula: Based on the aforementioned value function decomposition, an entropy regularization term is introduced to perform entropy regularization collaborative optimization. , in, R total This is the globally optimal strategy. V i (s) For the first i The local value function of an agent H(π i ) Let be the entropy of the role-playing strategy of the i-th agent d. For entropy weights.

[0013] In some embodiments of the present invention, by adjusting the entropy weight Implement the maximum entropy exploration mechanism; the entropy weight During the offline policy training process, the initial value is reduced according to a preset decay rule. It starts to decrease gradually from 0.

[0014] In some embodiments of the present invention, the role strategy library is obtained from pre-trained demonstrator samples; During the offline role strategy training process, if the convergence speed of the offline role strategy training is lower than a preset convergence speed threshold and / or the perturbation intensity of the training environment is higher than a preset threshold, the sample selection probability of selecting demonstrator action samples from the role strategy library is increased.

[0015] In some embodiments of the present invention, the generation of distributed control strategies for each intelligent agent further includes, during the online operation of the islanded microgrid, when a global frequency deviation is detected, each intelligent agent adjusts its distributed control strategy according to a dynamic anti-disturbance strategy.

[0016] In some embodiments of the present invention, the dynamic anti-interference strategy includes: When the absolute value of the global frequency deviation exceeds the first preset threshold, the weight of the maximum entropy exploration is temporarily increased; Calculate the comprehensive disturbance intensity index: , in, Δf For frequency deviation, This represents the rate of change of frequency deviation.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention proposes a multi-agent distributed architecture inspired by biological neural networks, constructs a frequency control scheme for isolated microgrids that integrates decentralized autonomous regulation, offline policy trainability, distributed collaborative optimization, and perturbation adaptive execution capabilities, and realizes a continuous control link from state perception and policy training to real-time regulation; it integrates a role-based training mechanism of maximum entropy exploration and value function decomposition and entropy regularization collaborative optimization, and innovates the entire process from architecture design, training optimization to real-time control, to achieve precise and stable frequency control of isolated microgrids.

[0018] A second aspect of the present invention provides a distributed frequency control system for an islanded microgrid, comprising: Decentralized Multi-Agent Control Architecture Module: Used to build a decentralized multi-agent control architecture based on multiple distributed power units, treating each power unit as an agent with independent decision-making capabilities; Role-based offline training module: This module is used to classify each agent into roles with different behavioral characteristics, and to train the role-based policy of each agent offline based on the role-based training mechanism, so as to obtain the role-based policy of each agent based on the global optimum. Value function decomposition and entropy regularization co-optimization module: used to decompose the role-based strategy into a value function to obtain the role-based strategy after value function decomposition; and to perform entropy regularization co-optimization on the role-based strategy after value function decomposition to generate distributed control strategies for each agent. Distributed online control execution module: used to generate autonomous power regulation amount for each agent based on local frequency deviation and the historical output status of each power unit during the online operation of the islanded microgrid, based on the distributed control strategy.

[0019] A third aspect of the present invention provides a distributed frequency control device for an islanded microgrid, characterized in that the device includes a computer device, the computer device including a processor and a memory, the processor storing computer instructions, and when the computer instructions are executed, the device implements the distributed frequency control method for the islanded microgrid.

[0020] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0021] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] In the attached diagram: Figure 1 This is a flowchart illustrating a distributed frequency control method for an islanded microgrid, as provided in an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of a distributed frequency control system for an islanded microgrid, provided as an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0027] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0028] Figure 1 This is a flowchart illustrating a distributed frequency control method for an islanded microgrid provided in an embodiment of the present invention.

[0029] Example 1, as Figure 1 As shown, this invention provides a distributed frequency control method for an islanded microgrid, the method comprising the following steps: S1. Construct a decentralized multi-agent control architecture based on multiple distributed power units, treating each distributed power unit as an agent with independent decision-making capabilities. S2. Divide each agent into roles with different behavioral characteristics, and train each agent's role policy offline based on the role-based training mechanism to obtain the role-based policy of each agent based on the global optimum. S3. Perform value function decomposition on the role-making strategy to obtain the role-making strategy after value function decomposition. S4. Perform entropy regularization collaborative optimization on the role-based strategy after value function decomposition to generate distributed control strategies for each agent. S5. During the online operation of the islanded microgrid, based on the distributed control strategy of each agent, each agent generates its autonomous power regulation based on the local frequency deviation and the historical output status of each distributed power unit.

[0030] By adopting the above scheme, the distributed frequency control method for isolated microgrids of the present invention constructs a decentralized multi-agent frequency control architecture, enabling each distributed power unit to generate control commands based solely on local operating information, avoiding dependence on a central controller and reducing the impact of communication link delays and link anomalies on the frequency control process. Based on this, an offline policy learning method based on a role-based training mechanism is introduced. Entropy enhancement constraints are used to improve the action diversity of the policy, allowing the agents to cover various operating conditions during the training phase and improving the policy's adaptability in complex isolated operating environments. Furthermore, through the synergistic optimization of value function decomposition and entropy regularization, the multi-agent system maintains policy convergence consistency in a high-dimensional action space and improves the stability of the policy network in generating control commands. Finally, during the operation of the isolated microgrid, each agent executes power regulation commands based on the distributed control strategy, achieving real-time response and continuous adjustment capabilities to frequency deviations.

[0031] In some embodiments of the present invention, in step S5, the autonomous power adjustment amount of each intelligent agent based on the local frequency deviation and the historical output state of each power unit is generated in the following way: Each agent acquires its local frequency deviation signal and the historical output status of each distributed power unit as current agent operating data, and uses the current agent operating data as the input vector of its distributed control strategy. The distributed control strategy outputs the autonomous power adjustment amount based on the input vector.

[0032] Preferably, this invention employs a multi-agent distributed architecture inspired by biological neural networks, namely a starfish-inspired distributed control architecture. This architecture centers on "local perception-global collaboration," designing each distributed power source (photovoltaic, energy storage, diesel generator, etc.) as an independent intelligent agent. Frequency control is achieved through a decentralized collaborative mechanism simulating the 12 legs of a starfish. The architecture comprises an agent layer, a communication layer, and a decision-making mechanism. Each power generation unit in the agent layer deploys a local policy network, collecting only local frequency deviations. It uses historical output data and does not rely on a central controller.

[0033] The state space of each agent is defined as the current frequency deviation, the integral frequency deviation, and the historical output, as shown in the following formula:

[0034] in, For real-time deviation, For the first The historical output sequence of an agent.

[0035] The communication layer only transmits global frequency offset signals. It does not transmit control commands; the communication protocol employs a lightweight local broadcast mechanism to ensure information transmission latency is <50ms. In the decision-making mechanism, each agent is based on a local policy network. Candidate adjustment of autonomous decision-making power The local policy network learns global cooperative rules through pre-training, namely:

[0036] The sum of the output adjustments of all agents must satisfy the power balance constraint, as shown in the following formula:

[0037] in, This represents the load fluctuation.

[0038] This embodiment addresses the inherent shortcomings of traditional centralized frequency control in isolated microgrids. Communication delays can cause severe frequency oscillations: the central controller relies on real-time data transmission, and signal lag leads to slow adjustments to load fluctuations, causing frequency to fluctuate repeatedly around the target value, or even exceed safe limits. Single-point failures at the central node are even more critical; if the central controller fails, diesel generators, energy storage units, and other equipment will lose coordination, potentially causing system frequency collapse. Furthermore, hierarchical coordination problems often lead to power overshoot; when the controller and power distributor malfunction, power generation may temporarily exceed demand, exacerbating equipment losses. This invention proposes treating each distributed power source, such as a diesel generator or energy storage unit, as an independent intelligent agent, mimicking the multi-legged cooperative mechanism of a starfish—a starfish without a central brain, its legs achieving global motion coordination through local perception and mechanical force transmission. In this control framework, each agent only needs to monitor local frequency deviations and historical output, learns the globally optimal strategy through a pre-training phase, and makes completely autonomous decisions during online operation. This architecture completely eliminates the dependence on a central controller and communication network, solving the frequency overshoot problem caused by controller and power distributor coordination failures in traditional hierarchical control. The architecture can maintain frequency stability even when communication is interrupted, and the cooperative efficiency of the 12 power generation units is highly consistent with the biological kinematic characteristics of the 12 legs of a starfish, verifying the adaptability of the biomimetic mechanism to high-dimensional control problems.

[0039] In some embodiments of the present invention, step S2, specifically includes the following steps for offline role-based strategy training for each agent based on the role-based training mechanism: The intelligent agent is assigned different roles: explorer, demonstrator, and leader; among which... The explorer generates explorer action samples based on random actions using a greedy strategy. The demonstrator generates demonstrator action samples based on the principle of a controller or distributor with preset performance and using a Gaussian noise detection strategy. The leader generates leader action samples based on policy fusion and using the OU noise detection strategy; Based on the Explorer action samples, Demonstrator action samples, and Leader action samples, a role strategy library containing action samples of multiple roles is constructed. The entropy of the role-based policy of each agent is calculated based on the policy corresponding to the sample in the role policy library.

[0040] Preferably, this invention employs a role-based training mechanism that integrates maximum entropy exploration. This mechanism addresses the convergence challenge of high-dimensional decision-making through multi-role differentiated exploration and entropy regularization enhancement, providing globally optimal strategies for agents in biomimetic architectures. Three types of virtual roles are constructed: explorer, demonstrator, and leader, each assigned a differentiated exploration strategy.

[0041] Explorers adopted - Greedy strategy ( =0.8), with an 80% probability of selecting the current optimal action and a 20% probability of random exploration, as shown in the following formula: The demonstrator employed a Gaussian noise strategy, adding Gaussian perturbations to the optimal action. The formula is shown below: .

[0042] The leader employs an OU noise strategy, generating time-dependent noise through an Ornstein-Uhlenbeck process to enhance strategy smoothness, as shown below: .

[0043] Introducing an entropy term into the policy output layer For each agent, the entropy of the role-based policy is calculated based on the policy corresponding to the sample in the role policy library, and the exploration randomness is dynamically adjusted, as shown below: .

[0044] This embodiment addresses the long-standing bottlenecks in deep reinforcement learning for microgrid frequency control—namely, low sample efficiency due to the high-dimensional decision space, fragile policy convergence, and insufficient adaptability to environmental disturbances—by proposing a complete entropy-enhanced exploration framework. The core breakthrough of this framework lies in deeply integrating the maximum entropy principle from information theory into the agent's learning mechanism. By dynamically adjusting the randomness of the policy, it fundamentally optimizes the balance between exploration and utilization. Traditional deep reinforcement learning methods often fall into local optima traps in complex microgrid environments with hybrid wind, solar, and energy storage systems. This is because the policy network tends to converge prematurely to a single action pattern, failing to effectively cover the optimal decision path in highly volatile scenarios. The entropy engine designed in this scheme embeds implicit random factors in the policy output layer, enabling the agent to maintain highly diverse action exploration capabilities in the early stages of training. As the learning process deepens, the system gradually reduces randomness through an adaptive temperature decay mechanism, ultimately leading to a precisely located globally optimal policy.

[0045] The implementation of this mechanism relies on a dual-channel collaborative architecture: First, an entropy monitoring module is introduced into the action generation layer to continuously quantify the dispersion of the strategy distribution, specifically the unpredictability of action selection, and transform it into an adjustment parameter for exploration intensity; second, a dual-path independent value evaluation network is constructed, adopting a pessimistic estimation principle, that is, always selecting the smaller value of the outputs of the two parallel Q networks as the target benchmark, completely eliminating the risk of strategy drift caused by overestimation of the value function. A specially designed temperature coefficient self-decay process enables the exploration rate to smoothly transition from the initial aggressive exploration state (80% of actions are dominated by entropy perturbations) to the precise utilization state in the stable operation stage (only 20% of actions retain random perturbations).

[0046] Based on the above dynamic adjustment exploration of randomness, in step S3, the value function decomposition is achieved using the following formula: The objective of the globally optimal policy is decomposed into the sum of the local value functions of each agent. , in, R total This is the globally optimal strategy. V i ( s ) is the first i The local value function of each agent.

[0047] In some embodiments of the present invention, step S4, which performs entropy regularization collaborative optimization on the role-based strategy after value function decomposition, is implemented using the following formula: Based on the aforementioned value function decomposition, an entropy regularization term is introduced to perform entropy regularization collaborative optimization. , in, R total This is the globally optimal strategy. V i (s) For the first i The local value function of an agent H(π i ) Let be the entropy of the role-playing strategy of the i-th agent d. For entropy weights.

[0048] The formula for the aforementioned co-optimization of value function decomposition and entropy regularization of the role-playing strategy defines an "ultimate objective function" for the entire multi-agent system. R total This objective function combines the two core requirements of "completing the task" and "maintaining innovation" to guide all agents to work together for optimization. V i (s) The higher the value, the greater the contribution of the agent to system stability (such as frequency recovery). By summing the local values ​​of the 12 agents, the system encourages the maximization of the sum of the local performance of all agents, thus naturally achieving global optimization. This reflects the specific significance of value function decomposition.

[0049] Furthermore, H(π i ) The higher the value, the more diverse and uncertain the behavior of the intelligent agent, that is, the stronger its exploratory nature; This determines the importance of "exploration" in the overall goal; α H(π i ) This encourages all agents to maintain a certain degree of randomness and exploratory nature, which is a manifestation of entropy regularization.

[0050] In some embodiments of the present invention, by adjusting the entropy weight Implement the maximum entropy exploration mechanism; the entropy weight During the offline policy training process, the initial value is reduced according to a preset decay rule. It starts to decrease gradually from 0.

[0051] Specifically, the entropy weight is set to a high value in the early stages of training and gradually reduced according to an adaptive decay mechanism as the training progresses. Specifically: α decays with each training round.

[0052] In some embodiments of the present invention, during the training process of the offline role policy training, demonstrator samples for guiding the learning of each agent are selected from the role policy library according to the frequency perturbation state based on the sample selection probability. If the convergence speed of the offline role policy training is lower than a preset convergence speed threshold and / or the perturbation intensity of the training environment is higher than a preset threshold, the sample selection probability of selecting demonstrator action samples from the role policy library is increased.

[0053] For example, during training, if a perturbation strength index is defined... :

[0054] in, For the rate of change of frequency deviation, when (With the threshold set to 0.1Hz / s), the sample weight of the presenter role is increased, as shown in the following formula:

[0055] in, The probability of selecting demonstrator samples increases from 0.2 to 0.8 as the perturbation increases, ensuring that high-value demonstrator samples dominate policy updates.

[0056] The above scheme, by appropriately increasing the selection probability of demonstrator samples during offline policy training, increases the proportion of high-value, stable trajectories among the samples participating in policy updates. This achieves two goals: First, when updating policy network parameters, it can refer more to the high-quality behaviors of low-frequency fluctuations and low-control-cost demonstrations provided by demonstrators, shortening policy convergence time and reducing ineffective exploration during training. Second, it retains the stability characteristics of demonstrator behavior in the policy after training convergence, effectively suppressing large frequency oscillations during actual islanded microgrid operation, and improving the dynamic stability and robustness of the system under disturbance conditions.

[0057] In some embodiments of the present invention, the generation of distributed control strategies for each agent further includes, during the online operation of the islanded microgrid, when a global frequency deviation is detected, each agent adjusts its distributed control strategy according to a dynamic anti-disturbance strategy.

[0058] In some embodiments of the present invention, the dynamic anti-interference strategy includes: When the absolute value of the global frequency deviation exceeds the first preset threshold, the weight of the maximum entropy exploration is temporarily increased; Calculate the comprehensive disturbance intensity index: , in, Δf For frequency deviation, This represents the rate of change of frequency deviation.

[0059] To better explain the role of the dynamic disturbance rejection strategy of the above technical solution in dealing with sudden disturbances during the online operation of islanded microgrids, the specific process is as follows: The dynamic disturbance rejection strategy adopted in this invention adaptively adjusts the role priority and entropy weight based on the frequency deviation intensity to maintain system stability during load fluctuations or new energy disturbances. Its control logic depends on the distributed decision-making and training mechanism of the preceding architecture.

[0060] This invention defines a disturbance intensity index :

[0061] in, This represents the rate of change of frequency deviation.

[0062] when When the Hz is >0.05, the entropy weight The formula for temporarily increasing exploration ability by 50% is shown below:

[0063] in, This is an indicator function; it is 1 if the condition is met, and 0 otherwise. The strategy update formula is then revised to:

[0064] The function's purpose is to detect whether the system deviates from its normal state in real time. Once an anomaly is detected, the algorithm's "enhanced exploration" mode is automatically activated, thereby improving the system's intelligence, adaptability, and recovery speed in dynamic disturbance environments.

[0065] The aforementioned method, by invoking pre-trained role policies and entropy control mechanisms, rapidly activates collaborative responses when disturbances occur, achieving a closed-loop process from architecture design to training optimization and real-time disturbance mitigation. In the multi-agent system architecture of isolated microgrids, the dynamic disturbance mitigation strategy utilizes trained policies to adjust role priorities and entropy weights to cope with disturbances, ultimately achieving stable control of the isolated microgrid.

[0066] This scheme constructs a role-based entropy collaborative optimization framework, achieving adaptation to complex environments through a three-level linkage mechanism. The multi-role sample generation layer virtualizes three types of functional agents: the Explorer injects Gaussian noise to drive high-risk action attempts; the Demonstrator mimics a classic PID controller to generate high-value experience samples; and the Leader aggregates heterogeneous experiences and outputs the final deployment strategy—forming a training sample pool covering the entire exploration spectrum. The entropy-role dynamic coupling layer: the Leader network synchronously calculates entropy weights during policy updates. When the system detects a frequency deviation exceeding a threshold (e.g., |), the leader weights are applied. Δf When the frequency exceeds 0.05Hz, an anti-disturbance mode is automatically triggered: the demonstrator sample weight ξ is drastically reduced from the baseline value of 0.8 to 0.3, while the entropy coefficient α is instantly increased by 150%, significantly enhancing the policy's stochastic exploration capability to cope with sudden disturbances. Distributed value decomposition layer: The global reward function is decoupled to the local agent, allowing each unit to optimize the individual entropy regularization term while considering the group's objective. This collaborative mechanism reduces the number of training rounds required for convergence from over 8000 rounds to 2200 rounds in a 200-node test system, lowering the adjustment cost by 0.017%, and maintaining a stable frequency deviation within 0.0037Hz even under minute-level fluctuations in wind and solar power, providing a scalable solution for high-proportion renewable energy access.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention constructs a decentralized multi-agent frequency control system, combining role-based policy training, maximum entropy constraint exploration, value function decomposition collaborative optimization, and disturbance adaptive control to achieve closed-loop autonomous control capability for isolated microgrid frequency control, from policy learning to real-time execution. This scheme utilizes local state input and a lightweight communication broadcast mechanism, enabling each agent to generate power regulation commands without a central controller, reducing communication dependence and single-point failure risks. Entropy-enhanced role-based offline training and batch iteration of the sample pool endow the policy with exploratory capabilities and trainability under complex operating conditions. Value function decomposition and entropy regularization constraints improve the consistency and convergence stability of distributed action decisions. Furthermore, a disturbance threshold-triggered parameter adaptive mechanism enables agents to continuously adjust and respond to disturbances during operation, thus forming a scalable distributed frequency regulation control process and achieving continuous control and online autonomous execution capabilities for isolated microgrids with multi-source collaboration.

[0068] This invention proposes a multi-agent distributed architecture inspired by biological neural networks, a role-based training mechanism that integrates maximum entropy exploration, and a collaborative optimization of value function decomposition and entropy regularization. It aims to revolutionize the entire process from architecture design and training optimization to real-time control, thereby achieving precise and stable frequency control of isolated microgrids.

[0069] Figure 2 This is a flowchart illustrating a distributed frequency control system for an islanded microgrid, provided in one embodiment of the present invention.

[0070] Example 2, as Figure 2 As shown, the present invention also provides a distributed frequency control system for islanded microgrids, comprising: Decentralized multi-agent control architecture module S11: Based on multiple distributed power units, a decentralized multi-agent control architecture is constructed, with each power unit acting as an agent with independent decision-making capabilities; Role-based offline training module S12: This module is used to classify each agent into roles with different behavioral characteristics. Based on the role-based training mechanism, it performs offline role policy training for each agent to obtain the role-based policy of each agent based on the global optimum. Value function decomposition and entropy regularization co-optimization module S13: used to decompose the role-based strategy using a value function to obtain the role-based strategy after value function decomposition; and to perform entropy regularization co-optimization on the role-based strategy after value function decomposition to generate distributed control strategies for each agent. Distributed online control execution module S14: During the online operation of the islanded microgrid, based on the distributed control strategy of each agent, each agent generates its autonomous power regulation based on the local frequency deviation and the historical output status of each distributed power unit.

[0071] Example 3: The present invention also provides a distributed frequency control device for an islanded microgrid. The device includes a computer device, which includes a processor and a memory. The processor stores computer instructions. When the computer instructions are executed, the device implements the distributed frequency control method for the islanded microgrid.

[0072] Example 4, as Figure 3 As shown, the present invention also provides an electronic device 100 for implementing a distributed frequency control method for islanded microgrids.

[0073] The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104.

[0074] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the distributed frequency control method for islanded microgrids described in the first aspect of the present invention by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0075] The memory 101 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0076] At least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 102 may be a microprocessor or any conventional processor. Processor 102 is the control center of electronic device 100, connecting various parts of electronic device 100 via various interfaces and lines.

[0077] The memory 101 in the electronic device 100 stores multiple instructions to implement a distributed frequency control method for an islanded microgrid, and the processor 102 can execute multiple instructions to achieve the following: A decentralized multi-agent control architecture is constructed based on multiple distributed power units, with each distributed power unit acting as an agent with independent decision-making capabilities. Each agent is divided into roles with different behavioral characteristics. Offline role policy training is performed on each agent based on the role-based training mechanism to obtain the role-based policy of each agent based on the global optimum. The role-playing strategy is decomposed into a value function to obtain the role-playing strategy after value function decomposition. Entropy regularization collaborative optimization is performed on the role-based strategy decomposed by the value function to generate distributed control strategies for each agent. During the online operation of the isolated microgrid, based on the distributed control strategy of each agent, each agent generates its own autonomous power regulation based on the local frequency deviation and the historical output status of each distributed power unit.

[0078] Example 5: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0079] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A distributed frequency control method for an islanded microgrid, characterized in that, The method includes the following steps: S1. Construct a decentralized multi-agent control architecture based on multiple distributed power units, treating each distributed power unit as an agent with independent decision-making capabilities. S2. Divide each agent into roles with different behavioral characteristics, and train each agent's role policy offline based on the role-based training mechanism to obtain the role-based policy of each agent based on the global optimum. S3. Perform value function decomposition on the role-making strategy to obtain the role-making strategy after value function decomposition. S4. Perform entropy regularization collaborative optimization on the role-based strategy after value function decomposition to generate distributed control strategies for each agent. S5. During the online operation of the islanded microgrid, based on the distributed control strategy of each agent, each agent generates its autonomous power regulation based on the local frequency deviation and the historical output status of each distributed power unit.

2. The distributed frequency control method for islanded microgrids according to claim 1, characterized in that, In step S5, each agent generates its autonomous power adjustment amount based on the local frequency deviation and the historical output status of each power unit in the following way: Each agent acquires its local frequency deviation signal and the historical output status of each distributed power unit as current agent operating data, and uses the current agent operating data as the input vector of its distributed control strategy. The distributed control strategy outputs the autonomous power adjustment amount based on the input vector.

3. The distributed frequency control method for islanded microgrids according to claim 1, characterized in that, In step S2, the specific steps for offline role-based policy training for each agent based on the role-based training mechanism include: The intelligent agent is assigned different roles: explorer, demonstrator, and leader; among which... The explorer generates explorer action samples based on random actions using a greedy strategy. The demonstrator generates demonstrator action samples based on the principle of a controller or distributor with preset performance and using a Gaussian noise detection strategy. The leader generates leader action samples based on policy fusion and using the OU noise detection strategy; Based on the Explorer action samples, Demonstrator action samples, and Leader action samples, a role strategy library containing action samples of multiple roles is constructed. The entropy of the role-based policy of each agent is calculated based on the policy corresponding to the sample in the role policy library.

4. The distributed frequency control method for islanded microgrids according to claim 3, characterized in that, In step S3, the value function decomposition is achieved using the following formula: The objective of the globally optimal policy is decomposed into the sum of the local value functions of each agent. ， in, R total The globally optimal strategy V i ( s ) is the first i The local value function of each agent.

5. The distributed frequency control method for islanded microgrids according to claim 4, characterized in that, In step S4, the entropy regularization collaborative optimization of the role-based strategy after value function decomposition is implemented using the following formula: Based on the aforementioned value function decomposition, an entropy regularization term is introduced to perform entropy regularization collaborative optimization. ， in, R total The globally optimal strategy V i (s) For the first i The local value function of an agent H(π i ) Let the entropy be the role-based strategy of the i-th agent. For entropy weights.

6. The distributed frequency control method for islanded microgrids according to claim 5, characterized in that, By adjusting the entropy weight Implement the maximum entropy exploration mechanism; the entropy weight During the offline policy training process, the initial value is reduced according to a preset decay rule. It starts to decrease gradually from 0.

7. The distributed frequency control method for islanded microgrids according to claim 6, characterized in that, During the offline role strategy training process, if the convergence speed of the offline role strategy training is lower than a preset convergence speed threshold and / or the perturbation intensity of the training environment is higher than a preset threshold, the sample selection probability of selecting demonstrator action samples from the role strategy library is increased.

8. The distributed frequency control method for islanded microgrids according to claim 7, characterized in that, The generation of distributed control strategies for each agent also includes, during the online operation of the islanded microgrid, when a global frequency deviation is detected, each agent adjusts its distributed control strategy according to a dynamic anti-disturbance strategy.

9. The distributed frequency control method for islanded microgrids according to claim 8, characterized in that, The dynamic anti-interference strategy includes: When the absolute value of the global frequency deviation exceeds the first preset threshold, the weight of the maximum entropy exploration is temporarily increased; Calculate the comprehensive disturbance intensity index: , in, Δf For frequency deviation, This represents the rate of change of frequency deviation.

10. A distributed frequency control device for an islanded microgrid, characterized in that, The device includes a computer device, which includes a processor and a memory. The processor stores computer instructions. When the computer instructions are executed, the device implements the distributed frequency control method for islanded microgrids as described in any one of claims 1 to 9.

Images