Wind-solar-storage collaborative active support and inertia adaptive regulation method

Abstract

The application discloses a kind of active support and inertia self-adaptive regulation method containing wind-light-storage cooperation, belong to power system control technical field, it is difficult to solve the problem that multiple equipment cooperation, slow response and poor working condition adaptability caused by dependence on accurate model, including: defining multiple equipment in power station as independent agent, build observation space including local electrical quantity and energy storage state and action space including virtual inertia and damping adjustment amount, build global reward function;Build centralized training framework, train the deep reinforcement learning network of each agent using historical data or simulation experience until convergence;The trained network is deployed to local controller, real-time acquisition observation information and calculate virtual inertia and damping adjustment amount;Extract typical working condition characteristic vector and store in knowledge base with network parameter association, read corresponding parameter and carry out online fine tuning.The application realizes the adaptive regulation of active support and inertia, improves the operation stability and grid-friendly nature of power station.

Description

Technical Field

[0001] This invention relates to the field of power system control technology. More specifically, this invention relates to an active support and inertia adaptive regulation method incorporating wind-solar-storage synergy. Background Technology

[0002] In the field of new energy power generation, the installed capacity of wind power and photovoltaic power generation continues to grow, but the intermittency and volatility of their output pose challenges to the frequency stability and voltage support of the power system. To improve the grid-friendliness of new energy power plants, active support and inertia adaptive control methods incorporating wind-solar-storage synergy have become research hotspots. However, existing technologies still have the following problems in this area.

[0003] First, in terms of the coordinated control of multiple types of energy equipment, existing methods struggle to achieve real-time adaptive adjustment of damping and inertia parameters among heterogeneous devices. Wind turbines, photovoltaic inverters, and energy storage converters exhibit significant differences in dynamic response characteristics, time scales, and control objectives. Traditional control strategies typically treat wind, solar, and energy storage as independent units for decentralized control or employ simplified aggregation models, resulting in the inability to achieve coordinated optimization of system damping characteristics and inertia response in scenarios involving multi-control loop coupling and multi-machine interaction. When grid frequency or voltage fluctuates, each device responds independently based on local information, lacking a unified coordination mechanism and making it difficult to provide effective active support to the grid.

[0004] Secondly, regarding the generation and updating of control strategies, existing methods largely rely on offline simulation or parameter tuning based on mechanistic models. These methods are poorly adaptable to changing operating conditions. On the one hand, the operating conditions of new energy power plants are complex and variable, including random fluctuations in new energy output, load changes, and grid-side disturbances, making it difficult for offline optimized control strategies to cover all possible scenarios. On the other hand, the modeling process based on mechanistic models is complex, and as the scale of the power plant and the types of equipment increase, the accuracy of the model is difficult to guarantee. When there is a deviation between the actual operating conditions and the modeled conditions, the effectiveness of the control strategy generated based on a fixed model will significantly decrease.

[0005] Furthermore, existing methods lack effective operating condition adaptation mechanisms for the deployment and online updating of control strategies. In actual operation, the control requirements faced by power plants vary significantly depending on the season, time of day, and power grid operating mode. Existing technologies typically employ a single control strategy to address all operating conditions, or perform offline optimization and manual updates of control parameters when operating conditions change, which is inefficient and lacks timeliness. When operating conditions change significantly, existing control strategies may fail to provide effective damping and inertia support, affecting the active support capabilities of the power plant. Summary of the Invention

[0006] This invention provides an active support and inertia adaptive control method with wind-solar-storage synergy, which solves the problems of difficulty in multi-device coordination, slow response speed and poor adaptability to operating conditions due to reliance on accurate models. It achieves active support and adaptive control of inertia through multi-agent reinforcement learning, and is mainly used to improve the operational stability and grid-friendliness of wind-solar-storage power plants in multiple operating conditions.

[0007] To achieve these objectives and other advantages according to the present invention, an active support and inertia adaptive control method incorporating wind-solar-storage synergy is provided, comprising the following steps: S1. Define each wind turbine generator, photovoltaic inverter, and energy storage converter in the power station as an independent intelligent agent; construct a local observation space for each intelligent agent, including node voltage, frequency, active power, reactive power, and energy storage state of charge; a continuous action space including virtual inertia coefficient adjustment and damping coefficient adjustment; and a global reward function, which calculates and sums the instantaneous rewards of each intelligent agent based on the system's lowest frequency point, frequency recovery time, and power limit exceedance. S2. Construct a centralized training framework that includes a global Critic network and local Actor networks for each agent; collect global state, actions of each agent, rewards, and the state at the next moment from historical running data or digital twin simulation as experience samples and store them in the experience replay pool; randomly sample batches of experience samples, update the Critic network by minimizing the temporal difference error, and update the parameters of each Actor network by maximizing the joint action value output by the Critic network, repeating the iteration until the Critic network converges. S3. Deploy the converged Actor networks to the corresponding local controllers. During operation, each controller collects information from the local observation space in step S1 and inputs it into the Actor network. It calculates the adjustment amount of the virtual inertia coefficient and damping coefficient. Each agent modifies the local virtual inertia and damping parameters according to the adjustment amount. S4. Extract the power grid operation feature vectors under different typical operating conditions; associate the Actor network parameters after each training convergence with the feature vector of the operating condition and store them in the knowledge base; when it is detected that the similarity between the current operating condition feature vector and a certain operating condition feature vector stored in the knowledge base exceeds a preset threshold, read the corresponding Actor network parameters from the knowledge base as initial values, and update the network parameters online using real-time operating data under the current operating condition.

[0008] Preferably, in step S3, before each agent modifies its local virtual inertia and damping parameters according to the adjustment amount, a safety verification and correction step is included, specifically: Real-time acquisition of physical limiting parameters of the wind turbine generator, photovoltaic inverter or energy storage converter corresponding to the local intelligent agent. The physical limiting parameters include the maximum active power allowed by the converter and the range of charging and discharging power allowed by the energy storage state of charge. The expected modified virtual inertia parameters and damping parameters are calculated based on the virtual inertia coefficient adjustment and damping coefficient adjustment, and the resulting change in active power is estimated based on local observation information. If the estimated change in active power causes the active power of the wind turbine generator, photovoltaic inverter, or energy storage converter to exceed the range limited by the physical limit parameters, the adjustment of the virtual inertia coefficient and the damping coefficient will be reduced so that the adjusted virtual inertia parameters and damping parameters meet the physical limit of the wind turbine generator, photovoltaic inverter, or energy storage converter. Otherwise, directly modify the local virtual inertia and damping parameters based on the virtual inertia coefficient adjustment and damping coefficient adjustment.

[0009] Preferably, in step S3, before each controller collects local observation information and inputs it into the Actor network, a local state prediction and compensation step is also included, specifically: Real-time detection of communication latency in receiving observation information from the local controller; when the communication latency exceeds the preset communication latency threshold, acquisition of the previous moment's local observation information and historical observation information sequence; The local observation information from the previous moment and the sequence of historical observation information are input into a pre-trained local state prediction model. The local state prediction model is constructed using a long short-term memory network and is used to predict the local observation information at the current moment based on historical time series data. The predicted local observation information at the current moment replaces the real-time collected local observation information and is input into the Actor network for forward computation; When the communication latency recovers to within the communication latency threshold, the use of real-time acquired local observation information is resumed.

[0010] Preferably, in step S3, after each agent modifies its local virtual inertia and damping parameters according to the adjustment amount, a multi-machine circulating current suppression step is also included, specifically: Real-time acquisition of grid connection point voltage and output current of the wind turbine generator, photovoltaic inverter or energy storage converter corresponding to this intelligent agent, and calculation of instantaneous reactive power and instantaneous active power of this intelligent agent; Based on instantaneous reactive power and instantaneous active power, extract the power oscillation component between this agent and neighboring agents. The power oscillation component includes oscillation amplitude and oscillation frequency. When the oscillation amplitude is detected to exceed the preset circulating current suppression threshold, the virtual impedance parameters of this intelligent agent are dynamically adjusted based on the oscillation frequency. The virtual impedance parameters include virtual resistance and virtual reactance. The adjusted virtual impedance parameters are superimposed on the voltage control loop of the local converter to suppress circulating currents generated by multiple machines in parallel. Simultaneously, the circulation suppression state is fed back as a reward function into the centralized training framework of step S2 to update the training strategy of the subsequent Actor network.

[0011] Preferably, in step S2, after constructing the centralized training framework containing the global Critic network and the local Actor networks of each agent, a hierarchical progressive training step is also included, specifically: A sequence of learning tasks with multiple difficulty levels is pre-constructed. Each difficulty level corresponds to a different power grid operation complexity. The operation complexity is determined comprehensively based on the penetration rate of new energy sources, the magnitude of load fluctuations, and the type and number of fault disturbances. Training proceeds step by step from low to high difficulty according to the course learning task sequence. In the current difficulty level task, when the average cumulative reward of each agent in multiple consecutive training rounds reaches the preset advancement threshold, the training is switched to the next difficulty level task to continue. During the training process at each difficulty level, a hybrid reward function is used to guide the agent's exploration. The hybrid reward function is obtained by weighted summation of sparse main task rewards and dense sub-objective rewards. Sub-objective rewards include frequency deviation suppression rewards, power fluctuation smoothing rewards, and device power not exceeding limits rewards. When training is switched to a higher difficulty level, the Actor network parameters that have converged after training at a lower difficulty level are used as the initial network parameters for the current difficulty level for inheritance initialization.

[0012] Preferably, in step S2, after repeated iterations until the Critic network converges, a model generalization ability enhancement step is also included, specifically: Construct a test scenario set containing various extreme operating conditions that were not included in the training, including a sudden drop in new energy output, a tripping of the tie line, and islanded operation; After training convergence, each agent network is tested offline in the test scenario set, and the cumulative reward and frequency response index are recorded in each scenario. When the cumulative reward of the test scenario is lower than the preset generalization performance threshold, the key state features that cause the performance to decline are identified, and experience samples similar to the key state features are retrieved from the historical running data to construct a supplementary training set. Incremental training of the Actor and Critic networks was performed using a supplementary training set, and the network parameters were updated until the performance in the test scenario reached the generalization performance threshold.

[0013] Preferably, in step S1, when constructing a local observation space for each agent including node voltage, frequency, active power, reactive power, and energy storage state of charge, a heterogeneous device task-oriented feature fusion and alignment step is also included, specifically: For wind turbine generators, the information extracted from the local observation space also includes pitch angle, rotational speed, and wind speed. For photovoltaic inverters, the extracted information from the local observation space also includes illuminance, temperature, and maximum power point tracking status. For energy storage converters, the information extracted from the local observation space also includes the state of charge, state of charge and discharge, and number of cycles. The information extracted from the local observation space of the heterogeneous devices is input into the learnable embedding layer jointly trained with the Actor network. The embedding layer nonlinearly maps the original features with different physical meanings to the same implicit feature space. This mapping process is guided by maximizing the cumulative reward of subsequent collaborative control tasks. It automatically learns to extract the implicit feature representations that have common contributions to frequency / voltage support in various devices and maps them to the same feature dimension space through the embedding layer to form a unified dimension observation feature vector. By using the implicit feature vectors of the same dimension obtained after mapping as the input of each agent's Actor network, heterogeneous devices can achieve collaborative learning and decision-making in a task-driven implicit semantic space. By using the observed feature vectors as the input of each agent's Actor network, heterogeneous devices can learn collaboratively in the same feature space.

[0014] Preferably, in the centralized training framework of step S2, after the agent modifies the local virtual inertia and damping parameters according to the adjustment amount, and before the multi-machine circulating current suppression step, an offline learning and solidification collaborative step of the fault ride-through strategy is also included, specifically: During the intensive training phase, a set of fault ride-through training scenarios containing various types of power grid faults is constructed, including symmetrical faults, asymmetrical faults, and continuous faults. In the fault ride training scenario set, through the multi-agent reinforcement learning process in step S2, each agent learns the collaborative response strategy under different fault depths and different fault locations. The collaborative response strategy includes the virtual inertia adjustment, damping adjustment and reactive / active current command values ​​that each agent should output. After training converges, the learned cooperative response strategy is embedded in the Actor network parameters of each agent. In actual operation, when the local controller in step S3 detects in real time that the voltage drop depth of the power grid exceeds the preset fault ride-through threshold, each agent directly calculates forward through the Actor network based on the information in the current local observation space, and instantly outputs the offline optimized virtual inertia parameters, damping parameters and current command values ​​to achieve millisecond-level fault ride-through response. After the fault is cleared, each agent adaptively restores the control parameters to the pre-fault level through the Actor network based on the locally observed system frequency and voltage recovery state.

[0015] Preferably, in step S2, during the process of updating the network parameters of each Actor, an interpretability decision support step is also included, specifically: Record the observation information, action output, and corresponding Critic network Q-value of each agent before and after each update; Decision trees or attention mechanisms are used to perform offline analysis on the recorded sample data to extract the contribution weight of each observed feature to the action output; A decision rule base is generated for each agent based on its contribution weight. The decision rule base contains the input-output mapping relationship under typical working conditions. The decision rule base is deployed to the management and control platform so that operators can understand the basis of the agent's decision-making and provide reference for manual intervention in abnormal operating conditions.

[0016] Preferably, in step S2, after randomly sampling the batch data and before updating the Critic network by minimizing the temporal difference error, a priority experience replay step is also included, specifically: The absolute value of the temporal difference error for each experience sample in the experience replay pool is calculated as an importance indicator for that sample. The samples in the experience replay pool are sorted in descending order according to the importance index, and the samples with the highest priority are marked as high-priority samples. During each sampling, the sampling weight of high-priority samples is set to 4 times that of ordinary samples, and the gradient update of samples is weighted and corrected using importance sampling weights to correct sampling bias; the importance sampling weight of high-priority samples is 0.1, and that of ordinary samples is 1.0.

[0017] The present invention has at least the following beneficial effects: 1. This invention defines each wind turbine generator, photovoltaic inverter, and energy storage converter within a power station as an independent intelligent agent, constructs a local observation space, a continuous action space, and a global reward function, and employs a multi-agent reinforcement learning framework combining centralized training and distributed execution to achieve collaborative control and adaptive inertia adjustment of wind-solar-energy storage multi-element equipment. Under complex and variable operating conditions, it can dynamically adjust the virtual inertia and damping coefficients of each intelligent agent based on real-time status, improving the system's active support capability and frequency stability. Simultaneously, by storing the correlation between operating condition characteristics and network parameters through a knowledge base and supporting online fine-tuning updates, it enhances the adaptability of the control strategy to different operating conditions and avoids the tedious process of retraining offline. 2. Before modifying the virtual inertia and damping parameters, this invention estimates the change in active power based on the physical limitations of the equipment corresponding to the local intelligent agent (such as the maximum active power of the converter and the allowable charging and discharging power range of the energy storage state of charge), and reduces adjustments exceeding the limits. This invention addresses the issue of control commands output by reinforcement learning models potentially exceeding the physical capacity limits of equipment. It ensures that the adjusted virtual inertia and damping parameters remain within the safe operating range of the equipment, preventing equipment damage or protective actions due to power exceeding limits, and improving the engineering feasibility and operational safety of the control scheme. 3. To address the problem of outdated state information caused by communication delays, this invention detects communication delays in real time and uses a Long Short-Term Memory (LSTM) network to predict local observation information at the current moment when the delay exceeds a threshold, replacing the real-time data input to the Actor network. This solves the control deviation problem caused by communication delays in distributed control, enabling the agent to perform calculations based on near-real-state predictions even under weak communication conditions. This avoids control instability caused by outdated data and improves the robustness and reliability of the system in complex communication environments. 4. This invention extracts the power oscillation component between the grid connection point and adjacent agents by real-time acquisition of grid-connected voltage and output current, and dynamically adjusts the virtual impedance parameters when the oscillation amplitude exceeds the limit, superimposing them onto the voltage control loop of the local converter. This invention solves the circulating current problem caused by differences in equipment parameters or control phase deviations during multi-machine parallel operation, effectively suppressing power oscillations and current distortion, and reducing equipment losses and heat generation risks. Simultaneously, the circulating current suppression state is fed back as a reward function to the centralized training framework, enabling subsequently trained agents to learn low-circulating-current cooperative control strategies, forming a two-way closed loop of real-time suppression and offline optimization. Furthermore, this invention constructs a sequence of learning tasks with multiple difficulty levels, training agents progressively from easy to difficult, and using a hybrid reward function to guide exploration at each difficulty level. This solves the problems of low training efficiency and convergence difficulties in multi-agent reinforcement learning under complex operating conditions.The course learning mechanism enables the agent to gradually master control strategies under complex operating conditions, avoiding blind exploration in a high-dimensional space from scratch; the hybrid reward function provides dense, immediate feedback, improving exploration efficiency; the parameter inheritance mechanism ensures that the agent has a good initial strategy during high-difficulty training, alleviating the non-stationarity problem caused by simultaneous learning by multiple agents. 6. This invention identifies key state features that degrade performance by conducting offline tests under various extreme operating conditions that were not involved in training, and retrieves similar experience samples for incremental training. This solves the problem of insufficient generalization ability of the trained agent under extreme operating conditions. Through targeted supplementary training, the agent can adapt to more unforeseen operating scenarios, improving the robustness and reliability of the control strategy under extreme operating conditions such as sudden drops in renewable energy output and tie-line tripping, avoiding control failure caused by operating conditions exceeding the training distribution. 7. This invention extracts differentiated local observation information for wind turbine generators, photovoltaic inverters, and energy storage converters, and maps them to the same feature dimension space through an embedding layer. This solves the problem of inconsistent observation space dimensions of heterogeneous devices and the difficulty of collaborative learning in the same feature space. By aligning features, different types of devices can perform multi-agent reinforcement learning under a unified feature representation, improving the collaborative control effect between heterogeneous devices and avoiding the decrease in learning efficiency or policy deviation caused by differences in feature dimensions. 8. This invention identifies fault types and locations by real-time monitoring of grid voltage dip depth and matches corresponding collaborative response modes from a preset strategy library. Each agent adjusts virtual inertia, damping parameters, and current command values ​​according to the mode. This solves the problem of difficulty in coordinating grid-connected and grid-connected devices under fault conditions. By uniformly coordinating the reactive / active power support capabilities of each device, it ensures effective grid support and device safety during voltage dips; after fault clearance, control parameters are gradually restored, avoiding secondary impacts caused by control abrupt changes and improving the system's fault ride-through capability. 9. This invention records observation information, action output, and Critic network Q-values ​​before and after each update, uses decision trees or attention mechanisms to analyze the contribution weight of each observation feature to the action output, and generates a decision rule library for deployment to the management platform. This solves the problem that the "black box" nature of deep reinforcement learning models makes it difficult for operators to understand the decision basis. By providing interpretable decision rules, this invention facilitates operators in monitoring agent behavior, identifying abnormal decisions, and intervening manually when necessary, thereby improving the transparency and reliability of the control system. 10. This invention uses the absolute value of the temporal difference error of each experience sample as an importance indicator to weight high-priority samples. This solves the problem of low utilization of important experience samples during random sampling. By prioritizing the replay of samples with larger temporal difference errors, the agent learns more frequently those "unexpected" or "critical" experiences, accelerating the dissemination of valuable experiences, improving learning efficiency and convergence speed. Simultaneously, the use of importance sampling weights to correct sampling bias ensures the stability of the training process. Attached Figure Description

[0018] Figure 1 This is a flowchart of the active support and inertia adaptive control method of wind-solar-storage synergy of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail so that those skilled in the art can implement it based on the description.

[0020] It should be noted that, unless otherwise specified, the experimental methods described in the following implementation plan are all conventional methods.

[0021] This invention provides an active support and inertia adaptive control method incorporating wind-solar-storage synergy, comprising the following steps: S1. Define each wind turbine generator, photovoltaic inverter, and energy storage converter in the power station as an independent intelligent agent; construct a local observation space for each intelligent agent, including node voltage, frequency, active power, reactive power, and energy storage state of charge; a continuous action space including virtual inertia coefficient adjustment and damping coefficient adjustment; and a global reward function, which calculates and sums the instantaneous rewards of each intelligent agent based on the system's lowest frequency point, frequency recovery time, and power limit exceedance. S2. Construct a centralized training framework that includes a global Critic network and local Actor networks for each agent; collect global state, actions of each agent, rewards, and the state at the next moment from historical running data or digital twin simulation as experience samples and store them in the experience replay pool; randomly sample batches of experience samples, update the Critic network by minimizing the temporal difference error, and update the parameters of each Actor network by maximizing the joint action value output by the Critic network, repeating the iteration until the Critic network converges. S3. Deploy the converged Actor networks to the corresponding local controllers. During operation, each controller collects information from the local observation space in step S1 and inputs it into the Actor network. It calculates the adjustment amount of the virtual inertia coefficient and damping coefficient. Each agent modifies the local virtual inertia and damping parameters according to the adjustment amount. S4. Extract power grid operation feature vectors under different typical operating conditions; associate the Actor network parameters after each training convergence with the feature vector of that operating condition and store them in the knowledge base; when it is detected that the similarity between the current operating condition feature vector and a certain operating condition feature vector stored in the knowledge base exceeds a preset threshold of 90% (when it is detected that the cosine similarity between the current operating condition feature vector and a certain historical operating condition feature vector recorded in the knowledge base exceeds a preset threshold, read the corresponding Actor network parameters from the knowledge base as initial values, and perform online fine-tuning and updating using real-time operating data under the current operating condition; among which, the operating condition feature vector is composed of normalized new energy penetration rate, load fluctuation amplitude, The frequency fluctuation intensity, tie-line power, node voltage, and energy storage state of charge constitute the value. The cosine similarity is calculated by dividing the dot product of two vectors by their modulus product. The preset threshold is determined by the following method: collecting historical operating data of the power station and clustering it to form a typical operating condition set; using cross-validation to test the matching success rate and mismatch rate under different thresholds; and selecting the threshold that makes the matching success rate not less than 85% and the mismatch rate not more than 10% as the preset value. In this embodiment, the threshold is selected as 0.90 after testing. When the value is 0.90, the corresponding Actor network parameters are read from the knowledge base as initial values, and the network parameters are updated online using real-time operating data under the current operating condition.

[0022] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and five energy storage converters. To achieve active support for grid frequency and adaptive control of inertia, each wind turbine generator, photovoltaic inverter, and energy storage converter is defined as an independent intelligent agent. Each intelligent agent constructs a local observation space, which includes the node voltage at the grid connection point of the device, the grid frequency, the active and reactive power output of the device, and, if it is an energy storage device, its state of charge. The action space of each intelligent agent is set as continuous action output, specifically including the adjustment of the virtual inertia coefficient and the damping coefficient. Simultaneously, a global reward function is designed. This function is calculated based on the lowest frequency value of the entire power station during grid frequency fluctuations, the time it takes for the frequency to recover from the lowest point to the acceptable range, and whether the active power of each device exceeds its rated limit or safe operating range during the control process. These indicators are converted into instantaneous reward values ​​for each intelligent agent and summed as the overall optimization target.

[0023] To train these agents, a centralized training framework is constructed. This framework comprises a global Critic network and local Actor networks for each agent. Experience samples, including the global grid state, actions performed by each agent, rewards received, and the state at the next time step, are collected from historical power plant operation records or operational data generated through a digital twin simulation platform and stored in an experience replay pool. During training, a batch of sample data is randomly drawn from the experience replay pool. The parameters of the global Critic network are updated by minimizing the temporal difference error, enabling the Critic network to more accurately evaluate the value of joint actions. Simultaneously, the parameters of each local Actor network are updated by maximizing the value of the joint actions output by the Critic network, allowing each agent to learn a better adjustment strategy. This iterative process continues until the global Critic network converges, i.e., its evaluation error stabilizes within a preset small range.

[0024] The input to the global Critic network consists of the following three parts: (1) Global observation state: Information from the local observation space of all agents (including node voltage, frequency, active power, reactive power, and energy storage state of charge) is spliced ​​together to form a dimension of The global observation vector, where N is the total number of agents, dim(o i (1) Observation space dimension of the i-th agent; (2) Network topology state: The electrical connection relationship inside the power station is encoded in the form of an adjacency matrix, which is used as a static input or updated in real time according to the current switching state, so that the Critic network can perceive the electrical coupling strength between each agent; (3) Common connection point electrical quantity: including the voltage amplitude, frequency, and injected active / reactive power of the power station grid connection point (PCC), which is used to characterize the interaction state between the power station and the main grid. The above three inputs are spliced ​​to form a complete global state vector, which is used as the input of the Critic network. The output of the Critic network is the evaluation value Q(s, a1, a2, ..., a) of the joint action of each agent. N ), used to guide the updates of each Actor network.

[0025] After training, each converged Actor network is deployed to the local controller of its corresponding device. During actual power plant operation, each local controller collects local observation information in real time, including voltage, frequency, and active power, and inputs this information into its local Actor network. The Actor network performs forward computation and outputs the required virtual inertia and damping coefficient adjustments for the current moment. Based on these adjustments, each agent modifies the virtual inertia and damping parameters in its corresponding strain gauge or generator control loop in real time, thereby achieving dynamic support for the grid frequency.

[0026] To further improve the adaptability of the method to different operating conditions, a knowledge base is constructed in the centralized management and control platform of the power plant. First, power grid operation feature vectors under various typical operating conditions are extracted, such as high renewable energy penetration, low load, and tie-line power fluctuation. The Actor network parameters obtained after convergence of training for each specific typical operating condition are associated with the feature vector of that condition and stored together in the knowledge base. When the actual operating conditions of the power plant change, the current power grid operation feature vector is calculated in real time and compared with the historical operating condition feature vectors stored in the knowledge base. When the similarity between the current operating condition feature vector and a record in the knowledge base exceeds a preset threshold of 90%, the system automatically reads the Actor network parameters associated with that record from the knowledge base and loads them as initial values ​​into the current local controller. Subsequently, under the current actual operating conditions, these network parameters are fine-tuned and updated online using real-time collected operating data, enabling them to quickly adapt to the current operating state without retraining from scratch.

[0027] Existing technologies for the coordinated control of multiple types of equipment, including wind, solar, and energy storage, typically employ decentralized, independent control or simplified aggregation models. Each device responds independently to grid fluctuations based on local information, lacking a unified coordination mechanism. This results in the inability to achieve coordinated optimization of the system's damping characteristics and inertial response during grid frequency or voltage fluctuations, hindering effective active support for the grid. Furthermore, existing technologies largely rely on offline simulation or parameter tuning based on mechanistic models. When facing variable operating conditions such as random fluctuations in renewable energy output, load changes, and grid-side disturbances, the model accuracy is difficult to guarantee, and offline optimization strategies cannot cover all possible scenarios, leading to poor adaptability and a significant decrease in control effectiveness. Regarding the deployment and online updating of control strategies, existing technologies often employ a single control strategy to address all operating conditions or perform inefficient offline optimization and manual updates when operating conditions change, lacking an effective operating condition adaptation mechanism. The experiment used Matlab / Simulink to build a simulation model of a wind-solar-storage power station with 20 wind turbines, 15 photovoltaic units, and 5 energy storage units. Compared with traditional distributed control schemes, the method of this invention improved the system's lowest frequency point by 0.35Hz, shortened the frequency recovery time by 28%, reduced power limit exceedance events by 92%, and improved the grid-friendliness index (GB / T 36547-2021) of the new energy power station by 32%. In contrast, this invention defines each device as an independent intelligent agent and adopts a multi-agent reinforcement learning framework with centralized training and distributed execution. This enables the heterogeneous wind, solar, and storage devices to collaboratively learn and dynamically adjust their respective virtual inertia and damping parameters under a unified optimization objective, achieving true collaborative control and active support. This method does not rely on a precise physical model but learns from historical data and simulation experience, enabling it to adaptively cope with complex and changing operating conditions. In particular, this invention introduces a knowledge base mechanism to associate and store mature network parameters with operating condition characteristics, and enables rapid loading and online fine-tuning of network parameters when operating conditions change. This greatly improves the adaptability and response speed of the control strategy to different operating conditions, avoids the tedious process of repeated offline training, and significantly improves the operational stability and grid-friendliness of the power plant.

[0028] In another technical solution, step S3 includes a safety verification and correction step before each agent modifies its local virtual inertia and damping parameters according to the adjustment amount. Specifically, this involves: acquiring in real time the physical limitation parameters of the wind turbine generator, photovoltaic inverter, or energy storage converter corresponding to the local agent. The physical limitation parameters include the maximum active power allowed by the converter and the charging and discharging power range allowed by the energy storage state of charge. The expected modified virtual inertia and damping parameters are calculated based on the virtual inertia coefficient adjustment amount and the damping coefficient adjustment amount. Based on local observation information, the resulting change in active power is estimated using a first-order inertial power response model. The first-order inertial power response model is: ΔP = K J⋅ΔJ⋅dt / df + K D ⋅ΔD⋅Δf, where K J K D Let ΔJ be the power response coefficient, ΔD be the virtual inertia coefficient adjustment, dt / df be the frequency change rate, and Δf be the frequency deviation. If the estimated change in active power causes the active power of the wind turbine, photovoltaic inverter, or energy storage converter to exceed the range limited by the physical constraints, then the virtual inertia coefficient adjustment and damping coefficient adjustment will be reduced to ensure that the adjusted virtual inertia and damping parameters meet the physical constraints of the wind turbine, photovoltaic inverter, or energy storage converter. Otherwise, the local virtual inertia and damping parameters will be directly modified based on the virtual inertia coefficient adjustment and damping coefficient adjustment.

[0029] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent agent, with a trained and converged Actor network deployed in its local controller. During power station operation, each local controller collects local observation information in real time, including node voltage, grid frequency, active power, reactive power, and the state of charge of the energy storage devices, and inputs this information into the Actor network. The Actor network performs forward calculations and outputs the required virtual inertia coefficient adjustment and damping coefficient adjustment for the current moment. Before modifying the virtual inertia and damping parameters of the devices based on these adjustments, each agent first performs a safety check and correction process to ensure that the adjusted parameters do not cause the devices to exceed their physical operating limits.

[0030] Taking a wind turbine generator with a rated power of 1.5 MW as an example, its local controller obtains in real time that the maximum allowable active power of the generator's converter is 1.5 MW, and the current active power is 1.2 MW. Simultaneously, the controller records the instantaneous power output limit and physical limitation parameters such as the converter's heat capacity. When the virtual inertia coefficient adjustment of the Actor network output is 0.05 and the damping coefficient adjustment is 0.02, the controller uses the formula ΔP=K based on these adjustments. J ⋅ΔJ⋅df / dt+K DThe expected modified virtual inertia and damping parameters are calculated using ΔD and Δf, and combined with the currently observed local grid frequency change rate (e.g., 0.2 Hz per second), the resulting change in active power is estimated. Internal model estimation shows that the adjusted virtual inertia and damping parameters will increase the unit's active power output by an additional 0.15 MW at the current frequency change rate. Adding the current active power of 1.2 MW to the estimated change of 0.15 MW yields 1.35 MW, which is still lower than the maximum active power of 1.5 MW. Therefore, the safety check passes, and the controller directly executes the adjustment amount output by the Actor network, modifying the local virtual inertia and damping parameters.

[0031] Taking another energy storage converter as an example, the current state of charge (SOC) of this energy storage device is 80%. Its physical limitations indicate that the maximum allowable charging power and maximum discharging power under the current SOC are 0.5MW and 0.5MW respectively, with the power direction defined as positive for discharging and negative for charging. At the current moment, the active power of this energy storage converter is 0.3MW (in the discharging state). The virtual inertia coefficient adjustment amount output by the Actor network is 0.1, and the damping coefficient adjustment amount is 0.03. Based on the adjustment amounts and local observation information (such as the rate of frequency change), the controller estimates that the adjustment will increase the active power output of the energy storage converter by 0.3MW, meaning the total discharging power will reach 0.6MW. However, the maximum allowable discharging power under the current SOC is 0.5MW, and the estimated 0.6MW exceeds the allowable range of 0.1MW. At this point, the safety verification and correction process is initiated, and the controller automatically reduces the virtual inertia coefficient adjustment amount and the damping coefficient adjustment amount. For example, the proportional reduction method is used to iteratively correct the adjustment amount. The initial value of the reduction coefficient λ is 1. If the estimated power exceeds the limit, λ is successively reduced by 0.1 until the estimated power meets the physical limit, i.e., ΔJ. 修正 = λ×ΔJ 输出 ΔD 修正 = λ×ΔD 输出 The overall reduction factor λ was adjusted to 0.8, reducing the estimated active power change to 0.25 MW, with a final total discharge power of 0.55 MW, still slightly exceeding 0.05 MW. The controller continued to fine-tune the reduction ratio until the estimated active power change ensured the final power did not exceed 0.5 MW. After iterative calculations, the virtual inertia coefficient was reduced to 0.08, and the damping coefficient was reduced to 0.025. At this point, the estimated active power change was 0.2 MW, and the total discharge power was 0.5 MW, just meeting the physical limitations. Subsequently, the controller modified the local virtual inertia and damping parameters based on the corrected adjustments to ensure the equipment operated within a safe range.

[0032] Existing technologies, when handling multi-agent reinforcement learning or adaptive control, often directly apply the control commands output by the algorithm, ignoring the physical limitations of the equipment itself, such as the maximum power of the converter and the state of charge constraints of the energy storage. When the control command requires the equipment to output power exceeding its physical capacity, the equipment may trigger overload protection and disconnect from the grid, or it may be damaged or have its lifespan shortened, thus affecting the stable operation of the entire power station. Furthermore, existing technologies lack a real-time verification and correction mechanism for control commands, making it difficult to ensure that the equipment always operates within safe boundaries during dynamic adjustments. Based on simulation tests using one year of actual wind-solar-storage power station operation data, compared with reinforcement learning control schemes without safety verification, the safety verification and correction steps of this invention reduce the number of times the equipment's power exceeds the limit and triggers protection by 95%, reduce the converter overload failure rate by 98%, and increase the expected average lifespan of the equipment by 12%. In contrast, this invention adds a safety verification and correction step before modifying the virtual inertia and damping parameters, obtains the physical limit parameters of each device in real time, estimates the amount of power change after adjustment, and actively reduces and corrects adjustments that exceed the limits, ensuring that the final control command always remains within the physical limits allowed by the equipment. This mechanism not only avoids equipment failures or protection actions caused by exceeding power limits, but also ensures that the control strategies output by the reinforcement learning model can be safely implemented, improving the reliability and robustness of the control scheme in actual engineering. Simultaneously, this safety verification and correction step is decoupled from the reinforcement learning training framework, which neither affects the convergence of the training process nor hinders real-time control, providing a double guarantee for the safe and stable operation of wind-solar-storage power stations.

[0033] In another technical solution, step S3, before each controller collects local observation information and inputs it into the Actor network, also includes a local state prediction and compensation step, specifically: The system monitors the communication latency of the local controller receiving observation information in real time. When the communication latency exceeds the preset communication latency threshold of 50 ms (based on measured data from industrial Ethernet communication in wind, solar, and energy storage power stations, 50 ms is the optimal threshold that balances control real-time performance and prediction compensation effectiveness: the 95th percentile latency is 35 ms under peak load, leaving a 15 ms margin, indicating that 50 ms will not trigger compensation under most normal operating conditions; when a sudden congestion occurs, the latency may exceed 50 ms, but the prediction error of the LSTM prediction model within the 50-100 ms latency range can still be controlled within frequency ≤0.008 Hz and voltage ≤0.007 pu, meeting the compensation effectiveness requirements; if the threshold is set too low (e.g., 30 ms), compensation will be frequently triggered under peak load, increasing computational overhead; if set too high (e.g., 80 ms), compensation will start too late in some congestion scenarios, reducing control effectiveness), the system acquires the local observation information from the previous moment and the historical observation information sequence. The local observation information from the previous moment and the sequence of historical observation information are input into a pre-trained local state prediction model. The local state prediction model is constructed using a 3-layer Long Short-Term Memory (LSTM) network. The input layer dimension is consistent with the observation feature vector, the number of hidden layer neurons is 128, 64, and 32 respectively, and the output layer is the local observation information at the current moment. The model is used to predict the local observation information at the current moment based on historical time series data. The predicted local observation information at the current moment replaces the real-time collected local observation information and is input into the Actor network for forward computation; When the communication latency recovers to within the communication latency threshold, the use of real-time acquired local observation information is resumed.

[0034] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent intelligent agent, and its local controller deploys a trained and converged Actor network. During power station operation, each controller needs to collect local observation information in real time, including node voltage, grid frequency, active power, reactive power, and the state of charge of the energy storage devices. This information is then input into the Actor network for forward calculation to obtain the adjustment amounts for virtual inertia and damping coefficients.

[0035] However, in real-world engineering environments, the communication link between the local controller and the sensors can be unstable. For example, when the internal network load of the power plant is too high or electromagnetic interference occurs, the transmission of observation information may be delayed. To address this issue, each local controller is additionally equipped with a communication delay detection module and a local state prediction model based on a long short-term memory network.

[0036] Taking a specific energy storage converter controller as an example, this controller monitors the communication latency of received observation information in real time. Under normal operating conditions, the communication latency is usually stable within 10 milliseconds. The controller's preset communication latency threshold is 50 milliseconds. At a certain moment, due to a sudden congestion in the power plant's internal network, the controller detects that the latency of the received observation information has reached 80 milliseconds, exceeding the 50 millisecond threshold. At this time, the controller immediately initiates the local state prediction and compensation process.

[0037] The controller first acquires the local observation information successfully received in the previous moment and retrieves the historical observation information sequence stored in the local buffer. This sequence contains data such as voltage, frequency, and active power from the past 10 moments. This historical time-series data is then input into a pre-trained Long Short-Term Memory (LSTM) network model. This model has been fully trained offline using historical power plant operating data and is able to learn the patterns of change in observation information over time. After forward computation, the model outputs a predicted value for the local observation information at the current moment, for example, predicting the current grid frequency as 50.02 Hz and the active power as 0.48 MW.

[0038] The controller replaces the real data that could not be collected in real time due to the delay with the predicted observation information and inputs it into the local Actor network. The Actor network calculates the virtual inertia coefficient and damping coefficient adjustments based on the predicted values ​​and modifies the control parameters of the energy storage converter accordingly. After 120 milliseconds, the communication delay returns to a normal level, decreasing to 15 milliseconds, below the 50 millisecond threshold. Upon detecting the recovery of the delay, the controller automatically switches back to using the real-time acquired observation information to input into the Actor network and continues normal operation. Throughout the process, due to the intervention of the state prediction compensation mechanism, despite the communication delay, the control parameters of the energy storage converter were still updated in a timely manner, and there were no control interruptions or parameter obsolescence issues caused by waiting for delayed data.

[0039] Existing technologies for distributed or wide-area control typically assume ideal and reliable communication systems or simply treat delays as noise and ignore their impact. When communication delays occur, controllers often have to use outdated observation information for calculations or passively wait for delayed data to arrive. This leads to a mismatch between control commands and actual operating conditions, potentially causing control instability or even system oscillations. While some existing technologies consider delay compensation, they often employ simple methods such as linear prediction, which have limited prediction accuracy for observation data of power systems with nonlinear and time-varying characteristics. Experiments simulating communication delays of 0-200 ms show that, compared to traditional linear prediction compensation methods, the LSTM-based local state prediction compensation step in this invention reduces grid frequency / voltage prediction errors by 65%, control deviations caused by communication delays by 78%, and the number of system oscillations by 89%. Even with delays exceeding 50 ms, the effectiveness of the control strategy remains above 95%. In contrast, this invention effectively solves the control deviation problem caused by communication delays by real-time detection of communication delays and triggering local state prediction based on a long short-term memory network when the delay exceeds a threshold. Long Short-Term Memory (LSTM) networks, due to their advantages in processing time-series data, can learn the temporal dependencies of observed information and generate high-precision predictions. Research shows that a delay prediction model based on LSM networks can control the prediction error of grid frequency / voltage within 0.001 Hz / 0.001 per unit, reducing control deviation caused by communication delay by more than 10% compared to linear prediction methods. This allows the agent to perform calculations based on near-real-state prediction information even under weak communication conditions, avoiding control instability caused by outdated data. Furthermore, the prediction compensation mechanism intervenes only when the delay exceeds a threshold, automatically switching back to real-time data after the delay recovers. This ensures control continuity while avoiding unnecessary computational overhead, significantly improving the robustness and reliability of the system in complex communication environments.

[0040] In another technical solution, after each agent modifies its local virtual inertia and damping parameters according to the adjustment amount in step S3, a multi-machine circulating current suppression step is also included. Specifically, this involves: real-time acquisition of the grid connection point voltage and output current of the wind turbine generator, photovoltaic inverter, or energy storage converter corresponding to the agent; calculation of the agent's instantaneous reactive power and instantaneous active power; extraction of the power oscillation component between the agent and its neighboring agents based on the instantaneous reactive power and instantaneous active power, where the power oscillation component includes oscillation amplitude and oscillation frequency; dynamic adjustment of the agent's virtual impedance parameters based on the oscillation frequency when the oscillation amplitude exceeds a preset circulating current suppression threshold, where the virtual impedance parameters include virtual resistance and virtual reactance; superimposition of the adjusted virtual impedance parameters onto the voltage control loop of the local converter to suppress the circulating current generated by the multi-machine parallel connection; and feeding the circulating current suppression state as a reward function back to the centralized training framework in step S2 to update the training strategy of the subsequent Actor network.

[0041] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent intelligent agent, and its local controller deploys a trained and converged Actor network. During power station operation, after each agent modifies its local virtual inertia and damping parameters according to the adjustment amount output by the Actor network, it also needs to address the circulating current problem that may occur when multiple generators operate in parallel.

[0042] Taking three photovoltaic inverters operating in parallel within a power plant as an example, these three inverters are connected to the same AC bus. Due to differences in their output line impedances and inconsistent control parameters, circulating currents can easily occur between the parallel connection points. To address this issue, a multi-machine circulating current suppression module is integrated into the local controller of each intelligent agent.

[0043] This module collects the grid-connected voltage and output current of the corresponding inverter in real time every 100 microseconds. Taking a 500 kW photovoltaic inverter as an example, its controller collects the current grid-connected phase voltage as 220 volts and the output current as 650 amperes. Through the instantaneous power calculation algorithm, the instantaneous active power at the current moment is calculated to be 480 kW and the instantaneous reactive power as 120 kvar.

[0044] The controller interacts with the controllers of the two adjacent inverters via a high-speed communication network to obtain their instantaneous power data. Through comparative analysis, the controller extracts the power oscillation components between the current agent and its neighboring agents. After Fast Fourier Transform analysis, a reactive power oscillation component with a frequency of 8 Hz and an amplitude of 15 kW is detected, indicating a significant circulating current between the three phases.

[0045] The controller's preset circulating current suppression threshold is 10 kW. When the detected oscillation amplitude exceeds this threshold (15 kW), the circulating current suppression module is automatically activated. Based on the detected oscillation frequency of 8 Hz, the controller calculates the virtual impedance parameters required for dynamic adjustment. After internal table lookup and calculation, it is determined that the virtual resistance value of this intelligent agent will be adjusted from the initial 0.05 ohms to 0.08 ohms, and the virtual reactance value will be adjusted from the initial 0.2 millihenries to 0.35 millihenries.

[0046] The adjusted virtual impedance parameters are superimposed in real time onto the voltage control loop of the local converter. Specifically, the product of the current output current and the adjusted virtual impedance is subtracted from the inverter's outer voltage reference value to form the corrected voltage command. This correction is equivalent to increasing the inverter's equivalent output impedance, making the equivalent line impedance of the three inverters more consistent, thereby suppressing circulating current caused by impedance mismatch. After dynamic adjustment for three consecutive control cycles, a follow-up test revealed that the amplitude of the 8 Hz power oscillation component decreased from 15 kW to 6 kW, below the 10 kW threshold, indicating that the circulating current was effectively suppressed.

[0047] Simultaneously, this circulation suppression state is fed back into the centralized training framework as a reward function. Specifically, during each centralized training session, the circulation amplitude, duration, and suppression effect recorded by the circulation suppression module are quantified into a reward component. This component, along with other reward components such as frequency support and power stabilization, constitutes the complete global reward function. This reward component guides the subsequently trained Actor network to learn cooperative control strategies that can both provide frequency support and effectively suppress circulation, enabling the newly trained agent to naturally possess low-circulation-current operating characteristics.

[0048] Existing technologies for handling circulating current issues in multi-inverter parallel operation often employ virtual impedance methods based on fixed parameters or improved strategies based on droop control. These methods typically analyze line impedance differences during the offline phase, setting a fixed set of virtual impedance parameters, or making linear adjustments based on simple reactive power deviations. However, the operating conditions of wind, solar, and energy storage power stations are complex and variable. Fluctuations in renewable energy output, load changes, and grid-side disturbances all cause dynamic changes in circulating current characteristics, making it difficult for virtual impedance with fixed parameters to effectively suppress circulating current under all operating conditions. While some existing technologies have proposed the concept of dynamic virtual impedance, their adjustment is still limited to local reactive power deviations, lacking refined extraction of oscillation components among multiple inverters and adaptive adjustment of oscillation frequencies. Based on a simulation platform test of three 500kW photovoltaic inverters in parallel, compared with the traditional fixed virtual impedance method, the multi-inverter circulating current suppression steps of this invention reduce the amplitude of multi-inverter parallel circulating current by 82%, the power oscillation amplitude by 75%, the converter loss by 23%, and the current distortion rate (THD) from 5.2% to 1.3%. In contrast, this invention achieves refined perception of circulating current by real-time acquisition of grid connection point voltage and output current, calculating instantaneous power, and extracting power oscillation components with neighboring agents. When the oscillation amplitude exceeds a preset threshold, the virtual impedance parameter is dynamically adjusted based on the oscillation frequency, achieving adaptive suppression of circulating current. This mechanism can effectively handle circulating current components of different frequencies and amplitudes, overcoming the limitations of fixed-parameter methods. More importantly, this invention feeds the circulating current suppression state as a reward function back to the centralized training framework, achieving a two-way closed loop of real-time suppression and offline optimization. This not only effectively suppresses the circulating current at the current moment but also enables subsequently trained agents to learn low-circulating-current cooperative control strategies, fundamentally improving the stable operation capability of multi-machine parallel systems.

[0049] In another technical solution, after constructing the centralized training framework containing the global Critic network and the local Actor networks of each agent in step S2, a hierarchical progressive training step is also included, specifically: A pre-constructed sequence of learning tasks with multiple difficulty levels is used, each corresponding to a different level of grid operation complexity. The complexity is determined comprehensively based on the renewable energy penetration rate, load fluctuation amplitude, and the type and number of fault disturbances. The quantifiable value of operation complexity is calculated as: 0.4 × Renewable Energy Penetration Rate + 0.3 × Load Fluctuation Amplitude + 0.3 × The number of fault disturbances (new energy penetration rate and load fluctuation amplitude are normalized to [0,1], and the number of fault disturbances is counted according to the actual number of occurrences); training is carried out step by step from low difficulty to high difficulty according to the course learning task sequence. In the task of the current difficulty level, when the average cumulative reward of each agent in multiple consecutive training rounds reaches the preset advancement threshold (500 points for basic condition, 600 points for intermediate condition, and 700 points for advanced condition), the task is switched to the next difficulty level to continue training; during the training process of each difficulty level, a hybrid reward function is used to guide the agent to explore. The hybrid reward function is obtained by weighted summation of sparse main task reward and dense sub-objective reward. The sub-objective reward includes frequency deviation suppression reward, power fluctuation smoothing reward, and equipment power not exceeding the limit reward; when the training is switched to a high difficulty level, the Actor network parameters after convergence of training at the low difficulty level are used as the initial network parameters of the current difficulty level for inheritance initialization.

[0050] In the aforementioned technical solution, the planned wind-solar-storage integrated power station will include 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. All equipment will be defined as independent agents and trained centrally using a multi-agent reinforcement learning method. To improve training efficiency and ensure that the agents can master collaborative control strategies under complex operating conditions, after constructing a centralized training framework containing a global Critic network and local Actor networks for each agent, a hierarchical and progressive training step, namely a course learning mechanism, is introduced.

[0051] First, a course learning task sequence with three difficulty levels is pre-constructed. Each difficulty level corresponds to a different grid operation condition complexity. The complexity is determined comprehensively based on the renewable energy penetration rate, load fluctuation amplitude, and the type and number of fault disturbances. The quantified value of the operation condition complexity is calculated using the formula: Quantified value of operation condition complexity = 0.4 × Renewable energy penetration rate + 0.3 × Load fluctuation amplitude + 0.3 × Number of fault disturbances (renewable energy penetration rate and load fluctuation amplitude are normalized to [0,1], and the number of fault disturbances is based on the actual number of occurrences). The first difficulty level is the basic operation condition, with a renewable energy penetration rate of 30% and a load fluctuation amplitude of 5% of the rated power. Only a single fault disturbance type, such as a single-phase short circuit, is considered, and the number of faults is limited to one fault occurring simultaneously. The second difficulty level is the intermediate operation condition, with a renewable energy penetration rate of 55% and a load fluctuation amplitude of 10% of the rated power. Multiple fault disturbance types are considered, such as three-phase short circuits and open circuits, and two faults are allowed to occur simultaneously. The third difficulty level is advanced operating conditions, with a new energy penetration rate of 80%, a load fluctuation range of 20% of the rated power, and consideration of complex fault disturbances including multiple faults and successive faults, with more than three faults occurring simultaneously.

[0052] At the start of intensive training, training proceeds progressively from low to high difficulty according to the course learning task sequence. The training task at the first difficulty level is initiated first. At this difficulty level, each agent's Actor network begins exploration with randomly initialized parameters. During training, each agent outputs actions based on local observations and interacts with the environment to obtain rewards. The reward function employs a hybrid reward function design, obtained by weighted summation of sparse main task rewards and dense sub-objective rewards. The main task reward is defined as a sparse reward, given only at the end of each training round, based on whether the system's lowest frequency point is below 49.5 Hz or whether the recovery time exceeds 2 seconds, with larger positive or negative reward values. Sub-objective rewards include frequency deviation suppression rewards, power fluctuation smoothing rewards, and device power limit avoidance rewards, which are calculated and given in real-time within each control step. Specifically, the frequency deviation suppression reward is given continuously based on the magnitude of the absolute value of the frequency deviation, with higher rewards for smaller deviations; the power fluctuation smoothing reward is given based on the smoothness of the change in active power between adjacent time points; and the device power limit avoidance reward is given with a negative penalty when the device's output power approaches its rated limit. In the hybrid reward function, the main task reward weight is set to 0.3, and the sum of the sub-target reward weights is set to 0.7.

[0053] Training continues, and every 100 training rounds, the average cumulative reward of all agents over the most recent 10 consecutive training rounds is calculated. When this average cumulative reward reaches the preset advancement threshold of 500 points, it indicates that the agent has mastered the basic control strategy for the current difficulty level, and automatically switches to the next difficulty level, the second difficulty level, for further training. When switching to the second difficulty level, all Actor network parameters converged in the first difficulty level are inherited and initialized as the initial network parameters for the current difficulty level, allowing the agent to learn more complex situations based on its existing experience. Similarly, in the second difficulty level, when the average cumulative reward over 10 consecutive training rounds reaches the advancement threshold of 600 points, the system switches to the third difficulty level, inheriting the network parameters converged in the second difficulty level. Finally, in the third difficulty level, when the average cumulative reward reaches 700 points and remains stable, all agents are considered to have completed training and obtained a cooperative control strategy suitable for complex situations.

[0054] Existing technologies for training multi-agent reinforcement learning models typically start from scratch under the most complex operating conditions. The agents must blindly explore in high-dimensional state and action spaces, often resulting in low exploration efficiency, slow convergence, or even failure to converge. This is especially problematic in multi-device collaborative scenarios like wind-solar-storage power stations, where simultaneously learning multiple objectives such as frequency support, power stabilization, and device limiting can easily lead to agents getting trapped in local optima, resulting in unstable training. Experiments using a Matlab / Simulink simulation model of a wind-solar-storage power station demonstrate that, compared to traditional methods that train directly under advanced complex operating conditions, this method reduces the number of training convergence rounds by 42%, shortens frequency recovery time by 16%, reduces power fluctuation amplitude by 18%, and improves multi-agent policy coordination by 35%. In contrast, this invention constructs a sequence of learning tasks with multiple difficulty levels, training the agents progressively from easy to difficult. This allows the agents to gradually accumulate basic control experience from simple operating conditions before transitioning to complex ones. A hybrid reward function is employed to guide exploration at each difficulty level, providing sparse main task rewards to clarify the final goal, while dense sub-goal rewards offer immediate feedback, accelerating the learning process. An inheritance-based initialization mechanism ensures the agent possesses a good initial strategy during high-difficulty training, preventing training crashes caused by environmental non-stationarity. Experiments show that using the hierarchical progressive training method of this invention, the number of training convergence rounds for the agent in the third difficulty condition is reduced by approximately 40% compared to methods that train directly from complex conditions. Furthermore, the final strategy exhibits improvements of over 15% in frequency support capability and power smoothing effect, significantly enhancing the stability of power plant operation.

[0055] In another technical solution, step S2, after repeated iterations until the Critic network converges, also includes a model generalization ability enhancement step, specifically: constructing a test scenario set containing various extreme operating conditions not involved in training, including sudden drops in new energy output, tie-line tripping, and islanded operation; conducting offline testing of each Actor network after training convergence in the test scenario set, recording the cumulative reward and frequency response index under each scenario; when the cumulative reward of the test scenario is lower than the preset generalization performance threshold of 850 points, identifying key state features that cause performance degradation through the random forest feature importance algorithm, retrieving experience samples similar to the key state features from historical operating data, and constructing a supplementary training set; using the supplementary training set to incrementally train the Actor network and Critic network, updating the network parameters until the performance in the test scenario reaches the generalization performance threshold.

[0056] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. All equipment is defined as independent intelligent agents and has undergone centralized training in multi-agent reinforcement learning. After repeated iterations until the global Critic network converges, a set of model generalization enhancement steps is implemented to ensure that the trained agent control strategy can operate stably under various extreme conditions.

[0057] First, a test scenario set containing various extreme operating conditions not included in the training dataset is constructed. This test scenario set covers three typical extreme operating conditions. The first condition is a sudden drop in renewable energy output, where wind power output drops from 80% to 20% of rated power and photovoltaic output drops from 90% to 10% of rated power within 10 seconds. The second condition is a tie-line trip, where the 220 kV tie-line connecting the power station to the main grid trips during grid-connected operation, causing the power station to switch to islanded operation mode. The third condition is islanded operation, where the power station, after disconnecting from the main grid, relies solely on its on-site wind, solar, and energy storage equipment to maintain the stable operation of the regional power grid, and the load fluctuation reaches 20% of the rated capacity. None of these three extreme operating conditions were included in the initial training dataset of the agent.

[0058] After training convergence, the Actor networks of each intelligent agent were subjected to offline testing in the aforementioned test scenarios. The tests employed a digital twin simulation platform to simulate the dynamic response process of the power plant under these three extreme operating conditions. In each test scenario, 10 test rounds were run, recording the cumulative reward and frequency response indicators for each round, including the lowest frequency point, the highest frequency point, and the time required for the frequency to recover to 49.5 Hz. Test results showed that in the scenario of a sudden drop in renewable energy output, the average cumulative reward was 780 points, lower than the preset generalization performance threshold of 850 points; the lowest frequency point dropped to 48.7 Hz, below the safety lower limit of 49.0 Hz. In the scenario of a tie-line trip, the average cumulative reward was 760 points, also below the 850-point threshold; the highest frequency point rose to 51.3 Hz, exceeding the safety upper limit of 50.5 Hz. In the islanded operation scenario, the average cumulative reward was 740 points, significantly lower than the threshold; the frequency recovery time was as long as 4.5 seconds, exceeding the preset 3-second recovery time requirement.

[0059] For scenarios where the cumulative reward falls below the generalization performance threshold, key state characteristics leading to performance degradation are further identified. In the scenario of a sudden drop in renewable energy output, the identified key state characteristics are a frequency change rate exceeding 0.5 Hz and total renewable energy output falling below 30% of rated capacity. In the tie-line tripping scenario, the key state characteristics are a grid connection point voltage amplitude falling below 0.7 per unit and a frequency deviation exceeding 0.3 Hz. In the islanded operation scenario, the key state characteristics are a state of charge of energy storage below 30% and a load power exceeding 50% of renewable energy output.

[0060] Based on the identified key state characteristics, experience samples with similar characteristics were retrieved from the historical operation data of the power plant. For the scenario of a sudden drop in renewable energy output, eight rapid fluctuation events in renewable energy output that occurred within the past year were retrieved from the historical database, and the corresponding voltage, frequency, and power time-series data were extracted to construct a supplementary training set. For the scenario of tie-line tripping, six tie-line fault simulation samples were selected from the offline data generated by the digital twin simulation platform, and a supplementary training set was also constructed. For the scenario of islanded operation, four islanded operation experience data were obtained from the operation records of other similar power plants as supplementary training material. These three supplementary training sets contain a total of 1800 experience samples.

[0061] The original Actor and Critic networks were incrementally trained using the supplementary training set. During training, samples from the supplementary training set were mixed with a portion of the samples from the original experience replay pool, with the sampling weight of the supplementary samples set to 0.6 and the sampling weight of the original samples set to 0.4. Fifty rounds of incremental training were performed with a learning rate of 0.0001, and validation was conducted again in the corresponding extreme operating condition test scenarios after each round. After 30 rounds of incremental training, the average cumulative reward in the scenario of a sudden drop in renewable energy output increased to 860 points, exceeding the 850-point threshold, and the lowest frequency recovered to 49.2 Hz. After 40 rounds of incremental training, the average cumulative reward in the scenario of a tie-line tripping reached 855 points, and the highest frequency decreased to 50.4 Hz. After 48 rounds of incremental training, the average cumulative reward in the scenario of islanded operation reached 845 points, and the frequency recovery time was shortened to 2.8 seconds. Incremental training terminated when the performance of all test scenarios reached the generalization performance threshold, resulting in agent network parameters with stronger generalization capabilities.

[0062] Existing technologies typically deploy trained models directly to real-world systems after multi-agent reinforcement learning training. However, historical or simulation data used during training often fails to cover all possible extreme operating conditions, such as rare but highly dangerous scenarios like sudden drops in renewable energy output, tie-line tripping, or islanded operation. When these untrained extreme conditions occur in actual operation, the agent's control strategies may fail, leading to frequency overruns, voltage instability, or even system collapse. In contrast, this invention proactively constructs a test scenario set containing various extreme conditions after training convergence, conducting offline testing on the agent to identify specific scenarios and key state characteristics where performance is insufficient. Based on these identification results, similar experience samples are retrieved from historical data to construct a supplementary training set for incremental training of the network, enabling the agent to learn targeted coping strategies under extreme conditions. This mechanism significantly improves the robustness and reliability of the agent in unforeseen scenarios. Experiments show that after incremental training, the cumulative reward of the agent under extreme conditions increases by more than 12% on average, and the frequency of events exceeding the limit is reduced by 70%. This effectively avoids control failures caused by operating conditions exceeding the training distribution, providing a strong guarantee for the safe and stable operation of wind, solar and energy storage power stations under extreme conditions.

[0063] In another technical solution, step S1, when constructing a local observation space for each intelligent agent including node voltage, frequency, active power, reactive power, and energy storage state of charge, also includes a heterogeneous device task-oriented feature fusion and feature alignment step. Specifically, for wind turbine generators, the extracted local observation information also includes pitch angle, rotational speed, and wind speed; for photovoltaic inverters, the extracted local observation information also includes illuminance, temperature, and maximum power point tracking status; for energy storage converters, the extracted local observation information also includes state of charge, charge / discharge status, and cycle count. The information extracted from the local observation space of the heterogeneous devices is input into the learnable embedding layer jointly trained with the Actor network. The embedding layer nonlinearly maps the original features with different physical meanings to the same implicit feature space. This mapping process is guided by maximizing the cumulative reward of subsequent collaborative control tasks. It automatically learns to extract the implicit feature representations that have common contributions to frequency / voltage support in various devices and maps them to the same feature dimension space through the embedding layer to form a unified dimension observation feature vector. By using the implicit feature vectors of the same dimension obtained after mapping as the input of each agent's Actor network, heterogeneous devices can achieve collaborative learning and decision-making in a task-driven implicit semantic space. By using the observed feature vectors as the input of each agent's Actor network, heterogeneous devices can learn collaboratively in the same feature space.

[0064] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent intelligent agent, and its local controller deploys a trained and converged Actor network. During power station operation, each agent needs to make control decisions based on local observation information. However, since the wind turbine generators, photovoltaic inverters, and energy storage converters are three types of heterogeneous devices, their observable local information differs significantly in type and dimension, making it difficult to directly perform collaborative learning and control within the same feature space.

[0065] To address this issue, a heterogeneous device task-oriented feature fusion process is introduced when constructing the local observation space for each agent in step S1. Taking a wind turbine generator in operation as an example, its local controller, in addition to collecting four common observations—node voltage, grid frequency, active power, and reactive power—also extracts three specific observations tailored to the characteristics of the wind turbine generator: pitch angle, rotational speed, and wind speed. At the current moment, the generator's recorded pitch angle is 12 degrees, rotational speed is 1500 revolutions per minute, and wind speed is 8.5 meters per second. Similarly, taking a photovoltaic inverter as an example, its local controller, while collecting the four common observations—node voltage, grid frequency, active power, and reactive power—also extracts three specific observations tailored to the characteristics of the photovoltaic device: illuminance, temperature, and maximum power point tracking status. At the current moment, the inverter's recorded illuminance is 820 watts per square meter, temperature is 35 degrees Celsius, and maximum power point tracking status is normal operation. Taking an energy storage converter as an example, its local controller, while collecting four common observations—node voltage, grid frequency, active power, and reactive power—extracts three additional specific observations tailored to the characteristics of the energy storage device: state of charge (SBC), state of charge / discharge (SDC), and number of cycles. At the current moment, the energy storage converter's SBC is 65%, its SDC is in the discharging state, and its cycle count is 1200. The dimensions of the raw observation information collected by these three types of devices differ. Wind turbine generators have seven raw features, photovoltaic inverters have seven raw features, but the content differs from wind turbine generators, and energy storage converters also have seven raw features, but the content differs from the former two. To achieve collaborative learning among heterogeneous devices in a unified feature space, a learnable embedding layer jointly trained with the Actor network is deployed in the local controller of each agent. For three types of equipment—wind turbine generators, photovoltaic inverters, and energy storage converters—three embedded networks with identical structures but independent parameters are constructed. Each embedded network contains two hidden layers: the first layer has 64 neurons, and the second layer has 32 neurons, ultimately outputting a 128-dimensional implicit feature vector.

[0066] Taking a wind turbine generator as an example, its 7-dimensional original feature vector is input into an embedding network, and after nonlinear mapping, it outputs a 128-dimensional implicit feature vector. Similarly, the original 7-dimensional features of photovoltaic inverters and energy storage converters are also mapped to the same 128-dimensional feature space through their respective parameter-independent but structurally identical embedding networks. This mapping process is guided by maximizing the cumulative reward of subsequent collaborative control tasks, and automatically learns to extract implicit feature representations that have common contributions to frequency / voltage support from various types of devices through joint training. In other words, although the embedding layer parameters of the three types of devices do not share, under the constraint of the same global reward function, each embedding layer is forced to learn to extract common features valuable to the collaborative control task, so that the output 128-dimensional feature vector is naturally aligned in the task-driven implicit semantic space.

[0067] The 128-dimensional implicit feature vector, processed by the embedding layer, is used as the input to the Actor network of each agent. The Actor network performs forward computation based on this unified-dimensional implicit feature, outputting virtual inertia coefficient adjustment and damping coefficient adjustment. Since the input features of all agents are mapped to the same implicit semantic space under task-driven conditions, the Actor network can learn cooperative control strategies based on the same feature representation. This enables three heterogeneous devices—wind turbine generators, photovoltaic inverters, and energy storage converters—to truly learn collaboratively in the same feature space, jointly optimizing their active support capabilities for the power grid.

[0068] Existing technologies for handling multi-energy power plants containing wind, solar, and energy storage typically employ decentralized independent control or simplified aggregation models. When using multi-agent reinforcement learning methods, the raw, heterogeneous observation information collected by each device is often directly used as input. This leads to inconsistencies in the observation space dimensions and feature meanings among wind turbines, photovoltaic inverters, and energy storage converters, making it difficult for Actor networks to perform effective collaborative learning. While some existing technologies attempt to model heterogeneous devices uniformly, they often rely on simple feature stitching or manual feature engineering, failing to fully explore the inherent correlations between the observation information of different devices. Based on simulation tests of collaborative control of heterogeneous wind, solar, and energy storage devices, compared with traditional feature stitching methods, this invention constructs a local observation space containing device-specific features for each agent and uses a learnable embedding layer jointly trained with the Actor network. This maps the raw observation information of heterogeneous devices to the same implicit semantic space under task-driven conditions, effectively solving the problems of inconsistent observation space dimensions and difficulty in collaborative learning among heterogeneous devices. This feature fusion mechanism enables multi-agent reinforcement learning for different types of devices under task-driven implicit semantic representations. The Actor network can learn better cooperative control strategies based on implicit feature vectors of a unified dimension. Research shows that through task-oriented feature fusion, the cooperative control effect among heterogeneous devices is significantly improved, and the overall frequency support capability and power fluctuation smoothing effect of the power station are significantly improved. This avoids the decline in learning efficiency or policy deviation caused by differences in feature dimensions, laying a solid foundation for the intelligent cooperative control of wind-solar-storage multi-energy complementary power stations.

[0069] In another technical solution, within the centralized training framework of step S2, after the agent modifies the local virtual inertia and damping parameters according to the adjustment amount, and before the multi-machine circulating current suppression step, an offline learning and solidification collaborative step for the fault ride-through strategy is also included, specifically: During the intensive training phase, a set of fault ride-through training scenarios containing various types of power grid faults is constructed, including symmetrical faults, asymmetrical faults, and continuous faults. In the fault ride training scenario set, through the multi-agent reinforcement learning process in step S2, each agent learns the collaborative response strategy under different fault depths and different fault locations. The collaborative response strategy includes the virtual inertia adjustment, damping adjustment and reactive / active current command values ​​that each agent should output. After training convergence, the learned collaborative response strategy is embedded in the Actor network parameters of each agent. In actual operation, when the local controller in step S3 detects that the grid voltage drop depth exceeds the preset fault ride-through threshold, each agent directly calculates forward through the Actor network based on the information in the current local observation space, and instantaneously outputs the offline optimized virtual inertia parameters, damping parameters and current command values ​​to achieve millisecond-level fault ride-through response. After the fault is cleared, each agent adaptively restores the control parameters to the pre-fault level through the Actor network based on the locally observed system frequency and voltage recovery status.

[0070] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent intelligent agent, and its local controller deploys a trained and converged Actor network. During power station operation, each intelligent agent needs to address the collaborative control problem under grid fault conditions to ensure that the power station has fault ride-through capability and can provide effective support to the grid.

[0071] To address this issue, an offline learning and solidification process for fault ride-through strategies is introduced into the centralized training framework in step S2. Specifically, during the centralized training phase, a fault ride-through training scenario set is first constructed, encompassing various grid fault types, including symmetrical faults (such as three-phase short circuits), asymmetrical faults (such as single-phase grounding and two-phase short circuits), and continuous faults (such as multiple fault impacts during reclosing). This training scenario set covers different fault depths (voltage drops to 0.2 pu, 0.5 pu, 0.8 pu, etc.), fault locations (in-station, near-station area, far-station area), and fault durations.

[0072] Under the fault-crossing training scenario set, through the multi-agent reinforcement learning process described in step S2, each agent autonomously learns a cooperative response strategy under different fault scenarios. During the learning process, each agent continuously optimizes its own strategy by interacting with the environment, ultimately learning a cooperative response behavior that maximizes the global reward function (including indicators such as voltage support effect during faults, equipment overcurrent suppression, and recovery speed after faults). The learned cooperative response strategy includes the virtual inertia adjustment, damping adjustment, and reactive / active current command values ​​that each agent should output.

[0073] Taking a three-phase short-circuit fault scenario in a training exercise as an example, a fault condition was set on the digital twin simulation platform where the voltage amplitude at the grid connection point suddenly dropped to 0.2 times the per-unit value for 150 milliseconds. In the initial stage, the responses of the various agents may be uncoordinated, leading to overcurrent or insufficient voltage support in some devices. After multiple rounds of iterative learning, the agents gradually mastered the following: when the voltage drops deeply, all devices should prioritize outputting reactive current to support the voltage, while rationally allocating active current according to their respective capacity limitations; after the fault is cleared, the control parameters should be smoothly restored to normal based on the voltage and frequency recovery status.

[0074] After training convergence, the learned cooperative response strategy is embedded into the Actor network parameters of each agent. At this point, the input to the Actor network is local observation information (including voltage, frequency, current, etc.), and the output is the offline optimized virtual inertia parameters, damping parameters, and current command values. This embedding process transforms the complex problem of fault scenario identification and policy matching into a direct input-to-output mapping, eliminating the need for online fault type identification and policy querying.

[0075] In actual operation, the local controller of each agent monitors the grid voltage in real time with a microsecond sampling period. When a voltage drop depth is detected to exceed the preset fault ride-through threshold (e.g., the voltage drops to below 0.5 times the per-unit value and lasts for more than 10 milliseconds), each agent directly performs forward calculations through the local Actor network based on the current local observation information, and instantly outputs the offline optimized virtual inertia parameters, damping parameters, and reactive / active current command values.

[0076] Taking a photovoltaic inverter as an example, when a voltage drop to 0.2 times the per-unit value is detected, its Actor network, based on local observation information such as current voltage, current, and active power, calculates forward to adjust the virtual inertia coefficient to 0.3 times the normal value, the damping coefficient to 0.5 times the normal value, the reactive current command value to 0.9 times the rated current, and the active current command value to be limited to within 0.2 times the rated current. Similarly, for an energy storage converter, its Actor network instantly outputs a reactive current command increased to 0.9 times and an active current limited to within 0.2 times, and automatically adjusts the discharge power within the allowable range based on the current state of charge (65%). For a wind turbine generator, in addition to adjusting reactive and active current, the commands output by its Actor network also appropriately reduce mechanical power input through pitch angle control, cooperating with the electronic control system to achieve power balance. The entire calculation process is completed in microseconds, achieving millisecond-level fault ride-through response, with a response speed comparable to traditional hardware-level solutions.

[0077] During the fault, each agent maintains the coordinated response parameters output by the Actor network, continuously injecting reactive current into the grid to support the voltage. After 150 milliseconds, the grid protection device trips and disconnects the faulty line, and the voltage begins to recover. At this point, each agent no longer uses a fixed recovery strategy, but adaptively restores its control parameters to pre-fault levels based on the real-time perception of system frequency and voltage recovery status by its local Actor network. The Actor network, having learned the optimal recovery trajectory during training, can strike a balance between rapid recovery and avoiding secondary impacts.

[0078] The entire fault-crossing collaborative process does not require unified coordination from a centralized monitoring platform. Each agent makes independent decisions based on local observation information. However, since the Actor networks of all agents are jointly trained under the same multi-agent reinforcement learning framework, their output control commands naturally contain collaborative information with other devices, thus realizing distributed collaborative response.

[0079] Existing technologies typically employ decentralized, independent control strategies when addressing fault ride-through issues, with each device responding independently based on local detection information. For example, photovoltaic inverters and wind turbine generators each have their own fault ride-through control logic, while energy storage systems may operate according to pre-set charging and discharging strategies, lacking coordination among them. When a grid fault occurs, this decentralized control can lead to some devices over-responding and exceeding capacity limits, while others under-respond and fail to provide effective support, or even control conflicts between devices. In contrast, this invention moves the complex fault identification and strategy matching process from the online stage at the time of fault occurrence to the offline training stage. This allows the agent to respond instantaneously based on the trained strategy when a fault occurs, retaining the advantages of multi-agent collaborative optimization while meeting the stringent response speed requirements of fault ride-through. Furthermore, since the Actor network of each agent has learned a smooth post-fault recovery strategy during the training phase, secondary shocks caused by abrupt changes in control parameters are avoided, significantly improving the fault ride-through capability of new energy power plants under grid faults and their proactive support capability for the grid.

[0080] In another technical solution, step S2, during the process of updating the network parameters of each Actor, also includes an interpretable decision-making assistance step, specifically: recording the observation information, action output, and corresponding Critic network Q value of each agent before and after each update; using a decision tree or attention mechanism with a depth of 5 and a minimum number of sample splits of 20 to perform offline analysis on the recorded sample data, extracting the contribution weight of each observation feature to the action output; generating a decision rule base for each agent based on the contribution weight, the decision rule base containing the input-output mapping relationship under typical working conditions; The decision rule base is deployed to the management and control platform so that operators can understand the basis of the agent's decision-making and provide reference for manual intervention in abnormal operating conditions.

[0081] In the aforementioned technical solution, the wind-solar-storage combined power station in actual operation includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. Each device is defined as an independent intelligent agent, and the Actor network of each agent is trained using a centralized training framework. During the updating of the parameters of each Actor network, to enhance the interpretability of the deep reinforcement learning model and enable operators to understand the decision-making basis of the agents and intervene manually when necessary, a set of interpretable decision support steps were implemented. During the centralized training phase, each time the Actor network parameters are updated iteratively, the system automatically records the observation information, action output, and corresponding Critic network Q value of each agent before and after each update. Taking a specific photovoltaic inverter agent as an example, the observation information recorded in a certain iteration includes a node voltage of 1.02 times the per-unit value, a grid frequency of 50.03 Hz, an active power of 0.45 MW, a reactive power of 0.03 Mvar, a solar irradiance of 780 W / m², and a temperature of 32 degrees Celsius. The agent's output actions include a virtual inertia coefficient adjustment of 0.04 and a damping coefficient adjustment of 0.01. Simultaneously, the Critic network's evaluation value (Q-value) for this joint action is recorded as 85.6. This recording covers complete decision data for all agents in each iteration, accumulating approximately 100,000 sample data points after 1000 iterations. These recorded sample data are then analyzed offline. A decision tree algorithm is used to train each agent's sample data separately. Taking the photovoltaic inverter agent as an example, various features from the observation information are used as input, and the virtual inertia coefficient adjustment in the action output is used as the prediction target to train a decision tree with a depth of 5. After training, the contribution weights of each observation feature to the action output are extracted by analyzing the splitting characteristics and information gain of each node in the decision tree. The analysis results show that the three observation features with the greatest impact on the virtual inertia adjustment of the photovoltaic inverter agent are, in order: grid frequency change rate (0.35), active power magnitude (0.28), and node voltage (0.18). For the damping coefficient adjustment, the features contributing the most are reactive power (0.32 weight), grid frequency deviation (0.25 weight), and energy storage state of charge (0.20 weight). Based on these analysis results, a decision rule base is generated for each agent. Taking this photovoltaic inverter as an example, the generated rule base contains input-output mapping relationships under typical operating conditions. For example, rule one stipulates that when the grid frequency is in the range of 50.00 to 50.05 Hz and the active power is less than 0.5 MW, the virtual inertia coefficient adjustment should be maintained between 0.03 and 0.05. Rule two stipulates that when the reactive power is greater than 0.05 Mvar and the node voltage is less than 0.98 times the per-unit value, the damping coefficient adjustment should be increased to above 0.02.Rule 3 stipulates that when the light intensity drops sharply from 800 watts per square meter to 500 watts with a rate of change exceeding 100 watts per second, the virtual inertia coefficient adjustment should be temporarily increased to 0.08 to cope with power fluctuations. These rules are stored in if-then format, forming a readable rule base containing over 200 rules. The generated decision rule base is deployed to the power plant's centralized management and control platform. The management and control platform displays the decision rules of each agent through a graphical interface, allowing operators to view the decision basis of any agent by clicking on it. For example, when a wind turbine outputs a large virtual inertia adjustment during frequency fluctuations, operators can query the rule base to find that this is because the current wind speed is 9 meters per second and the blade pitch angle is 8 degrees, triggering a response rule for a specific wind speed range, thus understanding the agent's decision logic.

[0082] In actual operation, when the monitoring system detects an anomaly in the decision-making of an agent—for example, an output action causing equipment power fluctuations to exceed the normal range—operators can immediately consult the rule base and compare the current operating condition with typical operating condition characteristics recorded in the rule base to identify whether the current decision deviates from the expected rules. If the decision is confirmed to be abnormal, operators can manually adjust the control parameters of the agent or switch to a standby control mode through the manual intervention interface of the management platform until the agent resumes normal decision-making. This manual intervention process is also recorded and used as a reference sample for subsequent training.

[0083] Existing technologies using deep reinforcement learning for power system control typically treat deep neural networks as black-box models. Operators can only see the input observations and output control commands, but cannot understand why the agent makes such decisions. This black-box characteristic leads to two main problems in actual power system operation: first, operators find it difficult to trust an inexplicable decision-making system, especially in critical control links involving grid safety; second, when the agent makes abnormal decisions, operators lack effective diagnostic tools, making it difficult to quickly locate the root cause of the problem and intervene. Actual power plant operation and maintenance tests show that after deploying a decision rule base, operators' understanding of the agent's decisions improved by 45%, the fault location time for abnormal operating conditions was reduced from an average of 120 seconds to 45 seconds, the success rate of manual intervention increased from 65% to 98%, and the efficiency of human-machine collaboration in the control system improved by 50%. In contrast, this invention records observation information, action outputs, and Q-values ​​during the training process, and uses interpretable algorithms such as decision trees to analyze the contribution weight of each observation feature to the action output offline, generating a readable decision rule base and deploying it to the management and control platform. This mechanism transforms the black-box model into a gray-box or even white-box model with some interpretability, enabling operators to intuitively understand the agent's decision-making basis under different operating conditions. When abnormal decisions occur, operators can quickly query the rule base, compare the current operating condition with typical rules, identify the problem, and intervene manually. Experiments show that after deploying interpretable decision assistance, operators' trust in the agent's control strategy increased by more than 40%, the average fault location time under abnormal operating conditions was reduced by 60%, and the transparency of the control system and human-machine collaboration capabilities were significantly improved.

[0084] In another technical solution, step S2, after randomly sampling batch data and before updating the Critic network by minimizing the temporal difference error, includes a priority experience replay step. Specifically, this involves: calculating the absolute value of the temporal difference error for each experience sample as an importance index for that sample; sorting the samples in the experience replay pool in descending order according to the importance index, and marking the top 10% of samples as high-priority samples; during each sampling, extracting high-priority samples with a probability four times higher than that of ordinary samples, and using importance sampling weights (0.1 for high-priority samples and 1.0 for ordinary samples) to perform weighted correction on the gradient update of the samples to correct sampling bias.

[0085] In the aforementioned technical solution, the actual operating wind-solar-storage combined power station includes 20 wind turbine generators, 15 photovoltaic inverters, and 5 energy storage converters. All devices are defined as independent agents, and a centralized training framework is used to train the Actor network for each agent. During training, to improve learning efficiency and convergence speed, a priority experience replay step is implemented after randomly sampling batch data and before updating the Critic network by minimizing the temporal difference error. First, an experience replay pool with a capacity of 100,000 experience samples is constructed. In each training iteration, experience samples generated by the agent's interaction with the environment are stored in the replay pool. Each sample includes the current global state, the actions performed by each agent, the immediate reward obtained, and the state at the next moment. The experience replay pool uses a circular queue for storage; when the number of samples reaches the upper limit, new samples overwrite the oldest samples. Before sampling batch data from the experience replay pool for training each time, the system first traverses all samples in the replay pool and calculates the absolute value of the temporal difference error for each experience sample. Temporal difference error represents the difference between the Critic network's evaluation of the current state-action pair and the actual reward plus the estimated value of the next state. For example, in one calculation, the absolute value of the temporal difference error is 0.85, while in another sample it is only 0.12. Based on this metric, the system sorts all samples in descending order of their absolute temporal difference error. A larger error indicates higher learning value for the current network, meaning it is more worthy of priority learning. Based on the sorting results, the top 10% of samples are marked as high-priority samples. For example, the top 10,000 samples in the replay pool with the highest absolute temporal difference error are assigned high-priority labels. In subsequent sampling, a weighted sampling strategy is used: each sample is assigned a sampling weight; high-priority samples have a weight of 4, and ordinary samples have a weight of 1. During sampling, weighted random sampling is performed based on the weights of each sample, making the probability of a high-priority sample being selected four times that of an ordinary sample. Simultaneously, importance sampling weights (0.1 for high-priority samples and 1.0 for ordinary samples) are used to correct gradient updates and eliminate sampling bias. After obtaining a batch of samples, gradient updates need to be corrected due to sampling probability bias. Therefore, the system calculates an importance sampling weight for each sampled. This weight is calculated based on the ratio of the sample's original sampling probability to its current sampling probability and is used to weight the gradient contribution of the sample when updating network parameters. For example, for samples drawn from the high-priority sample set, the importance sampling weight is set to 0.1 to correct the bias caused by oversampling; for samples drawn from ordinary samples, the importance sampling weight remains at 1.0.These weights are applied to the gradient updates of the Critic and Actor networks to ensure unbiased parameter updates. Taking a specific training iteration as an example, a batch of 128 samples is sampled from the experience replay pool, including 52 samples from the high-priority sample set and 76 samples from the ordinary sample set. When calculating the temporal difference error for each sample, the error value of the high-priority samples is multiplied by an importance sampling weight of 0.1, while the error value of the ordinary samples remains unchanged. Then, the weighted mean temporal difference error is calculated, and the parameters of the Critic network are updated by minimizing this mean. Simultaneously, when updating the Actor network, these weighted samples are also used to calculate the gradient. After one round of training iterations with priority experience replay, the key samples with larger temporal difference errors are learned more fully. After approximately 2000 training iterations, statistics show that after adopting the priority experience replay step, the frequency of learning high-value key samples increased by about 35%, and the number of iterations required for training convergence was reduced by about 25% compared to ordinary random sampling. Meanwhile, due to the introduction of importance sampling weights for bias correction, the training process maintained good stability and did not exhibit oscillations or divergences caused by sampling bias.

[0086] Existing technologies using experience replay mechanisms typically employ uniform random sampling to extract samples from the replay pool for training. This approach treats all experience samples equally, ignoring the significant differences in learning value among different samples. In multi-agent collaborative control training for wind-solar-storage power stations, a large number of samples come from stable operating states, with small temporal difference errors and limited contribution to network parameter updates. The truly valuable samples are often those from critical events such as fault conditions and sudden power fluctuations, which have larger temporal difference errors and contain richer learning information. Uniform random sampling results in a low probability of selecting these critical samples, requiring numerous iterations for effective learning, leading to low training efficiency. Experiments based on a wind-solar-storage multi-agent reinforcement learning training platform show that, compared to the traditional uniform random sampling experience replay method, the prioritized experience replay step of this invention increases the learning frequency of critical samples by 35%, reduces the number of training convergence rounds by 25%, improves the performance of the control strategy under fault conditions by 15%, and reduces the policy oscillation amplitude during training by 70%. In contrast, this invention uses the absolute value of the temporal difference error of each experience sample as an importance index to weight high-priority samples, enabling the agent to learn unexpected or critical experiences more frequently. This mechanism accelerates the dissemination of valuable experiences, allowing the network to absorb knowledge from key samples more quickly. Simultaneously, the use of importance sampling weights to weighted correct gradient updates effectively corrects distribution shifts caused by sampling biases, ensuring the stability of the training process. Experiments show that after adopting priority experience replay, within the same training rounds, the agent's performance on key tasks such as fault ride-through and power fluctuation suppression improves by approximately 15%, and the training convergence speed is accelerated by approximately 30%, significantly improving the training efficiency and final performance of multi-agent reinforcement learning in the collaborative control of wind, solar, and energy storage power stations.

[0087] Although the technical solutions of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. A method for active support and adaptive inertia control incorporating wind-solar-storage synergy, characterized in that, Includes the following steps: S1. Define each wind turbine generator, photovoltaic inverter and energy storage converter in the power station as an independent intelligent agent; construct a local observation space for each intelligent agent including node voltage, frequency, active power, reactive power and energy storage charge state; and a continuous action space including virtual inertia coefficient adjustment and damping coefficient adjustment. And a global reward function, which calculates and sums the instantaneous rewards for each agent based on the system's lowest frequency point, frequency recovery time, and power limit exceedance. S2. Construct a centralized training framework that includes a global Critic network and local Actor networks for each agent. Collect global state, actions of each agent, rewards, and the state at the next moment from historical operating data or digital twin simulation as experience samples and store them in the experience replay pool. Randomly sample a batch of empirical samples, update the Critic network by minimizing the temporal difference error, and update the parameters of each Actor network by maximizing the joint action value of the Critic network output. Repeat the iteration until the Critic network converges. S3. Deploy the converged Actor networks to the corresponding local controllers. During operation, each controller collects information from the local observation space in step S1 and inputs it into the Actor network. It calculates the adjustment amount of the virtual inertia coefficient and damping coefficient. Each agent modifies the local virtual inertia and damping parameters according to the adjustment amount. S4. Extract the power grid operation feature vectors under different typical operating conditions; associate the Actor network parameters after each training convergence with the feature vector of the operating condition and store them in the knowledge base; when it is detected that the similarity between the current operating condition feature vector and a certain operating condition feature vector stored in the knowledge base exceeds a preset threshold, read the corresponding Actor network parameters from the knowledge base as initial values, and update the network parameters online using real-time operating data under the current operating condition.

2. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S3, before each agent modifies its local virtual inertia and damping parameters according to the adjustment amount, a safety verification and correction step is also included, specifically: Real-time acquisition of physical limiting parameters of the wind turbine generator, photovoltaic inverter or energy storage converter corresponding to the local intelligent agent. The physical limiting parameters include the maximum active power allowed by the converter and the range of charging and discharging power allowed by the energy storage state of charge. The expected modified virtual inertia parameters and damping parameters are calculated based on the virtual inertia coefficient adjustment and damping coefficient adjustment, and the resulting change in active power is estimated based on local observation information. If the estimated change in active power causes the active power of the wind turbine generator, photovoltaic inverter, or energy storage converter to exceed the range limited by the physical limit parameters, the adjustment of the virtual inertia coefficient and the damping coefficient will be reduced so that the adjusted virtual inertia parameters and damping parameters meet the physical limit of the wind turbine generator, photovoltaic inverter, or energy storage converter. Otherwise, directly modify the local virtual inertia and damping parameters based on the virtual inertia coefficient adjustment and damping coefficient adjustment.

3. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S3, before each controller collects local observation information and inputs it into the Actor network, a local state prediction and compensation step is also included, specifically: Real-time detection of communication latency in receiving observation information from the local controller; when the communication latency exceeds the preset communication latency threshold, acquisition of the previous moment's local observation information and historical observation information sequence; The local observation information from the previous moment and the sequence of historical observation information are input into a pre-trained local state prediction model. The local state prediction model is constructed using a long short-term memory network and is used to predict the local observation information at the current moment based on historical time series data. The predicted local observation information at the current moment replaces the real-time collected local observation information and is input into the Actor network for forward computation; When the communication latency recovers to within the communication latency threshold, the use of real-time acquired local observation information is resumed.

4. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S3, after each agent modifies its local virtual inertia and damping parameters according to the adjustment amount, a multi-machine circulating current suppression step is also included, specifically: Real-time acquisition of grid connection point voltage and output current of the wind turbine generator, photovoltaic inverter or energy storage converter corresponding to this intelligent agent, and calculation of instantaneous reactive power and instantaneous active power of this intelligent agent; Based on instantaneous reactive power and instantaneous active power, extract the power oscillation component between this agent and neighboring agents. The power oscillation component includes oscillation amplitude and oscillation frequency. When the oscillation amplitude is detected to exceed the preset circulating current suppression threshold, the virtual impedance parameters of this intelligent agent are dynamically adjusted based on the oscillation frequency. The virtual impedance parameters include virtual resistance and virtual reactance. The adjusted virtual impedance parameters are superimposed on the voltage control loop of the local converter to suppress circulating currents generated by multiple machines in parallel. Simultaneously, the circulation suppression state is fed back as a reward function into the centralized training framework of step S2 to update the training strategy of the subsequent Actor network.

5. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S2, after constructing the centralized training framework containing the global Critic network and the local Actor networks of each agent, a hierarchical progressive training step is also included, specifically: A sequence of learning tasks with multiple difficulty levels is pre-constructed. Each difficulty level corresponds to a different power grid operation complexity. The operation complexity is determined comprehensively based on the penetration rate of new energy sources, the magnitude of load fluctuations, and the type and number of fault disturbances. Training proceeds step by step from low to high difficulty according to the course learning task sequence. In the current difficulty level task, when the average cumulative reward of each agent in multiple consecutive training rounds reaches the preset advancement threshold, the training is switched to the next difficulty level task to continue. During the training process at each difficulty level, a hybrid reward function is used to guide the agent's exploration. The hybrid reward function is obtained by weighted summation of sparse main task rewards and dense sub-objective rewards. Sub-objective rewards include frequency deviation suppression rewards, power fluctuation smoothing rewards, and device power not exceeding limits rewards. When training is switched to a higher difficulty level, the Actor network parameters that have converged after training at a lower difficulty level are used as the initial network parameters for the current difficulty level for inheritance initialization.

6. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S2, after repeated iterations until the Critic network converges, a step to enhance the model's generalization ability is also included, specifically: Construct a test scenario set containing various extreme operating conditions that were not included in the training, including a sudden drop in new energy output, a tripping of the tie line, and islanded operation; After training convergence, each agent network is tested offline in the test scenario set, and the cumulative reward and frequency response index are recorded in each scenario. When the cumulative reward of the test scenario is lower than the preset generalization performance threshold, the key state features that cause the performance to decline are identified, and experience samples similar to the key state features are retrieved from the historical running data to construct a supplementary training set. Incremental training of the Actor and Critic networks was performed using a supplementary training set, and the network parameters were updated until the performance in the test scenario reached the generalization performance threshold.

7. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S1, when constructing a local observation space for each agent, including node voltage, frequency, active power, reactive power, and energy storage state of charge, a heterogeneous device task-oriented feature fusion and alignment step is also included, specifically: For wind turbine generators, the information extracted from the local observation space also includes pitch angle, rotational speed, and wind speed. For photovoltaic inverters, the extracted information from the local observation space also includes illuminance, temperature, and maximum power point tracking status. For energy storage converters, the information extracted from the local observation space also includes the state of charge, state of charge and discharge, and number of cycles. The information extracted from the local observation space of the heterogeneous devices is input into the learnable embedding layer jointly trained with the Actor network. The embedding layer nonlinearly maps the original features with different physical meanings to the same implicit feature space. This mapping process is guided by maximizing the cumulative reward of subsequent collaborative control tasks. It automatically learns to extract the implicit feature representations that have common contributions to frequency / voltage support in various devices and maps them to the same feature dimension space through the embedding layer to form a unified dimension observation feature vector. By using the implicit feature vectors of the same dimension obtained after mapping as the input of each agent's Actor network, heterogeneous devices can achieve collaborative learning and decision-making in a task-driven implicit semantic space. By using the observed feature vectors as the input of each agent's Actor network, heterogeneous devices can learn collaboratively in the same feature space.

8. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 4, characterized in that, In the centralized training framework of step S2, after the agent modifies the local virtual inertia and damping parameters according to the adjustment amount, before the multi-machine circulating current suppression step, there is also an offline learning and solidification collaborative step of the fault ride-through strategy, specifically: During the intensive training phase, a set of fault ride-through training scenarios containing various types of power grid faults is constructed, including symmetrical faults, asymmetrical faults, and continuous faults. In the fault ride training scenario set, through the multi-agent reinforcement learning process in step S2, each agent learns the collaborative response strategy under different fault depths and different fault locations. The collaborative response strategy includes the virtual inertia adjustment, damping adjustment and reactive / active current command values ​​that each agent should output. After training converges, the learned cooperative response strategy is embedded in the Actor network parameters of each agent. In actual operation, when the local controller in step S3 detects in real time that the voltage drop depth of the power grid exceeds the preset fault ride-through threshold, each agent directly calculates forward through the Actor network based on the information in the current local observation space, and instantly outputs the offline optimized virtual inertia parameters, damping parameters and current command values ​​to achieve millisecond-level fault ride-through response. After the fault is cleared, each agent adaptively restores the control parameters to the pre-fault level through the Actor network based on the locally observed system frequency and voltage recovery state.

9. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S2, during the process of updating the network parameters of each Actor, an interpretability decision support step is also included, specifically: Record the observation information, action output, and corresponding Critic network Q-value of each agent before and after each update; Decision trees or attention mechanisms are used to perform offline analysis on the recorded sample data to extract the contribution weight of each observed feature to the action output; A decision rule base is generated for each agent based on its contribution weight. The decision rule base contains the input-output mapping relationship under typical working conditions. The decision rule base is deployed to the management and control platform so that operators can understand the basis of the agent's decision-making and provide reference for manual intervention in abnormal operating conditions.

10. The active support and inertia adaptive control method with wind-solar-storage synergy as described in claim 1, characterized in that, In step S2, after randomly sampling the batch data and before updating the Critic network by minimizing the temporal difference error, a priority empirical replay step is also included, specifically: The absolute value of the temporal difference error for each experience sample in the experience replay pool is calculated as an importance indicator for that sample. The samples in the experience replay pool are sorted in descending order according to the importance index, and the samples with the highest priority are marked as high-priority samples. During each sampling, the sampling weight of high-priority samples is set to 4 times that of ordinary samples, and the gradient update of samples is weighted and corrected using importance sampling weights to correct sampling bias; the importance sampling weight of high-priority samples is 0.1, and that of ordinary samples is 1.0.

Images