A source-network-load-storage coordination optimization method considering multiple heterogeneous flexible resources

Abstract

The application discloses a source network load storage coordination optimization method considering multiple heterogeneous flexible resources, comprising the following steps: generating a source load operation scene set based on wind power, photovoltaic and load prediction error, and determining the adjustable capacity boundary of various flexible resources at different time scales of day-ahead, day-in and real-time according to the scene set; then, a multi-time scale coordination optimization model is constructed to realize the collaborative scheduling of multiple types of resources such as micro gas turbines, hybrid energy storage and adjustable loads; further, a hybrid energy storage dynamic state constraint mechanism is introduced in the real-time stage to dynamically adjust the energy storage adjustment margin to maintain continuous response capability; finally, the model is jointly solved based on a multi-agent deep reinforcement learning method, and a cross-time scale coordination control strategy is generated through time scale action mapping, so that efficient, stable and economic operation of the source network load storage system is realized.

Description

Technical Field

[0001] This invention belongs to the field of power system flexible resource scheduling technology, specifically involving a source-grid-load-storage coordination optimization method that considers multiple heterogeneous flexible resources. Background Technology

[0002] As the proportion of new energy sources such as wind power and solar power continues to increase, the operation mode of the power system is shifting from the traditional "controllable power source tracking load" model to a new model characterized by "high fluctuations on both the source and load sides, and a significant increase in the difficulty of dynamic system balancing." Compared with traditional power sources, wind power and solar power output have stronger randomness, volatility, and intermittency. Coupled with factors such as the diversification of load-side energy consumption behavior, the growth of electric vehicles and flexible loads, the net load fluctuation of the system is more pronounced, leading to increased demand for reserve capacity, increased difficulty in real-time dispatch, and greater pressure on system regulation.

[0003] Under these conditions, relying solely on traditional peak-shaving and frequency regulation methods from conventional generating units is no longer sufficient to simultaneously meet the requirements of renewable energy consumption, grid security, and operational economy. Flexible resources such as energy storage, adjustable loads, and distributed adjustable power sources, due to their rapid response and flexible adjustment capabilities, are gradually becoming important means to enhance the system's flexible regulation capabilities. However, different types of flexible resources differ significantly in response speed, regulation capacity, operational constraints, and cost characteristics. Without a unified coordination mechanism, problems such as unreasonable resource allocation, insufficient utilization of regulation capacity, and decreased operational efficiency can easily arise.

[0004] Meanwhile, the scheduling optimization of high-proportion renewable energy systems has gradually evolved from a static optimization problem under a single time scale to a dynamic collaborative optimization problem involving multiple time scales, including day-ahead, intraday, and real-time. How to achieve unified modeling, collaborative adjustment, and dynamic optimization of various flexible resources under uncertain source-load conditions has become a key technical issue for improving renewable energy absorption capacity and system operational flexibility.

[0005] In the prior art, CN120341859A discloses a flexible resource adaptive configuration method based on multi-scenario fusion, which includes obtaining the predicted values ​​of wind power, photovoltaic power and load power of the regional power grid; constructing a scenario identification objective function with the goal of minimizing the weighted sum of flexibility characteristic indicators of power balance, peak shaving and frequency regulation scenarios; solving the scenario identification objective function based on scenario identification constraints to obtain a flexibility characteristic indicator vector; judging the scenario identification result based on the flexibility characteristic indicator vector; if the scenario is one or more of power balance, peak shaving and frequency regulation, constructing a two-stage adaptive configuration model with the goal of minimizing the capacity configuration of flexible resources on each side of source, grid, load and storage and minimizing the resource control losses on each side; solving the flexible resource configuration scale based on the scenario identification result and the first stage constraints; and solving the flexible resource operation plan based on the predicted values ​​of wind power, photovoltaic power and load power, the newly added flexible resource configuration scale, and the second stage deterministic and uncertain constraints.

[0006] Although a great deal of research has been conducted on new energy forecasting, energy storage control, demand response, and multi-timescale scheduling, the following problems still exist.

[0007] (1) Most existing methods predict and generate scenarios for a single object in wind power, photovoltaic or load, which makes it difficult to comprehensively consider the coupling relationship between wind power, photovoltaic and load, resulting in insufficient representativeness of the generated operation scenarios and difficulty in accurately reflecting the source-load fluctuation characteristics in actual operation.

[0008] (2) Existing technologies for flexible resources are mostly focused on single resource types such as energy storage or demand response. They lack a unified and coordinated regulation mechanism for multiple heterogeneous resources such as energy storage, adjustable load and adjustable power supply. This can easily lead to problems such as uncoordinated resource response, unreasonable allocation of regulation capacity and failure to fully utilize overall regulation potential.

[0009] (3) Although some existing technical solutions have carried out research on flexible resource allocation and multi-scenario optimization, they focus more on resource capacity allocation or static operation plan, lacking unified consideration of the coordination process of multiple time scales such as day-ahead, intraday and real-time. It is difficult to dynamically correct the resource adjustment strategy according to the source load prediction deviation and the changes in system operation status, thus affecting the system's real-time adjustment capability and operation stability.

[0010] (4) Existing multi-timescale optimization methods generally suffer from high computational complexity, insufficient real-time performance, and limited hierarchical coordination capabilities when dealing with high-dimensional states, multi-resource coupling, and uncertainties in new energy sources. In particular, during the continuous regulation of hybrid energy storage, situations such as energy storage state of charge shift, excessive use of local resources, and subsequent decline in regulation capabilities are prone to occur, thereby affecting the overall coordinated control effect of the system. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a source-grid-load-storage coordination optimization method that considers multiple heterogeneous and flexible resources.

[0012] The objective of this invention can be achieved through the following technical solutions: This invention provides a source-grid-load-storage coordination optimization method considering multiple heterogeneous and flexible resources, comprising the following steps: S1. Construct prediction models for wind power, photovoltaics, and loads, and generate a set of source-load operation scenarios based on prediction errors; S2. Based on the source load operation scenario set, determine the adjustable capability boundaries of various flexible resources at different time scales; S3. Based on the adjustable capability boundary of the aforementioned heterogeneous flexible resources, construct day-ahead, intraday, and real-time coordination optimization models, wherein: The current phase is used to determine the basic output plan and reserve capacity configuration; The intraday phase is used for rolling corrections based on source load forecast deviations; The real-time phase is used for rapid coordinated adjustment based on system power imbalance; S4. In the real-time phase, a hybrid energy storage dynamic state constraint mechanism is established to dynamically adjust the available adjustment margin of energy storage resources according to the system power imbalance and source load uncertainty. S5. Based on the multi-agent deep reinforcement learning method, the day-ahead, intraday and real-time coordinated optimization models are jointly solved, and a resource collaborative control strategy under multiple time scales is generated through the time scale action mapping mechanism to realize the coordinated and optimized operation of the source-grid-load-storage system.

[0013] Furthermore, S1 specifically includes: S11. Obtain historical wind power output data, photovoltaic power output data, load data and meteorological information, and perform missing value processing, outlier correction and normalization processing on the raw data to form a source-load prediction input dataset. S12. Based on the source-load prediction input dataset, construct prediction models for wind power, photovoltaic power, and load to obtain wind power prediction values, photovoltaic prediction values, and load prediction values ​​for the corresponding time periods; the prediction models are constructed based on the XGBoost method. S13. Construct a prediction error sequence based on actual operating data and prediction results, using the following formula: in, For a moment The prediction error; For a moment The actual operating value; For a moment The predicted value; S14. Based on the prediction error sequence, multiple sets of prediction error scenarios are generated using the Bootstrap resampling method, and the generated prediction errors are superimposed on the corresponding prediction values ​​to obtain the initial running scenario of the source load, as shown in the formula: in, For the first Individual source load operation scenarios; For the predicted results; For the first One prediction error scenario; This represents the initial total number of scenes; S15. Perform scenario reduction processing on the initial source load operation scenario to obtain a typical source load operation scenario set. The scenario reduction processing includes: Initial cluster centers are determined based on the distance between scenes, and iterative clustering is used to classify each running scene, where the distance between scenes is represented as: in, For the scene With Scene The distance between them; For the scene At any moment The value; For the scene At any moment The value; This represents the total number of time periods in the scheduling cycle. S16. Extract typical source-load operation scenarios based on scenario clustering results, and calculate the occurrence probability of each typical source-load operation scenario as the scenario input conditions for the subsequent multi-timescale coordinated optimization model.

[0014] Furthermore, the flexible resources include energy storage resources, adjustable loads, and adjustable power sources.

[0015] Furthermore, determining the adjustability boundaries of various flexible resources at different time scales based on the source load operation scenario set specifically includes: S21. Based on the typical source-load operation scenario set, obtain the net load fluctuation demand of the system at different time scales, and establish a collaborative response model of multiple heterogeneous flexible resources according to the response time, adjustment capacity and operation status of various flexible resources. S22. Determine the responsiveness of various flexible resources at the corresponding time scale. When the response time of a flexible resource meets the requirements of the current adjustment cycle and the resource status is within the allowable operating range, determine that the flexible resource participates in the current adjustment process. The formula is: in, For flexible resources In time scale Next moment The responsive state; For indicator functions; For flexible resources Response time; Time scale The corresponding maximum allowable response time; For flexible resources At any moment The running status; Time scale Flexible resources The allowed operating domain; the time scale includes the day-ahead time scale, intraday time scale, and real-time time scale; S23. Based on the responsiveness of various flexible resources, construct the system's adjustability constraints and determine the total adjustability the system can provide at different time scales, expressed as: in, For flexible resource aggregation; For flexible resources In time scale Next moment Adjustable power; Time scale Next moment The system regulation power requirement; S24. Based on the operating characteristics of energy storage resources, adjustable loads, and adjustable power sources, establish corresponding operating constraint models and adjustment capability constraint models respectively. Based on the operating constraint models and adjustment capability constraint models, obtain the adjustable capability boundaries of various flexible resources at different time scales, which serve as constraint input conditions for day-ahead, intraday, and real-time coordinated optimization models.

[0016] Furthermore, based on the operating characteristics of energy storage resources, adjustable loads, and adjustable power sources, corresponding operating constraint models and adjustment capacity constraint models are established respectively, specifically including: For energy storage resources, a state-of-charge constraint model is established, and the regulation capability boundary of the energy storage resources is determined based on the remaining capacity and charging / discharging capability. Its expression is as follows: in, For energy storage resources At any moment The state of charge; For energy storage resources The charge and discharge efficiency; For energy storage resources At any moment The charging and discharging power; The scheduling time interval; For energy storage resources Rated capacity; For adjustable loads, load regulation capacity constraints and regulation rate constraints are established to determine the adjustable capacity boundary of the adjustable load at different time scales. The expression for this boundary is: in, For adjustable load At any moment Adjustable power; For adjustable load Maximum adjustable capacity; For adjustable power supplies, upper and lower output limits and ramping constraints are established, and the adjustable capacity boundary is determined based on the current operating state. Its expression is as follows: in, Adjustable power supply At any moment ; output power; and Adjustable power supply The minimum and maximum output power.

[0017] Furthermore, the coordination optimization model for the current stage specifically includes: Based on the concentrated load forecasts and renewable energy forecasts for typical source-load operation scenarios, a day-ahead phase basic output plan and reserve capacity configuration model is constructed. The objective function is to minimize system operating cost. The objective function of the day-ahead phase coordinated optimization model includes micro gas turbine fuel cost, interruptible load response cost, and reserve equivalent resource response cost, expressed as: in, Optimize the objective function for the current phase; This is the set of scheduling times for the day before the current day; Fuel cost per unit power of micro gas turbines; For a moment The micro gas turbine is scheduled to be put into operation soon; For interruptible machine tool unit response cost; For a moment Interruptible machine tool response cut-off power; The unit response cost for backup equivalent resources; For a moment Backup equivalent resource response power; Based on the day-ahead source-load power balance relationship, the day-ahead coordinated output plan for various flexible resources is determined, where the system net load requirement is expressed as: in, For a moment The day-ahead load forecast; For a moment Renewable energy day-ahead forecasts; The constraints of the day-ahead stage coordinated optimization model include: system power balance constraints, upper and lower limits of micro gas turbine output constraints, micro gas turbine ramping constraints, interruptible machine tool response capacity constraints, and standby equivalent resource response capacity constraints.

[0018] Furthermore, the intraday coordination optimization model specifically includes: Based on the day-ahead phase coordination optimization results and source-load prediction error scenarios, a rolling correction is applied to the system net load fluctuation, and an intraday phase dynamic coordination optimization model is constructed. The objective function is to minimize the micro gas turbine regulation cost, energy storage response cost, flexible load regulation cost, and net load fluctuation deviation, expressed as: in, Optimize the objective function for the intraday phase; This is the set of scheduling times within the day; For a moment Adjust the output of the micro gas turbine within the day; The unit response cost of lithium-ion batteries; For a moment Lithium-ion battery response power; For the unit response cost of central air conditioning; For a moment Central air conditioning power adjustment; This is the weighting coefficient for peak shaving effect; For system source load difference; Based on the adjusted equivalent load and the combined output of wind and solar power, the net load deviation for each day period is determined, and its expression is as follows: in, For a moment Adjusted equivalent load; For a moment The combined efforts of wind and light; Based on the day-ahead basic output plan, the intraday rolling power balance relationship is established as follows: in, For a moment System net load power; For a moment Central air conditioning power adjustment; The constraints of the intraday phase coordination optimization model include: system power balance constraints, micro gas turbine deep adjustment constraints, micro gas turbine ramping constraints, energy storage state of charge constraints, energy storage charging and discharging power constraints, central air conditioning regulation capacity constraints, central air conditioning regulation rate constraints, and indoor temperature comfort constraints.

[0019] Furthermore, the coordination optimization model for the real-time stage specifically includes: in, Optimize the objective function for the real-time phase; For real-time scheduling time sets; Cost per unit power regulation for flywheel energy storage; For a moment Flywheel energy storage response power; For a moment Real-time response power of lithium-ion batteries; The remaining power deviation after adjustment; Based on the real-time power imbalance of the system, a real-time coordinated adjustment relationship between flywheel energy storage and lithium-ion batteries is established, expressed as: in, For a moment System power imbalance; Based on the intraday energy storage regulation results, the available regulation margin of lithium-ion batteries in real time is dynamically determined to ensure the coordinated allocation of energy storage resources during peak shaving and frequency regulation. The constraints of the real-time stage coordinated optimization model include: real-time power balance constraints of the system, available adjustment margin constraints of the lithium-ion battery, ramping constraints of the lithium-ion battery, output power constraints of the flywheel energy storage, and state of charge constraints of the flywheel energy storage.

[0020] Furthermore, S5 specifically includes: S51. Based on the day-ahead stage coordination optimization model, intraday stage coordination optimization model and real-time stage coordination optimization model, a multi-agent deep reinforcement learning joint decision-making framework is constructed. Different time scales correspond to different agents, and each agent generates resource adjustment actions according to the system operation status under the corresponding time scale. S52. Construct a multi-timescale state space, in which the day-ahead state quantities include the day-ahead load forecast, the day-ahead renewable energy forecast, the reserve capacity demand, and the adjustable capacity boundaries of various flexible resources; the intraday state quantities include source-load forecast error scenarios, system net load deviation, energy storage charge status, and flexible load operation status; and the real-time state quantities include system power imbalance, real-time energy storage charge status, and real-time adjustment margin. S53. Construct a multi-timescale action space, in which the day-ahead action includes the planned day-ahead output of the micro gas turbine, the interruptible load response power, and the reserve equivalent resource allocation capacity; the intraday action includes the intraday regulated output of the micro gas turbine, the lithium-ion battery charging and discharging power, and the central air conditioning regulation power; and the real-time action includes the flywheel energy storage response power and the real-time regulation power of the lithium-ion battery. S54. Based on the operational objectives of the coordinated optimization model at each time scale, construct a joint reward function; S55. Agents at each time scale iteratively update their policies based on the current state information and the joint reward function, and use an experience replay mechanism and a target network update mechanism to improve training stability in order to obtain the optimal adjustment policy at the corresponding time scale. S56. Establish a time-scale action mapping mechanism, using the basic output plan and reserve capacity configuration results generated in the day-ahead stage as the initial boundary conditions of the intraday stage coordination optimization model, and using the energy storage regulation results and flexible load regulation results generated in the intraday stage as the dynamic constraints of the real-time stage coordination optimization model. S57. Based on the multi-agent deep reinforcement learning model after training, output resource coordination control strategies at multiple time scales. Specifically, the day-ahead stage outputs the basic power output plan and reserve capacity configuration results, the intraday stage outputs the rolling adjustment power output plan and energy storage coordination adjustment results, and the real-time stage outputs the real-time coordination adjustment control commands of flywheel energy storage and lithium-ion batteries to achieve coordinated and optimized operation of the source-grid-load-storage system.

[0021] Furthermore, the joint reward function is formulated as follows: in, For a moment The combined reward value; To optimize the objective function value in the current phase; To optimize the objective function value for the intraday phase; To optimize the objective function value in the real-time phase; The remaining power deviation after adjustment; , , and These are the weight coefficients for the corresponding target items.

[0022] Compared with the prior art, the present invention has the following advantages: (1) In the prior art, most new energy scenario generation methods only predict and model errors for a single object in wind power, photovoltaic, or load. It is difficult to comprehensively consider the coupling relationship between wind power, photovoltaic, and load, as well as the temporal correlation of prediction errors. This easily leads to insufficient representativeness and high redundancy of the generated operating scenarios, which in turn affects the accuracy of subsequent backup configuration, rolling scheduling, and resource allocation strategies. This invention constructs wind power, photovoltaic, and load prediction models, and combines prediction error sequences, Bootstrap resampling methods, and scenario clustering reduction mechanisms to form a typical source-load operating scenario set, realizing a unified scenario representation and probabilistic description of source-load uncertainty. Through this technical solution, the fluctuation characteristics of new energy and load can be reflected more accurately, and the ability of source-load scenarios to approximate the actual operating state can be improved, thereby enhancing the scheduling rationality and system operation stability in the subsequent multi-timescale coordination and optimization process.

[0023] (2) In the prior art, most research on the regulation of flexible resources focuses on single resource objects such as energy storage or demand response, lacking a unified coordination mechanism for multiple heterogeneous resources such as energy storage resources, adjustable loads, and adjustable power sources. This leads to problems such as mismatched response speeds, unreasonable resource allocation sequences, and insufficient utilization of overall regulation capacity among different resources. This invention establishes a multi-heterogeneous flexible resource collaborative response model, and combines resource response time, regulation capacity, operating status, and dynamic operating boundaries to determine the adjustable capacity boundaries of various resources at different time scales, achieving unified modeling and collaborative regulation of multiple types of flexible resources. Through this technical solution, regulation tasks can be rationally allocated according to the dynamic characteristics of different resources, avoiding over-extraction of fast resources or lag in the response of slow resources, thereby improving the overall system regulation capacity and the utilization efficiency of flexible resources.

[0024] (3) Existing flexible resource allocation and scheduling methods mostly focus on resource capacity allocation or static operation plan generation, and do not adequately consider the coordination relationship between multiple time scales such as day-ahead, intraday, and real-time. They are difficult to dynamically correct scheduling strategies based on new energy fluctuations and source-load prediction deviations, leading to problems such as accumulated adjustment deviations and insufficient real-time control capabilities during actual system operation. This invention constructs a day-ahead, intraday, and real-time coordination optimization model and establishes a rolling transmission and dynamic mapping relationship between time scales, realizing the coordinated linkage between the basic output plan, rolling adjustment plan, and real-time control strategy. Through this technical solution, resource adjustment strategies can be dynamically adjusted according to changes in system operating status at different time scales, improving the system's adaptability to new energy fluctuations and load changes, reducing the risk of system power imbalance, and enhancing system operational stability and new energy absorption capacity.

[0025] (4) In the prior art, during the multi-timescale coordination and optimization process, hybrid energy storage resources often lack a unified dynamic state management mechanism, which easily leads to problems such as energy storage state of charge deviation, frequent charging and discharging of local energy storage units, and a decrease in subsequent regulation capability, thereby affecting the service life of energy storage and the continuous regulation capability of the system. This invention establishes a hybrid energy storage dynamic state constraint mechanism, dynamically adjusts the available regulation margin of energy storage resources according to the system power imbalance and source-load uncertainty, and combines the real-time stage energy storage collaborative regulation relationship to realize the coordinated division of labor between flywheel energy storage and lithium-ion batteries. Through this technical solution, the complementary advantages of different energy storage resources in terms of response speed and energy support capability can be fully utilized, reducing the problem of excessive use of energy storage resources and improving the service life of energy storage and the continuous regulation capability of the system.

[0026] (5) Existing multi-timescale optimization methods often suffer from high model complexity, insufficient real-time solution capability, and limited hierarchical coordination effect when facing high-dimensional states, multi-resource coupling, and uncertainties in new energy sources. These problems make it difficult to meet the online rapid decision-making needs in complex power grid scenarios. This invention introduces a multi-agent deep reinforcement learning joint solution method, constructing corresponding agents for different timescales (day-ahead, intraday, and real-time), and establishing a state space, action space, and joint reward mechanism to achieve joint training and collaborative decision-making of the multi-timescale coordinated optimization model. This technical solution effectively improves the online optimization solution capability and dynamic adaptive capability in complex scenarios, enhances the real-time performance and stability of multi-resource coordinated adjustment processes, and thus improves the overall economic efficiency and control efficiency of the system.

[0027] (6) Existing technologies often lack effective information transmission mechanisms between optimization models at different time scales, which can easily lead to problems such as the disconnect between day-ahead plans and real-time operating status, and the inability of intraday adjustment results to effectively support real-time control. This invention establishes a time-scale action mapping mechanism, using the basic output plan and reserve capacity configuration results generated in the day-ahead stage as boundary conditions for the intraday stage, and further using the energy storage adjustment results and flexible load adjustment results in the intraday stage as dynamic constraints for the real-time stage, thus achieving effective connection between optimization results at multiple time scales. This technical solution can reduce information distortion between hierarchical optimizations, improve the consistency and coordination between scheduling strategies at different time scales, and thereby further enhance the overall reliability and adjustment accuracy of the system. Attached Figure Description

[0028] Figure 1 This is a diagram illustrating the overall framework for source-grid-load-storage coordination optimization in an embodiment of the present invention. Figure 2 This is a diagram illustrating the multi-heterogeneous resource multi-timescale response structure of an embodiment of the present invention. Figure 3 This is a schematic diagram of a centralized training-distributed execution framework according to an embodiment of the present invention; Figure 4 This is a multi-timescale action mapping diagram according to an embodiment of the present invention; Figure 5 This is a flowchart of the MADDPG algorithm according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] Example 1: like Figure 1 As shown, this embodiment provides a source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources, and implements a technical solution for adaptive collaborative optimization across multiple time scales, including day-ahead, intraday, and real-time, to improve system regulation capacity, operational economy, and renewable energy consumption level. To address the above problems, this embodiment proposes a multi-time-scale collaborative regulation method involving multiple heterogeneous flexible resources, focusing on the following three technological innovations: (1) A source-load operation scenario generation method for unified coupling representation of wind power, photovoltaic and load is proposed. This method addresses the need for unified modeling of multi-source prediction errors by constructing a joint representation model of wind power, photovoltaic and load prediction deviations, which can simultaneously reflect the time correlation, fluctuation coupling and scenario evolution characteristics among multiple source-load variables. On this basis, a set of highly representative source-load operation scenarios suitable for scheduling optimization is formed, thereby improving the ability of scenario input to approximate the actual operating state and providing unified boundary conditions and decision-making basis for subsequent multi-resource coordinated regulation.

[0031] (2) A collaborative response modeling and system flexibility quantitative evaluation method for the optimal combination of multiple heterogeneous flexible resources is proposed. This method establishes a unified regulation characteristic model for multiple types of resources such as energy storage, adjustable load and distributed adjustable power source, and comprehensively considers the differentiated characteristics such as response speed, regulation capacity, duration, recovery characteristics and operating cost to form a collaborative response solution model for system regulation needs; further, a system flexibility evaluation model based on multi-dimensional performance indicators is constructed to quantitatively evaluate the comprehensive support capability under different resource combinations, and to provide a calculable and comparable evaluation basis for the optimal combination of multiple resources and the design of regulation strategies.

[0032] (3) A multi-timescale nested optimization method combining day-ahead, intraday, and real-time approaches is proposed, integrating dynamic state constraints of hybrid energy storage with multi-agent deep reinforcement learning. This method integrates day-ahead planning, intraday rolling correction, and real-time collaborative control into a unified model, and introduces a dynamic state-of-charge constraint model suitable for hybrid energy storage to enhance the sustainable utilization capability of energy storage during continuous adjustment. At the same time, it combines multi-agent deep reinforcement learning algorithms to achieve online adaptive decision-making and rapid solution for multiple resources in complex and uncertain scenarios, thereby enhancing the adaptability of the method to source-load fluctuations, resource state changes, and operation scenario switching, improving system adjustment flexibility, and reducing network operation losses.

[0033] Specifically, the following steps are included: S1. Construct prediction models for wind power, photovoltaics, and loads, and generate a set of source-load operation scenarios based on prediction errors; In one specific implementation, under conditions of high renewable energy integration, wind and solar power output exhibits significant randomness and volatility. Simultaneously, the load side is also highly uncertain due to user behavior, electric vehicle integration, and weather conditions. To improve the system scheduling plan's adaptability to source-load fluctuations, wind power, solar power, and load forecasting models are first established based on historical operating data and meteorological information. Furthermore, multi-scenario source-load operating data is generated by incorporating forecasting errors to reflect the system's changing characteristics under different operating conditions.

[0034] Historical wind power output data, photovoltaic power output data, load operation data, and meteorological data are acquired. Missing value repair, outlier removal, and normalization are performed on the raw data to form a predictive input dataset. Meteorological data includes information such as temperature, humidity, wind speed, solar irradiance, and weather type.

[0035] Based on this, the XGBoost algorithm is used to establish prediction models for wind power, photovoltaic power, and load points, and their expressions are as follows: in, For a moment The predicted value; For XGBoost prediction models; For a moment The input feature vector; For the first A regression tree; These are the weights of the leaf nodes in the corresponding regression tree; This represents the total number of regression trees.

[0036] After obtaining the point prediction results, to further reflect the uncertainties in new energy and load forecasting, statistical modeling of the prediction error is performed. The prediction error sequence is expressed as: To improve the ability of scene generation to characterize random fluctuations, the Bootstrap resampling method is used to sample historical error sequences with replacement, generating multiple sets of error scenes, the expression of which is as follows: in, For the first The scene at any moment The prediction error; For the Bootstrap resampling operator; This represents the number of historical samples. This represents the initial total number of scenes generated.

[0037] The generated prediction error is superimposed on the point prediction result to obtain the source load operation scenario: in, For the first Individual source load operation scenarios; This is the sequence of predicted results.

[0038] Since the initial number of scenes generated by Bootstrap is large, directly using them for subsequent optimization would significantly increase computational complexity. Therefore, an improved hierarchical K-means clustering method is used to reduce the number of scenes to extract typical scenes that can characterize the source load fluctuation characteristics.

[0039] Let the initial set of scenes be: P =[ P 1 , P 2 ,…, P M ], will the s Each scenario can be represented as P s =[ p s,1 , p s,2 ,…, p s,T The original number of cluster centers is counted as K 1.

[0040] Based on this, a preliminary selection was made. K One cluster center. First, the distance between the original scenes is calculated according to equation (6). dThe two scenes with the largest distance are then selected as the initial cluster centers. From the remaining scenes, the scene with the largest distance from the existing set of cluster centers is selected as the new center, until K1 initial cluster centers are formed. The distance between scenes is defined as: Each scenario is assigned to the cluster center closest to it. After one round of clustering, the corresponding objective function value can be obtained: In the formula: M s For the first s The category's runtime scenario value; Pjs For the first s Category 1 j One data volume; C s for s The center point of the class.

[0041] The dispersion of each cluster category is further calculated, and the category with the highest dispersion is selected. Then, within this category, the two farthest scenes are selected as new cluster centers, and clustering calculation is performed again to obtain the updated objective function value. The criterion is expressed as: At that point, return to continue the iteration; where d =( J (λ) - J (λ+1) ) / J (λ) ,for d 0 is a threshold set manually based on the changing characteristics. If... d ≤ d If the value is 0, the iteration stops and the final clustering result is output.

[0042] Based on the final clustering results, typical scenarios that can effectively characterize the load operation level after reduction are extracted, and the occurrence probability of each typical scenario is calculated.

[0043] To evaluate the accuracy of typical scenarios in representing the original source data, this invention uses root mean square error (RMSE) as an evaluation index. The smaller the value, the better the typical scenario obtained after scene reduction fits the original data.

[0044] In the formula, P ori s , t For the first s The original scene at time t The value,P typ c( s ), t Original scene s The typical scenario curve corresponding to the category at time... t The value.

[0045] Adjustable loads possess excellent flexibility and can quickly respond to grid regulation needs. By adjusting power consumption and time periods in real time, a certain degree of power supply and demand balance can be achieved, enhancing system operational stability and promoting the consumption of new energy sources and the economic optimization of grid operation.

[0046] Based on the generation of source load scenarios, an adjustable load response model is further established to characterize the dynamic characteristics of flexible loads participating in the system regulation process.

[0047] The central air conditioning load is described using an equivalent thermodynamic model. Load regulation is achieved by adjusting the set temperature, and its power regulation satisfies the following: in, For central air conditioning at all times The amount of power regulation; For air conditioning cooling efficiency; The equivalent thermal resistance of the room; This refers to the indoor temperature.

[0048] Central air conditioning systems must meet temperature constraints, power constraints, and adjustment rate constraints during operation to ensure user comfort and safe equipment operation.

[0049] Considering user comfort and demand response compensation costs, the incentive cost of central air conditioning is expressed as follows: in, For central air conditioning response costs; The unit power compensation coefficient; This is a comfort penalty factor. For user comfort temperature.

[0050] Industrial interruptible loads participate in system regulation by briefly interrupting the operation of some machine tools, and their response power satisfies: in, For interruptible machine tools at any time The response power; This represents the maximum responsive capacity.

[0051] At the same time, the response changes in adjacent time periods must meet the adjustment rate limit: In the formula: Δ P MT ( t For the factory at all times t Total load power (kW) that interruptible machine tools can respond to; P max MT is the maximum responsive load capacity declared by the factory to external parties; R MT This represents the maximum allowable change in interrupted power within adjacent scheduling cycles for the plant. Its incentive cost is significantly higher than that of air conditioning load, and it typically exhibits a stepped or convex function.

[0052] In the formula, C MT ( t The incentive cost is the amount of machine tool load that can be interrupted at any time. l MT Price for energy interruption compensation; u MT Compensation for fixed production losses due to a single interruption.

[0053] To simplify the modeling process, state variables such as remaining inventory of intermediate products and remaining oil in diesel engines can be uniformly equivalent to energy storage state of charge, thereby constructing a general equivalent energy storage model for EBE, as shown in the following equation.

[0054] In the formula: S EBE ( t )for t The equivalent charge capacity of EBE at any given time, i.e., the inventory of intermediate products or the remaining oil in the diesel engine; S EBE,max and S EBE,min These are the upper and lower limits of the equivalent load capacity, i.e., the inventory limit for intermediate products or the remaining fuel limit for diesel engines; Δ P EBE,dis The discharge power of the equivalent energy storage, i.e., the intermediate product supply rate or the diesel engine power generation power; Δ P EBE,dis,max and Δ P EBE,dis,min These are the upper and lower limits of the discharge power of the equivalent energy storage, respectively. or EBE,dis This refers to the equivalent energy storage discharge efficiency, which is the intermediate product conversion efficiency or the diesel engine power generation efficiency.

[0055] In the formula, C EBE (t (time) t Reserve resource incentive costs; S EBE ( t () represents the energy storage charging and discharging power; l EBE Cost compensation per unit power; u EBE This is the penalty coefficient for cycle loss.

[0056] S2. Based on the source load operation scenario set, determine the adjustable capability boundaries of various flexible resources at different time scales; In one specific implementation, such as Figure 2 As shown, to meet the dynamic adjustment needs of the system at the day-ahead, intraday, and real-time stages under conditions of high-proportion renewable energy access, a collaborative response mechanism for multiple heterogeneous and flexible resources at different time scales is established, combining wind power, photovoltaic, and load operation scenarios. Based on the source-load difference prediction results, the active power adjustment needs of the system at each stage are determined. Furthermore, considering the response time, adjustment capacity, and operating status of different resources, the availability of resources at corresponding time scales is dynamically evaluated, thus forming the adjustable capacity boundary at multiple time scales.

[0057] The system's regulation capability over a time scale s satisfies the following relationship: in, For flexible resource aggregation; For flexible resources In time scale Next moment The responsive state; For flexible resources At any moment Provided adjustable power; For the system on the time scale The demand for active power regulation.

[0058] The responsiveness of a resource is determined in the following ways: in, For flexible resources In time scale Next moment The responsive state; For indicator functions; For flexible resources Response time; Time scale The corresponding maximum allowable response time; For flexible resources At any moment The running status; Time scale Flexible resources The allowed operating domains; the time scales include day-ahead time scale, intraday time scale, and real-time time scale; Considering that when micro gas turbines participate in valley filling and peak shaving, as the output level exceeds the planned output, the costs of maintaining combustion stability, deteriorating operating conditions, and additional compensation all increase. Therefore, the cost of deep regulation typically increases with the depth of regulation. To characterize this characteristic, this invention uses a piecewise linear approach to describe the deep regulation cost of micro gas turbines. Assume the unit is in a certain time period... t The adjustment amount relative to the reference output is divided into: N If there are several adjustment intervals, then the cost of deep adjustment can be expressed as: In the formula, C deep mgt( t For micro gas turbines in time periods t The cost of deep adjustment; c deep m For the first m The unit adjustment cost corresponding to the segment adjustment depth; ∆ P up, down mgt, m ( t ) is the first m The actual downward adjustment amount undertaken by the segment; ∆ P up, down mgt, m, max( t () represents the maximum adjustment capacity of the corresponding segment. By setting an incremental segment cost coefficient, the operational characteristic that "the greater the adjustment depth, the higher the adjustment cost" can be reflected.

[0059] In the real-time regulation phase, to coordinate the collaborative operation of flywheel energy storage and lithium-ion batteries, a dynamic adaptive mechanism for reserved SOC in real-time regulation of hybrid energy storage is established. Since the power balance constraint is used to characterize the system frequency regulation demand in the real-time phase, the equivalent frequency regulation reserve capacity required by the lithium-ion battery is dynamically estimated based on the real-time power imbalance and source-load uncertainty, and further mapped to the corresponding SOC reserve margin. The expression is as follows: In the formula, ΔSOCFR b Leave a margin for battery frequency regulation; E rated b This refers to the rated energy capacity of a lithium-ion battery. For a moment t The equivalent frequency regulation reserve power demand calculated from the system power imbalance risk; the real-time power imbalance of the system. For normalized electricity prices; m ∆ is the electricity price impact coefficient; P imb ( t This is a net load uncertainty index that comprehensively considers load and renewable energy output fluctuations; k This is the uncertainty correction coefficient; α and β These are the weighting coefficients for the power imbalance term and the uncertainty term, respectively; w L and w R These are the uncertainty weights for the load side and the renewable energy side, respectively, and they typically satisfy... w L +w R = 1; T FR The frequency modulation time constant; s L、 s R The non-load and renewable output are respectively To further evaluate the overall flexibility of the system under the collaborative operation of multiple heterogeneous flexible resources, a multidimensional flexibility assessment method based on the Pareto frontier is introduced to quantitatively analyze the system's adjustment capability at different time scales.

[0060] At the time scale sss, the system flexibility state vector is represented as: In the formula, C (s) The effective regulation capacity (MW) that the system can provide reflects the maximum power regulation capability after multi-resource coordination; R (s) The equivalent response rate (MW / s) of the system measures the ability to quickly track power commands or frequency deviations. T (s) The longest duration (in seconds or minutes) during which the system can sustainably provide regulation reflects its resource support capabilities;V (s) The dynamic variability of the system output power trajectory reflects the distribution of currently available adjustable power on fast-response resources. The smaller the response time constant and the higher the proportion of available power, the stronger the dynamic variability of the system. Q (s) For system recoverability, Δ represents the time or energy required for the system to recover to a state where it can participate again after an adjustment. P r For resources r In time scale s Internally achievable power offset; E avail For available energy resources or the continuous capacity of the gas turbine; t r For resources r The minimum controllable adjustment time constant; t ref For reference high-frequency time scales; T ( s rec is the time it takes for resources to recover to a state where they can participate again after an adjustment is completed.

[0061] Because of the coupling and constraints among different flexibility indices, a Pareto non-dominant relation is introduced to describe the multi-resource collaborative adjustment capability. This relates to the flexibility states of two systems at the same time scale. and If the following conditions are met: Then it is called Dominate The system on the time scale s The Pareto flexibility frontier is defined as follows: In the formula, The set of optimal cooperative operating states that cannot be improved simultaneously by other flexible states (frontier); This represents the Pareto dominance relationship; each point on the frontier corresponds to a multi-resource coordinated adjustment strategy and its trade-off characteristics, clearly reflecting the complementarity and conflict between dimensions such as capacity, rate, and duration.

[0062] To quantify the overall level of system flexibility under different resource coordination methods, the Pareto front coverage volume is used as an evaluation index for system flexibility: in, For volume measurement in a multidimensional normalized flexibility space; The larger the value, the wider the feasible adjustment range of the system in the multidimensional flexibility space, and the stronger the overall flexibility of the system.

[0063] S3. Based on the adjustable capability boundary of multiple heterogeneous flexible resources, construct day-ahead, intraday, and real-time coordinated optimization models; In one specific implementation, to achieve coordinated regulation of multiple heterogeneous and flexible resources at different operational stages, a multi-timescale coordinated optimization model of "day-ahead, intraday, and real-time" is constructed. Different timescales correspond to different regulation objectives and uncertainty handling methods. Specifically, the day-ahead stage primarily formulates basic output plans and reserve capacity configuration schemes based on wind power, photovoltaic, and load forecast results; the intraday stage performs rolling corrections to system net load fluctuations based on forecast error scenarios; and the real-time stage completes rapid coordinated regulation based on the system's real-time power imbalance. These three stages are dynamically coupled through regulation capacity boundaries and energy storage capacity reservation mechanisms, thereby ensuring the coordinated operation capability of the source-grid-load-storage system at different timescales.

[0064] The day-ahead phase primarily focuses on optimizing system operational economy. Taking into account the operating costs of micro gas turbines, interruptible load response costs, and the cost of utilizing standby equivalent resources, a day-ahead coordinated optimization model is established, with the objective function expressed as: in, Optimize the objective function for the current phase; This is the set of scheduling times for the day before the current day; Fuel cost per unit power of micro gas turbines; For a moment The micro gas turbine is scheduled to be put into operation soon; For interruptible machine tool unit response cost; For a moment Interruptible machine tool response cut-off power; The unit response cost for backup equivalent resources; For a moment Backup equivalent resource response power; The current stage system power balance relationship is expressed as follows: in, For a moment Load day forecast; For a moment Renewable energy day-ahead forecast.

[0065] The output constraint of a micro gas turbine is expressed as: in, and These represent the minimum and maximum output of the micro gas turbine, respectively.

[0066] The ramp constraint for a micro gas turbine is expressed as: in, and These refer to the downhill ramp rate and the uphill ramp rate of the micro gas turbine.

[0067] The interruptible machine tool response capacity constraint is expressed as: in, This represents the maximum response capacity of an interruptible machine tool.

[0068] The standby equivalent resource response constraint is expressed as: in, Maximum response capacity of backup equivalent resources.

[0069] Based on the previous optimization results, the random fluctuations caused by source load prediction errors are further considered in the intraday stage. The changes in system net load are characterized through typical scenarios, and the output of various flexible resources is dynamically adjusted by rolling optimization.

[0070] The objective function for intraday optimization is expressed as: in, Optimize the objective function for the intraday phase; This is the set of scheduling times within the day; For a moment Adjust the output of the micro gas turbine within the day; The unit response cost of lithium-ion batteries; For a moment Lithium-ion battery response power; For the unit response cost of central air conditioning; For a moment Central air conditioning power adjustment; This is the weighting coefficient for peak shaving effect; For system source load difference; in, The equivalent load after adjustment; Contribute to the overall development of the landscape.

[0071] The intraday rolling power balance relationship is expressed as follows: in, This refers to the net load power of the system. Adjust the power of the central air conditioning system.

[0072] The depth adjustment constraint of a micro gas turbine is expressed as: in, This provides deep adjustment capability for micro gas turbines.

[0073] The vertical adjustment constraint of a micro gas turbine is expressed as: in, and These refer to the maximum downward adjustment capability and the maximum upward adjustment capability of the micro gas turbine, respectively.

[0074] The lithium-ion battery state-of-charge update model is represented as follows: in, For lithium-ion batteries at time The state of charge; The charge / discharge efficiency of lithium-ion batteries; The scheduling time interval; This refers to the rated capacity of the lithium-ion battery.

[0075] After considering the real-time frequency modulation requirement, the state-of-charge constraint of the lithium-ion battery is expressed as follows: in, and These are the lower and upper limits of the SOC (State of Charge) for lithium-ion batteries, respectively. Reserve SOC margin for real-time frequency modulation.

[0076] The central air conditioning regulation capacity constraint is expressed as: in, This is the maximum adjustment power of the central air conditioning system.

[0077] The central air conditioning regulation rate constraint is expressed as: in, This refers to the power adjustment rate of the central air conditioning system.

[0078] Indoor temperature comfort constraints are expressed as follows: in, and These are the lower and upper limits of the permissible indoor temperature, respectively.

[0079] The dynamic change model of indoor temperature is represented as follows: in, and These are the indoor temperature and the outdoor temperature, respectively. and These are the building thermal inertia parameters and the air conditioning regulation coefficient, respectively.

[0080] The real-time phase primarily focuses on rapidly adjusting the system's power imbalance. Due to the highly dynamic nature of the frequency modulation process, a dynamic frequency model is not explicitly established; instead, rapid power compensation is achieved through power balance constraints and energy storage reserves.

[0081] The objective function for real-time optimization is expressed as: in, Optimize the objective function for the real-time phase; For real-time scheduling time sets; Cost per unit power regulation for flywheel energy storage; For a moment Flywheel energy storage response power; For a moment Real-time response power of lithium-ion batteries; The remaining power deviation after adjustment; Constraints: In the formula, ∆P imb This represents the power imbalance of the system.

[0082] S4. In the real-time phase, establish a dynamic state constraint mechanism for hybrid energy storage, and dynamically adjust the available adjustment margin of energy storage resources according to the system power imbalance and source load uncertainty. In one specific implementation, considering the highly random, frequent, and continuously changing characteristics of real-time frequency regulation, hybrid energy storage is prone to deviations from its reasonable operating range in the continuous charge-discharge process. When the energy storage SOC approaches its upper or lower limit, it becomes difficult to maintain bidirectional power regulation capabilities, thus affecting the system's continuous frequency regulation effect and subsequent regulation capabilities. Therefore, based on the aforementioned regulation capability reservation mechanism, a dynamic state constraint mechanism for hybrid energy storage oriented towards the real-time stage is further established. Through flywheel energy storage self-recovery control and lithium-ion battery dynamic range management, coordinated control between energy storage state maintenance and real-time regulation capabilities is achieved.

[0083] Based on the long-term operating characteristics of hybrid energy storage and engineering application requirements, the SOC of lithium-ion batteries is divided into different functional operating ranges. The optimal SOC operating range is set at 0.40–0.70, within which the lithium-ion battery can maintain a high bidirectional regulation capability; the good SOC operating ranges are set at 0.25–0.40 and 0.70–0.85, where the energy storage still possesses a certain degree of unidirectional regulation capability; the charge recovery range is set at 0.20–0.25, used to guide the battery to gradually recover to normal operating conditions; the discharge recovery range is set at 0.85–0.90, used to reduce the risk of high SOC operation; the 0–0.20 range is the limited discharge range, and the 0.90–1.00 range is the limited charging range, to avoid overcharging and over-discharging affecting the energy storage lifespan.

[0084] Flywheel energy storage features fast response, long cycle life, and strong instantaneous power support, but it suffers from energy loss during standby and operation. Without long-term energy compensation, its State of Charge (SOC) will gradually decrease, weakening its subsequent frequency regulation capability. Therefore, during real-time adjustment, when the flywheel energy storage SOC deviates from the normal operating range, a low-power self-recovery control mechanism is introduced to dynamically correct the flywheel energy storage output power.

[0085] When system power deviation P e When the value is greater than 0, the flywheel energy storage discharge control strategy is expressed as follows: When system power deviation P e When <0, the flywheel energy storage charging control strategy is expressed as: in, The discharge power command under flywheel energy storage SOC constraint; This refers to the charging power command under the flywheel energy storage SOC constraint. This is the flywheel energy storage self-recovery coefficient, used to adjust the recovery speed; in this embodiment, it is set to 0.1. Rated power for flywheel energy storage; The raw power command for flywheel energy storage to participate in real-time regulation; Current SOC for flywheel energy storage; and These are the lower and upper limits of the permissible State of Charge (SOC) for flywheel energy storage, respectively.

[0086] While lithium-ion batteries possess good energy support capabilities, their lifespan is significantly affected by the State of Charge (SOC) operating range. When a battery is continuously in a high or low SOC state, it is prone to problems such as accelerated capacity decay, deterioration in consistency, and reduced cycle life. Therefore, it is crucial to establish a dynamic SOC range management mechanism that considers lifespan characteristics in real-time. This mechanism uses an SOC feedback coefficient to dynamically correct the charge and discharge power, gradually guiding the battery's SOC back towards its optimal operating range.

[0087] Let the optimal SOC reference value for lithium-ion batteries be: S ref b =0.55, then the discharge feedback coefficient and the charging feedback coefficient are defined as follows: In the formula, k d This is the lithium battery discharge feedback coefficient. k c Its charging feedback coefficient, S b This represents the current SOC value of the lithium-ion battery. m The self-recovery rate coefficient is set to 2 in this invention.

[0088] Under the dynamic range management mechanism, when the system power deviation... P e When the value is greater than 0, the lithium-ion battery discharge control strategy is expressed as follows: When system power deviation P e When <0, the lithium-ion battery charging control strategy is expressed as: in, This refers to the discharge power command under the state-of-charge (SOC) constraint of the lithium-ion battery. This refers to the charging power command under the state-of-the-art (SOC) constraint of the lithium-ion battery. This refers to the initial power command for lithium-ion batteries during the high-power regulation phase. The base recovery coefficient is set to 0.1 in this embodiment; This refers to the rated power of the lithium-ion battery. and These represent the lower and upper limits of the permissible state of charge (SOC) for lithium-ion batteries, respectively.

[0089] Through the aforementioned dynamic state constraint mechanism, the allocation of regulation capabilities between flywheel energy storage and lithium-ion batteries can be adaptively adjusted based on the degree of system power imbalance and the current operating state of the energy storage during real-time adjustment. The flywheel energy storage primarily undertakes high-frequency, rapid regulation tasks, while the lithium-ion battery provides continuous energy support. A dynamic SOC recovery mechanism prevents the energy storage from deviating from its optimal operating range for extended periods, thereby enhancing the continuous regulation capability of the hybrid energy storage system, slowing down energy storage lifespan degradation, and improving the real-time operational stability of the system.

[0090] S5. Based on the multi-agent deep reinforcement learning method, the day-ahead, intraday and real-time coordinated optimization models are jointly solved, and resource collaborative control strategies under multiple time scales are generated through the time scale action mapping mechanism to realize the coordinated and optimized operation of the source-grid-load-storage system. In one specific implementation, such as Figure 3 , Figure 5 As shown, to address the challenges of diverse flexible resources, significant differences in dynamic response characteristics, and strong operational environment uncertainty in multi-timescale peak shaving and frequency regulation, the day-ahead, intraday, and real-time coordinated regulation process is uniformly modeled as a constrained multi-agent Markov decision process. A multi-agent deep deterministic policy gradient (MADDPG) algorithm based on a centralized training-distributed execution (CTDE) architecture is used for joint solution. By learning consistent coordinated policies across timescales through a unified training framework, the day-ahead baseline plan, intraday rolling correction, and real-time rapid compensation remain coordinated at the policy level, thereby improving multi-resource coordinated regulation capabilities and system operational stability.

[0091] Considering system scale, network topology, and online computational complexity, the distribution network is divided into multiple operating zones. Each zone is configured with a zone-level agent responsible for the coordinated regulation of micro gas turbines, hybrid energy storage, central air conditioning, interruptible loads, and standby equivalent resources within that zone. The system zoning results and network topology remain unchanged throughout the scheduling cycle. During the centralized training phase, the evaluation network of each agent can share global system state and joint action information to assess the impact of multi-zone coordinated regulation behavior on overall operational performance. During the decentralized execution phase, each agent independently generates control actions based solely on locally available observation information within its zone, reducing communication pressure and meeting the online computational efficiency requirements of real-time scheduling.

[0092] During operation, system operation information is collected through measurement and control devices at key nodes of the power grid, including load levels, renewable energy output, regional power imbalance, real-time electricity prices, and energy storage status of charge. This information is then combined with dispatching levels to construct state descriptions for three stages: day-ahead, intraday, and real-time. For different types of flexible resources, state variables directly related to dispatching decisions are further extracted, including the adjustable capacity and ramp margin of micro gas turbines, the SOC and power margin of lithium-ion batteries and flywheel energy storage, the adjustable range of central air conditioning load, and the availability status of interruptible loads and standby equivalent resources. This forms a system state space covering characteristics across multiple time scales.

[0093] Subsequently, the state variables are normalized and structured to meet the requirements of Markov decision modeling. A time-scale action mapping mechanism is further introduced to achieve hierarchical activation and continuous maintenance of day-ahead, intraday, and real-time actions within a unified decision framework. Day-ahead actions are used to determine the baseline scheduling scheme; intraday actions are rolled over based on the day-ahead plan; and real-time actions perform rapid compensation for system power imbalances. This approach constructs a semantically unified state-action representation across different time scales, serving as a unified input to the multi-agent joint decision model.

[0094] Based on this, a regional multi-agent cooperative regulation model based on the MADDPG algorithm is established. The policy network of each regional agent generates control actions for various flexible resources within the region based on local observation information; the centralized evaluation network uses the global state of the system and joint actions during the training phase to uniformly evaluate the resource coordination within the region and the cross-regional regulation behavior, thereby characterizing the coupling relationship and overall regulation contribution of multiple regions and multiple resources at multiple time scales.

[0095] After centralized training, each regional agent operates independently in the distributed execution phase, implementing adjustment decisions according to time-scale action activation rules. In the day-ahead phase, a baseline scheduling scheme is established for micro gas turbines, interruptible loads, and standby equivalent resources. In the intraday phase, deep gas turbine regulation, flexible central air conditioning regulation, and rolling lithium-ion battery correction are introduced based on the day-ahead plan. In the real-time phase, flywheel energy storage and lithium-ion batteries, among other fast-response resources, perform rapid compensation based on system power deviations. Because the adjustment strategies at different time scales all originate from a unified training process, peak shaving and frequency regulation behaviors maintain consistency at the strategy level, thereby achieving an adaptive improvement in multi-regional, multi-resource collaborative adjustment capabilities.

[0096] To describe the multi-resource coordinated adjustment process, the system is modeled as a multi-agent Markov game, which is formally expressed as follows: in, A collection of intelligent agents; For the system state space; The first The local observation space and action space of an intelligent agent; Let be the state transition probability function; This is a system-level instant reward function; This is a discount factor used to balance the relationship between immediate returns and future returns.

[0097] System state is used to characterize the power grid's operating state, equipment operating state, and future regulation needs. Combining operational information at different time scales, the system state space and local observation space are defined as follows: in, These represent the environmental status, the planning status, and the resource status, respectively.

[0098] The environmental state is represented as: in, For partitioning At any moment The power imbalance; This is the load forecast value; This is a photovoltaic forecast value; This is a wind power forecast value; This is an indicator of net load uncertainty. To track deviations in the plan; This refers to the real-time electricity price.

[0099] The plan status is represented as: in, This is a reference plan for the current period; This is a reference plan for the day; This is a deviation from the plan's execution.

[0100] Resource status is represented as: in, This refers to the output power of a micro gas turbine. Power required to respond to interruptible loads; Output power as backup equivalent resources; Adjust the power of the central air conditioning system; and These are the output power of lithium-ion batteries and flywheel energy storage, respectively. and These are lithium-ion batteries and flywheel energy storage SOCs, respectively. Reserve SOC margin for frequency modulation of lithium-ion batteries.

[0101] For partition-level agents, the unified action vector is defined as follows: Among them, the previous action is defined as: In the formula, This represents the planned percentage of gas turbines in the day-to-day schedule; The percentage of interruptible load days that are called; The percentage of backup equivalent resources used in the previous day.

[0102] Intraday actions are defined as: in, This refers to the intraday correction factor for gas turbines; This is a correction factor for central air conditioning. This is a correction factor for lithium-ion batteries.

[0103] Real-time action is defined as: in, The real-time adjustment coefficient for flywheel energy storage; This is the real-time adjustment coefficient for lithium-ion batteries.

[0104] To achieve coordinated execution of actions at different time scales on a unified time axis, a time-scale action activation mechanism is introduced. For example... Figure 4 As shown, the activation variable for the current action is defined as follows: The intraday action activation variable is defined as: Real-time actions remain active at all times. The mapped day-ahead action is represented as: The mapped intraday action is represented as follows: Real-time actions are represented as: The final execution control command is represented as follows: in, , and These are the activation variables for day-to-day, intraday, and real-time actions, respectively. , and These represent the day-ahead, day-intraday, and real-time actions that take effect under the real-time step size after mapping. The control commands are ultimately executed by the partitioned intelligent agent; and These are the update cycles for actions taken the day before and within the day, respectively. This is a time-scale motion mapping function.

[0105] In a multi-agent Markov game, the system's state transition is determined by the joint actions of all agents, and the state transition process is represented as follows: in, u ( t )=[ u 1( t ), u 2( t ),…, u m ( t [] is for all intelligent agents to jointly execute actions; External random disturbances include uncertainties such as source load fluctuations and electricity price changes. Because a reinforcement learning approach based on environment interaction is used, the state transition probabilities do not need to be explicitly modeled but are implicitly generated by the environment simulation process.

[0106] To avoid policy conflicts between reward functions at different stages, a unified system-level reward function is constructed: in, For a moment The combined reward value; To optimize the objective function value in the current phase; To optimize the objective function value for the intraday phase; To optimize the objective function value in the real-time phase; The remaining power deviation after adjustment; , , and These are the weight coefficients for the corresponding target items.

[0107] To address the non-stationarity issue in multi-agent environments, a centralized evaluation network is constructed for each agent during the training phase. Its input includes the system's global state and the joint actions of all agents. in, For the first The state-action value function of an agent; To evaluate network parameters, a centralized evaluation network can characterize the impact of individual agent actions on the overall system performance, thus providing a stable value assessment signal for policy updates. During the decentralized execution phase, the evaluation network does not participate in online decision-making to meet the communication and computational constraints of actual operation.

[0108] The evaluation network is updated using temporal difference error, and its loss function is defined as: Among them, the target value Calculated from the target network: in, This is the discount factor.

[0109] The evaluation method for updating network parameters is as follows: in, To evaluate the network learning rate.

[0110] To improve training stability and sample utilization efficiency, a centralized experience replay buffer (D) is introduced to store multi-agent interaction samples: Training by randomly selecting small batches of samples can effectively reduce the temporal correlation between samples and alleviate the training oscillation problem.

[0111] Each agent is configured with a deterministic policy network that generates continuous control actions based on local observations: in, For the first Action network mapping function for each agent; These are the parameters for the action network. The action network employs a multi-layer fully connected neural network structure and introduces an action boundary mapping function in the output layer to ensure that the action satisfies physical constraints.

[0112] The action network is updated using the policy gradient method, whose gradient expression is: This update method enables the action network to generate scheduling decisions that maximize long-term benefits while satisfying system operational constraints.

[0113] The action network parameters are updated as follows: in, is the learning rate for the action network.

[0114] To enhance the exploratory capabilities during the initial training phase, random perturbations are superimposed on the motion output: in, To explore noise; The initial noise intensity; This is the noise attenuation coefficient.

[0115] To further improve training stability, a target network is introduced and a soft update method is used for parameter updates: in, These are the soft update coefficients for the target network, typically with small values, used to control the rate of change of the target network parameters. Soft updates effectively reduce the impact of drastic fluctuations in the target Q-value on the training process, thereby improving the model's convergence stability and robustness.

[0116] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A source-grid-load-storage coordination optimization method considering multiple heterogeneous and flexible resources, characterized in that, Includes the following steps: S1. Construct prediction models for wind power, photovoltaics, and loads, and generate a set of source-load operation scenarios based on prediction errors; S2. Based on the source load operation scenario set, determine the adjustable capability boundaries of various flexible resources at different time scales; S3. Based on the adjustable capability boundary of the aforementioned heterogeneous flexible resources, construct day-ahead, intraday, and real-time coordination optimization models, wherein: The current phase is used to determine the basic output plan and reserve capacity configuration; The intraday phase is used for rolling corrections based on source load forecast deviations; The real-time phase is used for rapid coordinated adjustment based on system power imbalance; S4. In the real-time stage, a hybrid energy storage dynamic state constraint mechanism is established to dynamically adjust the available adjustment margin of energy storage resources according to the system power imbalance and source load uncertainty. S5. Based on the multi-agent deep reinforcement learning method, the day-ahead, intraday and real-time coordinated optimization models are jointly solved, and a resource collaborative control strategy under multiple time scales is generated through the time scale action mapping mechanism to realize the coordinated and optimized operation of the source-grid-load-storage system.

2. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, S1 specifically includes: S11. Obtain historical wind power output data, photovoltaic power output data, load data and meteorological information, and perform missing value processing, outlier correction and normalization processing on the raw data to form a source-load prediction input dataset. S12. Based on the source-load prediction input dataset, construct prediction models for wind power, photovoltaic power, and load to obtain wind power prediction values, photovoltaic prediction values, and load prediction values ​​for the corresponding time periods; the prediction models are constructed based on the XGBoost method. S13. Construct a prediction error sequence based on actual operating data and prediction results, using the following formula: in, For a moment The prediction error; For a moment The actual operating value; For a moment The predicted value; S14. Based on the prediction error sequence, multiple sets of prediction error scenarios are generated using the Bootstrap resampling method, and the generated prediction errors are superimposed on the corresponding prediction values ​​to obtain the initial running scenario of the source load, as shown in the formula: in, For the first Individual source load operation scenarios; For the predicted results; For the first One prediction error scenario; This represents the initial total number of scenes; S15. Perform scenario reduction processing on the initial source load operation scenario to obtain a typical source load operation scenario set. The scenario reduction processing includes: Initial cluster centers are determined based on the distance between scenes, and iterative clustering is used to classify each running scene, where the distance between scenes is represented as: in, For the scene With scene The distance between them; For the scene At any moment The value; For the scene At any moment The value; This represents the total number of time periods in the scheduling cycle. S16. Extract typical source-load operation scenarios based on scenario clustering results, and calculate the occurrence probability of each typical source-load operation scenario as the scenario input conditions for the subsequent multi-timescale coordinated optimization model.

3. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, The flexible resources include energy storage resources, adjustable loads, and adjustable power sources.

4. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, The determination of the adjustability boundaries of various flexible resources at different time scales based on the source load operation scenario set specifically includes: S21. Based on the typical source-load operation scenario set, obtain the net load fluctuation demand of the system at different time scales, and establish a collaborative response model of multiple heterogeneous flexible resources according to the response time, adjustment capacity and operation status of various flexible resources. S22. Determine the responsiveness of various flexible resources at the corresponding time scale. When the response time of a flexible resource meets the requirements of the current adjustment cycle and the resource status is within the allowable operating range, determine that the flexible resource participates in the current adjustment process. The formula is: in, For flexible resources In time scale Next moment The responsive state; For indicator functions; For flexible resources Response time; Time scale The corresponding maximum allowable response time; For flexible resources At any moment The running status; Time scale Flexible resources The allowed operating domain; the time scale includes the day-ahead time scale, intraday time scale, and real-time time scale; S23. Based on the responsiveness of various flexible resources, construct the system's adjustability constraints and determine the total adjustability the system can provide at different time scales, expressed as: in, For flexible resource aggregation; For flexible resources In time scale Next moment Adjustable power; Time scale Next moment The system regulation power requirement; S24. Based on the operating characteristics of energy storage resources, adjustable loads, and adjustable power sources, establish corresponding operating constraint models and adjustment capability constraint models respectively. Based on the operating constraint models and adjustment capability constraint models, obtain the adjustable capability boundaries of various flexible resources at different time scales, which serve as constraint input conditions for day-ahead, intraday, and real-time coordinated optimization models.

5. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 4, characterized in that, Based on the operating characteristics of energy storage resources, adjustable loads, and adjustable power sources, corresponding operating constraint models and adjustment capacity constraint models are established, specifically including: For energy storage resources, a state-of-charge constraint model is established, and the regulation capability boundary of the energy storage resources is determined based on the remaining capacity and charging / discharging capability. Its expression is as follows: in, For energy storage resources At any moment The state of charge; For energy storage resources The charge and discharge efficiency; For energy storage resources At any moment The charging and discharging power; The scheduling time interval; For energy storage resources Rated capacity; For adjustable loads, load regulation capacity constraints and regulation rate constraints are established to determine the adjustable capacity boundary of the adjustable load at different time scales. The expression for this boundary is: in, For adjustable load At any moment Adjustable power; For adjustable load Maximum adjustable capacity; For adjustable power supplies, upper and lower output limits and ramping constraints are established, and the adjustable capacity boundary is determined based on the current operating state. Its expression is as follows: in, Adjustable power supply At any moment ; output power; and Adjustable power supply The minimum and maximum output power.

6. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, The coordination optimization model for the current stage specifically includes: Based on the concentrated load forecasts and renewable energy forecasts for typical source-load operation scenarios, a day-ahead phase basic output plan and reserve capacity configuration model is constructed. The objective function is to minimize system operating costs. The objective function of the day-ahead phase coordinated optimization model includes the micro gas turbine fuel cost, interruptible load response cost, and reserve equivalent resource response cost, expressed as: in, To optimize the objective function in the current phase; This is the set of day-ahead scheduling times; Fuel cost per unit power of micro gas turbines; For a moment The micro gas turbine is scheduled to be put into operation soon; For interruptible machine tool unit response cost; For a moment Interruptible machine tool response cut-off power; The unit response cost for backup equivalent resources; For a moment Backup equivalent resource response power; Based on the day-ahead source-load power balance relationship, the day-ahead coordinated output plan for various flexible resources is determined, where the system net load requirement is expressed as: in, For a moment The day-ahead load forecast; For a moment Renewable energy day-ahead forecasts; The constraints of the day-ahead stage coordinated optimization model include: system power balance constraints, upper and lower limits of micro gas turbine output constraints, micro gas turbine ramping constraints, interruptible machine tool response capacity constraints, and standby equivalent resource response capacity constraints.

7. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, The intraday coordination optimization model specifically includes: Based on the day-ahead phase coordination optimization results and source-load prediction error scenarios, a rolling correction is applied to the system net load fluctuation, and an intraday phase dynamic coordination optimization model is constructed. The objective function is to minimize the micro gas turbine regulation cost, energy storage response cost, flexible load regulation cost, and net load fluctuation deviation, expressed as: in, Optimize the objective function for the intraday phase; This is the set of scheduling times within the day; For a moment Adjust the output of the micro gas turbine within the day; The unit response cost of lithium-ion batteries; For a moment Lithium-ion battery response power; For the unit response cost of central air conditioning; For a moment Central air conditioning power adjustment; This is the weighting coefficient for peak shaving effect; For system source load difference; Based on the adjusted equivalent load and the combined output of wind and solar power, the net load deviation for each day period is determined, and its expression is as follows: in, For a moment Adjusted equivalent load; For a moment The combined efforts of wind and light; Based on the day-ahead basic output plan, the intraday rolling power balance relationship is established as follows: in, For a moment System net load power; For a moment Central air conditioning power adjustment; The constraints of the intraday phase coordination optimization model include: system power balance constraints, micro gas turbine deep adjustment constraints, micro gas turbine ramping constraints, energy storage state of charge constraints, energy storage charging and discharging power constraints, central air conditioning regulation capacity constraints, central air conditioning regulation rate constraints, and indoor temperature comfort constraints.

8. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, The coordination optimization model for the real-time phase specifically includes: in, Optimize the objective function for the real-time phase; For real-time scheduling time sets; Cost per unit power regulation for flywheel energy storage; For a moment Flywheel energy storage response power; For a moment Real-time response power of lithium-ion batteries; The remaining power deviation after adjustment; Based on the real-time power imbalance of the system, a real-time coordinated regulation relationship between flywheel energy storage and lithium-ion batteries is established, expressed as: in, For a moment System power imbalance; Based on the intraday energy storage regulation results, the available regulation margin of lithium-ion batteries in real time is dynamically determined to ensure the coordinated allocation of energy storage resources during peak shaving and frequency regulation. The constraints of the real-time stage coordinated optimization model include: real-time power balance constraints of the system, available adjustment margin constraints of the lithium-ion battery, ramping constraints of the lithium-ion battery, output power constraints of the flywheel energy storage, and state of charge constraints of the flywheel energy storage.

9. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 1, characterized in that, S5 specifically includes: S51. Based on the day-ahead stage coordination optimization model, intraday stage coordination optimization model and real-time stage coordination optimization model, a multi-agent deep reinforcement learning joint decision-making framework is constructed. Different time scales correspond to different agents, and each agent generates resource adjustment actions according to the system operation status under the corresponding time scale. S52. Construct a multi-timescale state space, in which the day-ahead state quantities include the day-ahead load forecast, the day-ahead renewable energy forecast, the reserve capacity demand, and the adjustable capacity boundaries of various flexible resources; the intraday state quantities include source-load forecast error scenarios, system net load deviation, energy storage charge status, and flexible load operation status; and the real-time state quantities include system power imbalance, real-time energy storage charge status, and real-time adjustment margin. S53. Construct a multi-timescale action space, in which the day-ahead action includes the planned day-ahead output of the micro gas turbine, the interruptible load response power, and the reserve equivalent resource allocation capacity; the intraday action includes the intraday regulated output of the micro gas turbine, the lithium-ion battery charging and discharging power, and the central air conditioning regulation power; and the real-time action includes the flywheel energy storage response power and the real-time regulation power of the lithium-ion battery. S54. Based on the operational objectives of the coordinated optimization model at each time scale, construct a joint reward function; S55. Agents at each time scale iteratively update their policies based on the current state information and the joint reward function, and use an experience replay mechanism and a target network update mechanism to improve training stability and obtain the optimal adjustment policy at the corresponding time scale. S56. Establish a time-scale action mapping mechanism, using the basic output plan and reserve capacity configuration results generated in the day-ahead stage as the initial boundary conditions of the intraday stage coordination optimization model, and using the energy storage regulation results and flexible load regulation results generated in the intraday stage as the dynamic constraints of the real-time stage coordination optimization model. S57. Based on the multi-agent deep reinforcement learning model after training, output resource collaborative control strategies at multiple time scales. Specifically, the day-ahead stage outputs the basic power output plan and reserve capacity configuration results, the intraday stage outputs the rolling adjustment power output plan and energy storage coordination adjustment results, and the real-time stage outputs the real-time collaborative adjustment control commands of flywheel energy storage and lithium-ion batteries, thereby realizing the coordinated and optimized operation of the source-grid-load-storage system.

10. The source-grid-load-storage coordination optimization method considering multiple heterogeneous flexible resources according to claim 9, characterized in that, The joint reward function is formulated as follows: in, For a moment The combined reward value; To optimize the objective function value in the current phase; To optimize the objective function value for the intraday phase; To optimize the objective function value in the real-time phase; The remaining power deviation after adjustment; , , and These are the weight coefficients for the corresponding target items.

Images