Hierarchical optimization and control method for electro-hydro coupling in polygeneration complementary systems

Abstract

The application discloses a hierarchical optimization regulation and control method and system for electric-hydrogen coupling in a multi-energy complementary system, relates to the technical field of multi-energy complementary system optimization regulation and control, and comprises the following steps: obtaining operation data of the multi-energy complementary system, aggregating the operation data based on the correlation of optimization targets to form a hierarchical optimization data set; constructing a corresponding state space model based on the hierarchical optimization data set, defining action spaces of each level, and designing a multi-objective reward function; inputting the state space model, the action space and the multi-objective reward function into a reinforcement learning intelligent agent to obtain a strategy model through training; based on the strategy model and the multi-objective reward function, each level strategy is cooperatively optimized to generate a global optimization strategy; inputting the global optimization strategy and the state space model into a prediction optimization module to generate an optimization scheduling scheme through rolling optimization; according to the optimization scheduling scheme, a scheduling instruction is generated after safety constraint inspection, and is sent to the system for execution; and the strategy model is updated based on operation feedback.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-energy complementary system optimal regulation, and more particularly to a hierarchical optimal regulation method for electric-hydrogen coupling in a multi-energy complementary system. BACKGROUND

[0002] In the operation process of a multi-energy complementary system, in order to fully utilize renewable energy, improve energy utilization efficiency, and take into account the economy, environmental protection, and reliability of the system, it is often necessary to achieve the coordinated optimal regulation of multiple energy forms such as electric energy and hydrogen energy. In particular, in an electric-hydrogen coupling system containing wind power, photovoltaic, electrolytic hydrogen production, hydrogen storage, fuel cells, and electrochemical energy storage, the operating environment presents strong time-varying and multi-uncertainty characteristics, and there is a significant coupling relationship between energy flows. Traditional single-energy optimization or static scheduling methods are difficult to achieve dynamic optimization of multiple objectives while ensuring system safety.

[0003] For example, the invention patent with the announcement number CN118381030A discloses a "multi-objective robust optimization method for power systems based on meta-multi-agent deep reinforcement learning", which discloses a two-stage robust optimization and multi-agent adversarial training scheme for a wind / light / fire hybrid power system. The scheme first constructs a multi-objective two-stage robust optimization model containing economic cost, carbon emission, and line active loss, etc. The fluctuation of renewable energy output is described by an uncertainty set, and the Lagrange dual method is used to transform it into a solvable form. On this basis, two types of heterogeneous agents are introduced to construct an adversarial optimization structure: one side minimizes the overall optimization objective, and the other side maximizes the two-stage target parameters to improve robustness; then, combined with the meta-learning framework (Reptile) and the adversarial strategy optimization (IPPO), training and fine-tuning are performed under different weight scenarios to obtain multiple Pareto optimal solutions for the scheduling optimization of the power system.

[0004] For the multi-objective scheduling optimization of the wind / light / fire hybrid power system, although robust optimization and meta multi-agent reinforcement learning are introduced to improve the adaptability to uncertainty, the modeling object is limited to the power system, and the energy coupling relationship between the electrolytic hydrogen production, hydrogen storage, fuel cell and the power system is not covered. The multi-agent structure of the method is the optimization of two agents in opposition, which lacks the collaborative modeling and strategy coordination mechanism among the subsystems in the hierarchical structure of the electricity-hydrogen multi-energy system. The optimization solution focuses on offline training and static Pareto solution generation in multiple weight scenarios, lacking real-time rolling scheduling capability combined with predictive optimization. At the same time, the robustness constraints are concentrated on renewable power output and power flow loss, and do not include physical constraints and safety constraints of the hydrogen storage, production and consumption process, which cannot effectively guarantee the global balance and safe operation of the system in the operation scenario with a high proportion of hydrogen energy participation. Therefore, when facing a multi-energy complementary system with complex electricity-hydrogen energy conversion, strong coupling constraints and dynamic operation characteristics, the existing scheme cannot realize the organic combination of hierarchical strategy optimization, global collaborative constraint satisfaction and real-time safety control, and cannot meet the efficient, stable and safe operation requirements of the electricity-hydrogen coupled system. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a hierarchical optimization regulation method for electricity-hydrogen coupling in a multi-energy complementary system, which solves the problem that the existing scheme lacks hierarchical strategy collaboration and real-time rolling scheduling under the condition of electricity-hydrogen multi-energy coupling, and cannot balance the actual operation requirements of economy, environmental protection and reliability by constructing a hierarchical state space model and a multi-objective reward function, and using a method combining multi-agent reinforcement learning and global collaborative game optimization.

[0006] To achieve the above object, the present application provides the following technical scheme:

[0007] The hierarchical optimization regulation method for electricity-hydrogen coupling in a multi-energy complementary system comprises the following steps: obtaining operation data of the multi-energy complementary system, aggregating through a mutual information clustering algorithm based on the time sequence correlation of the optimization target and the operation data, and forming a hierarchical optimization data set; constructing a state space model corresponding to each level based on the hierarchical optimization data set, defining an action space for each level, and designing a multi-objective reward function containing global collaborative constraint terms; inputting the state space model, the action space and the multi-objective reward function into the reinforcement learning agent of the corresponding level for training to obtain a strategy model of each level; performing collaborative optimization on the strategy of each level based on the strategy model of each level and the global collaborative constraint terms of the multi-objective reward function, and generating a global optimization strategy; inputting the global optimization strategy and the state space model into a predictive optimization module for rolling optimization to generate an optimization scheduling scheme; generating a scheduling instruction after safety constraint verification according to the optimization scheduling scheme, and issuing the scheduling instruction to the system for execution, and updating the strategy model based on the operation feedback.

[0008] In a preferred embodiment, the time sequence correlation based on the optimization target and the operation data is aggregated by a mutual information clustering algorithm to form a hierarchical optimization data set, and the specific steps are as follows: missing value completion, noise filtering and time synchronization processing are performed on the operation data of the multi-energy complementary system; the correlation coefficient between each operation data and the optimization target is calculated, and the operation data with a correlation exceeding a preset threshold is screened; the operation data is grouped based on a clustering algorithm to form a hierarchical optimization data set; after the hierarchical optimization data set is formed, the corresponding operation unit in the multi-energy complementary system is divided into each hierarchical unit based on the hierarchical optimization data set.

[0009] In a preferred embodiment, the multi-objective reward function includes a local optimization target item of each hierarchical unit and a global collaborative constraint item; the local optimization target item is constructed by a normalization and weighting method, and the weight coefficient is obtained by fitting historical operation data.

[0010] In a preferred embodiment, the construction method of the global collaborative constraint item includes: modeling the coupling relationship between each hierarchical unit in the multi-energy complementary system, establishing a coupling equation reflecting the energy, power and state interaction law between the hierarchical units; calculating the collaborative coefficient between the hierarchical units based on historical operation data, and performing outlier rejection and smoothing processing on the collaborative coefficient to reduce the noise influence; generating short-term operation state prediction results of each hierarchical unit using a prediction module, combining the prediction results, and adjusting the weight parameters of the collaborative coefficient using a rolling time domain optimization method to reflect the time-varying coupling characteristics of the system; the adjusted collaborative coefficient is substituted into the coupling equation to construct the global collaborative constraint item.

[0011] In a preferred embodiment, the state space model, the action space and the multi-objective reward function are input into the reinforcement learning agent of the corresponding level to train each level strategy model, and the specific steps are as follows: generating a state transition sequence based on the state space model and the action space; calculating the reward signal corresponding to the state transition sequence through the multi-objective reward function; combining the reward signal, training the agent using a reinforcement learning algorithm, updating the policy parameters, and obtaining each level strategy model.

[0012] In a preferred embodiment, the collaborative optimization of each level strategy to generate a global optimization strategy includes: calculating local control and scheduling schemes based on each level strategy model; inputting each local control and scheduling scheme and the global collaborative constraint item into a global optimizer, and the global optimizer uses a multi-agent collaborative game algorithm to adjust and combine each local scheme to obtain a global optimization strategy that meets the global constraint.

[0013] In a preferred embodiment, the inputting the global optimization strategy and the state space model into the prediction optimization module to perform rolling optimization to generate an optimized scheduling scheme comprises: based on the global optimization strategy and the state space model, combining real-time operation data of the multi-energy complementary system to construct a prediction optimization problem; using a rolling time domain optimization method to solve the prediction optimization problem to generate operation prediction results in a plurality of future time periods in each rolling period; extracting a scheduling scheme corresponding to the current rolling period from the operation prediction results as an optimized scheduling scheme output, and storing the operation prediction results for subsequent performance evaluation and strategy updating.

[0014] In a preferred embodiment, the generating scheduling instructions according to the optimized scheduling scheme after safety constraint verification specifically comprises: constructing a safety constraint set based on operation limits of each hierarchical unit, equipment rated parameters and backup margins; using a method combining sensitivity analysis and risk assessment models to evaluate the influence of the optimized scheduling scheme on system operation parameters and potential fault conditions; when the evaluation result shows that the optimized scheduling scheme violates the safety constraint or the risk index exceeds a preset threshold, modifying the optimized scheduling scheme to generate scheduling instructions meeting the safety constraint.

[0015] In a preferred embodiment, the updating the strategy model based on operation feedback specifically comprises: collecting actual operation data after executing the scheduling instructions, and comparing the actual operation data with operation prediction results generated in the prediction optimization module based on the state space model to obtain prediction errors; online correcting parameters of the state space model and the multi-objective reward function according to the prediction errors; and using the corrected state space model and the reward function for incremental training of the reinforcement learning intelligent agent to improve the adaptability of the strategy model to time-varying operation environment.

[0016] The technical effects and advantages of the hierarchical optimization and control method for electric-hydrogen coupling in the multi-energy complementary system of the application are as follows:

[0017] The application trains each hierarchical strategy model based on multi-agent reinforcement learning by constructing a hierarchical state space model, an action space and a multi-objective reward function containing global collaborative constraint terms, and coordinates each hierarchical strategy using a global collaborative game optimization method, thereby realizing hierarchical optimization and global collaborative control of the electric-hydrogen multi-energy system under complex time-varying and multi-uncertainty conditions; further, the online strategy updating mechanism based on operation feedback enhances the self-adaptive ability of the system to weather fluctuations, load changes and external disturbances; through safety constraint verification and scheme modification before scheduling execution, the operation safety and energy balance of equipment are ensured, and the problem that the existing electric-hydrogen coupled multi-energy system lacks hierarchical and global coordination, real-time rolling optimization and safety operation guarantee capability, and cannot simultaneously meet the actual operation requirements of economy, environmental protection and reliability is effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a hierarchical optimization regulation method for electric-hydrogen coupling in a multi-energy complementary system is provided for the embodiments of the present application.

[0019] Figure 2 A comparison curve schematic diagram before and after filtering of wind speed data is provided for the embodiments of the present application.

[0020] Figure 3 An improved K-means clustering result schematic diagram is provided for the embodiments of the present application.

[0021] Figure 4 A cumulative reward distribution schematic diagram of an economic intelligent agent in 50 random scenarios in a test set is provided for the embodiments of the present application.

[0022] Figure 5 A strategy convergence curve schematic diagram of a three-layer intelligent agent under a test scenario is provided. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] Embodiment 1, Figure 1 The hierarchical optimization regulation method for electric-hydrogen coupling in a multi-energy complementary system is provided, which comprises the following steps:

[0025] S1, obtaining operation data of a multi-energy complementary system, based on an optimization target and time sequence correlation of the operation data, performing aggregation through a mutual information clustering algorithm to form a hierarchical optimization data set;

[0026] S2, constructing a state space model corresponding to each level based on the hierarchical optimization data set, defining an action space of each level, and designing a multi-objective reward function containing a global collaborative constraint term;

[0027] S3, inputting the state space model, the action space and the multi-objective reward function into a reinforcement learning intelligent agent corresponding to each level to train, to obtain a strategy model of each level;

[0028] S4, performing collaborative optimization on the strategy of each level based on the strategy model of each level and the global collaborative constraint term of the multi-objective reward function, to generate a global optimization strategy;

[0029] S5, inputting the global optimization strategy and the state space model into a prediction optimization module to perform rolling optimization, to generate an optimization scheduling scheme;

[0030] S6, generating scheduling instructions according to the optimized scheduling scheme after safety constraint verification, issuing the scheduling instructions to the system for execution, and updating the strategy model based on running feedback.

[0031] The embodiment realizes the hierarchical optimization and global collaborative control of the electricity-hydrogen multi-energy system under complex time-varying and multi-uncertainty conditions by constructing a hierarchical state space model and a multi-objective reward function, training a hierarchical strategy by using multi-agent reinforcement learning, and combining global collaborative game optimization, predictive rolling optimization, and safety constraint verification. The self-adaptive ability of the system to weather fluctuations, load changes, and external disturbances is enhanced, which helps to ensure the safe operation of equipment and energy balance, and effectively solves the problem that the existing electricity-hydrogen coupled multi-energy system lacks hierarchical coordination, real-time optimization, and safe operation guarantee capability.

[0032] S1, obtaining operation data of a multi-energy complementary system, based on the time sequence correlation of the optimization target and the operation data, aggregating by using a mutual information clustering algorithm to form a hierarchical optimization data set.

[0033] In the embodiment, the hierarchical optimization data set is formed by aggregating based on the time sequence correlation of the optimization target and the operation data through the mutual information clustering algorithm, and the specific steps are as follows:

[0034] The operation data of the multi-energy complementary system is subjected to missing value completion, noise filtering, and time synchronization processing. Specifically:

[0035] Real-time data of each operating unit in the multi-energy complementary system is collected, including but not limited to: wind speed (m / s), power output (kW), pitch angle (°) of wind turbine generator set; irradiance (W / m²), panel temperature (℃), output power (kW) of photovoltaic array; current density (A / cm²), hydrogen production (Nm³ / h), operating temperature (℃) of electrolytic cell; pressure (MPa), hydrogen storage amount (Nm³) of hydrogen storage tank; output power (kW), hydrogen consumption rate (Nm³ / h) of fuel cell, etc.

[0036] During the missing value completion, the time series autocorrelation coefficient of the parameter is first calculated according to the formula: to judge the continuity, and the specific calculation formula is as follows:

[0037]

[0038] In the formula, y t is the observed value of the parameter at the t th time point, N is the length of the parameter sequence, and is the sample mean of the parameter.

[0039] If , it is defined that the continuity is strong, and cubic spline interpolation is used.​

[0040] If , define as the greater volatility, using a combination of historical 7-day time period mean and regression prediction method, the specific formula is as follows:

[0041]

[0042] Where , is the historical sample mean at the same time, is the sample through the regression model to get the prediction value at t time.

[0043] Further, the Kalman filter is used to smooth the sensor data, and the state transition model parameters are adjusted according to the device noise variance. For example, for the wind speed sensor, the measured noise variance is 0.5 (m / s) 2 , and the filter gain initial value is set to 0.15. Figure 2 The comparison curve of wind speed data before and after filtering is given, and it can be seen that the fluctuation amplitude is reduced.

[0044] Different devices have different sampling frequencies, and the data of different sampling frequencies are unified to a standard sampling interval, which is set to 5 minutes in this embodiment.

[0045] An optimization target signal including economy, environmental protection and reliability is established. The economic optimization target signal is the unit operation cost of the system, defined as the total marginal operation cost in time step t divided by the power served in the step, and the formula is:

[0046]

[0047] In the formula, and are the power purchased / sold by the grid in time step t, is the power generated by device i and used for energy supply, is the power of the electrolytic cell, is the time step, in hours, is the hydrogen production, is the power of the actual service load, and are the electricity price and the selling price in time step t, is the hydrogen energy income, is the variable operation and maintenance cost of the device, is the life loss, is the start-stop cost, is the equivalent electricity price cost, which is used to measure the power cost consumed by electrolysis, is the decrease of asset j health degree, is the on-off state of device k at step t, taking value 1 for on and 0 for off, To prevent small positive numbers from being zeroed, take 0.1 kWh.

[0048] The environmental optimization objective signal is carbon intensity, defined as the net emissions at step t divided by the served energy, formula:

[0049]

[0050] where, is the average emission factor of the grid, is the fossil fuel consumption, is its emission factor, is the emission factor of the electricity used for electrolysis, taking if the electrolysis electricity comes from purchased electricity, and a factor close to 0 if it comes explicitly from local renewable, is the emission reduction credit factor of the electricity sold.

[0051] The reliability optimization objective signal is energy continuity, defined as the served energy rate, formula:

[0052]

[0053] where, is the electricity demand.

[0054] Calculate the correlation coefficient between each operating data and the optimization objective, and screen out the operating data with a correlation exceeding a preset threshold. Specifically:

[0055] Calculate the Pearson correlation coefficient between each operating data and the above optimization objectives, and set the correlation threshold to 0.6. For each optimization objective, screen out the parameters that are significantly linearly correlated with it to form a set of first-level correlation parameters. Table 1 shows the correlation calculation results of some operating data and optimization objectives.

[0056] Table 1

[0057]

[0058] Group the operating data based on the clustering algorithm to form a hierarchical optimization data set. Specifically:

[0059] Perform improved clustering analysis based on mutual information on all operating parameters to identify the nonlinear association between parameters and optimization objectives. The mutual information calculation formula is as follows:

[0060]

[0061] In the formula, X is the set of values ​​for the running parameters, Y is the set of values ​​for the optimization objective, p(x,y) is the joint probability distribution of the parameter value x and the optimization objective value y, p(x) is the marginal probability distribution of the parameter value x, and p(y) is the marginal probability distribution of the optimization objective value y.

[0062] The clustering process is as follows:

[0063] 1) Calculate the mutual information values ​​between all parameters and each optimization objective;

[0064] 2) Sort the parameters corresponding to each optimization objective according to their mutual information values;

[0065] 3) Using the optimization objective as the cluster center, the parameters are grouped using an improved K-means clustering method;

[0066] 4) Each set of parameters is a secondary related parameter set.

[0067] Clustering results as follows Figure 3 As shown, the mutual information method is used to analyze the nonlinear correlation between the operating parameters and optimization objectives of a multi-energy complementary system. A clustering algorithm is then used to classify the parameters into three categories, corresponding to the three optimization objectives: economic efficiency, environmental friendliness, and reliability. The three star-shaped points in the figure represent the cluster centers for the three optimization objectives, and points of different colors indicate operating parameters with significant nonlinear correlations to their corresponding objectives. The horizontal and vertical axes represent the coordinates of the two principal components after dimensionality reduction using the mutual information features, facilitating visualization. Based on this clustering result, parameters with significant nonlinear correlations can be added to the datasets corresponding to the optimization objectives, forming the final hierarchical optimization dataset together with the linear correlation screening results. This ensures that the parameters within each stratum have a clear correlation with the optimization objectives, providing data support for subsequent hierarchical modeling and control based on the optimization objectives.

[0068] The improved K-means clustering method for grouping parameters specifically involves combining Euclidean distance and standardized mutual information dissimilarity as the distance metric for K-means clustering. The standardized formula for mutual information dissimilarity is as follows:

[0069]

[0070] The distance metric formula is as follows:

[0071]

[0072] In the formula, It is the mutual information value between sample i and sample j. and These are the minimum and maximum values ​​of the mutual information calculated among all sample pairs, respectively. It is the Euclidean distance between sample i and sample j.

[0073] After forming the hierarchical optimization dataset, the process further includes dividing the corresponding operating units in the multi-energy complementary system into hierarchical units based on the hierarchical optimization dataset. Specifically:

[0074] The set of primary correlation parameters, which characterizes the temporal correlation between operational data and optimization objectives based on Pearson correlation, and the set of secondary correlation parameters aggregated by the mutual information clustering algorithm are combined and categorized according to optimization objectives to form three hierarchical datasets: the economic dataset includes wind speed, photovoltaic power output, wind turbine power output, fuel cell hydrogen consumption rate, and hydrogen storage tank pressure; the environmental dataset includes electrolyzer operating temperature, hydrogen storage capacity, photovoltaic panel temperature, electrolyzer current density, and hydrogen production; and the reliability dataset includes hydrogen storage capacity, wind speed, photovoltaic power output, fuel cell output power, hydrogen storage tank pressure, and wind speed.

[0075] Furthermore, based on the parameter sources in the hierarchical dataset, the equipment is categorized into corresponding hierarchical units. Economic units include wind turbine generators, photovoltaic arrays, fuel cells, and hydrogen storage tanks; environmentally friendly units include electrolyzers, photovoltaic arrays, and hydrogen storage tanks; and reliability units include hydrogen storage tanks, fuel cells, wind turbine generators, and photovoltaic arrays.

[0076] Through the above steps, a structured hierarchical optimization dataset was formed, laying the data foundation for subsequent hierarchical modeling and optimization control.

[0077] S2, construct the state space model of the corresponding level based on the hierarchical optimization dataset, define the action space of each level, and design a multi-objective reward function that includes global collaborative constraints.

[0078] In this embodiment, the construction of the corresponding hierarchical state-space model based on the hierarchical optimization dataset specifically involves:

[0079] First, based on the three hierarchical datasets obtained from S1 (economic dataset, environmental dataset, and reliability dataset), a corresponding hierarchical state-space model is constructed. For each layer, the current quantities, predicted quantities, and statistics to be included in the state vector are first defined: for example, the state vector of the economic dataset includes not only the wind speed at the current moment. Light intensity Current output power of wind power and photovoltaic power , and equipment efficiency , It also includes short-term wind speed and light intensity forecasts. , Operational flags for wind turbine generators and photovoltaic power generation arrays , and the moving average and variance of the output power over the most recent 24 hours. , The state vector thus composed is:

[0080]

[0081] where the example dimension is 12 (6 current quantities, 2 predicted quantities, 2 flags, and 2 statistics). The environmental protection and reliability also adopt the composition method of "current quantity + predicted quantity + flag + statistic": the example state vector of environmental protection can be listed as:

[0082]

[0083] The example state vector of reliability can be listed as:

[0084]

[0085] In the above formula, is the input power of the electrolytic cell, is the hydrogen production, is the fuel cell output power, is the electrolytic cell efficiency, is the fuel cell efficiency, is the predicted electrolytic cell power at the next moment, is the predicted hydrogen production at the next moment, is the electrolytic cell operating state flag, is the fuel cell operating state flag, is the past 24-hour hydrogen production moving average, is the past 24-hour fuel cell power variance; is the current hydrogen storage device hydrogen storage state, is the current battery state of charge, is the hydrogen storage device hydrogen charging power, is the hydrogen storage device hydrogen discharging power, is the battery charging power, is the battery discharging power, is the predicted next moment hydrogen storage state, is the predicted next moment state of charge, is the hydrogen storage device operating state flag, is the battery operating state flag, is the past 24-hour battery SOC moving average, is the past 24-hour charging power variance.

[0086] It should be noted that the specific composition of the dimension value is determined by the designer according to the composition rule (current value, short-term prediction, running flag, statistical quantity) given in the embodiment, so the "12, 11, 12 dimensions" given in the embodiment is an exemplary specification example that can be adjusted by specific engineering implementation.

[0087] The state transition adopts a linear time-varying model to ensure interpretability and convenient fitting, and the formula is as follows:

[0088]

[0089] Wherein represents the hierarchical number, is the state vector of the lth layer at time t, is the action vector of the lth layer at time t, is a time-varying state transition matrix, is a time-varying input matrix, which is obtained by least squares or ridge regression fitting on a rolling window through historical data. The implementation details are as follows: collect historical samples (such as 30 days in the past with 5-minute sampling), solve the least squares problem with a regularization term, and the solution formula is as follows:

[0090]

[0091] Wherein is a regularization coefficient (an example is ), is the square of the Frobenius norm of the matrix ; after fitting, the matrices and are updated every hour or every day in the online phase, so as to reflect the aging of the device and the seasonal changes of the weather.

[0092] By constructing the state vector based on the current quantity, the predicted quantity, the running flag and the statistical quantity, and combining the linear time-varying model to describe the state transition, a state space model is constructed to describe the running state of each hierarchical unit and its dynamic evolution law, providing a unified mathematical basis for subsequent reinforcement learning training and global optimization control.

[0093] Further, the action space of each level is defined as:

[0094] The definition of the action space is directly based on the controllability of the corresponding device in the data set belonging to each layer in S1. Each component of the action vector takes a value within the physical constraint interval In engineering implementation, the upper and lower limits are first specified according to the actual constraints of the device (for example, the fan pitch angle , the photovoltaic tracking angle electrolyzer active power The normalized formula of the affine mapping is as follows:

[0095]

[0096] In the example configuration of the present embodiment, when constructing the action space, each controllable variable needs to be determined one by one according to the hierarchical units formed in the S1 step and the devices contained therein, and the physical meaning and control range thereof are explicitly determined. The economic unit contains a wind turbine generator, a photovoltaic array, a fuel cell and a hydrogen storage tank, and the corresponding action variables include a wind turbine pitch angle adjustment, a wind turbine power set value, a photovoltaic array tracking angle, a photovoltaic inverter output power set value, a fuel cell power set value and a hydrogen storage tank hydrogen output flow set value, and the action dimension is 6-dimensional. The environmental protection unit contains an electrolyzer, a photovoltaic array and a hydrogen storage tank, and the corresponding action variables include an electrolyzer power set value, an electrolyzer hydrogen production rate, a photovoltaic inverter output power set value, a hydrogen storage tank hydrogen charging flow and a hydrogen storage tank hydrogen discharging flow, and the action dimension is 5-dimensional. The reliability unit contains a hydrogen storage tank, a fuel cell, a wind turbine generator and a photovoltaic array, and the corresponding action variables include a hydrogen storage tank hydrogen charging power, a hydrogen storage tank hydrogen discharging power, a fuel cell power set value, a wind turbine power set value, a photovoltaic inverter power set value, a standby capacity reservation ratio and an active power exchange instruction with the power grid, and the action dimension is 7-dimensional. All action variables need to be subjected to value range constraints according to the rated power, operation limits and safety protection requirements of the devices, and are uniformly subjected to standardization processing and mapped to the [-1, 1] interval, so as to realize the comparability of different dimension variables in the optimization solution, and the control variables externally issued are projected back to the actual physical quantities according to the inverse mapping.

[0097] In the present embodiment, the multi-objective reward function includes a local optimization target item and a global collaborative constraint item of each hierarchical unit;

[0098] The local optimization target item is constructed by a normalization and weighting method, and the weight coefficient is obtained by fitting historical operation data, and specifically:

[0099] For any unit i, the local optimization target item is defined as:

[0100]

[0101] wherein represents the normalized score of the kth sub-target, and the value range is mapped to [0, 1]; the weight satisfies and The weights are determined by regression fitting of historical operating data supplemented by constrained optimization: using operating segments from the past 6 months as samples, the true key performance indicators (KPIs) for each segment are calculated (e.g., economic efficiency can be measured by average cost per hour, environmental efficiency by carbon emissions per unit time, reliability by failure probability, etc.). Then, a set of initial weights is obtained by constructing a least-squares optimization problem with non-negativity constraints and a sum-to-1 constraint. In the specific implementation, a quadratic programming solver or non-negativity least squares can be used, and the estimated values ​​are updated monthly in the online phase. Then, exponential weighted smoothing is used to update the current weights: if... The estimated weights obtained from this regression are the previous option weights. Then the current weight vector Calculate using the following formula:

[0102]

[0103] Among them, forgetting factor In this embodiment, 0.9 is used.

[0104] The method for constructing the global collaborative constraint term includes:

[0105] The coupling relationship between the hierarchical units in the multi-energy complementary system is modeled, and coupling equations reflecting the energy, power and state interaction laws between the hierarchical units are established.

[0106] The coordination coefficients between hierarchical units are calculated based on historical operating data, and outlier removal and smoothing are performed on the coordination coefficients to reduce the impact of noise.

[0107] The prediction module generates short-term operating state prediction results for each hierarchical unit. Based on the prediction results, the weight parameters of the coordination coefficient are adjusted using a rolling time-domain optimization method to reflect the time-varying coupling characteristics of the system.

[0108] Substitute the adjusted synergy coefficients into the coupling equation to construct a global synergy constraint term.

[0109] Specifically:

[0110] The coupling relationships between hierarchical units and the global cooperative constraints are expressed by coupling equations and cooperative coefficients. The coupling equations, which express the energy and power interaction between two units, can be represented in a general engineering form as follows:

[0111]

[0112] in The function represents the coupling of unit i to unit j at time t (e.g., the available electrical power input from the economic unit to the hydrogen energy conversion unit). For state-action to energy / power mapping (in implementation usually as directly measurable or modelized operators, e.g. ), coefficient , is a static scaling term or initial value of approximation coupling strength, which is adjusted later with historical data. The "violation" of coupling constraint can be defined as the difference between coupling output at a certain time and target coupling range (e.g. when available power is less than electrolyzer set power demand, it is considered as violation), and the corresponding penalty function takes quadratic form to get smooth gradient, the specific formula is as follows:

[0113]

[0114] where is the difference between actual coupling amount and allowed threshold (positive value represents violation), coefficient is the penalty strength, which is of the order of 100-1000 in engineering and is calibrated through simulation.

[0115] Synergy coefficient is used to represent the strength of "interdependence / synergy" between units and serves as the weight of global constraint term, its calculation uses the method of mutual information normalization: first select the key parameter set , representing unit i and j (e.g. economic key parameter set contains , , environmental key parameter set contains , ), then calculate mutual information and information entropy , , the definition formula is as follows:

[0116]

[0117] To ensure the robustness of mutual information estimation, for continuous variables, Kraskov k-NN method can be used to directly estimate mutual information; for scenarios that need to be discretized, first do equal frequency binning (e.g. 10 bins) and then use empirical probability estimation. The meaning of each variable in mutual information and information entropy calculation is as follows: if X is the value set of a certain variable, then is its probability density function;

[0118] The formula for calculating information entropy is as follows:

[0119]

[0120] After obtaining the time series of the synergy coefficient, abnormal points are removed according to the 3σ criterion, and then the time series is smoothed by a 24-hour moving average. If the control step is 5 minutes, the window is 288 sampling points. The smoothing processing reduces the influence of sensor noise.

[0121] To reflect the time-varying coupling characteristics of the system, in the rolling time domain, the weight parameter of the synergy coefficient is adjusted dynamically based on short-term load / meteorological prediction within a 4-hour prediction window. The adjustment strategy can use a Mamdani-type fuzzy logic controller: one of the inputs is the prediction error at time t , and the calculation formula is as follows:

[0122]

[0123] is the predicted value at time t, is the actual measured value, is a small positive number to prevent the denominator from being zero (here, the value is 10 -6 ); the other input is the system operation deviation , which represents the deviation of the key state of the system from the target value, for example, the normalized difference of the SOC deviation from the target value, or the proportion of the load that is not met within a certain time window.

[0124] Both input variables are divided into three fuzzy subsets: Low, Medium, and High, and are described by triangular membership functions. The specific parameters are obtained by statistical analysis of historical operation samples. Typical fuzzy rules include:

[0125] “if is High and is High, then the output weight adjustment factor takes a large positive value (such as +0.2)”;

[0126] “if is Low and is Low, then takes a small negative value (such as -0.2)”.

[0127] After fuzzy reasoning and centroid defuzzification, we get , and the weight update formula is:

[0128]

[0129] where represents the weight of the synergy coefficient at time t, represents the updated value at the next time. After updating, to ensure that the sum of all is 1, normalization processing is performed on all .

[0130] Substitute the adjusted synergy coefficient into the coupling equation, which is:

[0131]

[0132] where is the synergy coefficient between layer i and layer j at time t.

[0133] Further, the global synergy constraint term is constructed as is the accumulated weighted penalty of time series, which is formulated as

[0134]

[0135] The overall multi-objective reward is combined with the intra-layer weight to form the multi-objective reward function in normalized form, which is formulated as

[0136]

[0137] where (e.g. set to 0.3 as the default value (representing 30% relative weight for maintaining global coupling constraint and 70% for pursuing local performance). This expression form can not only adjust local sub-objective individually, but also control global consistency and safety.

[0138] It is noted that the specific elements and dimensions of state vector are determined according to the principle of “current quantity + short-term prediction + flag + statistical quantity”, and need to be consistent with the layer data set of S1; the mutual information estimation recommends using Kraskov k-NN estimator (k can be an integer real number of 4 to 6) for continuous variables, and using equal frequency 8 to 12 bins for discrete variables; the fitting , is added with appropriate regularization to prevent overfitting; the membership function parameters and rule base of fuzzy controller can be manually set from historical running curves and fine-tuned in online running using Bayesian optimization or simple grid search; the penalty coefficients and are determined after sensitivity analysis in simulation.

[0139] S3, input the state space model, action space and multi-objective reward function to the corresponding level of reinforcement learning agent for training to obtain the specific steps of each level policy model as follows:

[0140] Generate state transition sequence based on state space model and action space;

[0141] Calculate the reward signal corresponding to the state transition sequence through the multi-objective reward function; ​​

[0142] In combination with the reward signal, the reinforcement learning algorithm is used to train the agent, update the policy parameters, and obtain the policy model of each level.

[0143] In the embodiment, the above steps are specifically:

[0144] For the three hierarchical units obtained by S1, reinforcement learning agents based on the deep deterministic policy gradient algorithm are deployed. Each agent is composed of an Actor network and a Critic network. The Actor is responsible for outputting actions according to the current state, and the Critic is responsible for evaluating the value of the state-action pair. In this embodiment, the Actor network adopts a three-layer fully connected structure, with 256, 128, and 64 hidden layer neurons, and the activation function is ReLU. The output layer activation function is tanh to ensure that the action range is in [-1, 1]. The Critic network adopts a three-layer structure, with 512, 256, and 128 hidden layer neurons, and the second layer is fused with the action vector. The network weight uses Xavier initialization, and the learning rate is set to 1×10 -4 , and the optimizer is Adam.

[0145] To train the above agent, first, based on the state space model and action space constructed by S2, the state transition sequence is generated through environment simulation. In this embodiment, the Monte Carlo method is used to simulate the dynamic process of the multi-energy complementary system, and the simulation time step is 5 minutes. Each training round corresponds to 24 hours, i.e. 288 steps. The state transition uses the linear time-varying model in S2 to increase the diversity of training data. To cover different operating conditions, an experience pool with a capacity of 1000 operating scenarios is constructed, including typical working conditions such as low temperature and low light in winter, high temperature and high radiation in summer, peak load period, and renewable energy fluctuation. Table 2 gives some examples of scenarios.

[0146] Table 2

[0147]

[0148] During the training process, the instantaneous reward signal of the i-th level agent at each time step is calculated according to the multi-objective reward function , and the formula is as follows:

[0149]

[0150] where =0.5、 =0.3、 =0.2, is the safety constraint term at time t. If the action violates the safe operation boundary of the device (such as over-power operation of the electrolytic cell, over-pressure of the hydrogen storage tank), the instantaneous reward is reduced by 10, which is represented as =-10.

[0151] The training adopts an experience replay mechanism, and the experience buffer capacity is 50000, and 128 is randomly sampled from the experience buffer each time for updating The four tuples are subjected to batch gradient descent. To balance exploration and utilization, an e-greedy strategy is used in the first 1000 rounds, and e decreases from 1.0 to 0.01 (decay rate 0.995) round by round. The target network uses a soft update strategy τ = 0.001. The total number of training rounds is set to 5000, and each round contains 100 training rounds.

[0152] To deal with the coupling relationship between the three layers of agents, a multi-agent deep deterministic policy gradient (DDPG) framework is used for centralized training and distributed execution. The Critic network receives not only the state and action of the agent, but also the state and action of the other two agents when calculating the Q value, so as to deal with the non-stationary problem of the multi-agent environment. After each training round, the global reward is calculated and fed back to the Critic network update of each agent, so that each layer of policy satisfies the global coordination constraint while being locally optimal.

[0153] After training, each layer of policy model is independently tested, and the test set contains 500 running scenarios that have not appeared in training. The evaluation indicators include cumulative reward, convergence speed, action smoothness, and number of device constraint violations. Figure 4 The cumulative reward distribution of the economic agent in the test set of 50 random scenarios is given; Figure 5 The policy convergence curve of the three-layer agent in the test scenario is shown. It can be seen that after about 2000 rounds, the average reward of each layer of policy tends to be stable. Through grid search optimization of key parameters such as learning rate and exploration rate, the final policy model shows stable optimization and control ability under different seasons and load conditions.

[0154] S4, based on the global coordination constraint term of the multi-objective reward function and the policy model of each layer, the policy of each layer is optimized to generate a globally optimized policy.

[0155] In this embodiment, the collaborative optimization of each layer of policy to generate a globally optimized policy includes:

[0156] Based on the policy model of each layer, the local control and scheduling scheme is calculated respectively;

[0157] The local control and scheduling scheme and the global coordination constraint term are input into the global optimizer, and the global optimizer uses a multi-agent cooperative game algorithm to adjust and combine each local scheme to obtain a globally optimized strategy that meets the global constraint.

[0158] Specifically:

[0159] Based on the three types of policy models trained in S3, in each 5-minute control cycle, first, each hierarchical policy model independently calculates a local action vector, such as the economic unit output normalized fan pitch angle setting, photovoltaic inverter output setting, fuel cell power setting, hydrogen storage tank hydrogen discharge flow, etc.; the environmental protection unit outputs the electrolytic cell power setting, hydrogen production rate, photovoltaic inverter output setting, hydrogen storage tank hydrogen charging and discharging flow; the reliability unit outputs the hydrogen storage tank hydrogen charging and discharging power, battery charging and discharging power, standby capacity ratio, and active power exchange instruction with the grid, etc. These vectors are inversely mapped from the [-1, 1] interval to the actual physical value according to the action mapping method in S2 to obtain the local control and scheduling scheme.

[0160] The global optimizer is responsible for solving the coupling conflicts between different hierarchical units, such as power distribution conflicts, hydrogen supply and demand imbalances, etc. The global optimizer performs global collaborative optimization in the rolling horizon H, and its optimization objective function is to minimize the global cost function, which is expressed as follows:

[0161]

[0162] Among them, , , are the action vectors of the economic, environmental protection, and reliability hierarchical units, respectively; the local cost is obtained by negating the normalized local optimization objective item in S3, reflecting the economic cost, carbon emissions, or reliability loss of the unit.

[0163] The optimization solution adopts a multi-agent collaborative game algorithm, specifically the Nash equilibrium and Stackelberg model to find an equilibrium solution between each hierarchical unit, and the augmented Lagrange method is used to handle the power and hydrogen balance equation constraints. In the optimization solution based on multi-agent collaborative game, first, the three hierarchical units are regarded as game participants, and their strategy space is composed of the physical constraint interval of each action vector. The utility function is taken as the negative local cost, that is, plus the allocated part of the coupling cost item. The solution of the Nash equilibrium can use the iterative optimal response method: in each iteration, fix the strategies of the other two hierarchical units, minimize the utility function of this hierarchical unit alone, and update the action of this hierarchical unit until the difference between the strategies updated in two consecutive rounds is less than a given threshold or the iteration number reaches the upper limit. Under the Stackelberg model, a dominant hierarchical layer is first designated as the leader, and after fixing its strategy, the joint optimal response of the remaining two follower hierarchical layers is solved, and then the leader adjusts its own strategy according to the response of the follower hierarchical layers, and the process is repeated until convergence. The treatment of power balance and hydrogen balance equation constraints can be realized by augmented Lagrange method combined with alternating direction multiplier method, that is, the Lagrange multiplier and penalty term are updated simultaneously while updating the strategy of each hierarchical layer, to ensure that the final solution satisfies the equality constraints and converges to the game equilibrium point.

[0164] After the above iteration and coordination solution, the optimal control quantity combination satisfying the local performance requirements of each hierarchical layer and the global collaborative constraint conditions is obtained, and is output as a global optimization strategy, which is used to guide the subsequent rolling optimization and scheduling execution.

[0165] S5, input the global optimization strategy and state space model into the prediction optimization module to perform rolling optimization, and generate an optimized scheduling scheme including:

[0166] Based on the global optimization strategy and the state space model, combined with the real-time operation data of the multi-energy complementary system, a prediction optimization problem is constructed;

[0167] A rolling horizon optimization method is used to solve the prediction optimization problem, and in each rolling period, the operation prediction results of multiple future time periods are generated;

[0168] The scheduling scheme corresponding to the current rolling period is extracted from the operation prediction results, and is output as an optimized scheduling scheme, and the operation prediction results are stored for subsequent performance evaluation and strategy updating.

[0169] In the embodiment, the above steps are specifically:

[0170] The global optimization strategy output by S4 and the state space model constructed by S2 are jointly input into the prediction optimization module. The prediction optimization module takes the system state vector at the current time, the device action constraint and external prediction information (weather prediction, load prediction, electricity price prediction, etc.) as input, and constructs a prediction optimization problem. The prediction time domain length can be set according to the operation scene, for example, 4 hours, and the step length is consistent with the control period, to ensure the timeliness and accuracy of prediction and scheduling.

[0171] In each rolling period, the prediction optimization module first iteratively predicts the state changes of the system at future time instants using the state space model, considering the action reference given by the global optimization strategy and the influence of external disturbance prediction on renewable energy output and load changes. Subsequently, the predicted state sequence is combined with system operation constraints (such as device power upper and lower limits, energy storage SOC range, hydrogen pressure range, power balance and hydrogen balance conditions) to form a dynamic optimization problem with constraints. The objective of this problem is to minimize the comprehensive operation cost in the prediction time domain, including the sum of the weightless normalized costs corresponding to economic, environmental and reliability indicators, while ensuring that the constraint conditions are met throughout the prediction time domain.

[0172] The prediction optimization problem is solved using a rolling horizon optimization method. In each rolling period, the global optimizer first generates a control sequence for the future prediction time domain length of time instants, but only executes the first time instant control quantity and issues it as the current optimization scheduling scheme to the system execution unit. Subsequently, the system enters the next period, updates the real-time measured state quantity and external prediction data, and repeats the prediction-optimization-execution process. In this way, while ensuring the feasibility of short-term execution, the future scheduling plan can be continuously corrected based on the latest information, thereby realizing dynamic coordinated control of wind, light, electrolyzer, hydrogen storage tank, fuel cell and grid interaction and other multi-energy units.

[0173] To facilitate subsequent performance evaluation and strategy updating, the prediction optimization module stores the state trajectory, action sequence and constraint satisfaction in the prediction time domain into the operation database at the end of each rolling period. These data can be used for statistical analysis and can also be used as training samples for reinforcement learning agents to further improve the hierarchical strategy model and global optimization strategy in S3 and S4.

[0174] S6, according to the optimization scheduling scheme, generate scheduling instructions after safety constraint verification, issue to system execution, and update the strategy model based on operation feedback.

[0175] In this embodiment, the specific steps of generating scheduling instructions according to the optimization scheduling scheme after safety constraint verification are as follows:

[0176] Based on the operating limits of each hierarchical unit, device rated parameters and backup margins, a safety constraint set is constructed;

[0177] A method combining sensitivity analysis and risk assessment model is used to evaluate the influence of the optimization scheduling scheme on system operation parameters and potential fault conditions;

[0178] When the evaluation results show that the optimization scheduling scheme violates the safety constraints or the risk indicators exceed the preset threshold, the optimization scheduling scheme is modified to generate scheduling instructions that satisfy the safety constraints.

[0179] The updating strategy model based on operation feedback includes:

[0180] Collecting actual operation data after executing the scheduling instruction, and comparing with the operation prediction result generated in the prediction optimization module based on the state space model to obtain a prediction error;

[0181] Correcting the parameters of the state space model and the multi-objective reward function online according to the prediction error;

[0182] Using the corrected state space model and the reward function for incremental training of the reinforcement learning agent to improve the adaptability of the strategy model to the time-varying operation environment.

[0183] In the embodiment, the above steps are specifically:

[0184] First, according to the operation limits of each hierarchical unit in the system, the rated parameters of the equipment and the standby margin information, a safety constraint set is constructed, including the upper and lower limits of the power of the fan, photovoltaic, fuel cell, electrolytic cell, hydrogen storage tank, battery and grid interface, the pressure and temperature range of the hydrogen storage tank, the allowed interval of the battery state of charge, and the climbing rate and temperature rise limit of the key components. The optimization scheduling scheme generated in S5 is input into the safety constraint verification module, the influence of each control variable on the safety boundary is evaluated by using the sensitivity analysis method, and the potential failure probability and impact degree of the scheduling scheme in the future operation cycle are analyzed by combining the risk assessment model established based on historical failure statistics and operation state data. When the test result shows that a certain operation parameter will exceed the safety constraint or the risk index exceeds the preset threshold, the system corrects the corresponding control variable, for example, reduces the power instruction of the over-limit equipment, adjusts the energy storage charging and discharging strategy or switches the standby equipment, until the scheduling instruction that meets the safety constraint set is generated.

[0185] The corrected scheduling instruction is issued to the execution layer through the system control interface, including the fan pitch angle and power generation power setting, photovoltaic inverter output power, electrolytic cell and fuel cell power distribution, hydrogen storage tank hydrogen charging and discharging flow, battery charging and discharging power and grid power purchase and sale plan, etc. After the execution of the instruction, the operation data of each device and the overall state of the system are collected in real time, and the prediction error of the key operation parameters is obtained by comparing the prediction result generated by the prediction optimization module based on the state space model in S5 hour by hour, such as the prediction error of the renewable energy output, the prediction error of the hydrogen storage pressure and the SOC prediction error.

[0186] According to the prediction errors, the state space model parameters of S2 are corrected online, the state transition matrix and the input matrix in the system dynamic equation are updated, and the normalization parameters and the penalty term coefficients of each target item in the multi-objective reward function of S3 are adjusted, so that the reward calculation is more in line with the current operating conditions. The corrected state space model and the reward function are used to incrementally train the hierarchical reinforcement learning agent in S3, and a small batch of online updates are used to add the state-action-reward-next state four-tuple of the recent operating period to the experience buffer, and to perform fast iterative optimization of the policy model during idle computing periods. In this way, the policy model can gradually adapt to the time-varying characteristics of the external environment and the changes in the system operating conditions, and maintain the effectiveness and stability of the global optimization strategy in the long run.

[0187] The above formulas are dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula of the most recent real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0188] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product.

[0189] Those skilled in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0190] In addition, the functional modules in each embodiment of the present application can be integrated in one processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0191] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0192] Finally, the above is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A hierarchical optimization regulation method for electro-hydro coupling in a multi-energy complementary system, characterized in that, The method comprises the following steps: obtaining operation data of a multi-energy complementary system, the operation data comprising operation parameters of wind power generation, photovoltaic power generation, hydrogen production by electrolytic cell, hydrogen storage and fuel cell power generation; based on the time sequence correlation of the multiple optimization objectives of economy, environmental protection and reliability and the operation data, performing aggregation by using a mutual information clustering algorithm with the optimization objectives as clustering centers to form a hierarchical optimization data set, and dividing the operation units in the multi-energy complementary system into economy units, environmental protection units and reliability units according to the hierarchical optimization data set; based on the hierarchical optimization data set, constructing a state space model corresponding to each level, defining an action space of each level, designing a multi-objective reward function comprising a global collaborative constraint term, and constructing the global collaborative constraint term based on a coupling equation reflecting the energy and power interaction between the units at each level and a collaboration coefficient representing the collaboration strength between the units; training the multi-objective reward function by using a reinforcement learning agent to obtain a strategy model of each level; optimizing the strategy models of each level to obtain an optimized scheduling scheme; generating scheduling instructions according to the optimized scheduling scheme after safety constraint verification, issuing the scheduling instructions to the system for execution, and updating the strategy models based on operation feedback.

2. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 1, wherein, The optimization of the strategy models of each level to obtain the optimized scheduling scheme comprises: performing collaborative optimization on the strategy models of each level based on the strategy models of each level and the global collaborative constraint term in the multi-objective reward function to generate a global optimization strategy; inputting the global optimization strategy and the state space model into a prediction optimization module to generate the optimized scheduling scheme by using a rolling horizon optimization method.

3. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 2, wherein, The aggregation based on the time sequence correlation of the multiple optimization objectives of economy, environmental protection and reliability and the operation data by using the mutual information clustering algorithm with the optimization objectives as clustering centers to form the hierarchical optimization data set comprises the following steps: performing data processing on the operation data of the multi-energy complementary system, calculating the correlation coefficients between each operation data and the optimization objectives, and screening the operation data with a correlation exceeding a preset threshold; grouping the operation data based on the mutual information clustering algorithm to form the hierarchical optimization data set.

4. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 3, wherein, The multi-objective reward function comprises a local optimization objective term of each hierarchical unit and a global collaborative constraint term. The local optimization objective term is constructed by using a normalization and weighting method, and the weight coefficient is obtained by fitting historical operation data.

5. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 4, wherein, The construction method of the global collaborative constraint term comprises: establishing a coupling relationship equation between the hierarchical units in the multi-energy complementary system, the equation representing the energy interaction, power transmission and state correlation between the hierarchical units; calculating the collaboration coefficients between the hierarchical units based on historical operation data, and performing outlier rejection and smoothing processing on the collaboration coefficients; generating short-term operation state prediction results of each hierarchical unit by using a prediction module, adjusting the weight parameters of the collaboration coefficients by using a rolling horizon optimization method in combination with the prediction results; substituting the adjusted collaboration coefficients into the coupling equation to construct the global collaborative constraint term.

6. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 5, wherein, The steps of training the multi-objective reward function by using the reinforcement learning agent to obtain the strategy model of each level comprise: generating a state transition sequence based on the state space model and the action space; calculating a reward signal corresponding to the state transition sequence by using the multi-objective reward function; The reinforcement learning algorithm is used to train the agent in combination with the reward signal, and policy parameters are updated to obtain the policy model of each level.

7. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 6, wherein, The global optimization strategy is generated by optimizing each level policy in coordination, including: The local control and scheduling scheme is calculated based on each level policy model; The local control and scheduling scheme and global coordination constraint term are input into the global optimizer, and the global optimizer uses the multi-agent coordination game algorithm to adjust and merge each local scheme for consistency to obtain the global optimization strategy that meets the global constraint.

8. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 7, wherein, The global optimization strategy and state space model are input into the prediction optimization module to generate an optimized scheduling scheme through rolling horizon optimization, including: Based on the global optimization strategy and state space model, the real-time operation data of the multi-energy complementary system is combined to construct a prediction optimization problem; The rolling horizon optimization method is used to solve the prediction optimization problem to generate operation prediction results for multiple future time periods in each rolling period; The scheduling instruction corresponding to the current rolling period is extracted from the operation prediction results and used as the optimized scheduling scheme for execution. The complete operation prediction results are stored in the historical database for subsequent performance evaluation and strategy update.

9. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 8, wherein, The scheduling instruction is generated based on the optimized scheduling scheme after safety constraint verification, including: Based on the operating limits of each hierarchical unit, the rated parameters of the equipment, and the standby margin, a safety constraint set is constructed; The sensitivity analysis and risk assessment model are combined to evaluate the optimized scheduling scheme, and the system operating parameters and fault risk indicators are output; When the evaluation results show that the safety constraint set is violated or the risk indicators exceed the preset threshold, the optimized scheduling scheme is dynamically corrected, and the corrected scheduling instruction is output to the execution end.

10. The hierarchical optimization and control method for electro-hydro coupling in a multi-energy complementary system according to claim 9, wherein, The strategy model is updated based on the operation feedback, including: The actual operation data after executing the scheduling instruction is collected and compared with the operation prediction results generated in the prediction optimization module based on the state space model to obtain the prediction error; The parameters of the state space model and the multi-objective reward function are corrected online according to the prediction error; The corrected state space model and reward function are used for incremental training of the reinforcement learning agent to improve the adaptability of the strategy model to the time-varying operating environment.

Images