An A3C multi-agent parallel mechanism-based water-light-storage collaborative optimization scheduling method and system

Abstract

This invention discloses a hydro-solar-storage collaborative optimization scheduling method and system based on the A3C multi-agent parallel mechanism. The method includes: constructing a three-layer collaborative hydro-solar-storage optimization scheduling system operation scenario ("sensing-network-application"); constructing a joint scheduling model for a cascade hydropower station group with constraints such as hydropower station water balance, upstream and downstream hydraulic connections, water level limits, power generation flow limits, hydropower station output constraints, and channel constraints; constructing a dynamic intraday power output allocation model for hydro-solar-storage systems with the objective function of maximizing the total reward value of the optimization scheduling system, simulating the operation scenario and coupling relationship of hydropower, solar power, and storage stations; transforming the hydro-solar-storage optimization scheduling problem into a Markov process, and using the A3C algorithm to achieve efficient strategy exploration and learning, thereby obtaining an optimized scheduling method for the hydro-solar-storage optimization scheduling system. This invention can significantly improve the power absorption capacity of the hydro-solar-storage optimization scheduling system, reduce water and solar power curtailment, and ensure power supply in Xinjiang.

Description

Technical Field

[0001] This invention relates to the field of clean energy optimization scheduling technology, and in particular to a method and system for coordinated optimization scheduling of water, solar and energy storage based on the A3C multi-agent parallel mechanism. Background Technology

[0002] With the rapid development of the clean energy industry, the installed capacity and grid connection ratio of hydropower, solar power, and energy storage power stations have increased significantly. However, their independent operation mode is constrained by the upper limit of transmission channel capacity, which easily leads to a large amount of water and solar power curtailment, causing serious resource waste and economic losses, and restricting the grid system's ability to absorb clean energy. Therefore, there is an urgent need for a collaborative optimization scheduling method for hydropower, solar power, and energy storage based on the A3C (Asynchronous Advantage Action Evaluation) multi-agent parallel mechanism. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide a method and system for coordinated optimization scheduling of water, solar and storage based on the A3C multi-agent parallel mechanism.

[0004] Technical solution: The present invention provides a water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism, comprising the following steps:

[0005] The system constructs a three-layer collaborative hydro-solar-storage optimization scheduling system operation scenario, comprising: a perception layer for comprehensive and real-time perception of key operating parameters of the system; a network layer for ensuring low-latency transmission of data from the perception layer, monitoring data transmission stability and intercepting anomalies, performing preliminary cleaning, integration, and storage of massive amounts of perception data, and providing standardized data interfaces for optimization calculations in the application layer; and an application layer centered on the A3C algorithm-based hydro-solar-storage optimization scheduling system, combined with a supporting power plant information management system, to achieve fully automated scheduling from "data input - algorithm optimization - decision output," ultimately outputting cascade hydropower station reservoir capacity regulation plans, energy storage charging and discharging strategies, and photovoltaic power consumption schemes, supporting the collaborative optimization operation of the hydro-solar-storage optimization scheduling system.

[0006] A joint scheduling model for a cascade hydropower station group is constructed, including: taking the annual regulating hydropower station as the scheduling core of the cascade hydropower station group, and using the daily reservoir capacity change sequence within its scheduling cycle as input; combining the basic parameters and prediction data of each power station in the cascade hydropower station group; outputting the daily power generation sequence of the cascade hydropower station group within the scheduling cycle and the amount of water wasted converted to wastewater, and uploading it to the hydro-solar-storage optimization scheduling system;

[0007] A dynamic power output allocation model for hydropower, solar power, and energy storage is constructed, including: using the day as the scheduling cycle, and based on the water, solar, and energy storage operation data and constraints of the hydropower-solar-storage optimization scheduling system, a power output correction mechanism for water, solar, and energy storage is constructed; the daily power generation, total water wastage converted into electricity, hourly power output of the photovoltaic power station group, and upper limit of transmission channel capacity are obtained from the hydropower-solar-storage optimization scheduling system within the scheduling cycle; the obtained data are processed, and the hydropower output is used as the base load of the hydropower-solar-storage optimization scheduling system to obtain the pre-correction hydropower-solar superimposed power output curve;

[0008] The joint scheduling model of cascade hydropower station groups and the intraday dynamic output allocation model of hydro-solar-storage are transformed into Markov processes. The application layer of the hydro-solar-storage optimization scheduling system is designed, and the Markov process is solved using the A3C algorithm. This includes: constructing a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydro-solar-storage optimization scheduling system that integrates the A3C algorithm.

[0009] Furthermore, the constraints of the constructed joint dispatch model for the cascade hydropower station group include: hydropower station water balance constraints, upstream and downstream hydraulic connections, water level limits, power generation flow limits, and hydropower station output constraints; the output is expressed as:

[0010] ;

[0011] ;

[0012] in, For the first The total output of the tiered hydropower station group This refers to the number of power stations in a cascade hydropower station group. For the first The power output coefficient of a hydropower station For the first The first hydropower station Daily power generation flow For the first The first hydropower station The average water head of the day, The amount of electricity converted from water wastage during the scheduling cycle. To optimize the scheduling cycle in days for the hydro-solar-storage scheduling system, For the first The first hydropower station The daily water discharge rate To calculate the length of the time period.

[0013] Furthermore, a mechanism for correcting the output of water, solar, and energy storage will be constructed, including:

[0014] The typical daily power output periods of hydropower, solar power, and energy storage are divided into three periods: off-peak, secondary peak, and peak. During the off-peak period, the grid load is low and there is no solar power output. The hydropower-solar-storage optimized dispatch system mainly supplies power to the base load of hydropower. During the secondary peak period, solar power output is concentrated. During the peak period, solar power output gradually decreases to zero, while the grid load rises to its highest level of the day. It is necessary to rely on the output of hydropower and energy storage to ensure the load supply.

[0015] The upper limit of transmission channel capacity in the demand-adjustable mechanism is replaced with the upper limit of calculated channel capacity, and the hydropower output during the hydro-solar-storage power generation period is corrected for the first time; the upper limit of calculated channel capacity is expressed as follows:

[0016] ;

[0017] ;

[0018] ;

[0019] in, To optimize the scheduling system for hydro-photovoltaic-storage, the upper limit of the computing channel capacity is set. To optimize the power transmission channel capacity limit of the hydro-solar-storage scheduling system, This represents the maximum energy storage capacity of the energy storage power station group. The first optimized scheduling system for water, solar and energy storage Heavenly Hourly output required The first optimized scheduling system for water, solar and energy storage Heavenly Adjustable output per hour To optimize the power transmission channel capacity limit of the hydro-solar-storage scheduling system, The first optimized scheduling system for water, solar and energy storage Heavenly Hours of effort;

[0020] The total curtailment calculation for the hydro-solar-storage optimized scheduling system includes:

[0021] ;

[0022] in, The first optimized scheduling system for water, solar and energy storage Heavenly He always gave up light and effort when he was young; The first optimized scheduling system for water, solar and energy storage Heavenly The wasted power output during the first correction process in hours The first optimized scheduling system for water, solar and energy storage Heavenly The wasted power output during the second correction process;

[0023] The calculation of abandoned water and solar power includes:

[0024] ;

[0025] in, This refers to the amount of water and solar power wasted during the scheduling cycle, which is then converted into the amount of electricity wasted, i.e., penalty information. The total converted electricity from water wastage by the cascade hydropower stations within the scheduling cycle is obtained from the centralized control center of the hydropower-solar-storage optimization scheduling system. To calculate the length of the time period;

[0026] In the demand-adjustable mechanism, if the demand-adjustable electricity is less than the adjustable electricity on a given day, the photovoltaic power will be fully absorbed through the first power correction, eliminating the need to activate the energy storage station and reducing its operation and maintenance costs. If the demand-adjustable electricity is greater than the adjustable electricity on a given day, the energy storage station will be activated to participate in coordinated regulation, entering the second power correction phase. Specific correction measures include:

[0027] ;

[0028] in, For the first energy storage power station group Heavenly Hourly battery capacity The first optimized scheduling system for water, solar and energy storage Heavenly The output after the first correction in the hour For the first energy storage power station group Peak hours Discharge power per hour For the first energy storage power station group Daily electricity storage To calculate the length of the time period, For the first energy storage power station group Maximum daily energy storage capacity.

[0029] Furthermore, the objective function of the constructed hydro-solar-storage intraday dynamic power output allocation model is:

[0030] ;

[0031] in, To optimize the total reward value of the hydro-solar-storage scheduling system, As the reward coefficient, The reward information is the total power generation of the hydro-solar-storage optimized dispatch system within the dispatch cycle. This is the penalty coefficient; The penalty signal is used to calculate the amount of water and solar power wasted during the scheduling cycle. To calculate the length of the time period.

[0032] Furthermore, the joint scheduling model of the cascade hydropower station group and the intraday dynamic power output allocation model of hydropower-solar-storage systems are transformed into Markov processes, including:

[0033] State space S: The state space is the reservoir capacity sequence of the annual regulating hydropower station within the dispatch cycle, the th... The state before the next iteration is represented as follows: ,in This represents the daily reservoir capacity sequence of an annual regulating hydropower station, where the elements in the reservoir capacity must satisfy the following: , , These represent the minimum and maximum allowable storage capacity of the regulating reservoir during the annual scheduling cycle, respectively. The number of days in the scheduling cycle;

[0034] Action Space A:

[0035] No. The action taken in the next iteration is represented as follows: ,in This represents the daily reservoir capacity change sequence of an annual regulating hydropower station. Elements in the reservoir capacity change sequence must satisfy the following: , The action space must be consistent with the state space dimension to set the action amplitude threshold. In the early stage of training, action space elements are randomly selected, and subsequently, action space elements are selected according to the updates of the policy network and the value network.

[0036] Transfer function P:

[0037] Each time it runs, enter the first... The state before the next iteration and the Actions taken in the next iteration Agent output number The state before the next iteration The next state is represented as... The change of state is briefly described as state and actions Add the corresponding elements in the equation to obtain a new state. Changes in state are directly observed.

[0038] Reward function R:

[0039] The reward function is expressed as , As the reward coefficient, The first optimized scheduling system for water, solar and energy storage Heavenly Hours of effort, To calculate the length of the time period, The penalty coefficient is... The purpose of the water-solar-storage optimization scheduling model is to calculate the amount of water and solar power wasted during the scheduling cycle, which is the penalty information. The optimization goal of the water-solar-storage optimization scheduling model is to achieve the coordination between maximizing power consumption and reducing water and solar power wasted, and to promote the dual optimization of quantity and quality.

[0040] Furthermore, the application layer of the hydro-solar-storage optimized scheduling system is designed, including:

[0041] Worker: Set the number of Workers as needed;

[0042] Actor Network:

[0043] The Actor network is structured for a continuous action space, consisting of an input layer, an intermediate layer, and an output layer. It balances "exploration" and "exploitation," making the output action distribution more inclined towards "high-reward actions."

[0044] Input status After passing through the intermediate layer, the average value of the output motion is calculated. and action variance Mean of movement The mean of the multidimensional actions; the variance of the actions. For the variance of multidimensional actions, the action variance The larger the value, the greater the randomness of the actions, that is, the higher the "exploration level";

[0045] Critic Network:

[0046] The Critic network predicts the long-term cumulative reward expectation under a multidimensional reservoir capacity state sequence. Input status After passing through the intermediate layer, the long-term cumulative reward expectation is output. .

[0047] Furthermore, a mechanism for interaction between the local network and the global network is constructed, specifically as follows:

[0048] Advantage function:

[0049] ;

[0050] in, Indicates the state Select action The additional advantage compared to average level of movement; The action-value function, i.e., the state Select Action Expected future cumulative rewards; The state-value function represents the state value. The expected long-term cumulative reward;

[0051] The dominance function is approximated using time-series difference error:

[0052] ;

[0053] in, For timing difference error, yes Approximate calculation, It is to perform an action The reward received later It is a discount factor, representing the weight of future rewards; For state The expected long-term cumulative reward;

[0054] Optimize the Actor and Critic networks:

[0055] Simultaneously calculate the policy loss of the Actor and the value loss of the Critic to achieve synergistic optimization of the two;

[0056] Actor's policy loss: ,in Representative actions The logarithmic probability; This is an entropy regularization term to prevent the Actor from converging prematurely to a local optimum; the entropy coefficient... Control the intensity of "exploration". The entropy is the normal distribution; the value loss of the Critic: ;

[0057] The interaction between the local network and the global network is as follows:

[0058] After the loss values ​​of the Actor and Critic are added together, each Worker's local network calculates the gradient of the total loss with respect to all trainable parameters; the gradient is transmitted to the global network to update the global parameters, and then the local network synchronizes the latest parameters from the global network.

[0059] when When the value is greater than 0, the parameters of the Actor network are updated to increase the probability of selecting this strategy in the future, while reducing the exploration of this strategy and strengthening its utilization. At the same time, the Critic underestimates the value of the current state, so the parameters of the Critic network are updated to make the subsequent value prediction of this state closer to the actual reward.

[0060] Furthermore, the A3C algorithm is adapted, including:

[0061] Hardware compatibility:

[0062] An asynchronous training architecture based on GPU-CPU collaboration is built on the classic A3C architecture: the local network of the worker is computed using GPU, while the global network is shared using CPU.

[0063] Reward standardization:

[0064] Dynamic reward standardization is adopted to adapt to the current reward distribution, ensuring that the standardized value is between [-1, 1].

[0065] Automatic resource cleanup mechanism:

[0066] In PyTorch multi-process training, the synchronize() function in the CUDA library is called to ensure that the GPU's computing tasks are completed. After the tasks are completed, the empty_cache() cleanup function is called to release unused cached memory.

[0067] Furthermore, an optimized scheduling scheme for the hydro-solar-storage optimized scheduling system integrating the A3C algorithm is generated, and the specific steps are as follows:

[0068] Initialize the Mytest environment: state space S, action space A, reward function R, and termination condition MaxSteps;

[0069] Constructing a global network G and a local network of multiple Workers: Actor outputs the average action value. and action variance Critic outputs state value ;

[0070] Configure the SharedAdam optimizer and discount factor. Entropy coefficient ;

[0071] while the number of training iterations for multiple Workers has not reached MaxSteps;

[0072] Each Worker interacts independently with its local environment: based on its current state. Output actions via local network Import the cascade hydropower station joint commissioning model and the intraday dynamic power output allocation model of hydropower, solar power, and storage to obtain the new state. ,award ;

[0073] Real-time maintenance of dynamic standardization of rewards helps mitigate training fluctuations.

[0074] The local network calculates the loss, synchronizes the gradient to the global network, and pulls the latest parameters from the global network to update the local network;

[0075] Release cached GPU memory resources after training is complete;

[0076] Save the completed global network, plot the reward curve, and calculate the training time;

[0077] Load the global network in an independent environment and execute the complete process: based on the initial state. Output the reservoir capacity change plan of the annual regulating power station during the dispatch period;

[0078] By incorporating the annual reservoir capacity change scheme of the regulating power station into the cascade hydropower station group joint operation model and the daily dynamic output allocation model of hydropower, photovoltaic and storage power stations, hourly output and water and solar power curtailment information of hydropower, photovoltaic and storage power stations are obtained.

[0079] Write the output information to an Excel file and save it to a local path.

[0080] Another embodiment of the present invention provides a hydro-solar-storage collaborative optimization scheduling system based on the A3C multi-agent parallel mechanism, comprising:

[0081] The scenario construction module is used to build the operation scenario of the three-layer collaborative hydro-solar-storage optimization scheduling system, which includes: Perception layer: used for comprehensive and real-time perception of key operating parameters of the hydro-solar-storage optimization scheduling system; Network layer: used to ensure low-latency transmission of data from the perception layer, realize stability monitoring and anomaly interception of data transmission, complete the initial cleaning, integration and storage of massive perception data, and provide standardized data interfaces for optimization calculations in the application layer; Application layer: with the hydro-solar-storage optimization scheduling system based on the A3C algorithm as the core, combined with the supporting power station information management system, it realizes fully automated scheduling from "data input - algorithm optimization - decision output", and finally outputs the reservoir capacity regulation plan of cascade hydropower stations, energy storage charging and discharging strategies and photovoltaic power consumption schemes, supporting the collaborative optimization operation of the hydro-solar-storage optimization scheduling system.

[0082] The first model construction module is used to construct a joint scheduling model for a cascade hydropower station group. It includes: taking the annual regulating hydropower station as the scheduling core of the cascade hydropower station group, and taking the daily reservoir capacity change sequence within its scheduling cycle as input; combining the basic parameters and prediction data of each power station in the cascade hydropower station group; outputting the daily power generation sequence of the cascade hydropower station group within the scheduling cycle and the amount of water wasted converted to wastewater, and uploading it to the hydropower-solar-storage optimization scheduling system.

[0083] The second model construction module is used to construct a dynamic daily power output allocation model for hydropower, solar power, and energy storage. This includes: using the daily scheduling cycle as the scheduling period, constructing a power output correction mechanism for hydropower, solar power, and energy storage based on the operational data and constraints of the hydropower, solar power, and energy storage optimized scheduling system; obtaining the daily power generation, total water wastage equivalent power, hourly power output of the photovoltaic power station group, and upper limit of the transmission channel capacity from the hydropower, solar power, and energy storage optimized scheduling system within the scheduling cycle; processing the obtained data, and using the hydropower output as the base load of the hydropower, solar power, and energy storage optimized scheduling system to obtain the pre-correction hydropower and solar power superposition output curve.

[0084] The model solving module is used to transform the joint scheduling model of cascade hydropower station groups and the intraday dynamic power output allocation model of hydropower-solar-storage into Markov processes. It designs the application layer of the hydropower-solar-storage optimized scheduling system and uses the A3C algorithm to solve the Markov process. This includes: building a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydropower-solar-storage optimized scheduling system that integrates the A3C algorithm.

[0085] Another embodiment of the present invention also provides an electronic device, the device comprising:

[0086] Memory containing executable program code;

[0087] A processor coupled to the memory;

[0088] The processor calls the executable program code stored in the memory to execute the aforementioned water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism.

[0089] In another embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, which, when invoked, are used to execute the aforementioned water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism.

[0090] Beneficial effects: Compared with the prior art, the significant technical effects of this invention are as follows: The A3C algorithm has excellent convergence performance and can stably converge to the optimal scheduling strategy corresponding to the high reward value; Compared with other optimization algorithms (particle swarm optimization, SARSA), A3C (asynchronous advantage action evaluation algorithm) performs better in key indicators such as training time and power consumption, significantly improving the power generation efficiency of the hydro-solar-storage optimized scheduling system; Sensitivity analysis shows that the upper limit of transmission channel capacity has a significantly better effect on improving power consumption than the maximum number of training iterations, providing a quantitative basis for multi-department consultation to increase transmission channel capacity and further release the power consumption potential of the hydro-solar-storage optimized scheduling system. Attached Figure Description

[0091] Figure 1 This is a flowchart of the method of the present invention;

[0092] Figure 2 This is a schematic diagram of the operation scenario of the hydro-solar-storage optimized scheduling system;

[0093] Figure 3 This is a diagram showing the daily dynamic output allocation strategy for hydro-solar-storage systems;

[0094] Figure 4 This is a typical daily load curve for Xinjiang region in 2024;

[0095] Figure 5 This is the overall architecture diagram of the application layer.

[0096] Figure 6 It is a scatter plot of the rewards from 6000 training sessions;

[0097] Figure 7 It is a scatter plot of rewards from 12,000 training sessions. Detailed Implementation

[0098] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the invention.

[0099] Example 1:

[0100] like Figure 1 As shown, the hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism of the present invention includes the following steps:

[0101] S1, such as Figure 2 As shown, the operational scenario of constructing a three-layer collaborative water-solar-storage optimization scheduling system of "sensing-network-application" is as follows:

[0102] Perception Layer: As the core terminal layer for information acquisition in the hydro-solar-storage optimization scheduling system, it integrates irradiance sensors (real-time monitoring of photovoltaic power station illumination conditions), water level sensors (dynamically acquiring water level and flow data of cascade reservoirs), and multi-parameter sensors of the energy storage system (synchronously acquiring operating indicators such as voltage, current, and state of charge (SOC)). This enables comprehensive and real-time perception of key operating parameters of the hydro-solar-storage optimization scheduling system, providing basic data support for subsequent scheduling decisions.

[0103] Network Layer: This layer acts as a hub for information transmission and initial processing, consisting of an industrial local area network (LAN) covering the power plant's production area, a distributed network management system, and a cloud computing platform. The industrial LAN ensures low-latency transmission of sensor data, the distributed network management system monitors data transmission stability and intercepts anomalies, and the cloud computing platform performs initial cleaning, integration, and storage of massive amounts of sensor data, providing standardized data interfaces for optimized computation at the application layer.

[0104] Application Layer: This is the core functional layer for scheduling decision-making and management of the hydro-solar-storage optimization scheduling system. It takes the hydro-solar-storage optimization scheduling system based on the A3C algorithm as its core and combines it with the supporting power station information management system (including modules such as runoff prediction, meteorological prediction, operation status visualization, and scheduling scheme generation) to realize fully automated scheduling from "data input - algorithm optimization - decision output". Finally, it outputs the reservoir capacity regulation plan of cascade hydropower stations, energy storage charging and discharging strategies, and photovoltaic power consumption schemes to support the coordinated and optimized operation of the hydro-solar-storage optimization scheduling system.

[0105] S2. Construct a joint scheduling model for a cascade hydropower station group, including: taking the annual regulating hydropower station as the core of the scheduling of the cascade hydropower station group, and using the daily reservoir capacity change sequence within its scheduling cycle (usually 1 year) as input. Combine this with basic parameters and predicted data obtained from the basin control center, such as characteristic water level, output coefficient, power generation flow limit, output limit, "reservoir capacity-water level" curve, tailrace curve, head loss curve, and daily predicted runoff flow for each power station in the cascade hydropower station group. Based on this, a series of formulas are used to construct a joint scheduling model for the cascade hydropower station group, outputting the daily output sequence of the cascade hydropower station group within the scheduling cycle and the converted abandoned water volume, which is then uploaded to the hydro-solar-storage optimized scheduling system. Specifically:

[0106] The constraints and calculations of the constructed joint dispatch model for the cascade hydropower station group are as follows:

[0107] Hydropower station water balance constraints:

[0108] ;

[0109] in, , The first The first phase of the hydropower station Day, end of period The reservoir's water storage capacity in m 3 ; For the first The first hydropower station Daily inbound flow, m 3 / s; For the first The first hydropower station The outflow rate of the day, m 3 / s; For the first The first hydropower station Daily power generation flow, m 3 / s; For the first The first hydropower station Daily water discharge rate, m 3 / s; To calculate the duration, we set it to 86400 seconds.

[0110] Upstream and downstream hydraulic connections:

[0111] The water discharged from the upstream power station and the flow in the inter-station area together constitute the inflow runoff of the downstream power station.

[0112] ;

[0113] in, For the first The first hydropower station (the The downstream discharge of the upstream power station of the hydropower station, m 3 / s; For the first The first hydropower station to the first The flow rate of each hydropower station, m 3 / s;

[0114] Water level limit:

[0115] ;

[0116] in, For the first The first hydropower station in The water level at the time of the event, in meters (m); For the first The minimum permissible operating water level for each hydropower station is the dead water level, in meters. For the first The maximum permissible operating water level for each hydropower station is the normal storage water level, in meters.

[0117] Power generation flow limit:

[0118] ;

[0119] in, For the first The first hydropower station Daily power generation flow, m 3 / s; Take the first The first hydropower station The smaller of the minimum ecological flow rate per day and the minimum diversion flow rate required by the turbine, m 3 / s; For the first The first hydropower station The maximum permissible priming flow rate required for the Tianshui turbine, in m³ 3 / s;

[0120] Hydropower station output constraints:

[0121] ;

[0122] in, For the first The first hydropower station The minimum permissible output of the Tianshui Hydropower Station units, in kW; For the first The first The maximum permissible output of the Tianshui Hydropower Station units is kW; For the first The output coefficient of a hydropower station; For the first The first hydropower station Average head per day, in meters; For the first The first hydropower station The output of the day, kW;

[0123] Summary of calculation results:

[0124] ;

[0125] ;

[0126] in, For the first The total output of the cascade hydropower station group is kW; The number of power stations in a cascade hydropower station group; The electricity generated from water wastage during the scheduling cycle is expressed in kW·h. To optimize the scheduling cycle in days for the hydro-solar-storage scheduling system, ;

[0127] S3, such as Figure 3 A dynamic daily power output allocation model for hydropower, solar power, and energy storage was constructed, including: using a daily (24-hour) scheduling cycle, and based on the operational data and constraints of the hydropower, solar power, and energy storage optimized scheduling system, a power output correction mechanism for these three components was established. The daily power generation, total wastewater equivalent power, hourly power output of the photovoltaic power station group, and transmission channel capacity limits were obtained from the optimized scheduling system within the scheduling cycle. The acquired data was processed, and the hydropower output was used as the baseload of the optimized scheduling system to obtain the pre-correction hydropower-solar power output curve.

[0128] The specific steps are as follows:

[0129] Step 1: Division of Hydro-Solar-Storage Power Output Time Periods:

[0130] Referring to the "Typical Power Load Curves of Provincial Power Grids" published by the National Development and Reform Commission, and taking the power grid load in Xinjiang as an example, the logic for dividing the daily output periods of the hydro-solar-storage optimized dispatch system is determined: the peak-to-valley load difference in this region typically reaches 10%, with the high-load period concentrated between 10:00 and 22:00. For example... Figure 4 As shown, combining load timing characteristics and photovoltaic output patterns, a typical day is divided into three core output periods: 00:00-10:00 and 22:00-24:00 are low-peak periods, during which the grid load is low and there is no photovoltaic output, and the hydropower-solar-storage optimization dispatch system mainly supplies power based on hydropower base load; 10:00-18:00 is the secondary peak period, which is the concentrated period of photovoltaic output (accounting for more than 90% of the total photovoltaic output of the day), and the superposition of hydropower and photovoltaic output may exceed the upper limit of the transmission channel capacity, leading to concentrated curtailment of photovoltaic power; 18:00-22:00 is the peak period, during which photovoltaic output gradually decreases to zero, while the grid load rises to the highest point of the day (reaching the peak load around 21:00), requiring the output of hydropower and energy storage to ensure load supply;

[0131] Step 2: First Correction: "Need-to-Adjustable" Mechanism

[0132] A "demand-adjustable" mechanism is introduced: To alleviate the curtailment of solar power caused by the combined output of hydropower and solar power during peak periods, a "demand-adjustable" mechanism is introduced. When the upper limit of the transmission channel capacity is fixed, the demand-adjustable output refers to the extent to which hydropower output needs to be actively reduced to absorb the excess solar power output within the limit of the transmission channel capacity; the adjustable output refers to the amount of hydropower output that can be increased within the limit of the transmission channel capacity. Under this mechanism, hydropower output is reduced during peak periods to reduce solar power curtailment; and hydropower output is increased during peak and off-peak periods to adapt to changes in grid load.

[0133] ;

[0134] ;

[0135] in, The first optimized scheduling system for water, solar and energy storage Heavenly Required output per hour, kW; The first optimized scheduling system for water, solar and energy storage Heavenly Adjustable output per hour, kW; The upper limit of the power transmission channel capacity for the optimized scheduling system of hydro-solar-storage is kW; The first optimized scheduling system for water, solar and energy storage Heavenly Hours of effort.

[0136] The following improvements are made to the original "demand-adjustable" mechanism: An upper limit for the calculation channel capacity is introduced, the size of which is determined by:

[0137] ;

[0138] ;

[0139] ;

[0140] in, The upper limit of the computing channel capacity for the optimized scheduling system of hydro-photovoltaic-storage is kW; The maximum energy storage capacity of the energy storage power station group is kW;

[0141] The calculated channel capacity limit is only for calculation and has no practical significance; the original "demand-adjustable" transmission channel capacity limit is replaced with the calculated channel capacity limit.

[0142] Step 3: Second Correction: Energy Storage Station Storage and Discharge:

[0143] In the "demand-adjustable" mechanism, if the daily demand is less than the adjustable capacity, the photovoltaic power can be fully absorbed through the first adjustment, eliminating the need to activate the energy storage station and reducing its operation and maintenance costs. If the daily demand exceeds the adjustable capacity, hydropower regulation alone cannot fully release the channel capacity, and some photovoltaic power will still exceed the absorption capacity. In this case, the energy storage station needs to be activated to participate in coordinated regulation, entering the second output adjustment phase. Specific adjustment measures include:

[0144] ;

[0145] in, For the first energy storage power station group Heavenly Hourly energy storage capacity, kW; The first optimized scheduling system for water, solar and energy storage Heavenly The output after the first correction in hours, kW; For the first energy storage power station group Peak hours Discharge power per hour Take a fixed value of 4, that is, concentrate on the first The system is fully discharged within 4 hours during the peak period of the day, and the discharge power remains constant during this period, kW; For the first energy storage power station group Daily energy storage capacity, kW·h; To calculate the duration, a fixed value of 1 hour is used. For the first energy storage power station group Maximum daily energy storage capacity, kWh;

[0146] Step 4: Calculate the amount of water and solar power wasted:

[0147] The total curtailment calculation for the hydro-solar-storage optimized scheduling system includes:

[0148]

[0149] in, The first optimized scheduling system for water, solar and energy storage Heavenly He always gave up light and effort when he was young; The first optimized scheduling system for water, solar and energy storage Heavenly The power output generated during the first correction process, in kW; The first optimized scheduling system for water, solar and energy storage Heavenly The amount of wasted solar power generated during the second correction process. The optimized scheduling system for hydro-solar-storage power. Heavenly Even after the first correction, the total output per hour still exceeds the upper limit of the calculation channel. The output will be classified as the first part of curtailed solar power output; during the second correction process, the output between the upper limit of the calculation channel and the upper limit of the transmission channel capacity will exceed the first part of the energy storage power station group. Maximum daily storage capacity That part will be classified as the second part of abandoned light and energy contribution;

[0150] The calculation of abandoned water and solar power includes:

[0151] ;

[0152] in, The amount of water and solar power wasted during the scheduling cycle is calculated as the amount of electricity wasted, i.e., penalty information, in kW·h; The total converted electricity from water wastage of the cascade hydropower station group within the scheduling cycle, obtained from the centralized control center of the hydropower-solar-storage optimization scheduling system, is expressed in kW·h.

[0153] The objective function of the constructed hydro-solar-storage intraday dynamic power output allocation model is:

[0154] ;

[0155] in, The total reward value for optimizing the scheduling system of hydro-solar-storage; This is the reward coefficient; The total power generation of the hydro-solar-storage optimized dispatch system within the dispatch cycle is the reward information, expressed in kW·h. This is the penalty coefficient; The amount of water and solar power wasted during the scheduling cycle is converted into the amount of electricity wasted, i.e., penalty information, in kW·h.

[0156] S4. Transform the joint scheduling model of the cascade hydropower station group and the intraday dynamic output allocation model of hydropower-solar-storage into a Markov process (MDP). Design the application layer of the hydropower-solar-storage optimized scheduling system, and use the A3C algorithm to solve the Markov process. This includes: constructing a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydropower-solar-storage optimized scheduling system that integrates the A3C algorithm. The specific steps are as follows:

[0157] Step 1: Transform the joint scheduling model of the cascade hydropower station group and the intraday dynamic output allocation model of hydropower-solar-storage into a Markov process, that is, transform the hydropower-solar-storage optimization scheduling problem into a Markov process (MDP), including the state space S, action space A, transition function P, and reward function R, specifically:

[0158] The construction of MDPs in reinforcement learning is as follows:

[0159] State space S:

[0160] The state space represents the reservoir capacity sequence of an annual regulating hydropower station within its dispatch cycle. The state before the next iteration is represented as follows: ,in This represents the daily reservoir capacity sequence of an annual regulating hydropower station, where the elements in the reservoir capacity must satisfy the following: , , These represent the minimum and maximum allowable storage capacity of the regulating reservoir during the annual scheduling cycle, respectively. The scheduling cycle is in days, typically 365 or 366 days;

[0161] Action Space A:

[0162] No. The action taken in the next iteration is represented as follows: ,in This represents the daily reservoir capacity change sequence of an annual regulating hydropower station. Elements in the reservoir capacity change sequence must satisfy the following: , The threshold for the amplitude of the movement. The scheduling period is in days, typically 365 or 366. The action space must maintain the same dimension as the state space. Initially, action space elements are randomly selected; subsequently, based on updates to the policy and value networks, action space elements are selected appropriately.

[0163] Transfer function P:

[0164] Each time it runs, enter the first... The state before the next iteration and the Actions taken in the next iteration Agent output number The state before the next iteration The next state is the state that is currently in the next state. The next state can be represented as... The change of state is briefly described as state and actions Add the corresponding elements in the equation to obtain a new state. The state changes can be directly observed and do not need to be solved using a transfer function;

[0165] Reward function R:

[0166] The reward function can be expressed as: The optimization objective of the hydro-solar-storage optimization scheduling model proposed in this invention is to achieve a balance between maximizing power consumption and minimizing water and solar power curtailment, thereby promoting the dual optimization of both quantity and quality.

[0167] Step 2: Design the application layer of the water-solar-storage optimization scheduling system based on the A3C algorithm. The specific structure is as follows: Figure 5 As shown, the Markov process is solved to obtain the optimal scheduling scheme; this includes: a local network and global network interaction mechanism, adaptive processing of the A3C algorithm, and the generation of an optimized scheduling scheme for a water-solar-storage optimized scheduling system that integrates the A3C algorithm. Details are as follows:

[0168] The application layer of the water-solar-storage optimization scheduling system based on the A3C algorithm consists of the following parts:

[0169] Worker:

[0170] Each Worker acts as an intelligent agent. Debugging revealed that setting the number of Workers too low significantly impacts computational efficiency; setting it too high can lead to excessive CPU load due to frequent thread switching, and also makes it difficult to adapt to power production. Therefore, considering both the algorithm's generalizability and computational efficiency, the number of Workers was set to 50% of the current CPU threads.

[0171] Actor Network:

[0172] The Actor network is structured for a continuous action space, consisting of an input layer, an intermediate layer, and an output layer. It balances "exploration" and "exploitation," making the output action distribution more inclined towards "high-reward actions."

[0173] Input status After passing through the intermediate layer, the average value of the output motion is calculated. and action variance Mean of movement For example, 365. ) Mean of actions; Variance of actions The variance of the 365-dimensional actions, and the action variance. The larger the value, the greater the randomness of the actions, that is, the higher the "exploration level";

[0174] Critic Network:

[0175] The Critic network predicts the "long-term cumulative reward expectation" under a given 365-dimensional reservoir capacity state sequence. Input status After passing through the intermediate layer, the long-term cumulative reward expectation is output. ;

[0176] The mechanism for building interaction between the local network and the global network is as follows:

[0177] Advantage function:

[0178] ;

[0179] in, Indicates "in state" Select action "The additional advantages compared to 'average level of movement'"; The action-value function, i.e., the state Select Action Expected future cumulative rewards; The state-value function represents the state value. The expected long-term cumulative reward;

[0180] The dominance function can be approximated by the timing difference error:

[0181] ;

[0182] in, For timing difference error, yes Approximate calculation; It is to perform an action The reward received later; It is a discount factor, representing the weight of future rewards; For state The expected long-term cumulative reward;

[0183] Optimize the Actor and Critic networks:

[0184] Simultaneously calculate the policy loss of the Actor and the value loss of the Critic to achieve synergistic optimization of the two;

[0185] Actor's policy loss: ,in Representative actions The logarithmic probability; This is an entropy regularization term to prevent the Actor from converging prematurely to a local optimum; the entropy coefficient... Control the intensity of "exploration". The entropy is a normal distribution. The value loss of the Critic: ;

[0186] The interaction between the local network and the global network is as follows:

[0187] After the loss values ​​of the Actor and Critic are added together, each Worker's local network calculates the gradient of the total loss with respect to all trainable parameters; these gradients are transmitted to the global network to update the global parameters, and then the local network synchronizes the latest parameters from the global network.

[0188] when When the value is greater than 0 (action advantage is positive), the parameters of the Actor network are updated to increase the probability of selecting this strategy in the future, while reducing the exploration of this strategy and strengthening its utilization; at the same time, it means that the Critic underestimates the value of the current state, so the parameters of the Critic network are updated to make the subsequent value prediction of this state closer to the actual reward.

[0189] In this way, the experience of multiple workers can be shared, avoiding repeated exploration of effective strategies and accelerating algorithm convergence; it also prevents a single worker from getting stuck in local optima, and each worker's exploration is always based on the "latest results of collective wisdom".

[0190] The A3C algorithm is adapted as follows:

[0191] Hardware compatibility:

[0192] The original A3C was mainly based on single-threaded / multi-threaded training on CPU. This program builds an asynchronous training architecture that combines GPU and CPU based on the classic A3C: the local network of the worker is computed using GPU, and the global network is shared using CPU. Compared with pure CPU training, the efficiency is improved by 3-5 times.

[0193] Reward standardization:

[0194] At the beginning of training, the reward value range can be too large, leading to gradient distortion. To address this issue, dynamic reward standardization was designed. Unlike normalization, which uses a fixed maximum and minimum reward value as the range, dynamic standardization adapts to the current reward distribution, ensuring that the standardized value is around [-1,1], resulting in smoother gradient updates.

[0195] Automatic resource cleanup mechanism:

[0196] In PyTorch multi-process training, if the GPU memory of each worker is not actively cleaned up, it will lead to "memory not released after process ends," ultimately causing "CUDA out of memory" errors in subsequent training. Calling the `synchronize()` function in the CUDA library ensures that the GPU's computation task is completed, and after the task is finished, the `empty_cache()` function is called to release unused cached GPU memory. This mechanism precisely solves the most common resource management problem in multi-process reinforcement learning training, effectively ensuring the continuity and availability of the hydro-photovoltaic-storage optimized scheduling system program in power production.

[0197] The specific operation steps of the hydro-solar-storage optimization scheduling system integrating the A3C algorithm are as follows:

[0198] Step 1: Initialize the Mytest environment: state space S, action space A, reward function R, termination condition MaxSteps;

[0199] Step 2: Construct a global network G and a local network of multiple Workers: Actor (outputs the average action value) and action variance ) and Critic (output state value) );

[0200] Step 3: Configure the SharedAdam optimizer and discount factor Entropy coefficient ;

[0201] Step 4: while the number of training iterations for multiple Workers has not reached MaxSteps:

[0202] Step 5: Each Worker interacts independently with its local environment: based on the current state. Output actions via local network Import the cascade hydropower station joint commissioning model and the intraday dynamic power output allocation model of hydropower, solar power, and storage to obtain the new state. ,award ;

[0203] Step 6: Maintain dynamic standardization of rewards in real time to mitigate training fluctuations;

[0204] Step 7: The local network calculates the loss, synchronizes the gradient to the global network, and pulls the latest parameters from the global network to update the local network;

[0205] Step 8: Release cached GPU memory resources after training is complete;

[0206] Step 9: Save the completed global network, plot the reward curve, and calculate the training time;

[0207] Step 10: Load the global network in an isolated environment and execute the complete process: based on the initial state. Output the reservoir capacity change plan of the annual regulating power station during the dispatch period;

[0208] Step 11: Input the annual reservoir capacity change scheme of the regulating power station into the cascade hydropower station group joint operation model and the daily dynamic power output allocation model of hydropower, photovoltaic and storage power stations to obtain hourly power output and water and solar power curtailment information of hydropower, photovoltaic and storage power stations.

[0209] Step 12: Write the output information to an Excel file and save it to a local path. In summary, this invention takes the hydro-solar-storage optimized scheduling system as the research object, constructs a three-layer collaborative hydro-solar-storage optimized scheduling system operation scenario of "sensing-network-application"; constructs a joint scheduling model of cascade hydropower station groups and a daily dynamic power output allocation model of hydro-solar-storage, simulating the operation scenarios and coupling relationships of hydropower, solar power, and storage power stations; finally, the hydro-solar-storage optimized scheduling problem is transformed into a Markov Decision Process (MDP), and the A3C algorithm is used to achieve efficient exploration and learning of strategies. This method can significantly improve the power consumption of the hydro-solar-storage optimized scheduling system, reduce the phenomenon of water and solar power curtailment, and ensure the power supply in Xinjiang.

[0210] To verify the effectiveness of the method described in this invention, four hydropower stations, seven photovoltaic power stations, and three energy storage power stations in the Yarkand River Basin were used as research examples for analysis. The specific engineering parameters of the above 14 power stations are detailed in Table 1.

[0211] Table 1. Engineering Parameters for Hydropower, Solar Power, and Energy Storage Power Stations

[0212]

[0213] Furthermore, to evaluate the optimization scheduling effect of the algorithm used in this invention, it is compared with the following algorithms:

[0214] (1) Particle Swarm Optimization (PSO): A swarm intelligence optimization algorithm that simulates the foraging behavior of bird flocks. Through information sharing and cooperation among individuals in the group, the particles continuously adjust their positions and speeds in the solution space to find the optimal solution.

[0215] (2) Sarsa algorithm: an online reinforcement learning algorithm based on temporal difference learning. It updates the action value function by learning the experience of "state-action-next state-next action" to achieve iterative optimization of the policy and finally find the optimal policy that maximizes the cumulative reward.

[0216] Table 2 explains the optimization scheduling metrics for the A3C, SARSA, and PSO algorithms.

[0217] Table 2. Description of Optimized Scheduling Indicators for A3C, SARSA, and PSO Algorithms

[0218]

[0219] This example analysis uses the average annual power generation in Table 1 as the comparison standard. Simultaneously, measured data from hydroelectric, solar, and energy storage power stations between June 1, 2023, and May 31, 2024 are used for simulation calculations using A3C, PSO, and SARSA. The algorithm parameters are set as follows: reward weight is 1, penalty weight is 0.5, and action amplitude (m... 3 The maximum number of training iterations is 12,000, with alternative values ​​of 3,000, 6,000, and 9,000. The discount factor is 0.9, the entropy regularization coefficient of A3C is 0.005, the upper limit of the transmission channel is 2.1 million kW, and alternative values ​​are 180 kW and 240 kW.

[0220] First, based on the characteristics of the Yarkand River Basin in Xinjiang, a three-layer collaborative water-solar-storage optimization scheduling system operation scenario of "sensing-network-application" is constructed.

[0221] Secondly, based on the characteristics of the Yarkand River Basin hydropower station group, a joint dispatch model for the cascade hydropower station group is constructed. Hydropower station B, as an annual regulating station with strong regulation capacity, can be used as the dispatch core. Its daily reservoir capacity change sequence within the dispatch cycle is used as input, with the start time of the impoundment period as the starting point of the dispatch cycle, and its initial water level set at 1770m. Basic parameters and predicted data such as characteristic water levels, output coefficients, power generation flow limits, output limits, "reservoir capacity-water level" curves, tailrace curves, head loss curves, and daily predicted runoff flow are obtained from the Yarkand River Basin Central Control Center. Based on this, a multi-constraint coupled calculation model is constructed through a series of formulas, outputting the daily output sequence of the cascade hydropower station group within the dispatch cycle and the converted wastewater volume, which is then uploaded to the hydro-solar-storage optimized dispatch system.

[0222] The next step is to construct a dynamic daily power output allocation model for hydropower, solar power, and energy storage, using a 24-hour scheduling cycle. Based on the operational data and constraints of the hydropower, solar power, and energy storage systems within the optimized scheduling system, a power output correction mechanism for these systems will be established. The daily power generation, total converted water curtailment, hourly power output of the cascade hydropower station group, and the upper limit of transmission channel capacity will be obtained from the optimized scheduling system within the scheduling cycle. The acquired data will be processed, and the hydropower output will be used as the baseload of the optimized scheduling system to obtain the pre-correction hydropower-solar power output curve.

[0223] Referring to the "Typical Power Load Curves of Provincial Power Grids" issued by the National Development and Reform Commission, and taking the power grid load in Xinjiang as an example, the logic for dividing the daily output time of the hydro-solar-storage optimized dispatch system is determined: the peak-valley difference of the typical daily load in this region reaches 10%, and the high load value range is concentrated between 10:00 and 22:00. Based on the load timing characteristics and photovoltaic (PV) output patterns, a typical day is divided into three core output periods: 00:00-10:00 and 22:00-24:00 are low-peak periods, during which the grid load is low and there is no PV output, and the hydropower-PV-storage optimized dispatch system mainly supplies power based on hydropower base load; 10:00-18:00 is the secondary peak period, which is the concentrated period of PV output (accounting for more than 90% of the total PV output of the day), and the superposition of hydropower and PV output may exceed the upper limit of the transmission channel capacity, leading to concentrated curtailment of PV power; 18:00-22:00 is the peak period, during which PV output gradually decreases to zero, while the grid load rises to its highest point of the day (reaching the peak load around 21:00), requiring reliance on hydropower and energy storage output to ensure load supply;

[0224] A "demand-adjustable" mechanism is introduced: To alleviate the curtailment of solar power caused by the combined output of hydropower and solar power during peak periods, a "demand-adjustable" mechanism is introduced. When the upper limit of the transmission channel capacity is fixed, the demand-adjustable output refers to the extent to which hydropower output needs to be actively reduced to absorb the excess solar power output within the limit of the transmission channel capacity; the adjustable output refers to the amount of hydropower output that can be increased within the limit of the transmission channel capacity. Under this mechanism, hydropower output is reduced during peak periods to reduce solar power curtailment; and hydropower output is increased during peak and off-peak periods to adapt to changes in grid load.

[0225] The following improvements have been made to the original "demand-adjustable" mechanism: a calculated upper limit for transmission channel capacity has been introduced. The upper limit for transmission channel capacity in the original "demand-adjustable" mechanism has been replaced with a calculated upper limit for transmission channel capacity.

[0226] In the "demand-adjustable" mechanism, if the demand-adjustable electricity is less than the adjustable electricity on a given day, all photovoltaic power can be consumed through the first correction output without the need to activate the energy storage power station, thus reducing the operation and maintenance costs of the energy storage power station. If the demand-adjustable electricity is greater than the adjustable electricity on a given day, the channel capacity cannot be fully released by hydropower regulation alone, and some photovoltaic power will still exceed the consumption capacity. At this time, the energy storage power station needs to be activated to participate in the coordinated regulation and enter the second output correction stage.

[0227] Wasted water and wasted solar power are converted into wasted electricity.

[0228] Then, based on the above-constructed intraday dynamic power output allocation model for hydro-solar-storage, an optimized scheduling method for the hydro-solar-storage optimization scheduling system in the Yarkand River Basin is obtained.

[0229] To verify the performance of the hydro-solar-storage collaborative optimization scheduling method, experiments were conducted on the Python platform. The hardware environment for the simulation calculation was as follows: CPU: Intel(R) Core(TM) i7-14650HX; GPU: RTX 5060.

[0230] Observing the training process, initially, due to differences in the exploration abilities of different Workers, the reward values ​​showed a relatively obvious differentiation, such as... Figure 6 The reward levels vary significantly at the initial stages of different curves. Figure 7 The initial phase also exhibited significant fluctuations and differentiation in rewards, indicating that each Worker fully utilized its exploration of the solution space in the early stages of training, trying different strategies to find a better path. As training progressed, whether at 6000 or 12000 rounds, the reward curve gradually stabilized and approached higher reward values. Figure 6 In the middle, after about 4000-6000 steps, the reward for each curve stabilizes at 8.0 or above; Figure 7 In the subsequent training rounds, the reward remained stable at a good level of 8.0 or above. This indicates that although the exploration strategies of each Worker differed in the initial stage, through continuous training and optimization, the hydro-optical-storage optimization scheduling system can effectively integrate the learning outcomes of each Worker, ultimately converging the reward to a relatively ideal range. This demonstrates good convergence performance and verifies that the method of allocating 7 Workers for individual training based on CPU and GPU performance ensures both exploration diversity in the early stages of training and stable and excellent convergence results in the later stages.

[0231] A3C, SARSA, and PSO were used as the core algorithms to solve the problem of optimal scheduling of water-photovoltaic-storage systems.

[0232] The training time varies significantly among different algorithms. The PSO algorithm's training time increases dramatically with the number of training iterations, reaching 3948.9 seconds after 12000 iterations. This is because PSO optimizes by iteratively searching the solution space for particles, requiring calculations of particle positions and velocities in each round, leading to a significant accumulation of computational overhead with each round. The Sarsa algorithm's training time increases relatively more gradually, taking 316.8 seconds after 12000 iterations. As a single-agent temporal difference algorithm, its computational logic is relatively simple, but it still experiences some computational accumulation with the number of training rounds. The A3C algorithm, on the other hand, demonstrates significant efficiency, requiring only 57.3 seconds after 12000 rounds. This is attributed to its multi-agent asynchronous parallel training mechanism, where multiple workers can simultaneously interact with the environment and update parameters, greatly improving training efficiency and effectively reducing overall training time, a particularly pronounced advantage with a higher number of training rounds.

[0233] Different algorithms showed varying performance in improving power output stability during wet and dry seasons. PSO and SARSA, both simple algorithms, yielded similar stability ratios but limited solution space exploration, achieving stability ratios of 0.033 and 0.02296 respectively after 12,000 iterations. The A3C (Asynchronous Advantage Actor-Critic) algorithm, however, demonstrated significant advantages. Its stability ratio continuously decreased from 0.002 at 3,000 training iterations to a mere 0.00129 at 12,000 iterations, exhibiting not only the lowest initial stability ratio but also the most significant downward trend with increasing training iterations. The A3C algorithm, with multiple workers undergoing independent training, greatly enhanced the exploration of the solution space, ultimately achieving the best results in optimizing power output stability during wet and dry seasons, effectively maintaining a stable output state across the two-year wet and dry season variations.

[0234] As the number of training iterations increased from 3,000 to 12,000, the power generation of all three algorithms showed an upward trend, indicating that each algorithm, through continuous training, could optimize the power generation capacity of the hydro-solar-storage optimized scheduling system. The A3C algorithm showed the most significant improvement in power generation. At 3,000 training iterations, its power generation was approximately 9.178 billion kWh. As the number of training iterations gradually increased to 6,000, 9,000, and 12,000, the power generation successively increased to 9.452 billion kWh, 9.627 billion kWh, and 9.724 billion kWh, maintaining a stable and rapid growth. While the PSO and SARSA algorithms also maintained growth in overall growth rate and final power generation, their results were significantly lower than those of the A3C algorithm. Furthermore, during training, it was found that the average total power generation of hydropower stations in the training sessions of A3C, SARSA, and PSO were 5.433 billion, 5.318 billion, and 5.851 billion kWh, respectively, with PSO algorithm showing a significant advantage in total power generation. The average total power generation of photovoltaic energy storage in the training sessions of A3C, SARSA, and PSO were 4.063 billion, 3.748 billion, and 3.030 billion kWh, respectively. A comprehensive comparison shows that during the training process, the A3C algorithm, leveraging its multi-threading advantage, actively explores power generation strategies to increase the absorption of photovoltaic energy storage, maximizing overall power generation. SARSA, as a single-agent temporal difference algorithm, faces relatively greater difficulty in balancing exploration and utilization. PSO relies on iterative search of particles in the solution space; in the complex power generation optimization problem of a hydro-photovoltaic-energy storage optimization scheduling system, its search efficiency and accuracy are inferior to A3C, resulting in a gap in power generation optimization performance.

[0235] The higher the percentage change in the proportion of energy stored in wasted light across the A3C, SARSA, and PSO algorithms at different training iterations, the better the energy storage utilization of wasted light. Overall, as the number of training iterations increases from 3000 to 12000, the proportion of energy stored in wasted light for all three algorithms increases, indicating that each algorithm can improve the level of energy storage utilization of wasted light to some extent after training. Specifically, the A3C algorithm shows the most significant increase in the proportion of energy stored in wasted light and consistently maintains a leading position. At 3000 training iterations, its proportion is 0.318, and subsequently increases to 0.339, 0.341, and 0.347 for 6000, 9000, and 12000 training iterations, respectively, showing a stable and continuous increase. While PSO and SARSA also show overall growth, their overall growth rate and final proportion are lower than those of the A3C algorithm. In summary, the A3C algorithm can efficiently explore the scheduling strategy space of the hydro-solar-storage optimization scheduling system, learn the optimal mode of coordinating curtailed solar power and energy storage more accurately, and demonstrate a better ability to utilize curtailed solar power as energy storage.

[0236] The annual power generation of the hydro-solar-storage optimized dispatch system is equivalent to the carbon dioxide emissions from a thermal power plant (unit: 10,000 tons). Overall, as the number of training iterations increases from 3,000 to 12,000, the carbon dioxide emission reductions corresponding to all three algorithms show an upward trend. The A3C algorithm shows the most significant and consistently high emission reduction. Under the "dual carbon" goal, vigorously developing clean energy is a key path to achieving this goal. As an important form of clean energy integration, the hydro-solar-storage optimized dispatch system should strive to reduce dependence on traditional thermal power, reduce carbon dioxide emissions, and promote the transformation of the energy system towards low-carbon and clean energy.

[0237] Increasing the upper limit of transmission channel capacity and the number of training sessions all promote the absorption of electricity by the hydro-solar-storage optimized dispatching system. In the example with a transmission channel capacity limit of 2.4 million kW and 12,000 training sessions, the hydro-solar-storage optimized dispatching system achieved a maximum electricity absorption of 9.936 billion kWh. The improvement of transmission channels has a more significant effect on the absorption of electricity by the hydro-solar-storage optimized dispatching system. This is mainly because increasing the upper limit of transmission channel capacity enhances the "potential" of electricity absorption by the hydro-solar-storage optimized dispatching system, thereby improving the overall absorption level. On the other hand, increasing the number of training sessions is a process of continuously exploring the "potential" of the hydro-solar-storage optimized dispatching system, continuously refining the operation plan, and improving the overall absorption level.

[0238] Example 2:

[0239] A collaborative optimization scheduling system for water, solar, and energy storage based on the A3C multi-agent parallel mechanism is also provided, including:

[0240] The scenario construction module is used to build the operation scenario of the three-layer collaborative hydro-solar-storage optimization scheduling system, which includes: Perception layer: used for comprehensive and real-time perception of key operating parameters of the hydro-solar-storage optimization scheduling system; Network layer: used to ensure low-latency transmission of data from the perception layer, realize stability monitoring and anomaly interception of data transmission, complete the initial cleaning, integration and storage of massive perception data, and provide standardized data interfaces for optimization calculations in the application layer; Application layer: with the hydro-solar-storage optimization scheduling system based on the A3C algorithm as the core, combined with the supporting power station information management system, it realizes fully automated scheduling from "data input - algorithm optimization - decision output", and finally outputs the reservoir capacity regulation plan of cascade hydropower stations, energy storage charging and discharging strategies and photovoltaic power consumption schemes, supporting the collaborative optimization operation of the hydro-solar-storage optimization scheduling system.

[0241] The first model construction module is used to construct a joint scheduling model for a cascade hydropower station group. It includes: taking the annual regulating hydropower station as the scheduling core of the cascade hydropower station group, and taking the daily reservoir capacity change sequence within its scheduling cycle as input; combining the basic parameters and prediction data of each power station in the cascade hydropower station group; outputting the daily power generation sequence of the cascade hydropower station group within the scheduling cycle and the amount of water wasted converted to wastewater, and uploading it to the hydropower-solar-storage optimization scheduling system.

[0242] The second model construction module is used to construct a dynamic daily power output allocation model for hydropower, solar power, and energy storage. This includes: using the daily scheduling cycle as the scheduling period, constructing a power output correction mechanism for hydropower, solar power, and energy storage based on the operational data and constraints of the hydropower, solar power, and energy storage optimized scheduling system; obtaining the daily power generation, total water wastage equivalent power, hourly power output of the photovoltaic power station group, and upper limit of the transmission channel capacity from the hydropower, solar power, and energy storage optimized scheduling system within the scheduling cycle; processing the obtained data, and using the hydropower output as the base load of the hydropower, solar power, and energy storage optimized scheduling system to obtain the pre-correction hydropower and solar power superposition output curve.

[0243] The model solving module is used to transform the joint scheduling model of cascade hydropower station groups and the intraday dynamic power output allocation model of hydropower-solar-storage into Markov processes. It designs the application layer of the hydropower-solar-storage optimized scheduling system and uses the A3C algorithm to solve the Markov process. This includes: building a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydropower-solar-storage optimized scheduling system that integrates the A3C algorithm.

[0244] Example 3:

[0245] An electronic device, the device comprising:

[0246] Memory containing executable program code;

[0247] A processor coupled to the memory;

[0248] The processor calls the executable program code stored in the memory to execute the above-described hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism.

[0249] The device may include: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to perform the steps of the method described in Embodiment 1.

[0250] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, memory. Such program modules include, but are not limited to, operating a water-optical-storage optimization scheduling system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in Embodiment 1 of this invention.

[0251] The processor executes various functional applications and data processing by running programs stored in memory, such as the method provided in Embodiment 1 of the present invention.

[0252] Example 4:

[0253] A computer-readable storage medium storing computer instructions, which, when invoked, are used to execute a water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism as described above.

[0254] The computer storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used or combined with an instruction to execute a water-optical-storage optimization scheduling system, apparatus, or device.

[0255] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in conjunction with a water-solar-storage optimization scheduling system, apparatus, or device, executed by instructions.

[0256] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0257] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0258] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the above-described method operations, but can also perform related operations in the methods provided in any embodiment of the present invention.

Claims

1. A water-light-storage collaborative optimization scheduling method based on an A3C multi-agent parallel mechanism, characterized in that, Includes the following steps: The system constructs a three-layer collaborative hydro-solar-storage optimization scheduling system operation scenario, comprising: a perception layer for comprehensive and real-time perception of key operating parameters of the system; a network layer for ensuring low-latency transmission of data from the perception layer, monitoring data transmission stability and intercepting anomalies, performing preliminary cleaning, integration, and storage of massive amounts of perception data, and providing standardized data interfaces for optimization calculations in the application layer; and an application layer centered on the A3C algorithm-based hydro-solar-storage optimization scheduling system, combined with a supporting power plant information management system, to achieve fully automated scheduling from "data input - algorithm optimization - decision output," ultimately outputting cascade hydropower station reservoir capacity regulation plans, energy storage charging and discharging strategies, and photovoltaic power consumption schemes, supporting the collaborative optimization operation of the hydro-solar-storage optimization scheduling system. A joint scheduling model for a cascade hydropower station group is constructed, including: taking the annual regulating hydropower station as the scheduling core of the cascade hydropower station group, and using the daily reservoir capacity change sequence within its scheduling cycle as input; combining the basic parameters and prediction data of each power station in the cascade hydropower station group; outputting the daily power generation sequence of the cascade hydropower station group within the scheduling cycle and the amount of water wasted converted to wastewater, and uploading it to the hydro-solar-storage optimization scheduling system; A dynamic power output allocation model for hydropower, solar power, and energy storage is constructed, including: using the day as the scheduling cycle, and based on the water, solar, and energy storage operation data and constraints of the hydropower-solar-storage optimization scheduling system, a power output correction mechanism for water, solar, and energy storage is constructed; the daily power generation, total water wastage converted into electricity, hourly power output of the photovoltaic power station group, and upper limit of transmission channel capacity are obtained from the hydropower-solar-storage optimization scheduling system within the scheduling cycle; the obtained data are processed, and the hydropower output is used as the base load of the hydropower-solar-storage optimization scheduling system to obtain the pre-correction hydropower-solar superimposed power output curve; The joint scheduling model of cascade hydropower station groups and the intraday dynamic output allocation model of hydro-solar-storage are transformed into Markov processes. The application layer of the hydro-solar-storage optimization scheduling system is designed, and the Markov process is solved using the A3C algorithm. This includes: constructing a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydro-solar-storage optimization scheduling system that integrates the A3C algorithm.

2. The A3C multi-agent parallel mechanism-based water-light-storage collaborative optimization scheduling method according to claim 1, characterized in that, The constraints of the constructed joint scheduling model for the cascade hydropower station group include: hydropower station water balance constraints, upstream and downstream hydraulic connections, water level limits, power generation flow limits, and hydropower station output constraints; the output is expressed as: ； ； in, For the first The total output of the tiered hydropower station group This refers to the number of power stations in a cascade hydropower station group. For the first The power output coefficient of a hydropower station For the first The first hydropower station Daily power generation flow For the first The first hydropower station The average water head of the day, The amount of electricity converted from water wastage during the scheduling cycle. To optimize the scheduling cycle in days for the hydro-solar-storage scheduling system, For the first The first hydropower station The daily water discharge rate To calculate the length of the time period.

3. The hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, Constructing a power output correction mechanism for water, solar, and energy storage, including: The typical daily power output periods of hydropower, solar power, and energy storage are divided into three periods: off-peak, secondary peak, and peak. During the off-peak period, the grid load is low and there is no solar power output. The hydropower-solar-storage optimized dispatch system mainly supplies power to the base load of hydropower. During the secondary peak period, solar power output is concentrated. During the peak period, solar power output gradually decreases to zero, while the grid load rises to its highest level of the day. It is necessary to rely on the output of hydropower and energy storage to ensure the load supply. The upper limit of transmission channel capacity in the demand-adjustable mechanism is replaced with the upper limit of calculated channel capacity, and the hydropower output during the hydro-solar-storage power generation period is corrected for the first time; the upper limit of calculated channel capacity is expressed as follows: ； ； ； in, To optimize the scheduling system for hydro-photovoltaic-storage systems, the upper limit of the computational channel capacity is set. To optimize the power transmission channel capacity limit of the hydro-solar-storage scheduling system, This represents the maximum energy storage capacity of the energy storage power station group. The first optimized scheduling system for water, solar and energy storage Heavenly Hourly output required The first optimized scheduling system for water, solar and energy storage Heavenly Adjustable output per hour To optimize the power transmission channel capacity limit of the hydro-solar-storage scheduling system, The first optimized scheduling system for water, solar and energy storage Heavenly Hours of effort; The total curtailment calculation for the hydro-solar-storage optimized scheduling system includes: ； in, The first optimized scheduling system for water, solar and energy storage Heavenly He always gave up light and effort when he was young; The first optimized scheduling system for water, solar and energy storage Heavenly The wasted power output during the first correction process in hours The first optimized scheduling system for water, solar and energy storage Heavenly The wasted power output during the second correction process; The calculation of abandoned water and solar power includes: ； in, This refers to the amount of water and solar power wasted during the scheduling cycle, which is then converted into the amount of electricity wasted, i.e., penalty information. The total converted electricity from water wastage by the cascade hydropower stations within the scheduling cycle is obtained from the centralized control center of the hydropower-solar-storage optimization scheduling system. To calculate the length of the time period; In the demand-adjustable mechanism, if the demand-adjustable electricity is less than the adjustable electricity on a given day, the photovoltaic power will be fully absorbed through the first power correction, eliminating the need to activate the energy storage station and reducing its operation and maintenance costs. If the demand-adjustable electricity is greater than the adjustable electricity on a given day, the energy storage station will be activated to participate in coordinated regulation, entering the second power correction phase. Specific correction measures include: ； in, For the first energy storage power station group Heavenly Hourly battery capacity The first optimized scheduling system for water, solar and energy storage Heavenly The output after the first correction in the hour For the first energy storage power station group Peak hours Discharge power per hour For the first energy storage power station group Daily electricity storage To calculate the length of the time period, For the first energy storage power station group Maximum daily energy storage capacity.

4. The water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The objective function of the constructed hydro-solar-storage intraday dynamic power output allocation model is: ； in, To optimize the total reward value of the hydro-solar-storage scheduling system, As the reward coefficient, The reward information is the total power generation of the hydro-solar-storage optimized dispatch system within the dispatch cycle. This is the penalty coefficient; The penalty signal is used to calculate the amount of water and solar power wasted during the scheduling cycle. To calculate the length of the time period.

5. The hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The joint scheduling model of cascade hydropower station groups and the intraday dynamic output allocation model of hydropower-solar-storage power plants are transformed into Markov processes, including: State space S: The state space is the reservoir capacity sequence of the annual regulating hydropower station within the dispatch cycle, the th... The state before the next iteration is represented as follows: ,in This represents the daily reservoir capacity sequence of an annual regulating hydropower station, where the elements in the reservoir capacity must satisfy the following: , , These represent the minimum and maximum allowable storage capacity of the regulating reservoir during the annual scheduling cycle, respectively. The number of days in the scheduling cycle; Action Space A: No. The action taken in the next iteration is represented as follows: ,in This represents the daily reservoir capacity change sequence of an annual regulating hydropower station. Elements in the reservoir capacity change sequence must satisfy the following: , The action space must be consistent with the state space dimension to set the action amplitude threshold. In the early stage of training, action space elements are randomly selected, and subsequently, action space elements are selected according to the updates of the policy network and the value network. Transfer function P: Each time it runs, enter the first... The state before the next iteration and the Actions taken in the next iteration Agent output number The state before the next iteration The next state is represented as... The change of state is briefly described as state and actions Add the corresponding elements in the equation to obtain a new state. Changes in state are directly observed. Reward function R: The reward function is expressed as , As the reward coefficient, The first optimized scheduling system for water, solar and energy storage Heavenly Hours of effort, To calculate the length of the time period, The penalty coefficient is... The purpose of the water-solar-storage optimization scheduling model is to calculate the amount of water and solar power wasted during the scheduling cycle, which is the penalty information. The optimization goal of the water-solar-storage optimization scheduling model is to achieve the coordination between maximizing power consumption and reducing water and solar power wasted, and to promote the dual optimization of quantity and quality.

6. The hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The application layer of the hydro-solar-storage optimization scheduling system is designed, including: Worker: Set the number of Workers as needed; Actor Network: The Actor network is structured for a continuous action space, consisting of an input layer, an intermediate layer, and an output layer. It balances "exploration" and "exploitation," making the output action distribution more inclined towards "high-reward actions." Input status After passing through the intermediate layer, the average value of the output motion is calculated. and action variance Mean of movement The mean of the multidimensional actions; the variance of the actions. For the variance of multidimensional actions, the action variance The larger the value, the greater the randomness of the actions, that is, the higher the "exploration level"; Critic Network: The Critic network predicts the long-term cumulative reward expectation under a multidimensional reservoir capacity state sequence. Input status After passing through the intermediate layer, the long-term cumulative reward expectation is output. .

7. The hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The mechanism for building interaction between the local network and the global network is as follows: Advantage function: ； in, Indicates the state Select action The additional advantage compared to average level of movement; The action-value function, i.e., the state Select Action Expected future cumulative rewards; The state-value function represents the state value. The expected long-term cumulative reward; The dominance function is approximated using time-series difference error: ； in, For timing difference error, yes Approximate calculation, It is to perform an action The reward received later It is a discount factor, representing the weight of future rewards; For state The expected long-term cumulative reward; Optimize the Actor and Critic networks: Simultaneously calculate the policy loss of the Actor and the value loss of the Critic to achieve synergistic optimization of the two; Actor's policy loss: ,in Representative actions The logarithmic probability; This is an entropy regularization term to prevent the Actor from converging prematurely to a local optimum; the entropy coefficient... Control the intensity of "exploration". The entropy is the normal distribution; the value loss of the Critic: ; The interaction between the local network and the global network is as follows: After the loss values ​​of the Actor and Critic are added together, each Worker's local network calculates the gradient of the total loss with respect to all trainable parameters; the gradient is transmitted to the global network to update the global parameters, and then the local network synchronizes the latest parameters from the global network. when When the value is greater than 0, the parameters of the Actor network are updated to increase the probability of selecting this strategy in the future, while reducing the exploration of this strategy and strengthening its utilization. At the same time, the Critic underestimates the value of the current state, so the parameters of the Critic network are updated to make the subsequent value prediction of this state closer to the actual reward.

8. The water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The A3C algorithm is adapted, including: Hardware compatibility: An asynchronous training architecture based on GPU-CPU collaboration is built on the classic A3C architecture: the local network of the worker is computed using GPU, while the global network is shared using CPU. Reward standardization: Dynamic reward standardization is adopted to adapt to the current reward distribution, ensuring that the standardized value is between [-1, 1]. Automatic resource cleanup mechanism: In PyTorch multi-process training, the synchronize() function in the CUDA library is called to ensure that the GPU's computing tasks are completed. After the tasks are completed, the empty_cache() cleanup function is called to release unused cached memory.

9. The hydro-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism according to claim 1, characterized in that, The following are the specific steps to generate an optimized scheduling scheme for a water-solar-storage optimized scheduling system that integrates the A3C algorithm: Initialize the Mytest environment: state space S, action space A, reward function R, and termination condition MaxSteps; Constructing a global network G and a local network of multiple Workers: Actor outputs the average action value. and action variance Critic outputs state value ; Configure the SharedAdam optimizer and discount factor. Entropy coefficient ; while the number of training iterations for multiple Workers has not reached MaxSteps; Each Worker interacts independently with its local environment: based on its current state. Output actions via local network Import the cascade hydropower station joint commissioning model and the intraday dynamic power output allocation model of hydropower, solar power, and storage to obtain the new state. ,award ; Real-time maintenance of dynamic standardization of rewards helps mitigate training fluctuations. The local network calculates the loss, synchronizes the gradient to the global network, and pulls the latest parameters from the global network to update the local network; Release cached GPU memory resources after training is complete; Save the completed global network, plot the reward curve, and calculate the training time; Load the global network in an independent environment and execute the complete process: based on the initial state. Output the reservoir capacity change plan of the annual regulating power station during the dispatch period; By incorporating the annual reservoir capacity change scheme of the regulating power station into the cascade hydropower station group joint operation model and the daily dynamic output allocation model of hydropower, photovoltaic and storage power stations, hourly output and water and solar power curtailment information of hydropower, photovoltaic and storage power stations are obtained. Write the output information to an Excel file and save it to a local path.

10. A hydro-solar-storage collaborative optimization scheduling system based on the A3C multi-agent parallel mechanism, characterized in that, include: The scenario construction module is used to build the operation scenario of the three-layer collaborative hydro-solar-storage optimization scheduling system, namely "sensing-network-application". It includes: Sensing layer: used for comprehensive and real-time sensing of key operating parameters of the hydro-solar-storage optimization scheduling system; Network layer: used to ensure low-latency transmission of data from the sensing layer, realize stability monitoring and anomaly interception of data transmission, complete the initial cleaning, integration and storage of massive sensing data, and provide standardized data interfaces for optimization calculations in the application layer; Application layer: with the hydro-solar-storage optimization scheduling system based on the A3C algorithm as the core, combined with the supporting power station information management system, to realize the fully automated scheduling from "data input-algorithm optimization-decision output", and finally output the reservoir capacity regulation plan of cascade hydropower stations, energy storage charging and discharging strategy and photovoltaic power consumption scheme, supporting the collaborative optimization operation of the hydro-solar-storage optimization scheduling system. The first model construction module is used to construct a joint scheduling model for a cascade hydropower station group. It includes: taking the annual regulating hydropower station as the scheduling core of the cascade hydropower station group, and taking the daily reservoir capacity change sequence within its scheduling cycle as input; combining the basic parameters and prediction data of each power station in the cascade hydropower station group; outputting the daily power generation sequence of the cascade hydropower station group within the scheduling cycle and the amount of water wasted converted to wastewater, and uploading it to the hydropower-solar-storage optimization scheduling system. The second model construction module is used to construct a dynamic daily power output allocation model for hydropower, solar power, and energy storage. This includes: using the daily scheduling cycle as the scheduling period, constructing a power output correction mechanism for hydropower, solar power, and energy storage based on the operational data and constraints of the hydropower, solar power, and energy storage optimized scheduling system; obtaining the daily power generation, total water wastage equivalent power, hourly power output of the photovoltaic power station group, and upper limit of the transmission channel capacity from the hydropower, solar power, and energy storage optimized scheduling system within the scheduling cycle; processing the obtained data, and using the hydropower output as the base load of the hydropower, solar power, and energy storage optimized scheduling system to obtain the pre-correction hydropower and solar power superposition output curve. The model solving module is used to transform the joint scheduling model of cascade hydropower station groups and the intraday dynamic power output allocation model of hydropower-solar-storage into Markov processes. It designs the application layer of the hydropower-solar-storage optimized scheduling system and uses the A3C algorithm to solve the Markov process. This includes: building a local network and a global network interaction mechanism, adapting the A3C algorithm, and generating an optimized scheduling scheme for the hydropower-solar-storage optimized scheduling system that integrates the A3C algorithm.

11. An electronic device, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when invoked, are used to execute a water-solar-storage collaborative optimization scheduling method based on the A3C multi-agent parallel mechanism as described in any one of claims 1-9.

Images