Reservoir water supply scheduling optimization method and system based on deep reinforcement learning

Abstract

The application is a reservoir water supply scheduling optimization method and system based on deep reinforcement learning, relating to the technical field of reservoir scheduling data management. Under the unified upstream and downstream topology and flood control-target / constraint framework, the step-by-step optimization and particle swarm optimization are combined to determine the staging and step structure, and the scheduling model is constructed by deep reinforcement learning. The continuous outflow generated by the model is shaped according to the step outflow rule and the pre-storage and pre-discharge stage division, and the executable boundaries (minimum step distance, maximum climbing rate, etc.) of each period are loaded respectively to generate few and stable step instructions. The executable check of the target requirements is implemented, and if it can be executed, it is issued and monitored throughout the process to complete the online self-adaptation of the scheduling constraints in terms of degree, energy consumption and deviation. If it cannot be executed, it forms a check feedback to drive rearrangement and re-solution. This process can significantly reduce small and high-frequency instructions, protect the gate pump, stabilize the hydraulic process, reduce energy consumption, and ensure that the strategy is interpretable.

Description

Technical Field

[0001] This invention relates to the field of reservoir scheduling data management technology, and in particular to a reservoir water supply scheduling optimization method and system based on deep reinforcement learning. Background Technology

[0002] A mature framework for reservoir water supply scheduling optimization based on deep reinforcement learning has been established. The system first collects state data such as reservoir water level, capacity, and inflow through the Internet of Things and monitoring stations, as well as demand data from downstream domestic, industrial, and agricultural sectors, and constraint data such as flood control and dead water levels. After preprocessing and feature engineering, a structured time-series dataset is formed. Then, the scheduling problem is transformed into a Markov decision process, defining a state space including real-time system parameters, an action space including water supply allocation, a multi-objective comprehensive reward function, and a state transition function based on water balance. Adaptive deep deterministic policy gradient models are constructed. The models are first trained offline based on historical data, and the parameters are optimized by combining experience replay and other mechanisms. After reaching the target, a real-time scheduling scheme is deployed to generate the scheme.

[0003] For example, the Chinese invention patent with announcement number CN117236478B discloses a multi-objective multi-reservoir scheduling optimization method based on Transformer-improved deep reinforcement learning. This method first obtains basic data on reservoir operation, and then establishes an optimization scheduling model for multi-objective reservoirs. The optimization objectives are to maximize power generation, minimize AAPFD value, and maximize water supply revenue during the scheduling period. Further, under constraints, the algorithm is optimized, including encoding, decoding, and training processes. Using average water year data as an example, the monthly power generation flow and water supply flow of each reservoir are optimized. Under the premise of ensuring the safe operation of each reservoir, the method increases power generation and water supply revenue while reducing AAPFD value.

[0004] For example, Chinese invention patent CN117787631A discloses a method for scheduling and risk warning of raw water systems based on deep reinforcement learning, which includes the following steps: establishing a Markov game model for minimizing the total energy consumption of water intake pumping stations and minimizing the sediment content of water intake, and designing the corresponding state space, action space and reward function; acquiring historical operation data and hydrological data, and constructing a scheduling environment model based on a preset hydraulic model and MLP neural network; training a deep reinforcement learning agent through the scheduling environment model and the agent PPO algorithm, and improving the generalization performance based on domain randomization to obtain the agent policy; constructing a seq2seq model, and making multi-step predictions of river sediment content and reservoir water supply based on historical operation data, and generating a risk warning model by combining the trained agent policy.

[0005] The above-mentioned technology has the following technical problems:

[0006] The current reservoir water supply scheduling model based on deep reinforcement learning is a closed-loop design centered around data input, state modeling, action output, and feedback optimization. It first collects reservoir water level and supply-demand data to construct a state space, then transforms the scheduling objective into a reward function. The optimal strategy is learned through models such as deep deterministic policy gradient, and finally, the water supply allocation instruction for a given time period is output. This process focuses on optimizing supply-demand balance and water resource utilization. When generating actions, the model is guided solely by the optimal scheduling objective, assuming that water supply can be infinitely subdivided and equipment can respond in real time. This results in output instructions being mostly small-step, high-frequency adjustments, causing a disconnect between the model's instructions and the actual operating capabilities of the field equipment. This leads to problems such as gate vibration and wear, pump thermal shock, protection tripping, and increased energy consumption. Summary of the Invention

[0007] To address the technical problem in existing technologies where output commands are mostly small-step, high-frequency adjustments, leading to a disconnect between model commands and the actual operational capabilities of field equipment, this invention provides a reservoir water supply scheduling optimization method and system based on deep reinforcement learning. The technical solution is as follows:

[0008] On the one hand, a reservoir water supply scheduling optimization method based on deep reinforcement learning is provided. This method includes: Step 1: Determining the upstream and downstream topological relationship of the reservoir, summarizing flood control objectives and constraints, and selecting stepwise optimization algorithm and particle swarm optimization algorithm as the basis for constructing the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning; Step 2: Performing stepwise processing on the continuous outflow variables output by the reservoir water supply scheduling optimization model based on the stepwise outflow rule, dividing the reservoir flood regulation process into several stages based on the pre-storage and pre-release rule, and configuring differentiated scheduling constraint rules for each stage; Step 3: Obtaining the target requirements output by the reservoir water supply scheduling optimization model, and performing an executability check on the target requirements. If it is determined to be executable, a target instruction is issued, and the execution process of the target instruction is monitored and evaluated, thereby adjusting the scheduling constraint rules of the reservoir water supply scheduling optimization model. If it is determined to be unexecutable, a verification feedback is generated.

[0009] On the other hand, a reservoir water supply scheduling optimization system based on deep reinforcement learning is provided. This system applies a reservoir water supply scheduling optimization method based on deep reinforcement learning. The system includes: a model building module, used to determine the upstream and downstream topological relationship of the reservoir, summarize the flood control objectives and constraints, and select stepwise optimization algorithm and particle swarm optimization algorithm as the basis for constructing the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning; a model constraint module, used to perform stepwise processing on the continuous outflow variables output by the reservoir water supply scheduling optimization model based on the stepwise outflow rule, divide the reservoir flood regulation process into several stages based on the pre-storage and pre-release rule, and configure differentiated scheduling constraint rules for each stage; and a model adjustment module, used to obtain the target requirements output by the reservoir water supply scheduling optimization model, and perform executability verification on the target requirements. If it is determined to be executable, the target instruction is issued, the execution process of the target instruction is monitored and evaluated, and the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted. If it is determined to be inexecutable, verification feedback is generated.

[0010] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0011] (1) This invention determines the phased and stepped structure by combining stepwise optimization and particle swarm optimization under a unified upstream and downstream topology and flood control-target / constraint framework, and constructs a scheduling model using deep reinforcement learning; the continuous outflow generated by the model is shaped according to the stepped outflow rules and the pre-storage and pre-release stages, and different executable boundaries (minimum step distance, maximum slope rate, etc.) are loaded for each stage to generate fewer and more stable step instructions; the feasibility of the target requirements is checked, and if it can be executed, it is issued and monitored throughout the process, and the scheduling constraints are adapted online by the completion degree, energy consumption and deviation; if it cannot be executed, a check feedback is formed to drive re-arrangement and re-solution. This process can significantly reduce small and high-frequency instructions, protect gate pumps, stabilize the hydraulic process and reduce energy consumption.

[0012] (2) This invention uses a phased loading and adaptively correctable set of executable boundary parameters, and prioritizes dam protection > peak shifting > pre-discharge, and uses a mechanism of interruptible cooling for safety events to automatically shape the continuous targets generated by the model into a small number of stable step instructions. This avoids small and frequent fine-tuning of the opening, reduces the impact and wear of gate opening and closing, reduces thermal shock and false tripping of pumping stations, and makes the process line smoother and reduces false alarms. The operational efficiency index only drives the boundary to tighten or loosen when the data update conditions are met, so that the boundary changes synchronously with the equipment status and hydraulic conditions. It will not set the executable boundary set too conservatively to affect the response, nor will it set the executable boundary set too sensitively to cause sawtooth jitter during the opening change process. It jointly verifies the upper limit of opening, downstream permission and head difference and other safety constraints to ensure that the energy consumption per unit water volume decreases, the start-up and shutdown losses are reduced, and the equipment life and water supply reliability are improved simultaneously. At the same time, the parameter-rule-instruction full-link traceability makes it easy to quickly switch to a more conservative and executable strategy in the case of ice period, power restriction or equipment degradation.

[0013] (3) In this invention, the completion rate is measured in real time during step execution. The abnormality is determined based on the number of mismatched steps. When the proportion of abnormality / warning is too high, the torque is first distinguished as abnormal. If abnormal, the speed is reduced globally (the ramp rate is uniformly reduced) and a warning is issued. Otherwise, the opening degree of the problematic gate is fixed and removed from the adjustable range. The remaining healthy gates are allocated according to their weights without exceeding the minimum step distance, maximum ramp rate, minimum duration and capacity limit, so as to maintain the total flow balance. On the one hand, the unreachable small step fine adjustment is automatically merged to suppress the sawtooth jitter caused by high frequency back-and-forth and commands, and significantly reduce the opening and closing impact and wear. On the other hand, under high-risk conditions such as ice season, the system can quickly derate and operate in a steady state to avoid triggering the protection trip due to excessive sensitivity and repeated command changes. At the same time, the execution overlap and interpretability are improved, and the energy consumption is lower and the risk is more controllable. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the reservoir water supply scheduling optimization method based on deep reinforcement learning provided in an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of the reservoir water supply scheduling optimization system based on deep reinforcement learning provided in an embodiment of the present invention;

[0017] Figure 3This is a schematic diagram of the peak shape of the pre-storage and pre-release rule model provided in an embodiment of the present invention;

[0018] Figure 4 This is a schematic diagram of the basic peak shape of the third-order flood control outflow provided in an embodiment of the present invention;

[0019] Figure 5 This is a schematic diagram of the basic peak shape of the fifth-order flood control outflow provided in an embodiment of the present invention;

[0020] Figure 6 This is a schematic diagram of the basic peak shape of the three-stage flood control outflow (without pre-discharge period) provided in an embodiment of the present invention;

[0021] Figure 7 This is a schematic diagram of the basic peak shape of the five-order flood control outflow (without pre-discharge period) provided in an embodiment of the present invention;

[0022] Figure 8 This is a schematic diagram of the basic peak shape of the nine-order flood control outflow (without pre-discharge period) provided in the embodiment of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0024] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0025] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0026] This invention provides a reservoir water supply scheduling optimization method based on deep reinforcement learning, as follows: Figure 1 The flowchart shown is for a reservoir water supply scheduling optimization method based on deep reinforcement learning. The processing flow of this method may include the following steps:

[0027] Step 1: Determine the upstream and downstream topological relationships of the reservoir, summarize the flood control objectives and constraints, and select the stepwise optimization algorithm and particle swarm optimization algorithm as the basis for constructing the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning.

[0028] In one example embodiment, this method uses the connectivity between the reservoir and the downstream water system as its framework. First, a trainable scheduling environment is established: the reservoir capacity-water level relationship H=L(V) is used to calculate between the reservoir capacity V(t) and the water level H(t). That is, given the reservoir capacity V (m³), it is converted into the corresponding water level H (m) through the function H=L(V), where V(t) is the reservoir capacity at time t and H(t) is the water level at time t. Water balance is established according to the inflow Q_in(t), the planned release Q_rel(t), the overflow Q_spill(t), and the ecological discharge Q_eco(t) at time t: V(t+Deltat)=V(t)+[Q_in(t)−Q_rel(t)−Q_spill(t)−Q_eco(t)]*Deltat, where Deltat is the sampling interval. External limits are defined by the downstream permissible flow rate Q_lim(t) and the safety control water level H_safe at time t. Abstracting this into a reinforcement learning problem: State(t) at time t = {H(t), V(t), Q_in(t), Q_lim(t), stage label, prediction window features…} (describing the hydraulics and constraints inside and outside the reservoir); Action(t) at time t = {total gate opening e(t) at time t, number of pump units n_pump(t) at time t, allocation coefficient beta_i(t) at time t, i = 1…m, and…} =1 (mapped to the relative flow rate over time and its distribution via hydraulic relationships).

[0029] To obtain a reasonably structured candidate solution, stepwise optimization is first used to determine the phases (pre-release / peak shift / dam protection), the number of steps, and their order. Then, particle swarm optimization (PSO) is used to perform global optimization on the continuous parameters (target water release q_k, duration, bridging duration, etc. for each step) within the phase. Based on this, the initial strategy and search boundary for reinforcement learning are given.

[0030] The strategy learning uses algorithms such as DDPG / PPO to minimize the weighted cost. The objective function J is: −ɑ×water supply revenue + ŋ×energy cost + Ψ×over-limit penalty + δ×smoothing penalty. Water supply revenue is obtained by summing the actual water supply of each water user × the time-sharing unit water value (or weight) according to the time period, and deducting the demand gap penalty and (non-flood season) water abandonment penalty. Energy cost is estimated by pump power and head. ɑ, ŋ, Ψ and δ are all weighting factors. Over-limit penalty is imposed for H(t)>H_safe or Q_rel(t)>Q_lim(t). Smoothing penalty uses total variation of actions to suppress high-frequency small steps. To ensure executability, actions are projected onto the executable boundary before being issued: minimum step size, minimum adjustable opening, maximum ramp rate (maximum allowable opening change per unit time), minimum duration, cooling time, opening and overflow limits, maximum allowable head difference, and available pump power collectively define the feasible domain; when icing / blockage or a decline in health occurs (denominated by a derating factor in the operational performance index), the executable boundary is dynamically tightened. This forms an integrated modeling process from topology and constraints to stepwise optimization, DRL learning strategies, and boundary projection into steps. All parameters have physical meaning and can be directly calibrated by monitored quantities or design values, approximating the global optimum while outputting a small number of stable and executable step commands.

[0031] Taking into account the scale of this project and the requirements for computational efficiency, the stepwise optimization algorithm and the particle swarm optimization algorithm were selected as the optimization algorithms for the project, as shown in Table 1, which compares the algorithms. Table 1 shows the descriptions, advantages and disadvantages of several algorithms.

[0032] Table 1 Algorithm Comparison Table

[0033]

[0034] Step 2: Based on the stepped outflow rule, the continuous outflow variables output by the reservoir water supply scheduling optimization model are processed into steps. Based on the pre-storage and pre-release rule, the reservoir flood regulation process is divided into several stages, and differentiated scheduling constraint rules are configured for each stage.

[0035] It should be explained that the improved model based on stepped outflow inevitably has large fluctuations in outflow after optimization. Therefore, we consider improving the outflow variable, that is, quantifying and merging the target changes within each sampling window, compressing multiple small changes into a small number of stable steps. The improved model based on pre-storage and pre-discharge rules aims to reduce the peak value of the restored flood at the control target station. The flood regulation process is divided into a pre-discharge period, a peak staggering period, and a dam protection period. Through certain rules, we achieve the goal of increasing the discharge during the pre-discharge period, reducing the outflow during the peak staggering period, and increasing the outflow during the dam protection period to quickly reduce the final water level of the flood regulation.

[0036] The two model improvements mentioned above directly address the pain points of overly fine-grained instructions, frequent changes, and mismatch with on-site constraints: First, based on the quantification and merging of stepped outflow, the continuous outflow, which is prone to large and high-frequency fluctuations after optimization, is shaped within each sampling window according to executable boundaries such as minimum step size, maximum ramp rate, minimum duration, and cooling. This compresses multiple small modifications into a few stable steps, significantly reducing the jaggedness of instruction frequency and amplitude, reducing opening and closing impacts and wear, and improving execution overlap. Second, based on the phased rules of pre-storage and pre-release, with the goal of reducing peak flow and lowering the final water level at the control station, the process is divided into a pre-release period (appropriately increasing outflow to create storage capacity), a peak-shifting period (reducing outflow to reduce peak flow under permissible conditions), and a dam-protection period (rapidly increasing outflow to ensure safety). Different executable boundaries and priorities are loaded according to each period, avoiding frequent minor adjustments to track the optimal level from a strategic perspective, while ensuring that safety and downstream permissibility are not exceeded. The combination of the two approaches not only transforms the calculated optimal values ​​into step-by-step instructions, but also achieves quantifiable improvements in peak window control, final water level drop, energy consumption, and operation and maintenance costs, thus resolving the contradiction between detailed instructions and unreachable on-site conditions from the source.

[0037] Specifically, based on the pre-storage and pre-release rules, the flood control process of the reservoir is divided into several stages, which include the pre-release period, the peak-shaving period, and the dam protection period.

[0038] The pre-release period is triggered under the following conditions: the reservoir water level has reached or exceeded the preset pre-release start water level, and the water level safety margin is greater than or equal to the defined water level safety margin. The duration of the above conditions is greater than or equal to the defined duration, and the reservoir status is not within the cooling time window. The defined water level safety margin is stored in the database and is a value used to determine whether the pre-release period or the dam protection period is triggered. The defined duration is stored in the database and refers to the maximum allowed duration.

[0039] Cooling-off window refers to a fixed period of time after a period switch or major step adjustment has just been completed, during which switching is prohibited again to prevent repeated switching / fine-tuning due to short-term fluctuations.

[0040] Based on practical requirements, the determination of the discharge flow process during the pre-release period must generally consider and meet the following conditions: (a) the water level must be lowered to the designated safe level within the specified pre-release period; (b) for safety, ease of operation, and feasibility, the reservoir water level should remain as stable and gradual as possible during the discharge process, avoiding steep rises and falls; (c) the discharge flow at any given moment cannot exceed the reservoir's maximum discharge capacity at that moment; (d) the maximum discharge flow during the discharge process should meet the downstream flood control requirements, i.e., it cannot exceed the maximum downstream inflow; (e) the maximum drop in reservoir water level per unit time should be within a reasonably permissible range. How to reasonably determine the discharge flow process during the pre-release period, while ensuring the above conditions, and thus scientifically guide and formulate reservoir scheduling plans, is of paramount practical significance. Traditional methods relying on experience or simple trial-and-error approaches are insufficient in terms of both accuracy and practicality.

[0041] The pre-release start water level is lower than the safety control water level.

[0042] During the pre-release period, the discharge is gradually increased; it's impossible to release a large flow rate at the beginning. The minimum water level for pre-release can be constrained by the lower limit of the dynamic control zone for flood control levels. The discharge process generally involves a gradual increase, or an increase followed by a period of stability.

[0043] The triggering conditions for the staggered peak period are as follows: the flood peak index is greater than or equal to the flood peak threshold. The flood peak index refers to the dimensionless comprehensive value obtained by normalizing and weighting the intensity (relative multi-year percentile / design value) of the inflow water and the change characteristics of the rise rate. The flood peak threshold refers to the maximum value allowed by the flood peak index. The downstream constraint margin is greater than or equal to the defined downstream constraint margin. The downstream constraint margin refers to the difference between the upper limit of the permissible flow at the control section and the planned outflow at the current time. The defined downstream constraint margin refers to the minimum value allowed by the downstream constraint margin. The duration of the above conditions is greater than or equal to the defined duration and the reservoir status is not within the cooling time window.

[0044] Based on actual conditions, a scientific and reasonable peak-shaving scheduling scheme is adopted, including optimized scheduling, real-time scheduling, and predictive scheduling. This ensures the safe operation of water conservancy projects and prevents accidents. Based on historical data and meteorological conditions, the inflow volume to reservoirs over a future period is predicted, and the timing and volume of upstream water inflow are reasonably forecasted to avoid excessively high reservoir water levels during peak periods. Advanced scheduling algorithms are employed to achieve optimal allocation and rational utilization of water resources.

[0045] If the downstream water inflow is large, the discharge needs to be gradually reduced to stagger the peak flow downstream. The discharge process is generally a gradual reduction, or a reduction followed by a constant flow. If the downstream water inflow is small, the dam protection period begins directly.

[0046] During the dam protection period, the triggering conditions are: the reservoir water level has reached or exceeded the preset safety control water level, or the water level safety margin is less than the defined water level safety margin, or a safety risk event occurs downstream or on the project side (such as the flow rate at the control section exceeding the permissible limit). The duration of the above conditions is greater than or equal to the defined duration and the reservoir status is not within the cooling time window.

[0047] After the peak flow period ends or the reservoir exceeds a certain water level, it is necessary to rapidly increase the reservoir discharge to ensure that the reservoir does not exceed its maximum water level. The discharge process is generally gradual or increases and then remains unchanged.

[0048] The defined values ​​and thresholds that appear in several stages are uniformly stored in the database in the form of a parameter library, forming a structured configuration table (fields include: parameter name, definition and calculation method, unit, default value and value range, effective object / stage, data source, version number, effective time, approver and remarks), and support the retention of historical versions. The formulation method follows the principles of security first, compliance constraints, data support, verifiability and rollback.

[0049] Figure 3 This is a schematic diagram of the pre-storage and pre-discharge rule model provided in this embodiment of the invention. The horizontal axis t represents time in minutes, which is the timeline of the scheduling process. The vertical axis q represents the outflow value from the reservoir to the downstream in m³ / min. T1 is the switching point from the pre-discharge period to the staggered peak period, and T2 is the switching point from the staggered peak period to the dam protection period. The stepped shape of the broken line represents the stepped outflow rule, which is not a continuous small-amplitude adjustment, but a small number of stable steps with sufficient duration. Adjacent steps must meet executable constraints such as minimum step distance, maximum slope rate, minimum duration, and cooling. Pre-discharge period: first emptying the reservoir. Step ① in the figure represents raising the outflow to a higher but controlled level to reserve reservoir capacity for subsequent peaks. Staggered peak period: reducing the superposition of downstream peaks. Step ② shows the downward adjustment of outflow (as low as possible within the downstream permissible range) to avoid downstream flood peaks. Dam protection period: ensuring the safety of the dam and reservoir area. Step ③ shows the increase of outflow to a higher level to quickly return to the final water level or eliminate the risk of exceeding limits.

[0050] In one example implementation, the rule execution logic is as follows:

[0051] (1) When the pre-release period, peak-shifting period, and dam protection period all exist;

[0052] The specific implementation method is shown in Table 2, which outlines the pre-storage and pre-release rules.

[0053] Table 2 Pre-storage and Pre-release Rules

[0054]

[0055] Figure 4 This is a schematic diagram of the basic peak shape of the third-order flood control outflow provided in an embodiment of the present invention. Figure 5This is a schematic diagram of the basic peak shape of the fifth-order flood control outflow provided in this embodiment of the invention. The horizontal axis t represents time in minutes, which is the timeline of the scheduling process. The vertical axis q represents the outflow value of the reservoir downstream in m³ / min. The core logic of the rule is as follows: at the switching times T1 and T2, the entire process is divided into three segments: the pre-release period, the peak-shaving period, and the dam protection period. A constant outflow value q is determined for each segment, so that it is within the allowable outflow value range of that period. Y_Qmin is the minimum allowable outflow value during the pre-release period, Y_Qmax is the maximum allowable outflow value during the pre-release period, C_Qmin is the minimum allowable outflow value during the peak-shaving period, C_Qmax is the maximum allowable outflow value during the peak-shaving period, B_Qmin is the minimum allowable outflow value during the dam protection period, B_Qmax is the maximum allowable outflow value during the dam protection period, and Tmax is the end time. If the value exceeds the limit, it is truncated at the boundary. Subsequently, a peak-shaped orderliness check is performed, forcibly satisfying the shape constraint that the outflow value in the middle section (peak-off period) is the lowest, and the outflow values ​​at both ends (pre-discharge period and dam protection period) are relatively higher. When the magnitude relationship between adjacent periods does not conform to this constraint, it is corrected by exchanging or adjusting the target values ​​of adjacent sections. After the above value selection and shape correction, the three steps are then arranged and smoothed according to the executable boundaries (including the verification of minimum step distance, maximum ramp rate, minimum duration, cooling time, and capacity / safety upper limit, and setting bridging transitions at points requiring rapid changes), ultimately forming a sequence of quantifiable step-shaped outflow commands. This process ensures that the outflow process not only meets the stage objectives and safety constraints, but also possesses a stable and executable engineering form.

[0056] (2) Only during the off-peak period and the dam protection period;

[0057] When the downstream peak value is relatively early, the peak-shifting period after reverse calculation may also be relatively early, resulting in no pre-release period. In this case, the reservoir will directly store water to shift the peak, and the implementation method is shown in Table 3, which shows the pre-storage and pre-release rules (no pre-release period):

[0058] Table 3. Pre-storage and Pre-release Rules (No Pre-release Period)

[0059]

[0060] Figure 6 This is a schematic diagram of the basic peak shape of the three-stage flood control outflow (without pre-discharge period) provided in an embodiment of the present invention. Figure 7 This is a schematic diagram of the basic peak shape of the five-order flood control outflow (without pre-discharge period) provided in an embodiment of the present invention. Figure 8This is a schematic diagram of the basic peak shape of the nine-order flood control outflow (without pre-discharge period) provided in this embodiment of the invention. The horizontal axis t represents time in minutes, which is the timeline of the scheduling process. The vertical axis q represents the outflow value of the reservoir downstream in m³ / min. The entire process is divided into only two segments: the peak-shaving period and the dam-protection period. The period before the switching time (denoted as T2) is the peak-shaving period, and the period after that until the end of the window Tmax is the dam-protection period. Each segment has a fixed target outflow q, and each must fall within the allowable outflow value range of that segment. If it exceeds the limit, it is truncated according to the boundary. Subsequently, the peak shape ordering of the two target values ​​is checked, and the shape constraint that the outflow value of the peak-shaving segment should not be higher than the outflow value of the dam-protection segment is enforced. If it is not satisfied, it is corrected by exchanging or adjusting the target values ​​of adjacent segments. After the above value selection and correction, the flow is then arranged and smoothed according to the executable boundaries (minimum step size, maximum ramp rate, minimum duration, cooling, and capacity / safety upper limit checks, and bridging transitions are set as needed), ultimately forming a stepped outflow command sequence containing only two segments. This demonstrates the steady-state, executable scheduling logic under conditions without pre-discharge, ensuring that even without the pre-discharge reservoir clearing stage, the goals of mid-term peak shaving and end-stage safety can still be achieved, while meeting engineering and safety constraints.

[0061] Specifically, the configuration of different scheduling constraint rules for each stage refers to: different executable boundary parameter sets for each stage; the executable boundary parameter set refers to the parameterized set of field equipment and safety constraints, which is used to convert the target requirements generated by the reservoir water supply scheduling optimization model into step instructions; the parameters in the executable boundary parameter set include, but are not limited to, minimum step distance, maximum gradient rate, and cooling time window.

[0062] When multiple conditions are met simultaneously, the priority of the dam protection period is higher than that of the peak-shaving period, which in turn is higher than that of the pre-release period. The designated rules are based on the escalating risks and costs and regulatory safety: the dam protection period is directly related to the structural safety of the dam and hard constraints such as flood control limits and control sections. Once these limits are exceeded, irreversible consequences such as overtopping, excessive structural stress, and energy dissipation failure may occur. Its legal and engineering responsibilities are the highest and must be prioritized. The purpose of the peak-shaving period is to reduce the downstream flood peak while meeting safety requirements. It is a secondary level of flood control and disaster prevention optimization and is subject to conditions such as downstream permissible flow rates. The pre-release period is a proactive reservoir capacity preparation and benefit optimization, with strong reversibility and time flexibility.

[0063] In the event of a safety risk incident, the cooling window can be interrupted and the dam protection period can be entered directly. However, such situations are irreversible or costly. If we continue to wait for the cooling period to end, we may miss the response window and cause serious consequences such as overtopping and excessive structural stress. Therefore, we must immediately switch to a conservative strategy: increase outflow, tighten minimum step distance and maximum ramp rate, and extend minimum duration and cooling, without exceeding capacity and downstream permitting conditions, in order to quickly bring the reservoir area back to a safe state and comply with engineering regulations and regulatory requirements.

[0064] This application first divides the flood control process into a pre-release period, a peak-shaving period, and a dam-protection period based on clearly defined thresholds. Then, it loads differentiated executable boundary parameter sets for each period, ensuring that the continuous objectives generated by deep reinforcement learning are stably suppressed into fewer and more stable step-by-step instructions before issuance. Simultaneously, by prioritizing the dam-protection period > peak-shaving period > pre-release period and using strong triggering rules for interruptible cooling of safety events, it ensures that a conservative strategy is immediately implemented when engineering risks arise, avoiding overstepping of limits, opening and closing impacts, and increased energy consumption due to waiting for cooling or frequent fine-tuning. As a result, the execution frequency is significantly reduced, the overlap of instruction completion is improved, equipment wear and false trips are reduced, and the entire process is parameterized. Compared with existing technologies, this solution no longer outputs a continuous curve solely guided by the optimal algorithm objective, nor does it rely solely on static rules based on empirical thresholds. Instead, it embeds the phased thresholds—executable boundaries—priority / interruption into the reinforcement learning closed loop, deeply coupling strategy optimization with on-site executability to achieve feasible scheduling.

[0065] Step 3: Obtain the target requirements output by the reservoir water supply scheduling optimization model and check the feasibility of the target requirements. If it is determined to be feasible, issue the target instruction, monitor and evaluate the execution process of the target instruction, and adjust the scheduling constraint rules of the reservoir water supply scheduling optimization model accordingly. If it is determined to be unfeasible, generate verification feedback.

[0066] Although the target requirements in this application may cover multiple variables such as outflow process, distribution ratio, pump start-up and shutdown and number of units, in order to directly address the core of on-site feasibility and safety constraints, this application focuses on the gate opening in the target requirements.

[0067] Specifically, the feasibility of the target requirements is checked. The specific check process is as follows: S31, analyze the execution status of the current instruction. If the execution is completed and the reservoir status is not within the cooling time window, proceed to S33; otherwise, proceed to S32. S32, check whether the priority of the stage specified in the target requirements is lower than the priority of the stage specified in the current instruction. If it is not lower, proceed to S33; if it is lower, do not issue the target instruction and continue executing the current instruction. S33, use a language framework (such as Python) to parse whether the differences between the gate openings in the target requirements and the corresponding gate openings are equal. If the gate opening is greater than or equal to the minimum step distance, then execute S34; otherwise, locate the abnormal gate opening, i.e., the difference between the gate opening and the current corresponding gate opening is less than the minimum step distance, mark it as an abnormal gate opening, and provide opening feedback. That is, by accurately transmitting the gate opening deviation that does not meet the minimum step distance constraint, the scheduling system helps to identify the conflict point between the instruction and the physical constraint of the equipment in a timely manner, and avoids invalid gate action or equipment damage due to blindly issuing instructions; S34, arrange the target requirements based on the executable boundary parameter set, issue the target instruction after the arrangement is completed, and monitor and evaluate the execution process of the target instruction.

[0068] This process filters unreasonable instructions layer by layer from three dimensions: execution timing, priority, and equipment constraints. This effectively avoids problems such as wear and tear and tripping of field equipment caused by instruction conflicts and over-constraint operation, ensuring stable equipment operation. Simultaneously, it ensures that scheduling instructions are issued only at appropriate times, improving instruction execution success rate and scheduling efficiency, and avoiding resource waste. Compared with existing technologies, its advantage lies in breaking through the limitations of traditional verification that only focuses on a single constraint (such as only checking equipment parameters). It constructs a full-chain verification system of timing-priority-equipment constraints-parameter orchestration, taking into account both the priority ranking of scheduling objectives and deeply integrating the actual operating status and physical boundaries of equipment. This makes scheduling instructions more secure and executable, achieving precise matching between scheduling optimization objectives and field equipment capabilities, rather than relying solely on the theoretically optimal output of the model.

[0069] Furthermore, the target requirements are orchestrated based on the executable boundary parameter set. The specific orchestration process is as follows: extract the minimum step distance and maximum ramp rate from the executable boundary parameter set; obtain the flow capacity of each gate and compare it with the reference flow capacity corresponding to the executable boundary parameter set, and mark the comparison result as a correction coefficient; construct a physical model or numerical model (such as a CFD fluid dynamics model) consistent with the actual gate and river channel scale to simulate the water flow state under different water levels and opening degrees, and calculate the actual gate flow capacity through experimental data. The flow capacity is essentially the volume of water flowing through the gate per unit time, so the volumetric flow rate unit is used, and cubic meters per second is preferred in engineering.

[0070] The correspondence between the executable boundary parameter set and the reference overcurrent capacity is based on the dual requirements of anchoring the legality of scheduling instructions and constructing a baseline for safe equipment operation. This ensures that the constraint effect of the parameter set on reservoir scheduling is both theoretically reasonable and practically instructive. The reference overcurrent capacity is a benchmark value determined by design standards, industry specifications, or equipment factory parameters. The executable boundary parameter set must be set in accordance with this benchmark value (e.g., matching the corresponding gate opening adjustment range according to the reference overcurrent capacity). If boundary parameters are defined independently of the reference overcurrent capacity, there may be a risk of compliance with the action but overcurrent exceeding the limit (e.g., the gate adjustment step distance meets the requirements, but the actual overcurrent corresponding to the opening is far beyond the safety threshold). Therefore, the correspondence between the two is a necessary prerequisite for ensuring that scheduling instructions are legal in both the dimensions of action feasibility and flow safety, and provides a unified benchmark framework for subsequent comparison with the actual overcurrent capacity and generation of correction coefficients.

[0071] If the correction factor is greater than 1, increase the minimum step size by multiplying the minimum step size by the correction factor. After correction, mark it as the target step size and the maximum gradient as the target gradient. If the correction factor is equal to 1, mark the minimum step size as the target step size and the maximum gradient as the target gradient. If the correction factor is less than 1, mark the minimum step size as the target step size and decrease the maximum gradient by multiplying the maximum gradient by the correction factor. After correction, mark it as the target gradient.

[0072] The core of this correction logic design is based on the deviation between the actual current-carrying capacity and the reference current-carrying capacity of the gate reflected by the correction coefficient. It is specifically adapted to the actual operating state of the equipment to solve the scheduling risk problem caused by the mismatch between general boundary parameters and individual equipment performance in the existing technology. When the correction factor is greater than 1, it indicates that the actual current-carrying capacity of the gate is better than the reference value. In this case, increasing the minimum step distance can expand the single adjustment range while ensuring current-carrying safety, avoiding frequent fine-tuning due to excessively small step distances and improving scheduling efficiency. When the correction factor is equal to 1, it means that the actual performance of the gate is consistent with the reference standard. There is no need to adjust the boundary parameters. The original minimum step distance and maximum ramp rate can be directly used to ensure that the scheduling instructions meet the normal safe operation requirements of the equipment. When the correction factor is less than 1, it means that the actual current-carrying capacity of the gate is weaker than the reference value (such as due to wear and aging leading to a decrease in current-carrying efficiency). If the original maximum ramp rate is still used for adjustment, there is a risk that the current will exceed the equipment's load-bearing limit in a short period of time. Therefore, the maximum ramp rate needs to be reduced to limit the rate of opening adjustment, while retaining the original minimum step distance to avoid excessive compression of adjustment accuracy. This prevents equipment damage caused by overcurrent exceeding the limit and ensures the executability of scheduling instructions, ultimately achieving a precise match between the boundary parameters and the actual safety performance of the equipment and avoiding operational hazards.

[0073] By using the target step distance and target ramp rate corresponding to each gate, the valve opening requirements in the target requirements are arranged into a gate opening sequence that conforms to the executable boundary.

[0074] Taking the scheduling scenario of the No. 1 agricultural water supply gate of a reservoir as an example, suppose the target requirement is to adjust the gate opening from the current 10% to 35%, and after prior adjustments, the target step distance is determined to be 5% / time, and the target ramp rate is 10% / hour (i.e., the total opening adjustment within each hour does not exceed 10%). When arranging the gate opening sequence, the total adjustment amount is first divided according to the target step distance: the gate needs to complete a 25% opening increase, which can be divided into 5 adjustments based on the target step distance of 5% / time (the single adjustment range is not less than the minimum executable step distance), each adjustment being 5%. Then, the adjustment rhythm is verified in conjunction with the target ramp rate: since the total adjustment within each hour cannot exceed 10%, if each adjustment is 5%, then an adjustment is performed every 30 minutes (once every 30 minutes, 5% each time, accumulating to 10% adjustment per hour, which does not exceed the ramp rate limit). The final gate opening sequence is as follows: 15% to 20% to 25% to 30% to 35% (adjusted once every 30 minutes). This sequence accurately achieves the target opening of 35% while strictly adhering to the constraints of the target step distance and target ramp rate. It avoids frequent actions due to excessively small single adjustment range or equipment shock caused by excessively fast adjustment rate, ensuring that each operation meets the gate's safe operating boundary. The minimum step distance is the lower limit of a single adjustment determined by the gate's physical structure, and the maximum ramp rate is the upper limit of adjustment per unit time determined by the equipment's operating performance.

[0075] Specifically, the arrangement of target requirements based on the executable boundary parameter set also includes the correction of the executable boundary parameter set. The specific correction process is as follows: obtain the operating efficiency index of the reservoir gate and compare it with the historical adjacent operating efficiency index; the operating efficiency index of the reservoir gate is the quantitative data obtained by analyzing the set of safety parameters collected within the time interval between the current instruction and the previous instruction; the historical adjacent operating efficiency index refers to the operating efficiency index obtained by analyzing the previous instruction and the current instruction as the time interval.

[0076] If a data update condition exists, the executable boundary parameter set is modified based on the reservoir gate's operational efficiency index to actively adapt the parameter constraints to the gate's performance degradation. If no such condition exists, there is no need to modify the executable boundary parameter set.

[0077] In one example embodiment, based on the reservoir gate operation efficiency index, the executable boundary parameter set needs to be corrected. This requires establishing a mapping logic linking the direction of efficiency index change with the direction of parameter adjustment. For instance, when the efficiency index decreases by 15% compared to historical values, the maximum ramp rate is reduced from 10% / hour to 7% / hour. This is because excessively rapid gate adjustment after performance degradation can lead to unstable flow and increased mechanical shock; reducing the rate can mitigate equipment overload risks. The target step distance is reduced from 5% / time to 3% / time. Due to decreased response accuracy, reducing the single adjustment amplitude can improve opening control accuracy and prevent further increases in head loss caused by overshoot. The allowable opening deviation threshold is increased from ±2% to ±3%. Because performance degradation makes it difficult for the gate to accurately reach the target opening, appropriately relaxing the deviation threshold can avoid frequent triggering of mismatch step markers, reduce unnecessary command corrections, and simultaneously match the current actual gate control capabilities, ensuring a smooth scheduling process.

[0078] The data update condition refers to the fact that the difference between the historical adjacent operating efficiency index and the operating efficiency index of the reservoir gate is less than the operating efficiency index threshold difference; where the operating efficiency index threshold difference is the minimum value allowed in the database to quantify the difference between the historical adjacent operating efficiency index and the operating efficiency index of the reservoir gate.

[0079] This correction logic effectively avoids the problems of frequent minor instruction overload and failure to identify key deviations caused by the disconnect between instructions and reality. It judges the gate's performance stability by using the difference in operational efficiency index, and only corrects the executable boundary parameter set when performance changes significantly. This avoids the scheduling model outputting a large number of fragmented, small-amplitude minor instructions due to frequent parameter adjustments, reducing mechanical wear caused by frequent start-ups, shutdowns, and fine-tuning. Simultaneously, it reduces the data processing load on the scheduling system and improves the consistency and efficiency of instruction execution. Relying on the difference comparison-condition triggering correction mechanism, the executable boundary parameters are always anchored to the gate's real-time operating performance, fundamentally reducing the problem of instructions deviating from reality. In practice, this avoids repeated deviations between instructions and execution results due to parameter lag, thereby reducing redundant minor instructions generated to correct deviations and ensuring that dispatch instructions focus on core water supply targets rather than getting caught in a cycle of deviation-correction-re-deviation; thirdly, by using clear data update conditions (no correction if the difference is less than a threshold), gate performance changes are transformed into quantifiable judgment criteria, avoiding the obscuring of key deviations (such as subtle anomalies in the early stages of equipment performance degradation) by too many minor instructions, enabling the dispatch system to more accurately identify significant changes in gate performance, promptly resolve potential risks through parameter correction, and ensure the stability and safety of reservoir water supply dispatch.

[0080] The operational efficiency index of a reservoir gate refers to the set of safety parameters related to the safety and feasibility of the reservoir gate collected during the safety monitoring period. The parameters are normalized and dimensionless according to a unified standard, and a weight vector is set according to the degree of influence of the parameters on safety and operational stability. The normalized parameters are then weighted and summarized, and the reciprocal of the summation result is the operational efficiency index of the reservoir gate. The safety monitoring period is the time interval between the current command and the previous command.

[0081] In one example embodiment, the set of safety parameters includes energy consumption per unit flow rate, head loss before and after the gate, and abnormal response rate. Energy consumption per unit flow rate refers to the electrical energy consumed to transport (or lift) a unit volume of water within the safety monitoring period, reflecting the energy efficiency level of the water supply, and is obtained by comprehensive analysis of data collected from electricity meters and flow meters respectively. Head loss before and after the gate refers to the specific energy difference between the upstream and downstream of the gate, reflecting local hydraulic loss and throttling intensity, and is associated with cavitation, vibration, and structural stress risks, and is obtained by analysis of data collected from upstream and downstream water level gauges. Abnormal response rate refers to the degree of abnormality in the actual opening of all gates reaching the expected value, characterizing the unreachability of commands and execution abnormalities, and is obtained from the position feedback of the hoist.

[0082] It needs to be explained that the head loss before and after the gate is the core correlation node. An increase in its value means that there is blockage, wear or deviation in the gate's flow channel, which will directly lead to an increase in the resistance when the water flows through the gate. In order to maintain the target flow rate, more electrical energy needs to be consumed to drive the gate adjustment or the water pump pressurization, thereby increasing the energy consumption per unit flow rate. At the same time, excessive head loss may be accompanied by water flow turbulence, which impacts the gate's mechanical structure (such as transmission components and limit devices), causing problems such as delayed arrival, overshoot or jamming after the gate receives the opening command, thus increasing the abnormal arrival response rate.

[0083] The core reason for choosing these three parameters to construct the safety parameter set is that the energy consumption per unit flow rate directly reflects the economic efficiency and equipment load of the gate operation, and can reveal the aging and inefficiency of the mechanical system; the head loss before and after the gate directly reflects the unobstructedness and flow stability of the flow channel, and is the core link between energy consumption and response performance, which can provide early warning of flow anomalies; the on-time response anomaly rate accurately characterizes the reliability of the gate's execution commands, and is directly related to the execution effect of the scheduling commands. The three parameters have a clear causal relationship, and can quantify the gate's safety and executability from different dimensions. By weighting and summarizing, the one-sidedness of single-parameter evaluation can be avoided, ensuring that the operational efficiency index can comprehensively and objectively reflect the overall operating status of the gate.

[0084] The operational efficiency index of reservoir gates is used to characterize the safe operation and performance of the gates under current working conditions.

[0085] Specifically, the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted. The specific adjustment process is as follows: If the duration of the abnormal proportion exceeding the limit is longer than the defined duration, it indicates that there is a systematic mismatch between the scheduling constraint rules and the overall equipment operation capacity of the reservoir, resulting in repeated deviations that cannot be eliminated by local adjustments. Therefore, based on the preset reduction range in the database, the target ramp rate of all gates in the reservoir is reduced, and the regulation rate requirement is reduced at the global level. This makes the constraint rules adapt to the actual performance of most gates, curbs the continuous spread of systematic deviations, avoids cumulative damage to equipment caused by long-term overload operation, and ensures the stability of overall scheduling.

[0086] Conversely, several warning gates are identified, indicating severe short-term execution deviations and insufficient stability. Continued participation in regulation may exacerbate flow fluctuations or trigger equipment risks. Their openings are then fixed, meaning the current measured gate opening is locked to a constant value. Several warning gates are removed from the adjustable set, and several abnormal gates are identified. The target ramp rate of these abnormal gates is restored to the minimum ramp rate. The deviation of the abnormal gates is relatively minor (the number of mismatch steps does not exceed the limit), but local execution fluctuations exist. The core reason may be that the current target ramp rate is slightly higher than its actual stable operating capacity. Restoring them to the minimum ramp rate (the lower limit of the safest and most stable regulation rate for the equipment) can enhance their operational stability by reducing the regulation rate, reducing subsequent deviations, while retaining their ability to participate in scheduling, avoiding excessive restrictions that affect the overall regulation flexibility.

[0087] Several normal state gates are located, and the flow gap is obtained by calculating the difference between the target total flow and the current actual adjustable total flow. Under the condition of satisfying the executable boundary parameter set, the flow gap is distributed to several normal state gates to form a new gate opening sequence, so that the total flow in the target command remains unchanged. With the flow gap filling as the goal and the executable boundary parameters as constraints, the adjustment potential of the normal state gates is quantitatively allocated. Specifically, firstly, based on the current opening degree, target step distance, target ramp rate, and flow capacity curve of the gates in normal state, the adjustable flow range of each gate (i.e., the range of additional flow that a single gate can handle under safety constraints) is constructed. Then, using linear programming or heuristic algorithms (such as genetic algorithms), with the minimization of the total flow gap as the objective function, the additional flow share that each gate in normal state needs to handle is calculated under the constraints that the adjustable flow range of each gate, the opening adjustment amplitude does not exceed the target step distance, and the unit time adjustment rate does not exceed the target ramp rate. Finally, based on the correspondence between flow and opening degree (such as by back-calculation through the flow capacity formula), the allocated flow is converted into a specific opening degree increment, which is embedded into the original adjustment sequence to form a new gate opening degree sequence, ensuring that the total flow is consistent with the target command and that each adjustment step meets the executable boundary.

[0088] By handling gates in different states in a tiered manner (isolating early warning gates, reducing speed of abnormal gates, and allocating gaps to normal gates), the interference of local equipment deviations on the overall scheduling can be precisely resolved. This avoids the inability to execute target instructions due to a single gate malfunction, ensuring the consistency between the total water supply flow and the scheduling target, and solving the problem of disconnect between instructions and execution. Differentiated strategies are adopted for different durations of abnormal exceeding limits (global speed reduction if timeout, local adjustment if no timeout). This can eliminate the risk of systemic constraint mismatch through global speed reduction, and preserve the adjustment efficiency of normal equipment through local optimization, reducing resource waste or safety hazards, and solving the problem of mismatch between constraint rules and actual equipment capacity. All adjustments are based on executable boundary parameters (such as step distance and ramp rate constraints when allocating gaps to normal gates), which can ensure that the newly generated opening sequence is always within the safe operating range of the equipment, preventing gate overload, overcurrent, and other faults caused by forcibly filling flow gaps, solving the problem of conflict between scheduling targets and equipment safety, and ultimately achieving a synergistic effect of achieving water supply targets, equipment safety and stability, and optimal scheduling efficiency.

[0089] Furthermore, the execution process of the target instruction is monitored and evaluated. The specific monitoring process is as follows: within a certain step distance, if the cumulative deviation between the actual climbing rate and the target climbing rate is greater than the defined deviation value, it is marked as a mismatch step, and the number of mismatch steps is counted. The defined deviation value refers to the maximum value allowed by the cumulative deviation value stored in the database. The acquisition of the cumulative deviation value needs to be achieved through three steps: real-time monitoring, hourly calculation of deviation, and summation. First, during the period when the gate performs a single step adjustment (e.g., within 30 minutes of adjusting by 5% step distance). The actual opening value is collected at a fixed time period (e.g., every 5 minutes). The actual climbing rate for each period is calculated (e.g., if the actual opening increases from 10% to 11% within 5 minutes, the actual climbing rate is 1% / 5 minutes). Next, the target climbing rate (e.g., the target climbing rate is 0.8% / 5 minutes) is subtracted from the actual climbing rate for each period to obtain the instantaneous deviation value for that period. Finally, the instantaneous deviation values ​​for all periods within the adjustment step are summed algebraically, and the result is the cumulative deviation value between the actual climbing rate and the target climbing rate.

[0090] If a gate does not have any mismatch steps during the execution of the target instruction, it is marked as a normal state gate; if a gate has mismatch steps during the execution of the target instruction, and the number of mismatch steps is less than or equal to the defined number of mismatch steps, it is marked as an abnormal state gate; if a gate has mismatch steps during the execution of the target instruction, and the number of mismatch steps is greater than the defined number of mismatch steps, it is marked as a warning state gate; the defined number of mismatch steps refers to the maximum allowed number of mismatch steps stored in the database.

[0091] The total percentage of abnormal and warning gates among all gates is calculated. If the total percentage exceeds the defined percentage, it is marked as an abnormal percentage exceeding the limit, and the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted. The defined percentage refers to the maximum allowed value of the total percentage of abnormal and warning gates among all gates stored in the database.

[0092] If the total proportion is less than or equal to the defined proportion value, the scheduling constraint rules of the reservoir water supply scheduling optimization model will not be adjusted.

[0093] Furthermore, the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted. This also includes optimizing the reservoir water supply scheduling optimization model after the target instruction is executed. The specific optimization process is as follows: obtain the execution consistency rate of the target instruction and compare it with the defined execution consistency rate; whereby the defined execution consistency rate refers to the lower limit value of the execution consistency rate stored in the database.

[0094] If the execution consistency rate of the target instruction is less than or equal to the defined execution consistency rate, then the shearing coefficient in the reservoir water supply scheduling optimization model is reduced. That is, the execution consistency rate is divided by the defined execution consistency rate and multiplied by the shearing coefficient, thereby reducing the shearing coefficient in the reservoir water supply scheduling optimization model and thus optimizing the reservoir water supply scheduling optimization model. If the execution consistency rate of the target instruction is greater than the defined execution consistency rate, then there is no need to optimize the reservoir water supply scheduling optimization model.

[0095] The execution consistency rate is a core indicator that quantifies the degree of agreement between target instructions and actual on-site execution results. It directly reflects the effectiveness of reservoir scheduling instructions from model output to equipment execution. First, key control parameters such as gate opening sequence and outflow rate are extracted from the target instructions as assessment benchmarks. Then, through an automated monitoring system, data such as actual gate opening and actual outflow rate at each assessment node are collected synchronously at the instruction time granularity. Subsequently, a deviation allowable threshold is set (e.g., opening deviation ≤ ±2%) to determine whether the actual value of each node is consistent with the target value. Finally, the execution consistency rate is calculated as the number of consistent execution nodes / the total number of assessment nodes × 100%.

[0096] In reservoir water supply scheduling optimization models, the shear coefficient is a core parameter used to constrain the aggressiveness of the model's output commands. Its main function is to limit and filter the initial scheduling schemes generated by the model (such as gate opening adjustment range, outflow rate changes, etc.), preventing the model from outputting aggressive commands that exceed the actual carrying capacity of the equipment due to excessive pursuit of theoretical optimization goals (such as maximizing water resource utilization). For example, when the model calculates based on supply and demand data that the gate opening needs to be increased from 20% to 60% within one hour, the shear coefficient will shear this adjustment range according to the equipment's safety boundary (such as a maximum ramp rate of 10% / hour), correcting it to a reasonable scheme that meets the constraints. Essentially, it balances the model's optimization goals with the operational feasibility of the regulating valve. When the target command execution consistency rate is less than or equal to the defined execution consistency rate, it indicates a large deviation between the model's output commands and the actual execution results. The core reason is usually that the current shear coefficient is set too wide, resulting in insufficient constraints on the model's aggressive commands, causing the commands to exceed the equipment's capacity and become unexecutable. Reducing the shearing coefficient at this point enhances its constraint on the initial model scheme, forcing the model to generate a more conservative scheduling scheme that is closer to the actual operating boundary of the equipment. This reduces the deviation between instructions and execution results, improves the consistency rate of subsequent target instructions, and achieves dynamic adaptation between model optimization goals and on-site executability.

[0097] It should be explained that this application only adjusts the constraints during operation, without directly modifying the objective function or arbitrarily rewriting the instruction curve. Appropriately tightening / relaxing the constraints can project the theoretical optimum into the field-executable optimum in real time without changing the optimization objective and strategy direction. This prioritizes ensuring that the limits are not exceeded and that there is no overload, and improves stability and lifespan by suppressing small high-frequency actions, thereby reducing energy consumption and the risk of false tripping.

[0098] Figure 2 This is a schematic diagram of the reservoir water supply scheduling optimization system based on deep reinforcement learning provided in an embodiment of the present invention. It includes a model building module, a model constraint module, a model adjustment module, and a database. The model building module is connected to the model constraint module, the model constraint module is connected to the model adjustment module, and the model building module, model constraint module, and model adjustment module are all connected to the database.

[0099] The database is used to store the parameters involved in the reservoir water supply scheduling optimization system based on deep reinforcement learning. It needs to be designed according to functional classification and association mapping principles, and then establish the relationship between modules (such as the historical matching records of the operation efficiency index and the corresponding constraint parameters, and the timestamp association between instruction data and execution feedback data). It also ensures that the data storage logic is adapted to the parameter calling and iterative optimization needs of the deep reinforcement learning model, so as to realize the orderly management and efficient retrieval of data.

[0100] The model building module is used to determine the upstream and downstream topological relationships of the reservoir, summarize the flood control objectives and constraints, and select stepwise optimization algorithm and particle swarm optimization algorithm as the basis for constructing the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning. The model constraint module is used to perform stepwise processing on the continuous outflow variables output by the reservoir water supply scheduling optimization model based on the stepwise outflow rule, and divide the reservoir flood regulation process into several stages based on the pre-storage and pre-release rule, configuring differentiated scheduling constraint rules for each stage. The model adjustment module is used to obtain the target requirements output by the reservoir water supply scheduling optimization model, and perform executability verification on the target requirements. If it is determined to be executable, the target instruction is issued, the execution process of the target instruction is monitored and evaluated, and the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted accordingly. If it is determined to be inexecutable, verification feedback is generated.

[0101] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A reservoir water supply scheduling optimization method based on deep reinforcement learning, characterized in that, The method includes: Step 1: Determine the upstream and downstream topological relationship of the reservoir, summarize the flood control objectives and constraints, and select the stepwise optimization algorithm and particle swarm optimization algorithm as the basis for constructing the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning. Step 2: Based on the stepped outflow rule, the continuous outflow variables output by the reservoir water supply scheduling optimization model are processed into steps. Based on the pre-storage and pre-release rule, the reservoir flood control process is divided into several stages, and differentiated scheduling constraint rules are configured for each stage. The stages include the pre-release period, the peak-shaving period, and the dam protection period. Step 3: Obtain the target requirements output by the reservoir water supply scheduling optimization model and check the feasibility of the target requirements. If it is determined to be feasible, issue the target instruction, monitor and evaluate the execution process of the target instruction, and adjust the scheduling constraint rules of the reservoir water supply scheduling optimization model. If it is determined to be unfeasible, generate verification feedback. The feasibility verification of the target requirements is carried out as follows: S31. Analyze the execution status of the current instruction. If the execution is completed and the reservoir status is not within the cooldown time window, proceed to S33; otherwise, proceed to S32. S32. Is the priority of the stage specified in the target requirement lower than the priority of the stage specified in the current instruction? If it is not lower, proceed to S33; if it is lower, do not issue the target instruction and continue to execute the current instruction. S33. Are the differences between the gate openings in the target requirements and the current gate openings all greater than or equal to the minimum step distance? If yes, then execute S34. If not, then locate the abnormal gate openings and provide opening feedback. S34. Arrange the target requirements based on the executable boundary parameter set, issue the target instructions after the arrangement is completed, and monitor and evaluate the execution process of the target instructions; The monitoring and evaluation process of the target instruction execution is specifically as follows: If the cumulative deviation between the actual climbing rate and the target climbing rate is greater than the defined deviation value within a certain step distance, it is marked as a mismatch step, and the number of mismatch steps is counted. If a gate does not have a mismatch step during the execution of the target command, it is marked as a normal gate. If a gate has a mismatch step during the execution of the target instruction, and the number of mismatch steps is less than or equal to the defined number of mismatch steps, it is marked as an abnormal state gate. If a gate has mismatch steps during the execution of the target command, and the number of mismatch steps is greater than the defined number of mismatch steps, then it is marked as a gate in a warning state. The total percentage of gates in abnormal and early warning states among all gates is calculated. If the total percentage is greater than the defined percentage, it is marked as an abnormal percentage exceeding the limit, and the scheduling constraint rules of the reservoir water supply scheduling optimization model are adjusted. If the total proportion is less than or equal to the defined proportion value, the scheduling constraint rules of the reservoir water supply scheduling optimization model will not be adjusted. The adjustment process for the scheduling constraint rules of the reservoir water supply scheduling optimization model is as follows: If the duration of the abnormal proportion exceeding the limit is longer than the defined duration, the target ramp rate of all gates in the reservoir will be reduced. Conversely, several early warning state gates are located and their openings are fixed. Several early warning state gates are removed from the adjustable set, several abnormal state gates are located, the target ramp rate of several abnormal state gates is restored to the minimum ramp rate, several normal state gates are located, the flow gap is obtained, and under the condition of satisfying the executable boundary parameter set, the flow gap is distributed to several normal state gates to form a new gate opening sequence, so that the total flow in the target command remains unchanged.

2. The method of claim 1, wherein, The reservoir flood control process is divided into several stages based on the pre-storage and pre-release rules. The specific division process is as follows: The pre-release period is triggered by the following conditions: the reservoir water level has reached or exceeded the preset pre-release start water level, and the water level safety margin is greater than or equal to the defined water level safety margin. The duration of the above conditions is greater than or equal to the defined duration and the reservoir status is not within the cooling time window. The triggering conditions for the staggered peak period are: the flood peak index is greater than or equal to the flood peak threshold, the downstream constraint margin is greater than or equal to the defined downstream constraint margin, the duration of the above conditions is greater than or equal to the defined duration, and the reservoir status is not within the cooling time window. The dam protection period is triggered by the following conditions: the reservoir water level has reached or exceeded the preset safety control water level, or the water level safety margin is less than the defined water level safety margin, or a safety risk event occurs downstream or on the project side. The duration of the above conditions is greater than or equal to the defined duration and the reservoir status is not within the cooling time window.

3. The reservoir water supply scheduling optimization method based on deep reinforcement learning according to claim 1, characterized in that, The differentiated scheduling constraint rules configured for each stage specifically refer to: Each stage corresponds to a different set of executable boundary parameters; The executable boundary parameter set refers to the parameterized set of field equipment and safety constraints, which is used to convert the target requirements generated by the reservoir water supply scheduling optimization model into step instructions that can be implemented on site. When multiple conditions are met simultaneously, the priority of the dam protection period is higher than that of the peak shifting period, and the priority of the peak shifting period is higher than that of the pre-release period. When a safety risk event occurs, the cooling-off period is interrupted, and the dam protection period begins immediately.

4. The reservoir water supply scheduling optimization method based on deep reinforcement learning according to claim 1, characterized in that, The orchestration process based on the executable boundary parameter set for the target requirements is as follows: Obtain the minimum step size and maximum ramp rate from the set of executable boundary parameters; Obtain the current capacity of each gate and compare it with the reference current capacity corresponding to the executable boundary parameter set, and mark the comparison result as a correction coefficient; If the correction factor is greater than 1, then increase the minimum correction step size. After the correction is completed, mark it as the target step size and mark the maximum grade rate as the target grade rate. If the correction factor is equal to 1, then the minimum step size is marked as the target step size, and the maximum gradeability is marked as the target gradeability. If the correction factor is less than 1, then mark the minimum step size as the target step size, reduce the correction maximum gradient, and mark it as the target gradient after the correction is completed; By using the target step distance and target ramp rate corresponding to each gate, the valve opening requirements in the target requirements are arranged into a gate opening sequence that conforms to the executable boundary.

5. The reservoir water supply scheduling optimization method based on deep reinforcement learning according to claim 4, characterized in that, The orchestration of target requirements based on the executable boundary parameter set also includes modifying the executable boundary parameter set. The specific modification process is as follows: Obtain the operational efficiency index of the reservoir gates and compare it with the historical operational efficiency index of adjacent gates; Determine if there are data update conditions. If so, adjust the executable boundary parameter set based on the reservoir gate's operational efficiency index. If it does not exist, there is no need to modify the executable boundary parameter set; The data update condition refers to the fact that the difference between the historical adjacent operating efficiency index and the operating efficiency index of the reservoir gate is less than the operating efficiency index definition difference. The operational efficiency index of the reservoir gate refers to the collection of safety parameters related to the safety and feasibility of the reservoir gate during the safety monitoring period. The parameters are then normalized and their dimensions are eliminated according to a unified standard. A weight vector is set according to the degree of influence of the parameters on safety and operational stability. The normalized parameters are then weighted and summarized. The reciprocal of the summation result is the operational efficiency index of the reservoir gate. The operational efficiency index of the reservoir gate is used to characterize the safe operation and execution efficiency of the gate under the current working conditions.

6. The reservoir water supply scheduling optimization method based on deep reinforcement learning according to claim 1, characterized in that, The adjustment of the scheduling constraint rules of the reservoir water supply scheduling optimization model also includes optimizing the reservoir water supply scheduling optimization model after the target instruction is executed. The specific optimization process is as follows: Obtain the execution consistency rate of the target instruction and compare it with the defined execution consistency rate; If the execution consistency rate of the target instruction is less than or equal to the defined execution consistency rate, then the shearing coefficient in the reservoir water supply scheduling optimization model is reduced, thereby optimizing the reservoir water supply scheduling optimization model. If the execution consistency rate of the target instruction is greater than the defined execution consistency rate, then there is no need to optimize the reservoir water supply scheduling optimization model.

7. A reservoir water supply scheduling optimization system based on deep reinforcement learning, employing the reservoir water supply scheduling optimization method based on deep reinforcement learning as described in any one of claims 1-6, characterized in that: include: The model building module is used to determine the upstream and downstream topological relationship of the reservoir, summarize the flood control objectives and constraints, and select the stepwise optimization algorithm and particle swarm optimization algorithm as the basis for building the reservoir water supply scheduling optimization model, thereby establishing a reservoir water supply scheduling optimization model based on deep reinforcement learning. The model constraint module is used to perform step-like processing on the continuous outflow variables output by the reservoir water supply scheduling optimization model based on the step-like outflow rules. Based on the pre-storage and pre-release rules, the reservoir flood regulation process is divided into several stages, and differentiated scheduling constraint rules are configured for each stage. The model adjustment module is used to obtain the target requirements output by the reservoir water supply scheduling optimization model and to check the feasibility of the target requirements. If it is determined to be feasible, the target instruction is issued, and the execution process of the target instruction is monitored and evaluated, thereby adjusting the scheduling constraint rules of the reservoir water supply scheduling optimization model. If it is determined to be unfeasible, a verification feedback is generated.

Images