An artificial intelligence-based inter-basin water transfer project optimal scheduling method

Abstract

The application discloses a kind of based on artificial intelligence's cross-basin water diversion engineering optimization scheduling method, belong to cross-basin water diversion optimization technical field.The problems that the traditional cross-basin water diversion engineering optimization scheduling method in prior art is difficult to generate optimal scheduling plan due to easy strategy deviation;The application collects the storage capacity, flow, operation constraint and historical scheduling record of each node of test area reservoir, forms global dynamic embedding, and constructs reservoir node and water flow pipeline into graph structure, generates global embedding matrix as state input, trains basic global strategy network, forms global water diversion strategy network, divides global water diversion strategy network into macroscopic layer and microscopic layer, jointly trains by sharing global embedding information, forms layered water diversion strategy, converts global multi-step return into short-term and medium-term weighted return, generates scheduling optimization strategy.The application improves the global optimization and fine control of cross-reservoir water diversion, and can be applied to cross-basin water diversion planning.

Description

Technical Field

[0001] This invention relates to the field of inter-basin water transfer optimization technology, specifically to an artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects. Background Technology

[0002] Inter-basin water transfer projects typically involve multiple reservoirs, water transfer pipelines, and complex upstream and downstream dependencies. Their operational status is influenced by various factors, including inflow processes, reservoir regulation methods, water conveyance capacity, and time-varying constraints. In actual scheduling, time delays in the flow process are common between reservoirs, and the response of downstream reservoirs to upstream scheduling operations is often not immediate. Furthermore, the storage and release characteristics of reservoirs exhibit significant nonlinearity, and water transfer operations are subject to multiple constraints such as safe water levels, ecological flow, and gate capacity. This results in a highly coupled and significantly delayed overall system dynamic. Traditional data-driven scheduling methods typically rely on local or short-term results as the basis for strategy adjustments during training. When using such traditional methods to train inter-basin water transfer systems, the overall state across the entire basin cannot be directly observed. The strategy can only be updated based on local feedback. Therefore, upstream and downstream response delays, water transition times at intermediate nodes, and constraint triggering factors are all mistakenly attributed by the model to improper scheduling decisions. After continuous training, the strategy will continuously shift to adapt to local immediate feedback, failing to maintain a correct judgment of the global true state, ultimately causing the strategy to gradually deviate from the global optimum.

[0003] In summary, there is a need for an artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects. Summary of the Invention

[0004] A brief overview of the invention is given below to provide a basic understanding of certain aspects of it. It should be understood that this overview is not an exhaustive summary of the invention. It is not intended to identify key or essential parts of the invention, nor is it intended to limit the scope of the invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description that follows.

[0005] In view of this, in order to solve the problem that traditional inter-basin water transfer project optimization scheduling methods in the prior art are prone to strategy deviations, making it difficult to generate optimal scheduling plans, this invention provides an artificial intelligence-based inter-basin water transfer project optimization scheduling method.

[0006] The technical solution is as follows: An artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects, comprising the following steps:

[0007] S1. Collect the water storage, flow, operational constraints and historical scheduling records of each reservoir node in the pilot area, and use the node embedding method to convert the state of each node into a high-dimensional vector to form a global dynamic embedding of the pilot area;

[0008] S2. Based on global dynamic embedding, the reservoir nodes and water flow pipelines are constructed as a graph structure, and a lightweight graph convolutional network is used to propagate information on node embedding to generate a global embedding matrix that reflects the coupling relationship of the reservoir.

[0009] S3. The global embedding matrix is ​​used as the state input of the basic global policy network to train the basic global policy network. Real-time data is introduced in actual operation for meta-reinforcement learning fine-tuning to form a global water diversion policy network that can adapt to environmental changes.

[0010] S4. The global water transfer strategy network is divided into a macro layer and a micro layer. The macro layer outputs the overall water transfer target across reservoirs based on the global embedding matrix. The micro layer generates specific operation actions for each reservoir based on the overall water transfer target across reservoirs and the real-time global embedding matrix. The macro layer and the micro layer share global embedding information for joint training to form a complete hierarchical water transfer strategy.

[0011] S5. Based on the hierarchical water transfer strategy, the global multi-step reward is compromised into a short-to-medium-term weighted reward, and the delay information and reservoir coupling constraints are incorporated into the reward calculation. The result is input into the global water transfer strategy network for gradient update, generating a global-hierarchical scheduling optimization strategy.

[0012] Furthermore, in S1, the real-time data collected from each node of the pilot reservoir is integrated and synchronously marked according to the time sequence;

[0013] The multidimensional state vector of each node is mapped to a high-dimensional feature space. The corresponding embedding vector is generated through a node embedding algorithm, and a temporal feature matrix is ​​constructed to maintain the dynamic change of node state over time. The embedding vectors of each node are fused to form a global dynamic embedding that can reflect the coupling and operational dependence of the reservoir in the pilot area.

[0014] node At time step Embedded vector Represented as:

[0015]

[0016] in, For nodes At time step The state vector contains water storage capacity, flow rate, operational constraints, and historical scheduling records. The weight matrix is ​​a learnable matrix. For learnable bias, For non-linear activation functions, superscript Indicates the number of iterations.

[0017] Furthermore, step S2 includes the following steps:

[0018] S21. Based on the global dynamic state embedding, set edge weights and construct a graph structure of reservoir nodes and water flow paths according to the upstream and downstream relationship of the reservoir and the watershed topology.

[0019] In step S21, each reservoir node within the pilot reservoir area is used as a graph structure. nodes This indicates that the water flow pipelines and water diversion channels between reservoirs will be used as the structural diagram. edge express;

[0020] Edge weights are set based on flow capacity or pipeline water carrying capacity, and a graph connectivity matrix is ​​established according to the upstream and downstream relationships of the reservoir and the watershed topology. Its elements are ,in, Let M be the set of real numbers, and M be the total number of nodes. For nodes To the node The water flow capacity of the pipelines between them This represents the largest pipeline water carrying capacity in the entire basin.

[0021] Label the nodes with operational constraints, historical scheduling records, and traffic limits, and output graph structure data. ;

[0022] S22. Iterate through the embedding vectors in the graph structure data, weight the results, and integrate them to generate a global embedding matrix of reservoir coupling relationships;

[0023] In step S22, the embedding vectors in the graph structure data are input into a lightweight graph convolutional network, and iteratively updated through multiple rounds of graph convolutional propagation to obtain the iteratively updated node vectors.

[0024] No. layer to the first The iterative update process of a layer is represented as follows:

[0025]

[0026] in, For nodes At time step No. Layer node vectors, For nodes The set of neighboring nodes, Let i be the degree of node i. For the first The weight matrix of the layer, For nodes At time step No. Layer node vectors;

[0027] The node vectors updated in each iteration are normalized, and the historical dynamics of the node states are modeled using time series encoding. A node feature weighting mechanism, i.e., an attention mechanism, is introduced, combining attention weights. Based on flow capacity, the importance of water diversion, and operational constraints, the nodes are... At time step No. Layer node vector Assign different weights, among which, To generate a weighted node vector for the total number of iterations. , ;

[0028] Attention weight Represented as:

[0029]

[0030] in, Constraints such as flow limits for nodes. These are learnable parameters for the attention mechanism. This indicates a feature concatenation operation. It is a function;

[0031] By integrating multiple iterations and all weighted node vectors, a global embedding matrix is ​​generated that includes reservoir node states, flow constraints, upstream and downstream dependencies, and pipeline coupling relationships.

[0032] Furthermore, step S3 includes the following steps:

[0033] S31. Input the global embedding matrix into the state encoding module of the basic global policy network, perform feature parsing and dimension regularization on it, and use the state constructed from the parsed state vector as the input to the basic global policy network for training, thereby obtaining the trained basic global policy network:

[0034] In S31, the status includes the adjustable water volume, discharge volume and cross-basin allocation instructions of each reservoir. The action constraint module combines reservoir safety restrictions, pipeline capacity and upstream and downstream relationships to screen executable actions.

[0035] The state-action value is calculated using a reinforcement learning algorithm. ,in, This represents the learnable parameters in the value network or policy network. Let be the state at time t. The action to be taken at time t, combined with the target value and loss function The parameters of the policy network are iteratively updated to obtain the trained basic global policy network.

[0036] Target value Represented as:

[0037]

[0038] in, As a discount factor, This represents the instantaneous reward at time t. This represents the state of the system at the next moment after performing the current action. Indicates the candidate action variable for the next moment;

[0039] loss function Represented as:

[0040]

[0041] in, This indicates calculating the expectation over multiple training samples. This represents the square of the prediction error;

[0042] The parameter update method is as follows:

[0043]

[0044] in, , This represents the gradient of the loss function with respect to the parameters during the policy network parameter update phase;

[0045] S32. Real-time status is obtained through data collection and transformation, and the trained basic global policy network is fine-tuned to form a global water diversion policy network;

[0046] In S32, real-time reservoir status data of the target area is collected during the actual operation phase and converted into an input format consistent with the global embedding matrix.

[0047] The real-time state is input into the trained base global policy network, and the adaptive loss is calculated on new data using a meta-reinforcement learning algorithm. formula Quickly update strategy parameters, among which, Based on the basic network parameters, The meta-learning rate, This represents the gradient of the loss function with respect to the parameters during the policy parameter update phase;

[0048] During the fine-tuning process, a real-time simulation environment is constructed using small-scale online samples to perform constraint consistency checks and upstream and downstream coupling verification on the action decisions updated by meta-learning.

[0049] Based on the trend of parameter changes after fine-tuning, the convergence of the policy is monitored, and the global water regulation policy network with completed meta-reinforcement learning fine-tuning is output.

[0050] Furthermore, step S4 includes the following steps:

[0051] S41. The macroscopic layer of the global water transfer strategy network is adopted, with the real-time global embedding matrix as input and the overall water transfer target across reservoirs as output.

[0052] In step S41, the macro layer receives the real-time global embedding matrix, analyzes the water status, flow changes, and coupling relationships of each reservoir node, and combines the weight information updated by the fine-tuned strategy network to comprehensively evaluate the upstream and downstream dependency structure, watershed supply and demand difference, pipeline water capacity, and operational constraints, and calculates the overall water transfer target vector across reservoir nodes. ,in:

[0053]

[0054] in, For time step Real-time global embedding matrix, For macro-level policy functions, For the total amount of water transferred across river basins, Assign a proportion to the nodes. Prioritize node scheduling;

[0055] S42. Using the micro-layer of the global water transfer strategy network, the overall water transfer target across reservoir nodes is input, and the specific operation actions of each reservoir node are generated by combining the real-time global embedding matrix.

[0056] S43. The macro-layer and micro-layer policy networks share a global embedding matrix. A joint loss function is used to simultaneously optimize the macro-layer and micro-layer. Gradient updates are used to improve the overall performance of the hierarchical policy. Multi-step simulations are performed on the water regulation actions generated during joint training to form a complete hierarchical water regulation policy.

[0057] In S43, the joint loss function Represented as:

[0058]

[0059] in, For the overall water diversion target error at the macro level, For micro-level motion constraint deviation, The loss of consistency between macro goals and micro actions , and These are the weighting coefficients.

[0060] Furthermore, in step S5, the delayed response, nonlinear constraints, and coupling relationships of each reservoir are incorporated into the reward function to calculate the immediate reward. ;

[0061] Instant rewards Represented as:

[0062]

[0063] in, For nodes At time step t, the current water storage level Let i be the target water storage capacity. For nodes To the node The actual scheduled traffic, , , This represents the corresponding penalty weight coefficient. This indicates a constraint penalty for exceeding the pipeline's water carrying capacity. This is a penalty for failing to meet the timing requirements of delayed responses from upstream and downstream suppliers.

[0064] The operational actions of the tiered water diversion strategy are calculated in time series to obtain multi-step returns, and the long-term target is compromised into a short-to-medium-term weighted return. ;

[0065] Short-to-medium term weighted returns Represented as:

[0066]

[0067] in, ], As a discount factor, For short- to medium-term time steps, For the future Instant rewards for each step;

[0068] Finally, the gradient descent learning optimization method is used to iteratively update the loss function of the hierarchical policy network, and the optimized global-hierarchical scheduling policy is output.

[0069] The beneficial effects of this invention are as follows: By mapping the state of each reservoir node to a high-dimensional vector and forming a global dynamic state embedding, combined with graph structure modeling of the coupling relationship between reservoirs, this invention achieves a comprehensive perception of the overall state of the reservoir system; by using a lightweight graph convolutional network to propagate node information, it can accurately capture upstream and downstream dependencies and nonlinear flow constraints, thereby improving the scheduling strategy's ability to understand complex coupling relationships; by training the basic global policy network and performing meta-reinforcement learning fine-tuning in actual operation, the strategy can quickly adapt to changing hydrological conditions and operating environments, achieving hierarchical optimization of cross-reservoir water transfer objectives and local operational actions; hierarchical collaborative training of macro and micro layers and incorporating delay information and coupling constraints into reward calculation help the strategy maintain stability and rationality under multi-step rewards, significantly improving the policy deviation problem caused by local feedback in traditional methods; overall, this method can achieve global optimization and fine control of cross-reservoir water transfer, significantly improving the reliability and scientific nature of water transfer decisions. Attached Figure Description

[0070] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0071] Figure 1 This is a flowchart illustrating an artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects. Detailed Implementation

[0072] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0073] refer to Figure 1 This embodiment describes an artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects, specifically including the following steps:

[0074] S1. Collect the water storage, flow, operational constraints and historical scheduling records of each reservoir node in the pilot area, and use the node embedding method to convert the state of each node into a high-dimensional vector to form a global dynamic embedding of the pilot area;

[0075] S2. Based on global dynamic embedding, the reservoir nodes and water flow pipelines are constructed as a graph structure, and a lightweight graph convolutional network is used to propagate information on node embedding to generate a global embedding matrix that reflects the coupling relationship of the reservoir.

[0076] S3. The global embedding matrix is ​​used as the state input of the basic global policy network to train the basic global policy network. Real-time data is introduced in actual operation for meta-reinforcement learning fine-tuning to form a global water diversion policy network that can adapt to environmental changes.

[0077] S4. The global water transfer strategy network is divided into a macro layer and a micro layer. The macro layer outputs the overall water transfer target across reservoirs based on the global embedding matrix. The micro layer generates specific operation actions for each reservoir based on the overall water transfer target across reservoirs and the real-time global embedding matrix. The macro layer and the micro layer share global embedding information for joint training to form a complete hierarchical water transfer strategy.

[0078] S5. Based on the hierarchical water transfer strategy, the global multi-step reward is compromised into a short-to-medium-term weighted reward, and the delay information and reservoir coupling constraints are incorporated into the reward calculation. The result is input into the global water transfer strategy network for gradient update, generating a global-hierarchical scheduling optimization strategy.

[0079] Furthermore, in S1, the real-time data collected from each node of the pilot reservoir is integrated and synchronously marked according to the time sequence;

[0080] The multidimensional state vector of each node is mapped to a high-dimensional feature space. The corresponding embedding vector is generated through a node embedding algorithm, and a temporal feature matrix is ​​constructed to maintain the dynamic change of node state over time. The embedding vectors of each node are fused to form a global dynamic embedding that can reflect the coupling and operational dependence of the reservoir in the pilot area.

[0081] node At time step Embedded vector Represented as:

[0082]

[0083] in, For nodes At time step The state vector contains water storage capacity, flow rate, operational constraints, and historical scheduling records. The weight matrix is ​​a learnable matrix. For learnable bias, For non-linear activation functions, superscript Indicates the number of iterations.

[0084] Specifically, in the time-series feature matrix, for nodes At time step The state vector is represented as:

[0085]

[0086] in, This refers to the water storage capacity (reservoir capacity). For inbound traffic, To generate traffic, For operational constraints, this specifically includes upper and lower limits of the reservoir's safe water storage capacity (i.e., , To constrain the lower limit of the safe water storage capacity (reservoir capacity), The upper limit of the safe water storage capacity (reservoir capacity) and the ecological flow constraint (i.e.) wait;

[0087] The original multidimensional state of a node and its coupling relationship are mapped and aggregated in the same expression, as shown in the formula:

[0088]

[0089] In the formula, Example 1 Each node at time... Embedded vector, For the first Each node at time... The original state vector is composed of water storage, flow rate, operational constraint parameters, and historical scheduling characteristics (which can be taken as...) in this embodiment. ), The feature mapping matrix, Bias vector;

[0090] A feature matrix is ​​constructed for time series data, where rows represent time steps and columns represent the state dimensions of each node. The dynamic changes in node state over time are preserved through normalization, standardization, and time series encoding.

[0091] The embedding vectors of each node are fused to form a global dynamic embedding that can reflect the coupling and operational dependence of the reservoir in the pilot area.

[0092] The embedding vectors of each node are fused according to the watershed topology. Weighted averaging, attention weighting, or graph convolution operations can be used to incorporate the upstream and downstream dependencies, pipeline coupling, and scheduling constraints between nodes into the fusion weights. The fused result forms the global dynamic state embedding matrix of the pilot area. The data structure is the number of nodes × embedding dimension × time step, which not only preserves the individual state information of each node, but also reflects the coupling relationship between reservoirs and the overall water transfer dependency.

[0093] Furthermore, step S2 includes the following steps:

[0094] S21. Based on the global dynamic state embedding, set edge weights and construct a graph structure of reservoir nodes and water flow paths according to the upstream and downstream relationship of the reservoir and the watershed topology.

[0095] In step S21, each reservoir node within the pilot reservoir area is used as a graph structure. nodes This indicates that the water flow pipelines and water diversion channels between reservoirs will be used as the structural diagram. edge express;

[0096] Edge weights are set based on flow capacity or pipeline water carrying capacity, and a graph connectivity matrix is ​​established according to the upstream and downstream relationships of the reservoir and the watershed topology. Its elements are ,in, Let M be the set of real numbers, and M be the number of nodes. For nodes To the node The water flow capacity of the pipelines between them This represents the largest pipeline water carrying capacity in the entire basin.

[0097] Label the nodes with operational constraints, historical scheduling records, and traffic limits, and output graph structure data. ;

[0098] S22. Iterate through the embedding vectors in the graph structure data, weight the results, and integrate them to generate a global embedding matrix of reservoir coupling relationships;

[0099] In step S22, the embedding vectors in the graph structure data are input into a lightweight graph convolutional network, and iteratively updated through multiple rounds of graph convolutional propagation to obtain the iteratively updated node vectors.

[0100] No. layer to the first The iterative update process of a layer is represented as follows:

[0101]

[0102] in, For nodes At time step No. Layer node vectors, For nodes The set of neighboring nodes, For nodes degree, For the first The weight matrix of the layer, For nodes At time step No. Layer node vectors;

[0103] The node vectors updated in each iteration are normalized, and the historical dynamics of the node states are modeled using time series encoding. A node feature weighting mechanism, i.e., an attention mechanism, is introduced, combining attention weights. Based on flow capacity, the importance of water diversion, and operational constraints, the nodes are... At time step No. Layer node vector Assign different weights, among which, To generate a weighted node vector for the total number of iterations. , Highlighting the coupled information of key nodes and high-risk traffic;

[0104] Attention weight Represented as:

[0105]

[0106] in, Constraints such as flow limits for nodes. These are learnable parameters for the attention mechanism. This indicates a feature concatenation operation. It is a function;

[0107] By integrating multiple iterations and all weighted node vectors, a global embedding matrix is ​​generated that includes reservoir node states, flow constraints, upstream and downstream dependencies, and pipeline coupling relationships.

[0108] Specifically, in this embodiment, the time series coding method introduces GRU (Gated Recurrent Unit) cells to calculate the nodes. At time step Node states encoded by time series ,in, Indicates a gating mechanism. Represents a node At time step No. Layer node vectors, For nodes At time step Node status encoded in time series;

[0109] First, the feature vector of each node in the aforementioned graph structure, including water storage capacity, flow rate, operational constraints, and historical scheduling information, is converted into a high-dimensional vector through a node embedding method and input into a lightweight graph convolutional network. In each iteration, the node receives information from its neighboring nodes and updates its own embedding vector through weighted summation or attention mechanism to reflect the coupling state of the upstream and downstream reservoirs. The number of iterations is set according to the scale of the reservoir network and the depth of information transmission, such as 3-5 rounds, to ensure that the influence of upstream and downstream can be fully propagated while controlling the amount of computation.

[0110] In this embodiment, the number of network layers, the number of attention heads, the number of message passing times, and the neighbor sampling strategy of the lightweight graph convolutional network are controlled. The specific steps are as follows:

[0111] Based on the network size and number of nodes, select an appropriate number of graph convolutional layers and attention heads to avoid excessive smoothing or gradient vanishing problems caused by overly deep networks. For example, a 2-3 layer GCN can be used, with 2-4 attention heads per layer; at the same time, neighbor nodes are sampled in each round of message passing to limit the maximum number of neighbors, which ensures information coverage while reducing computational complexity.

[0112] The node vectors updated in each iteration are normalized, and the historical dynamics of the node states are modeled using time series coding methods.

[0113] After each iteration, the node vectors are normalized to unify their dimensions, eliminating differences between different nodes and features. Then, the node vectors are arranged according to the time series, and the dynamic changes of node states over historical time are modeled using time series coding methods to reflect the nonlinear changes in reservoir storage and discharge over time and the delayed response characteristics of upstream and downstream areas.

[0114] A node feature weighting mechanism is introduced, assigning different weights to node embeddings based on flow capacity, water transfer importance, and operational constraints. This highlights the coupling information of key nodes and high-risk flows. Feature weights are applied to each node embedding vector, with the weights determined based on the node's key indicators in the whole basin water transfer. For example, nodes with large capacity reservoirs, inter-basin water transfer nodes, and high upstream-downstream connectivity have higher weights. At the same time, the node embedding vectors are dynamically weighted in conjunction with the current flow share, the frequency of operational constraint triggering, and historical scheduling risk events, in order to highlight the information of key nodes that have a significant impact on the global water transfer strategy.

[0115] By integrating the node vectors from multiple iterations and weighting, a global embedding matrix is ​​generated that includes the reservoir node status, flow constraints, upstream and downstream dependencies, and pipeline coupling relationships.

[0116] The node vectors after each iteration are integrated into a matrix structure according to the time step and node number, while retaining node features, edge features and weighting information, forming a global embedding matrix that can be input into the policy network. The matrix dimension is the number of nodes × feature dimension × number of time steps, which reflects the current state of each reservoir node, as well as the upstream and downstream dependence, pipeline coupling and historical dynamics, providing complete and computable global state information for subsequent policy network training.

[0117] Furthermore, step S3 includes the following steps:

[0118] S31. Input the global embedding matrix into the state encoding module of the basic global policy network, perform feature parsing and dimension regularization on it, and use the state constructed from the parsed state vector as the input to the basic global policy network for training, thereby obtaining the trained basic global policy network:

[0119] In S31, the status includes the adjustable water volume, discharge volume and cross-basin allocation instructions of each reservoir. The action constraint module combines reservoir safety restrictions, pipeline capacity and upstream and downstream relationships to screen executable actions.

[0120] The state-action value is calculated using a reinforcement learning algorithm. ,in, This represents the learnable parameters in the value network or policy network. The state at time t represents the overall operating state of the entire system at time t, which may include the water storage capacity, inflow rate, and outflow rate of each node. The action taken at time t, i.e., the current water diversion decision, may include the outflow rate, discharge rate, and water diversion priority of a certain node, and is based on feedback signals, constraint feedback, and changes in coupling state, combined with the target value. and loss function The parameters of the policy network are iteratively updated to obtain the trained basic global policy network.

[0121] Target value Represented as:

[0122]

[0123] in, As a discount factor, This represents the instantaneous reward at time t. This represents the state of the system at the next moment after performing the current action. Indicates the candidate action variable for the next moment;

[0124] loss function Represented as:

[0125]

[0126] in, This indicates calculating the expectation over multiple training samples. This represents the square of the prediction error;

[0127] The policy network parameter update method is as follows:

[0128]

[0129] in, For learning rate, This represents the gradient of the loss function with respect to the parameters during the policy network parameter update phase. This parameter update method is used to iteratively optimize the basic global policy network parameters during the training phase.

[0130] S32. Obtain the real-time state through collection and transformation, and fine-tune the trained basic global policy network;

[0131] In S32, real-time reservoir status data of the target area is collected during the actual operation phase and converted into an input format consistent with the global embedding matrix.

[0132] The real-time state is input into the trained base global policy network, and the adaptive loss is calculated on a small amount of new data using a meta-reinforcement learning algorithm. formula The strategy parameters are updated quickly. This parameter formula is used for rapid adaptive updates during actual operation. Based on the basic network parameters, The meta-learning rate, This represents the gradient of the loss function with respect to the parameters during the policy parameter update phase;

[0133] During the fine-tuning process, a real-time simulation environment is constructed using small-scale online samples to perform constraint consistency checks and upstream and downstream coupling verification on the action decisions updated by meta-learning.

[0134] Based on the trend of parameter changes after fine-tuning, the convergence of the policy is monitored, and the global water regulation policy network with completed meta-reinforcement learning fine-tuning is output.

[0135] Specifically, in S31, a set of action candidates is generated for each node, including water diversion volume, discharge volume, cross-basin allocation ratio and operation instructions. At the same time, in the action constraint module, the current water level safety threshold, the maximum water flow capacity of the pipeline, the gate opening range and the upstream and downstream dependencies are combined to select executable actions, ensuring that each action satisfies the local reservoir safety and follows the coupling relationship of the entire basin.

[0136] Each state-action pair is input into the value evaluation module. The state-action value function is calculated using a reinforcement learning algorithm. Combined with the reward signal, which includes constraints such as water balance index, upstream and downstream scheduling coordination degree, and pipeline load balance, the network parameters are forward propagated to calculate the loss. Then, the weights are updated through back propagation to achieve iterative optimization of the policy network.

[0137] Multi-scenario simulation environments are constructed using historical hydrological data, meteorological forecasts, and pipeline load information, including dry seasons, wet seasons, and extreme water inflow events. These simulations simulate water flow between reservoirs and upstream and downstream responses, execute policy network output actions, and update node states.

[0138] Important coupling state change sequences, sudden flow fluctuations, and constraint trigger records during training are cached in the experience replay pool. A certain proportion of data is randomly sampled and input into the network for each training session. Through multiple iterations, the stability of training is improved, while ensuring that the policy can learn the global scheduling rules under different hydrological and operational constraints.

[0139] During the training iteration, the action distribution and gradient change trend of the policy output are monitored in real time. When the action distribution is found to be too concentrated or the gradient update is unstable, the network parameters are calibrated by weight regularization or learning rate adjustment. After the training iteration is completed and the policy network parameters converge, the network structure and weights obtained by the final training are saved as the basic global policy network. The basic global policy network that has been trained can generate water regulation actions that can be performed by each reservoir with the global embedding matrix as input.

[0140] In S32, the reservoir storage capacity, inflow, outflow, gate opening and operation constraints of each reservoir are collected in real time through the reservoir field monitoring system and SCADA data interface. The sampling frequency can be set to 15 minutes or 1 hour. After the data is cleaned, outlier is removed and time series is labeled, it is mapped according to the node embedding dimension and time series format during the training of the basic global strategy network to form an input consistent with the global embedding matrix during the training phase.

[0141] The preprocessed real-time global embedding matrix is ​​input into the basic global policy network, and a meta-reinforcement learning algorithm is used to perform fast gradient updates based on the latest hydrological conditions and operational constraints. During the update process, only a small amount of recent time period data is used to reduce the risk of overfitting. The network parameters are adjusted through mini-batch gradient iteration, so that the policy can quickly adapt to the current environmental changes without destroying the original global scheduling rules.

[0142] Online samples from the fine-tuning phase are input into a lightweight simulation model to simulate water flow and pipeline constraints between reservoirs. The consistency of the strategy output actions is verified based on water level safety thresholds, gate flow capacity, and upstream-downstream dependencies. The simulation environment can be generated by combining historical flow sequences with current real-time data. By advancing through continuous time steps, the system checks whether the reservoir status after the action execution meets safety constraints and records upstream-downstream response lags and scheduling coordination indicators to provide feedback signals for fine-tuning.

[0143] During the fine-tuning iteration, the stability of the action distribution and gradient change trend of the monitoring strategy network output are monitored. When the change in action distribution and reward after several consecutive rounds of updates is lower than the preset threshold, the fine-tuning is determined to be converged. The finally updated network parameters are saved as a global water diversion strategy network, which can generate water diversion operations that can be performed by each reservoir based on the real-time global embedding matrix.

[0144] Furthermore, step S4 includes the following steps:

[0145] S41. The macroscopic layer of the global water transfer strategy network is adopted, with the real-time global embedding matrix as input and the overall water transfer target across reservoirs as output.

[0146] In step S41, the macro layer receives the real-time global embedding matrix, analyzes the water status, flow changes, and coupling relationships of each reservoir node, and combines the weight information updated by the fine-tuned strategy network to comprehensively evaluate the upstream and downstream dependency structure, watershed supply and demand difference, pipeline water capacity, and operational constraints, and calculates the overall water transfer target vector across reservoir nodes. ,in:

[0147]

[0148] in, For time step Real-time global embedding matrix, For macro-level policy functions, For the total amount of water transferred across river basins, Assign a proportion to the nodes. Prioritize node scheduling;

[0149] S42. Using the micro-layer of the global water transfer strategy network, the overall water transfer target across reservoir nodes is input, and the specific operation actions of each reservoir node are generated by combining the real-time global embedding matrix.

[0150] S43. The macro-layer and micro-layer policy networks share a global embedding matrix. A joint loss function is used to simultaneously optimize the macro-layer and micro-layer. Gradient updates are used to improve the overall performance of the hierarchical policy. Multi-step simulations are performed on the water regulation actions generated during joint training to form a complete hierarchical water regulation policy.

[0151] In S43, the joint loss function Represented as:

[0152]

[0153] in, For the overall water diversion target error at the macro level, For micro-level motion constraint deviation, The loss of consistency between macro goals and micro actions , and These are the weighting coefficients.

[0154] Specifically, the macro layer receives the embedded matrix output by the fine-tuned global water diversion strategy network through an interface. This matrix contains the current water storage, inflow, outflow, scheduling constraints, and upstream and downstream coupling information of each reservoir node. The macro module decodes the embedded vectors, restores the high-dimensional features to the key hydrological indicators of each reservoir, and identifies the dynamic dependencies and flow transmission characteristics between nodes based on time series encoding, providing complete system state information for overall water diversion decision-making.

[0155] The macro-level layer uses the weight parameters of the policy network to perform a weighted summation of the inputs of each node, forming a total water transfer demand forecast for each basin and inter-basin pipeline; at the same time, it considers upstream and downstream dependencies, real-time flow capacity, water storage safety level, ecological flow constraints and historical operation data to limit and correct the water transfer capacity of each reservoir; based on the comprehensive assessment, it calculates the total water transfer volume across reservoirs, the water transfer ratio allocated by basin and node, and the scheduling priority.

[0156] The time series coding module of the policy network is used to integrate multi-step hydrological forecast information into the embedding vector. The forward propagation is used to generate water storage change forecasts and flow scheduling schemes for each future time period. Considering the response lag between upstream and downstream and the nonlinear characteristics of the basin, the macro layer outputs the overall water transfer target across reservoirs in continuous time steps.

[0157] The micro-level layer receives macro-level water diversion targets and a global embedding matrix through an interface. This embedding matrix contains real-time water storage, inflow, outflow, and coupling relationship information for each reservoir. The micro-level module parses each reservoir node, decomposing the global target into the local adjustable water volume limit, discharge range, and executable operation time window. At the same time, it combines the reservoir's safe water level, gate capacity, pipeline water flow capacity, and upstream and downstream dependency constraints to provide a data foundation for generating local water diversion actions.

[0158] The parsed local state vector is input into the micro-policy network, which calculates the optimal operation action for each reservoir through forward propagation, including the specific water output, scheduling start and end times, and operation sequence. The micro-policy network combines historical scheduling records and multi-step report information to perform weighted optimization of the action, while preserving the dependency features between coupled nodes.

[0159] The micro-level performs constraint verification on the generated action sequence, including whether the water output exceeds the safe capacity, whether the scheduling time matches the pipeline water supply capacity, and whether the operation sequence conforms to the upstream and downstream dependency logic. For actions that do not meet the constraints, they are corrected through backtracking adjustment or approximate optimization algorithms to make the operation actions meet all safety and scheduling restrictions. Finally, the set of executable operation instructions for each reservoir is output.

[0160] During joint training, the generated global embedding matrix is ​​simultaneously input into the macro-layer and micro-layer policy networks. This embedding matrix contains the real-time water storage, flow changes, operational constraints, and upstream and downstream coupling relationships of each reservoir node. The macro-layer generates the overall water transfer target across reservoirs based on the global embedding, and the micro-layer generates the specific operational actions of each reservoir using the same embedding matrix. By sharing the embedding matrix, the consistency between the macro-target and the micro-action in terms of time, space, and constraints is ensured.

[0161] A joint loss function is defined, which includes the macro-level water diversion target error, the micro-level action constraint bias, and the multi-step reward trade-off index. The gradient of the macro-level output and the micro-level action generation error are calculated together, and the weight parameters of the macro-level and micro-level are updated using backpropagation. During the training process, multiple iterations are carried out in combination with changes in hydrological conditions, pipeline load levels, and upstream and downstream dependencies.

[0162] After each round of joint training, the cross-reservoir target output by the macro layer and the operational actions generated by the micro layer are input into the multi-step water transfer simulation module to simulate reservoir water storage changes, flow transmission, and pipeline load under different hydrological conditions and operational constraints. The joint strategy output is verified by recording the water transfer status of each node, upstream and downstream response delays, and constraint triggering. The strategy network parameters are adjusted according to the simulation results and iterated until the action sequence meets the constraint requirements in multiple scenarios, thereby forming a complete training hierarchical water transfer strategy.

[0163] Furthermore, in step S5, the delayed response, nonlinear constraints, and coupling relationships of each reservoir are incorporated into the reward function to calculate the immediate reward. ;

[0164] Instant rewards Represented as:

[0165]

[0166] in, For nodes At time step t, the current water storage level Let i be the target water storage capacity. For nodes To the node The actual scheduled traffic, , , This represents the corresponding penalty weight coefficient. This indicates a constraint penalty for exceeding the pipeline's water carrying capacity. This is a penalty for failing to meet the timing requirements of delayed responses from upstream and downstream suppliers.

[0167] The operational actions of the tiered water diversion strategy are calculated in time series to obtain multi-step returns, and the long-term target is compromised into a short-to-medium-term weighted return. ;

[0168] Short-to-medium term weighted returns Represented as:

[0169]

[0170] in, ], As a discount factor, For short- to medium-term time steps, For the future Instant rewards for each step;

[0171] Finally, the gradient descent learning optimization method is used to iteratively update the loss function of the hierarchical policy network, and the optimized global-hierarchical scheduling policy is output.

[0172] Specifically, in the action sequence generated by the stratified water transfer strategy, the water transfer volume, discharge volume, and cross-basin allocation operations of each reservoir are recorded at time steps, and the real-time return of each step is calculated by combining hydrological forecasts, upstream and downstream dependence, and pipeline capacity. A weighted compromise method is adopted to convert the long-term water transfer target into short- and medium-term returns according to the attenuation coefficient, and these returns are superimposed with the real-time returns to form a multi-step comprehensive return. The weight coefficient can be set according to the reservoir capacity ratio, basin importance, and scheduling cycle length.

[0173] The delayed response, nonlinear constraints, and coupling relationships of each reservoir are incorporated into the reward function, and the policy network parameters are updated through weighted updating.

[0174] Factors such as upstream and downstream response delays of the reservoir, nonlinear storage and release characteristics of water diversion operations, gate flow constraints, and pipeline water carrying capacity are incorporated into the reward function.

[0175] The reward function is expressed as:

[0176]

[0177] in, Let be the total reward value at time t. This is a water diversion revenue item, used to represent water supply satisfaction, power generation revenue, or water resource utilization efficiency. This is the delayed response cost term, which reflects the impact of the reservoir's delayed response to dispatch instructions; This is a constraint and coupling penalty term used to measure the degree of water level exceeding limits, flow exceeding limits, and imbalance in upstream and downstream coupling. These are the weighting coefficients for the overall reward value, the delayed response cost, and the constraint and coupling penalty, respectively.

[0178] The weighted coefficients reflect the contribution or penalty to the overall water diversion target; the comprehensive reward value is calculated for each action sequence, and the reward signal is fed back to the policy network. The weight parameters of the macro and micro layers are adjusted by gradient calculation to ensure that the network considers both the global objective and local constraints during the optimization process.

[0179] The hierarchical policy network is iteratively updated using the gradient descent learning optimization method, and the optimized global-hierarchical scheduling policy is output.

[0180] In each round of optimization, the calculated multi-step reward and weighted reward are used as the loss function input to the policy network, and the gradient is calculated through backpropagation. The network parameters are updated in batches using mini-batch training or time-series data sampling, and the learning rate is dynamically adjusted to control the convergence speed. During the iteration process, the action distribution, policy output error, and reward changes of each reservoir are recorded to ensure the stability of the training process and gradually optimize the feasibility of the stratified water diversion strategy under different hydrological and operational constraints. After the iterative training reaches the convergence condition, the weight parameters of the macro-layer and micro-layer policy networks and the optimal state of the joint loss function are saved. The final stratified water diversion strategy is then generated.

[0181] Although the invention has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. An artificial intelligence-based method for optimizing the scheduling of inter-basin water transfer projects, characterized in that, Includes the following steps: S1. Collect the water storage, flow, operational constraints and historical scheduling records of each reservoir node in the pilot area, and use the node embedding method to convert the state of each node into a high-dimensional vector to form a global dynamic embedding of the pilot area; S2. Based on global dynamic embedding, the reservoir nodes and water flow pipelines are constructed as a graph structure, and a lightweight graph convolutional network is used to propagate information on node embedding to generate a global embedding matrix that reflects the coupling relationship of the reservoir. S3. The global embedding matrix is ​​used as the state input of the basic global policy network to train the basic global policy network. Real-time data is introduced in actual operation for meta-reinforcement learning fine-tuning to form a global water diversion policy network that can adapt to environmental changes. S4. The global water transfer strategy network is divided into a macro layer and a micro layer. The macro layer outputs the overall water transfer target across reservoirs based on the global embedding matrix. The micro layer generates specific operation actions for each reservoir based on the overall water transfer target across reservoirs and the real-time global embedding matrix. The macro layer and the micro layer share global embedding information for joint training to form a complete hierarchical water transfer strategy. S5. Based on the hierarchical water transfer strategy, the global multi-step reward is compromised into a short-to-medium-term weighted reward, and the delay information and reservoir coupling constraints are incorporated into the reward calculation. The result is input into the global water transfer strategy network for gradient update, generating a global-hierarchical scheduling optimization strategy.

2. The method for optimizing the scheduling of inter-basin water transfer projects based on artificial intelligence according to claim 1, characterized in that, In step S1, the real-time data collected from each reservoir node in the pilot area is integrated and synchronously marked according to the time sequence. The multidimensional state vector of each node is mapped to a high-dimensional feature space. The corresponding embedding vector is generated through a node embedding algorithm, and a temporal feature matrix is ​​constructed to maintain the dynamic change of node state over time. The embedding vectors of each node are fused to form a global dynamic embedding that can reflect the coupling and operational dependence of the reservoir in the pilot area. node At time step Embedded vector Represented as: ； in, For nodes At time step The state vector contains water storage capacity, flow rate, operational constraints, and historical scheduling records. The weight matrix is ​​a learnable matrix. For learnable bias, For non-linear activation functions, superscript Indicates the number of iterations.

3. The method for optimizing the scheduling of inter-basin water transfer projects based on artificial intelligence according to claim 2, characterized in that, S2 includes the following steps: S21. Based on the global dynamic state embedding, set edge weights and construct a graph structure of reservoir nodes and water flow paths according to the upstream and downstream relationship of the reservoir and the watershed topology. In step S21, each reservoir node within the pilot reservoir area is used as a graph structure. nodes This indicates that the water flow pipelines and water diversion channels between reservoirs will be used as the structural diagram. edge express; Edge weights are set based on flow capacity or pipeline water carrying capacity, and a graph connectivity matrix is ​​established according to the upstream and downstream relationships of the reservoir and the watershed topology. Its elements are: ； in, Let M be the set of real numbers, and M be the total number of nodes. For nodes To the node The water flow capacity of the pipelines between them This represents the largest pipeline water carrying capacity in the entire basin. Label the nodes with operational constraints, historical scheduling records, and traffic limits, and output graph structure data. ; S22. Iterate through the embedding vectors in the graph structure data, weight the results, and integrate them to generate a global embedding matrix of reservoir coupling relationships; In step S22, the embedding vectors in the graph structure data are input into a lightweight graph convolutional network, and iteratively updated through multiple rounds of graph convolutional propagation to obtain the iteratively updated node vectors. No. layer to the first The iterative update process of a layer is represented as follows: ； in, For nodes At time step No. Layer node vectors, For nodes The set of neighboring nodes, For nodes degree, For the first The weight matrix of the layer, For nodes At time step No. Layer node vectors; The node vectors updated in each iteration are normalized, and the historical dynamics of the node states are modeled using time series encoding. A node feature weighting mechanism, i.e., an attention mechanism, is introduced, combining attention weights. Based on flow capacity, the importance of water diversion, and operational constraints, the nodes are... At time step No. Layer node vector Assign different weights, among which, To generate a weighted node vector for the total number of iterations. , ; Attention weight Represented as: ； in, Constraints such as flow limits for nodes. These are learnable parameters for the attention mechanism. This indicates a feature concatenation operation. It is a function; By integrating multiple iterations and all weighted node vectors, a global embedding matrix is ​​generated that includes reservoir node states, flow constraints, upstream and downstream dependencies, and pipeline coupling relationships.

4. The method for optimizing the scheduling of inter-basin water transfer projects based on artificial intelligence according to claim 3, characterized in that, S3 includes the following steps: S31. Input the global embedding matrix into the state encoding module of the basic global policy network, perform feature parsing and dimension regularization on it, and use the state constructed from the parsed state vector as the input to the basic global policy network for training, thereby obtaining the trained basic global policy network: In S31, the status includes the adjustable water volume, discharge volume and cross-basin allocation instructions of each reservoir. The action constraint module combines reservoir safety restrictions, pipeline capacity and upstream and downstream relationships to screen executable actions. The state-action value is calculated using a reinforcement learning algorithm. ,in, This represents the learnable parameters in the value network or policy network. Let be the state at time t. The action to be taken at time t, combined with the target value and loss function The parameters of the policy network are iteratively updated to obtain the trained basic global policy network. Target value Represented as: ； in, As a discount factor, This represents the instantaneous reward at time t. This represents the state of the system at the next moment after performing the current action. Indicates the candidate action variable for the next moment; loss function Represented as: ； in, This indicates calculating the expectation over multiple training samples. This represents the square of the prediction error; The parameter update method is as follows: ； in, For learning rate, This represents the gradient of the loss function with respect to the parameters during the policy network parameter update phase; S32. Real-time status is obtained through data collection and transformation, and the trained basic global policy network is fine-tuned to form a global water diversion policy network; In S32, real-time reservoir status data of the target area is collected during the actual operation phase and converted into an input format consistent with the global embedding matrix. The real-time state is input into the trained base global policy network, and the adaptive loss is calculated on new data using a meta-reinforcement learning algorithm. formula The strategy parameters are updated quickly. This parameter formula is used for rapid adaptive updates during actual operation. Based on the basic network parameters, The meta-learning rate, This represents the gradient of the loss function with respect to the parameters during the policy parameter update phase; During the fine-tuning process, a real-time simulation environment is constructed using small-scale online samples to perform constraint consistency checks and upstream and downstream coupling verification on the action decisions updated by meta-learning. Based on the trend of parameter changes after fine-tuning, the convergence of the policy is monitored, and the global water regulation policy network with completed meta-reinforcement learning fine-tuning is output.

5. The method for optimizing the scheduling of inter-basin water transfer projects based on artificial intelligence according to claim 4, characterized in that, S4 includes the following steps: S41. The macroscopic layer of the global water transfer strategy network is adopted, with the real-time global embedding matrix as input and the overall water transfer target across reservoirs as output. In step S41, the macro layer receives the real-time global embedding matrix, analyzes the water status, flow changes, and coupling relationships of each reservoir node, and combines the weight information updated by the fine-tuned strategy network to comprehensively evaluate the upstream and downstream dependency structure, watershed supply and demand difference, pipeline water capacity, and operational constraints, and calculates the overall water transfer target vector across reservoir nodes. ,in: ； in, For time step Real-time global embedding matrix, For macro-level policy functions, For the total amount of water transferred across river basins, Assign a proportion to the nodes. Prioritize node scheduling; S42. Using the micro-layer of the global water transfer strategy network, the overall water transfer target across reservoir nodes is input, and the specific operation actions of each reservoir node are generated by combining the real-time global embedding matrix. S43. The macro-layer and micro-layer policy networks share a global embedding matrix. A joint loss function is used to simultaneously optimize the macro-layer and micro-layer. Gradient updates are used to improve the overall performance of the hierarchical policy. Multi-step simulations are performed on the water regulation actions generated during joint training to form a complete hierarchical water regulation policy. In S43, the joint loss function Represented as: ； in, For the overall water diversion target error at the macro level, For micro-level motion constraint deviation, The loss of consistency between macro goals and micro actions , and These are the weighting coefficients.

6. The method for optimizing the scheduling of inter-basin water transfer projects based on artificial intelligence according to claim 5, characterized in that, In step S5, the delayed response, nonlinear constraints, and coupling relationships of each reservoir are incorporated into the reward function to calculate the immediate reward. ; Instant rewards Represented as: ； in, For nodes At time step t, the current water storage level Let i be the target water storage capacity. For nodes To the node The actual scheduled traffic, , , This represents the corresponding penalty weight coefficient. This indicates a constraint penalty for exceeding the pipeline's water carrying capacity. This is a penalty for failing to meet the timing requirements of delayed responses from upstream and downstream suppliers. The operational actions of the tiered water diversion strategy are calculated in time series to obtain multi-step returns, and the long-term target is compromised into a short-to-medium-term weighted return. ; Short-to-medium term weighted returns Represented as: ； in, ], As a discount factor, For short- to medium-term time steps, For the future Instant rewards for each step; Finally, the gradient descent learning optimization method is used to iteratively update the loss function of the hierarchical policy network, and the optimized global-hierarchical scheduling policy is output.

Images