Drainage basin flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning

Abstract

The invention discloses a basin flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning, and the method is characterized in that the method comprises the following steps: collecting historical flood water quality observation data and historical flood hydrological data; performing standardization processing on the historical flood hydrological data to obtain a comparison sequence and a reference sequence, and performing dimensionless processing; calculating a dynamic grey correlation coefficient and a weighted grey correlation degree by using the comparison sequence after dimensionless processing and the reference sequence, and sequencing to form a weighted grey correlation degree sequence; screening out an effective historical flood field by using the weighted grey relational degree sequence, and performing dynamic weight optimization based on a long short-term memory network and a reinforcement learning agent based on a TD3 algorithm; and calculating by using the historical flood water quality observation data and the historical flood weight to obtain final predicted flood water quality data. The prediction method provided by the invention can realize intelligent detection of water quality in the flood process, and is accurate, real-time and self-adaptive.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hydrological prediction and water quality monitoring, and particularly relates to a basin flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning. BACKGROUND

[0002] In the management of basin hydrology and water environment protection, the prediction of water quality change during flood has always been a complex and challenging task. Traditional water quality prediction methods mostly rely on hydrological and hydrodynamic models or statistical methods based on historical data, which have the following obvious shortcomings: first, traditional hydrological models are heavily dependent on physical mechanism modeling and a large number of parameter calibration, and have poor adaptability and limited real-time prediction accuracy in the case of sudden rainstorm flood; second, existing methods often separate the prediction of flood process and water quality, and do not fully consider the dynamic coupling relationship between flood evolution and water quality change; in addition, most prediction models lack the ability of online learning and adaptive adjustment to real-time monitoring information, and are difficult to cope with data missing or abnormal situations, resulting in lag or distortion of the prediction results.

[0003] In recent years, some researches have tried to introduce machine learning methods (such as LSTM) for hydrological and water quality prediction, which has improved the prediction ability to some extent, but still has limitations: first, such methods usually rely on fixed model structure and weights, and are difficult to dynamically optimize according to real-time flood evolution state; second, existing methods do not fully explore the similarity of multiple historical floods, and lack the ability of dynamic weighting of key influencing factors; third, the use of real-time feedback information in the prediction process is insufficient, and the online learning and calibration of model parameters are not realized, which limits the generalization ability and stability of the model in changing environment.

[0004] Therefore, there is an urgent need for an intelligent prediction method that can integrate multi-source information, have dynamic optimization and real-time learning ability, and effectively couple the flood process and water quality indicators, to improve the accuracy, real-time performance and reliability of joint prediction of flood water quality. SUMMARY

[0005] The purpose of the present application is to overcome the deficiencies in the prior art, and to provide an intelligent prediction method for basin flood water quality based on dynamic weight optimization and deep reinforcement learning, so as to realize intelligent detection of water quality during flood, and to be accurate, real-time and adaptive.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] On the one hand, the present application provides an intelligent prediction method for basin flood water quality based on dynamic weight optimization and deep reinforcement learning, characterized by the following steps:

[0008] Collect historical flood water quality observation data and historical flood hydrological data;

[0009] The historical flood hydrological data is standardized to obtain a comparison sequence and a reference sequence, and the comparison sequence and the reference sequence are dimensionless processed;

[0010] The comparison sequence and the reference sequence after dimensionless processing are used to calculate a dynamic grey correlation coefficient;

[0011] The dynamic grey correlation coefficient is used to calculate the weighted grey correlation degree of the comparison sequence to the reference sequence, and the weighted grey correlation degree is sorted to form a weighted grey correlation degree sequence;

[0012] The weighted grey correlation degree sequence is used to screen out effective historical flood fields, and a long short-term memory network and a TD3 algorithm-based reinforcement learning agent are used to dynamically optimize the historical flood weight of the effective historical flood field;

[0013] The historical flood water quality observation data and the historical flood weight are used to calculate the final predicted flood water quality data.

[0014] Further, the starting point of each historical flood rainfall is taken as the time origin, the flood similarity index set is extracted, including the rising flow, the hourly cumulative rainfall sequence and the hourly cumulative water sequence, to form a comparison sequence, and the expression is:

[0015] ;

[0016] Wherein, is the comparison sequence; is the rising flow of the i-th historical flood; is the hourly cumulative rainfall sequence of the comparison sequence; is the hourly cumulative water sequence of the comparison sequence; x=1,2,…,72; i is the i-th historical flood, N is a positive integer;

[0017] According to the different flood rising time t0, the flood rising flow, the hourly cumulative rainfall sequence and the hourly cumulative water sequence are obtained by automatic collection or manual observation of the water and rainfall station, and the stage cumulative water in the flood forecast period is obtained by using the flood forecast data, to form a reference sequence, and the expression is:

[0018] ;

[0019] Wherein, is the reference sequence; is the current measured flood rising flow; is the hourly cumulative rainfall sequence of the reference sequence; The hourly accumulated water sequence of the reference sequence; y = 1, 2, …, The subscript is the flood forecasting period, t0 is the flood rising time, ≤72.

[0020] Further, the comparison sequence and the reference sequence are dimensionless, and the expression is:

[0021] ;

[0022] ;

[0023] ;

[0024] Wherein, is the dimensionless comparison sequence, j ∈ {flow, rainfall, water volume} three types of index dimensions, j = 1, 2, …, ; is the data of the jth similarity index of the ith flood in the comparison sequence; is the data of the jth similarity index in the reference sequence; is the dimensionless reference sequence.

[0025] Further, the absolute difference between the dimensionless comparison sequence and the reference sequence is calculated , and the expression is:

[0026] ;

[0027] The maximum value , the minimum value , and the expression is:

[0028] ;

[0029] ;

[0030] Wherein, represents the minimum value of the series; represents the maximum value of the series;

[0031] The absolute difference , the maximum value , the minimum value , and the dynamic grey correlation coefficient are calculated, and the expression is:

[0032] ;

[0033] Wherein, the expression of time attenuation resolution coefficient is as follows: The expression of time attenuation resolution coefficient is as follows:

[0034] ;

[0035] The expression of time attenuation resolution coefficient is as follows: The prediction time is as follows:

[0036] Further, the weighted grey correlation degree of each index is calculated by using dynamic grey correlation coefficient The expression is as follows:

[0037] ;

[0038] The expression of time attenuation resolution coefficient is as follows: The weighted grey correlation degree of the ith flood is as follows: The weight of each index is as follows, wherein the weight of flow is 0.4, the weight of rainfall is 0.3, and the weight of water quantity is 0.3;

[0039] The weighted grey correlation degrees of n comparison sequences to the same reference sequence are arranged in size order to form a weighted grey correlation degree sequence.

[0040] Further, the effective historical flood field number A is screened out by using the weighted grey correlation degree sequence, and the expression is as follows:

[0041] ;

[0042] The historical floods corresponding to the first A weighted grey correlation degrees in the weighted grey correlation degree sequence are taken as the similar flood set.

[0043] Further, the initial weight of historical flood is generated by using long short-term memory network LSTM, including the following steps:

[0044] The historical flood hydrological data of the effective historical flood field is extracted, a standardized 216-dimensional feature vector is constructed for each historical flood, and a real-time flood state vector is constructed;

[0045] The standardized 216-dimensional feature vector and the real-time flood state vector are stacked to form an input matrix of long short-term memory network;

[0046] The standardized input matrix is input into the bidirectional long short-term memory network for high-dimensional spatio-temporal feature extraction;

[0047] The attention weight of each time step in the flood process is calculated based on the spatio-temporal attention mechanism, and the weighted context feature vector is generated by using the obtained attention weight of each time step;

[0048] The context vector is input into a fully connected neural network for transformation, and after normalization by the softmax function, the initial probability distribution weights of historical floods are obtained.

[0049] Furthermore, the expression for the standardized 216-dimensional feature vector is:

[0050] ;

[0051] in, A standardized 216-dimensional feature vector; The flood surge flow is the flood surge flow sequence, which consists of the flood surge flow value. It consists of 72 copies. After filtering Hourly cumulative rainfall sequence of a historical flood; After filtering The hourly cumulative water volume sequence of a historical flood; x=1,2,…,72; For the first A historic flood, ; Represented as a 216-dimensional real number space;

[0052] The expression for the real-time flood state vector is:

[0053] ;

[0054] in, Let the real-time flood state vector be the vector; for the real-time flood state vector... t>t0+ During this period, the cumulative rainfall and cumulative water volume The input is filled with the historical average or zero value to meet the neural network’s requirement for a fixed input size and ensure that the dimension is always 216.

[0055] The expression for the input matrix X of a Long Short-Term Memory (LSTM) network is:

[0056] ;

[0057] The standardized input matrix X is fed into a bidirectional long short-term memory network for high-dimensional spatiotemporal feature extraction, as shown in the following expression:

[0058] ;

[0059] ;

[0060] ;

[0061] in, is the input at the current time step t; and are the hidden states of the forward and backward LSTMs at the previous time step t-1 and the next time step t+1, respectively; and are the cell states of the forward and backward LSTMs at the previous time step t-1 and the next time step t+1, respectively; and are the hidden states of the forward and backward LSTMs at the current time step t; and are the cell states of the forward and backward LSTMs at the current time step t; is the final feature representation at the t-th time step, which is concatenated from the hidden states of the forward and backward LSTMs at the current time step t; the Tanh function is used as the cell state activation function;

[0062] The spatio-temporal attention mechanism is introduced to calculate the attention weight at each time step, which focuses on the key period of the flood process. The expression of the attention weight at each time step is:

[0063] ;

[0064] where, is the normalized attention weight at the t-th time step; is the exponential function; is the sum of the exponential scores of all time steps; is the energy score at the t-th time step, whose expression is:

[0065] ;

[0066] where, is the trainable feature score vector; is the hyperbolic tangent activation function; is the trainable weight matrix; is the trainable bias vector;

[0067] The expression of the context feature vector is:

[0068] ;

[0069] The context vector is input into a fully connected neural network for transformation, whose expression is:

[0070] ;

[0071] in, This is the original weight score vector; This is the first layer weight matrix; This is the first layer bias vector; This is the weight matrix for the second layer; This is the second layer bias vector; It is a linear rectifier unit. It is an S-shaped function that outputs the original weight score vector. ;

[0072] The initial probability distribution weights of historical floods, obtained after normalization using the softmax function, are expressed as follows:

[0073] ;

[0074] in, For the first Normalized initial weights for historical floods; For the first The original weighted score vector of a historical flood.

[0075] Furthermore, a reinforcement learning agent based on the TD3 algorithm is employed to normalize the initial weights. Real-time optimization includes the following steps:

[0076] First, construct the state space, and at each optimization time... State vector It consists of the following four-dimensional features, expressed as follows:

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] in, Let be the absolute error of the prediction at time k; Predict the flow rate value for the hydrological model at time k; Let be the measured flow rate at time k; This represents the measured rate of change in flow rate; The attenuation coefficient is... ; For the filtered first Weighted grey relational degree of a historical flood; The weighted grey relational degree after time decay;

[0082] Secondly, the reinforcement learning agent generates actions by the Actor policy network, and updates the historical flood weights:

[0083] The action output by the reinforcement learning agent at time k is the adjustment amount of the historical flood weight , which is constrained in the range of [-0.1, 0.1] and expressed as:

[0084] ;

[0085] The action is generated by the Actor policy network , which is expressed as:

[0086] ;

[0087] wherein represents the weight adjustment amount suggested by the Actor policy network for the i-th historical flood; ;

[0088] The historical flood weight is updated according to the following formula:

[0089] ;

[0090] wherein represents the old weight of the i-th historical flood at time k; is a truncation function to ensure that the calculated new weight does not exceed the reasonable range of [0, 1]; represents the new weight of the i-th historical flood at time k+1; ;

[0091] The weight normalization processing is performed on , which is expressed as:

[0092] ;

[0093] wherein represents the sum of all historical flood weights after updating;

[0094] Further, the reward function R is designed to optimize the prediction accuracy and process line smoothness, which is expressed as:

[0095] ;

[0096] ;

[0097] wherein is the weight coefficient of the error penalty term, ; is the second derivative of the observed flow process line;​ a weight coefficient of the smoothness penalty term,

[0098] Finally, the network updates the Critic network, adopts the double Q network structure, and synchronously updates the Actor network, the target network soft update coefficient is 0.005; the exploration strategy adopts the Ornstein-Uhlenbeck noise, the noise parameter is 0.1; the agent synchronously updates the strategy network parameters once every 6 hours.

[0099] Further, the final predicted flood water quality data at each moment is calculated by multiplying the historical flood water quality observation data by the corresponding historical flood weight, and the expression is:

[0100]

[0101] wherein, the final predicted flood water quality data, the historical flood water quality data of the first field historical flood at the moment t;

[0102] Error determination is performed on the obtained final predicted flood water quality data:

[0103] Real-time monitoring prediction absolute error when , the historical maximum prediction absolute error, the model state is determined to be mismatched, and the recalibration mechanism is triggered immediately.

[0104] Compared with the prior art, the present application has the following beneficial effects:

[0105] The watershed flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning provided by the present application significantly improves the comprehensive performance of watershed flood water quality prediction through multi-technology fusion and mechanism innovation, which is embodied in the following aspects:

[0106] 1) High prediction accuracy;

[0107] The dynamic grey correlation analysis quantifies the similarity between the historical flood and the current process, and the bidirectional long short-term memory network (LSTM) deep extracts high-dimensional spatiotemporal features by using the integrated attention mechanism, which provides accurate basis for weight allocation, and fundamentally improves the accuracy of water quality parameter prediction.

[0108] 2) Strong real-time adaptive ability;

[0109] ​​The introduction of a reinforcement learning agent based on the TD3 algorithm can dynamically adjust the historical flood weight according to real-time monitoring data (such as prediction error and flow rate of change); this online optimization mechanism enables the model to adapt to the dynamic changes of the flood process, effectively reducing the lag of the prediction results.

[0110] 3) Smooth and stable prediction process;

[0111] The designed reward function considers both prediction accuracy and process line smoothness during optimization, avoiding the problem of dramatic fluctuations in prediction values that may occur in traditional methods, making the prediction results more reasonable and reliable, and having strong practicality.

[0112] 4) Good system robustness;

[0113] The built-in recalibration mechanism can automatically trigger when the prediction error is out of limits (model mismatch), reset the network state and reinitialize the optimization process, so that the system can quickly recover to a reliable state, enhancing the model's self-repairing ability and long-term running stability in abnormal conditions.

[0114] 5) Comprehensive use of multi-source information: the prediction method provided by the present application organically couples flood hydrological characteristics and water environmental quality indicators, achieving coordinated simulation and prediction of flood processes and water pollution changes, and providing a comprehensive technical means for solving water environmental risk early warning in complex environments.

[0115] The present application not only significantly improves the accuracy and real-time performance of flood water quality prediction, but also enhances the adaptability and reliability of the model, and has good application prospect and popularization value. BRIEF DESCRIPTION OF DRAWINGS

[0116] Figure 1 A flowchart of a basin flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning provided by the present application is shown. DETAILED DESCRIPTION

[0117] The present application will be further described below in conjunction with the drawings. The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0118] Referring to Figure 1 The steps of a basin flood water quality intelligent prediction method based on dynamic weight optimization and deep reinforcement learning of the present application are as follows:

[0119] Step 1) Collect historical flood water quality observation data and historical flood hydrological data, and standardize the flood data to obtain a comparison sequence and a reference sequence; the specific steps include:

[0120] The historical flood water quality observation data are collected, including dissolved oxygen, pH, total phosphorus, total nitrogen, chemical oxygen demand and permanganate index.

[0121] The flood similarity index set is extracted with the starting point of rainfall of each historical flood as the time origin, including the rising flow, the hour-level cumulative rainfall sequence and the hour-level cumulative water sequence, to form a comparison sequence, so that the flood duration of the flood data is 72 hours, the time period is 1 hour, and the expression is as follows:

[0122] ;

[0123] Among them, is the comparison sequence; is the rising flow of the i-th historical flood; is the hour-level cumulative rainfall sequence of the comparison sequence; is the hour-level cumulative water sequence of the comparison sequence; x=1,2,…,72; i is the i-th historical flood, N is a positive integer.

[0124] According to the different rising time t0 of the flood (such as 6h, 12h after rainfall, etc.), the rising flow, the hour-level cumulative rainfall sequence and the hour-level cumulative water sequence are obtained through automatic collection or manual observation of the water and rainfall station, the stage cumulative water in the flood prediction period is obtained by using the flood forecast data, a reference sequence is formed, and the expression is as follows:

[0125] ;

[0126] Among them, is the reference sequence; is the current measured rising flow of the flood; is the hour-level cumulative rainfall sequence of the reference sequence; is the hour-level cumulative water sequence of the reference sequence; y=1,2,…, ; subscript is the flood prediction period, t0 is the rising time of the flood, ≤72.

[0127] Step 2) Dimensionless processing is performed on the comparison sequence and the reference sequence; the specific steps include:

[0128] The dimensionless processing is performed on the comparison sequence and the reference sequence, and the expression is as follows:

[0129] ;

[0130] ;

[0131] ;

[0132] wherein, is the dimensionless comparison sequence, j∈{flow, rainfall, water volume} three types of index dimensions, j = 1, 2, …, ; is the data of the jth similarity index of the ith flood in the comparison sequence; is the data of the jth similarity index in the reference sequence; is the dimensionless reference sequence.

[0133] Step 3) Calculate the dynamic grey correlation coefficient using the dimensionless comparison sequence and the reference sequence; specifically including:

[0134] Calculate the absolute difference value of the dimensionless comparison sequence and the reference sequence , the expression is as follows:

[0135] ;

[0136] Take the maximum value , the minimum value , the expression is as follows:

[0137] ;

[0138] ;

[0139] wherein, represents taking the minimum value of the series; represents taking the maximum value of the series.

[0140] Using the absolute difference value , the maximum value , the minimum value , calculate the dynamic grey correlation coefficient , the dynamic grey correlation coefficient expression is as follows:

[0141] ;

[0142] wherein, is the time decay resolution coefficient (the traditional grey correlation analysis uses a fixed resolution coefficient, and the present application improves it to a model that decays with time, highlights recent influences, weakens long-term influences, and dynamically adjusts resolution ability), the expression of the time decay resolution coefficient is as follows:

[0143] ;

[0144] wherein, is the prediction time.

[0145] Step 4) Calculate the weighted grey correlation degree using the dynamic grey correlation coefficient, and sort it; specifically including:

[0146] Using the dynamic grey correlation coefficient , the weighted grey correlation degree of each index is calculated, and the expression is as follows:

[0147] ;

[0148] Among them, is the weighted grey correlation degree of the ith flood; is the weight of each index: the weight of flow is 0.4, the weight of rainfall is 0.3, and the weight of water is 0.3;

[0149] Arrange the weighted grey correlation degrees of n comparison sequences to the same reference sequence in order of size to form a weighted grey correlation degree sequence.

[0150] Step 5) Dynamic weight optimization prediction of flood water quality based on long short-term memory network and reinforcement learning agent; The specific steps include:

[0151] Step 51) Use the weighted grey correlation degree sequence to filter out the effective historical flood field number A, and the expression is as follows:

[0152] ;

[0153] Take the first A weighted grey correlation degrees in the weighted grey correlation degree sequence as the corresponding historical flood set; Generate the initial weight of the historical flood through the long short-term memory network LSTM, as follows:

[0154] Step 511) Input feature construction and standardization;

[0155] Extract the A historical flood data after screening, and construct a standardized 216-dimensional feature vector for each historical flood, as follows:

[0156] ;

[0157] Among them, is the standardized 216-dimensional feature vector; is the flood rising flow, and the flood rising flow sequence is composed of 72 times of flood rising flow values ; is the hourly cumulative rainfall sequence of the a historical flood after screening; is the hourly cumulative rainfall sequence of the a historical flood after screening ; x = 1, 2, …, 72; is the historical flood, ; is expressed as a 216-dimensional real number space;

[0158] The real-time flood state vector is constructed, and the expression is as follows:

[0159] ;

[0160] wherein, is the real-time flood state vector; for the real-time flood state vector t>t0+ The cumulative rainfall and cumulative water volume of the period (i.e. the future period beyond the currently known range) are filled with historical same period mean values or zero values to meet the requirement of fixed input size of the neural network, ensuring that the dimension is always 216; Stack and

[0161] to form the input matrix X of the long short-term memory network, and the expression is as follows: ;

[0162] ;

[0163] Step 512) bidirectional long short-term memory network feature extraction;

[0164] The normalized input matrix X is input into the bidirectional long short-term memory network for high-dimensional spatiotemporal feature extraction, and the expression is as follows:

[0165] ;

[0166] ;

[0167] ;

[0168] wherein, is the input of the current time step; and are the hidden states of the forward long short-term memory network and the reverse long short-term memory network at the previous time step t-1 and the next time step t+1, respectively; and are the cell states of the forward long short-term memory network and the reverse long short-term memory network at the previous time step t-1 and the next time step t+1, respectively; and are the hidden states of the forward long short-term memory network and the reverse long short-term memory network at the current time step t, respectively; and are the cell states of the forward long short-term memory network and the reverse long short-term memory network at the current time step t, respectively;​ The final feature representation at time step t is formed by concatenating the hidden states of the forward long short-term memory network and the backward long short-term memory network at the current time step t; the Tanh function is used as the cell state activation function.

[0169] Step 513) Attention mechanism weighting;

[0170] A spatiotemporal attention mechanism is introduced to calculate the attention weight at each time step, in order to focus on the key periods of the flood process, as shown in the following expression:

[0171] ;

[0172] in, Let be the normalized attention weights at time step t; It is an exponential function; This is the sum of the exponential fractions for all time steps (t=1 to 72); The energy fraction at time step t is expressed as follows:

[0173] ;

[0174] in, Trainable feature score vectors; It is the hyperbolic tangent activation function; This is a trainable weight matrix; It is a trainable bias vector;

[0175] Using the attention weights obtained above for each time step Generate a weighted context feature vector The expression is as follows:

[0176] ;

[0177] Step 514) Fully connected layer and weight normalization;

[0178] context vector The input is transformed into a fully connected neural network, as shown in the following expression:

[0179] ;

[0180] in, This is the original weight score vector; This is the first layer weight matrix; This is the first layer bias vector; This is the weight matrix for the second layer; This is the second layer bias vector; It is a linear rectifier unit. Sigmoid function, the output of the original weight score vector , the initial probability distribution weight of historical flood after normalization by the softmax function, the expression is as follows:

[0181] ;

[0182] Wherein, is the normalized initial weight of the first field historical flood; is the original weight score vector of the first field historical flood;

[0183] Step 52) weight dynamic optimization based on reinforcement learning agent;

[0184] The reinforcement learning agent based on TD3 algorithm is adopted to optimize the normalized initial weight in real time, as follows:

[0185] Step 521) constructing state space;

[0186] At each optimization time , the state vector is composed of the following four-dimensional features, the expression is as follows:

[0187] ;

[0188] ;

[0189] ;

[0190] ;

[0191] Wherein, is the predicted absolute error at k time; is the hydrological model predicted flow value at k time; is the measured flow value at k time; is the measured flow rate of change; is the decay coefficient, ; is the weighted grey correlation degree of the first field historical flood after screening; is the weighted grey correlation degree after time decay;

[0192] Step 522) output action and execute network strategy;

[0193] The reinforcement learning agent generates actions through its Actor policy network to update the historical flood weight, the specific process is as follows:

[0194] The action output by the reinforcement learning agent at time k Adjustment amount for historical flood weights The constraint is within the range of [-0.1, 0.1], and the expression is as follows:

[0195] ;

[0196] Actions are generated by the Actor policy network. The following results are generated:

[0197] ;

[0198] in, This represents the weight adjustment amount suggested by the Actor policy network for the a-th historical flood.

[0199] Historical flood weights are updated using the following formula:

[0200] ;

[0201] in, No. The old weight of a historical flood at time k; This is a truncation function to ensure that the calculated new weights do not exceed a reasonable range of [0,1]. For the first The new weight of the historical flood at time k+1;

[0202] right The weight normalization process is performed as follows:

[0203] ;

[0204] in, This is the sum of the weights of all historical floods after the update;

[0205] Step 523) Design the reward function;

[0206] Design a reward function R to optimize prediction accuracy and process line smoothness, as shown in the following expression:

[0207] ;

[0208] ;

[0209] in, The weighting coefficients for the error penalty term. ; The approximate second derivative of the measured flow rate process line; The weighting coefficients for the smoothness penalty term. ;

[0210] Step 524) Perform network updates and exploration;

[0211] The network update uses a Critic network with a dual-Q network structure, and is updated synchronously with the Actor network. The target network soft update coefficient is 0.005.

[0212] The exploration strategy uses Ornstein-Uhlenbeck noise with noise parameter σ=0.1; the agent updates the strategy network parameters every 6 hours.

[0213] Step 53) Dynamic prediction and recalibration;

[0214] Step 531) Flood water quality prediction;

[0215] Using the historical water quality observation data obtained in step 1), including dissolved oxygen (DO), pH, total phosphorus, total nitrogen, chemical oxygen demand (COD), and permanganate index, the final predicted flood water quality data at each moment is calculated by multiplying the historical flood water quality observation data by the corresponding historical flood weights, as shown in the following expression:

[0216] ;

[0217] in, To ultimately predict flood water quality data, For the first Measured flood water quality data at time t for a historical flood event;

[0218] Step 532) Trigger recalibration;

[0219] Real-time monitoring of prediction absolute error ,when hour If the error is the largest prediction error in history, it is determined to be a model state mismatch, and a recalibration mechanism is immediately triggered: the hidden state of the long short-term memory network is reset, the weights are initialized and optimized.

[0220] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent prediction of water quality in watershed floods based on dynamic weight optimization and deep reinforcement learning, characterized in that, Includes the following steps: Collect historical flood water quality observation data and historical flood hydrological data; The historical flood hydrological data are standardized to obtain comparison sequences and reference sequences, and the comparison sequences and reference sequences are then dimensionless. The dynamic grey relational coefficient is calculated using the dimensionless comparison sequence and the reference sequence. The weighted grey correlation degree of the comparison sequence to the reference sequence is calculated using the dynamic grey correlation coefficient, and the weighted grey correlation degree is sorted to form a weighted grey correlation degree sequence. The effective historical flood fields are selected by using the weighted grey relational sequence, and the historical flood weights of the effective historical flood fields are dynamically optimized based on a long short-term memory network and a reinforcement learning agent based on the TD3 algorithm. The final predicted flood water quality data is obtained by using the historical flood water quality observation data and the historical flood weights.

2. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 1, characterized in that, Using the starting point of each historical flood rainfall event as the time origin, a flood similarity index set is extracted, including the initial flow rate, hourly cumulative rainfall sequence, and hourly cumulative water volume sequence, forming a comparison sequence, expressed as: ； in, For comparison sequences; The initial rise flow rate of the i-th historical flood; For comparison, hourly cumulative rainfall sequences; This is a comparison sequence of hourly cumulative water volume; x = 1, 2, ..., 72; i represents the i-th historical flood. N is a positive integer; Depending on the flood rise time t0, the flood rise flow, hourly cumulative rainfall sequence, and hourly cumulative water volume sequence are obtained through automatic data collection or manual observation from hydrological and rainfall monitoring stations. Using flood forecast data, the stage-specific cumulative water volume within the flood forecast period is obtained to form a reference sequence, expressed as: ； in, For reference sequence; This represents the current measured flood surge flow rate; The hourly cumulative rainfall series serves as a reference series. The hourly cumulative water volume sequence is the reference sequence; y=1,2,…, ; Subscript The flood forecast period is defined as t0, where t0 is the moment the flood begins to rise. ≤72.

3. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 1, characterized in that, The comparison sequence and the reference sequence are dimensionless, expressed as follows: ； ； ； in, The comparison sequence is dimensionless, with j ∈ {flow, rainfall, water volume} as the three index dimensions, j = 1, 2, ... ; To compare the data of the j-th similarity index of the i-th flood in the sequence; The data is for the j-th similarity index in the reference sequence; This is the reference sequence after dimensionless processing.

4. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 3, characterized in that, Calculate the absolute difference between the dimensionless comparison sequence and the reference sequence. The expression is: ； Extract the maximum value. Minimum value The expression is: ； ； in, This indicates taking the minimum value in a series; This indicates taking the maximum value in the series; Using absolute difference maximum value Minimum value Calculate the dynamic grey relational coefficient The expression is: ； in, The time decay resolution coefficient, The expression is as follows: ； in, For predicting time.

5. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 4, characterized in that, Using dynamic grey relational coefficient The weighted grey relational degree of each indicator is calculated, and the expression is: ； in, Let be the weighted grey relational degree of the i-th flood; The weights for each indicator are as follows: flow rate has a weight of 0.4, rainfall has a weight of 0.3, and water volume has a weight of 0.

3. The weighted grey relational degrees of n comparison sequences relative to the same reference sequence are arranged in order of magnitude to form a weighted grey relational degree sequence.

6. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 5, characterized in that, Using a weighted grey relational sequence, the effective historical flood field frequency A is selected, expressed as follows: ； Take the historical floods corresponding to the first A weighted grey relational degrees in the weighted grey relational degree sequence as the similar flood set.

7. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 6, characterized in that, The initial weights for historical floods are generated using a Long Short-Term Memory (LSTM) network, including the following steps: Extract historical flood hydrological data from effective historical flood fields, construct a standardized 216-dimensional feature vector for each historical flood, and construct a real-time flood state vector; The standardized 216-dimensional feature vector is stacked with the real-time flood state vector to form the input matrix of the long short-term memory network. The standardized input matrix is ​​fed into a bidirectional long short-term memory network for high-dimensional spatiotemporal feature extraction. The attention weights at each time step during the flood process are calculated based on the spatiotemporal attention mechanism, and the weighted context feature vectors are generated using the obtained attention weights at each time step. The context vector is input into a fully connected neural network for transformation, and after normalization by the softmax function, the initial probability distribution weights of historical floods are obtained.

8. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 6, characterized in that, The expression for the standardized 216-dimensional feature vector is: ； in, A standardized 216-dimensional feature vector; The flood surge flow is the flood surge flow sequence, which consists of the flood surge flow value. It consists of 72 copies. After filtering Hourly cumulative rainfall sequence of a historical flood; After filtering The hourly cumulative water volume sequence of a historical flood; x=1,2,…,72; For the first A historic flood, ; Represented as a 216-dimensional real number space; The expression for the real-time flood state vector is: ； in, Let the real-time flood state vector be the vector; for the real-time flood state vector... t>t0+ During this period, its cumulative rainfall and cumulative water volume The input is filled with the historical average or zero value to meet the neural network’s requirement for a fixed input size and ensure that the dimension is always 216. The expression for the input matrix X of a Long Short-Term Memory (LSTM) network is: ； The standardized input matrix X is fed into a bidirectional long short-term memory network for high-dimensional spatiotemporal feature extraction, as shown in the following expression: ； ； ； in, Input for the current time step; and These are the hidden states of the forward long short-term memory network and the backward long short-term memory network at the previous time step t-1 and the next time step t+1, respectively. and The cells represent the forward long short-term memory network and the backward long short-term memory network at the previous time step t-1 and the next time step t+1, respectively. and These are the hidden states of the forward long short-term memory network and the backward long short-term memory network at the current time step t, respectively. and These represent the cell states of the forward long short-term memory network and the backward long short-term memory network at the current time step t, respectively. The final feature representation at time step t is formed by concatenating the hidden states of the forward long short-term memory network and the backward long short-term memory network at the current time step t; the Tanh function is used as the cell state activation function. A spatiotemporal attention mechanism is introduced to calculate the attention weight at each time step, focusing on the critical periods of the flood process. The expression for the attention weight at each time step is as follows: ； in, Let be the normalized attention weights at time step t; It is an exponential function; It is the sum of the exponential fractions of all time steps; The energy fraction at time step t is expressed as: ； in, Trainable feature score vectors; It is the hyperbolic tangent activation function; This is a trainable weight matrix; It is a trainable bias vector; Context feature vector The expression is: ； context vector The input is transformed into a fully connected neural network, and the expression is: ； in, This is the original weight score vector; This is the first layer weight matrix; This is the first layer bias vector; This is the weight matrix for the second layer; This is the second layer bias vector; It is a linear rectifier unit. It is an S-shaped function, outputting the original weight score vector. ; The initial probability distribution weights of historical floods, obtained after normalization using the softmax function, are expressed as follows: ； in, For the first Normalized initial weights for historical floods; For the first The original weighted score vector of a historical flood.

9. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 8, characterized in that, A reinforcement learning agent based on the TD3 algorithm is used to normalize the initial weights. Real-time optimization includes the following steps: First, construct the state space, and at each optimization time... State vector It consists of the following four-dimensional features, expressed as follows: ； ； ； ； in, Let be the absolute error of the prediction at time k; Predict the flow rate value for the hydrological model at time k; Let be the measured flow rate at time k; This represents the measured rate of change in flow rate; The attenuation coefficient is... ; For the filtered first Weighted grey relational degree of a historical flood; The weighted grey relational degree after time decay; Secondly, reinforcement learning agents are used to generate actions through the Actor policy network to update historical flood weights: The action output by the reinforcement learning agent at time k Adjustment amount for historical flood weights The constraint is within the range of [-0.1, 0.1], and the expression is: ； Actions are generated by the Actor policy network. The generation is represented as: ； in, Indicate that the Actor policy network is the first... The recommended weighting adjustment for historical flood events; Historical flood weights are updated using the following formula: ； in, No. The old weight of a historical flood at time k; This is a truncation function to ensure that the calculated new weights do not exceed a reasonable range of [0, 1]. For the first The new weight of the historical flood at time k+1; right The weight normalization process is performed, and the expression is: ； in, This is the sum of the weights of all historical floods after the update; Furthermore, a reward function R is designed to optimize prediction accuracy and process line smoothness, expressed as: ； ； in, The weighting coefficients for the error penalty term. ; The second derivative of the measured flow process line; The weighting coefficients for the smoothness penalty term. ; Finally, the network update uses a Critic network with a dual-Q network structure, updated synchronously with the Actor network, and the target network soft update coefficient is 0.005; the exploration strategy uses Ornstein-Uhlenbeck noise with noise parameter σ=0.1; the agent updates the strategy network parameters synchronously every 6 hours.

10. The intelligent water quality prediction method for watershed floods based on dynamic weight optimization and deep reinforcement learning according to claim 9, characterized in that, The final predicted flood water quality data at each moment is calculated by multiplying historical flood water quality observation data by the corresponding historical flood weights, as expressed in the following expression: ； in, To ultimately predict flood water quality data, For the first Measured flood water quality data at time t from a historical flood event; Error assessment was performed on the final predicted flood water quality data obtained: Real-time monitoring of prediction absolute error ,when hour, If the error is the largest absolute error in history, it is determined to be a model state mismatch, and the recalibration mechanism is immediately triggered.

Images