A river water quality simulation method, device, medium and product based on an LSTM-Transformer hybrid model

Abstract

The application discloses a river water quality simulation method and device based on an LSTM-Transformer hybrid model, a medium and a product, relates to the field of river water quality simulation, and comprises the following steps: acquiring and preprocessing multi-parameter water quality monitoring data, screening model input features and model output features, and extracting and normalizing model input sequences and model output sequences; the normalized model input sequences and model output sequences are divided into a training sample set, a verification sample set and a test sample set; an LSTM-Transformer hybrid model is constructed, the trained LSTM-Transformer hybrid model is obtained by using the training sample set and the verification sample set, the simulation result of the water quality parameter is obtained in combination with the test sample set, the problems of information attenuation and memory confusion of a single LSTM model and local capture deviation of a Transformer are effectively alleviated, and therefore the water quality simulation effect is improved.

Description

Technical Field

[0001] This application relates to the field of river water quality simulation, and in particular to a method, equipment, medium and product for river water quality simulation based on an LSTM-Transformer hybrid model. Background Technology

[0002] Rivers are the primary carriers of global freshwater resources, and changes in their water quality directly determine drinking water safety, agricultural irrigation efficiency, industrial water sustainability, and the stability of aquatic ecosystems. Therefore, achieving accurate simulation and prediction of river water quality parameters is of significant scientific importance and engineering application value for river water quality management and the protection of aquatic ecosystems.

[0003] Traditional watershed hydrological and water quality models based on physical processes, such as the Soil and Water Assessment Tool (SWAT) and the MIKE series models, are grounded in the fundamental principles of hydrology, hydrodynamics, and physics. They can achieve refined simulation and prediction of water quality processes under different watershed characteristics and topographic conditions. However, these models typically face challenges such as large data requirements, complex parameters, and difficulties in calibration. Furthermore, their accuracy is highly dependent on high-resolution topographic, meteorological, and land use data.

[0004] Currently, deep learning technology is increasingly being applied to the field of water environment due to its excellent data analysis and mining capabilities and good applicability. In recent years, data-driven models such as Artificial Neural Networks (ANN) and Long Short-Term Memory (LSTM) networks have received widespread attention and research in water quality simulation and prediction. These methods do not require the pre-construction of complex physical or biochemical reaction mechanism equations; they can automatically mine and learn the potential mapping relationships between input and output variables, and achieve accurate simulation and prediction of water quality parameters based on the learned patterns. They have the advantages of rapid model construction, relatively simple model structure, ease of mastery, and easy generalization.

[0005] LSTM, with its excellent time series modeling capabilities, has become one of the most widely used models in the field of water quality simulation. Its core gating mechanism—a ternary control system consisting of an input gate, a forget gate, and an output gate—can dynamically regulate the flow and retention of information, achieving effective memorization of time series information. A Chinese invention patent (application number 2024115165725) provides a river water quality simulation and prediction method based on IGWO-LSTM, using the Improved GreyWolf Optimizer (IGWO) algorithm for LSTM model parameter optimization. It utilizes LSTM to capture long-term dependencies and periodic changes in time series data to achieve river water quality simulation and prediction. However, because this model uses a recursive calculation method, information decay and memory confusion occur as the processed sequences are continuously stacked. In recent years, the Transformer model has shown promising application prospects in the field of water quality simulation. The model uses a self-attention mechanism, which can directly capture complex dependencies between any two time steps, breaking through the traditional recursive computation paradigm. Chinese invention patent (application number 2024112829527) provides a method for simulating and predicting water quality parameters based on PatchSavitsky-Golay filtering and Transformer model. It utilizes the self-attention mechanism of Transformer model to handle the nonlinear characteristics of time series, thereby enhancing the model's ability to capture long-term dependencies in the series and improving the accuracy and reliability of water quality parameter simulation and prediction. However, because its self-attention mechanism treats all time steps as equal, it weakens the bias of time order and has a deviation in capturing local details. Summary of the Invention

[0006] The purpose of this application is to provide a method, device, medium and product for river water quality simulation based on the LSTM-Transformer hybrid model, so as to alleviate the problems of information decay and memory confusion in long-range dependencies of single LSTM, as well as the problem of detail capture bias in local feature extraction of Transformer, thereby improving the water quality simulation effect.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for simulating river water quality based on an LSTM-Transformer hybrid model, including: S1. Acquire multi-parameter water quality monitoring data from the target watershed stations and preprocess the data to obtain a dataset; S2, filter the model input features and model output features based on the dataset; S3, extract the model input sequence and model output sequence based on the model input features and the model output features, and normalize the model input sequence and model output sequence to obtain the normalized model input sequence and the normalized model output sequence; S4, the normalized model input sequence and the normalized model output sequence are divided into a training sample set, a validation sample set and a test sample set; S5. Construct an LSTM-Transformer hybrid model that includes an LSTM module, a Transformer module, and a fully connected layer module, and perform iterative training using the training sample set and the validation sample set to obtain the trained LSTM-Transformer hybrid model. S6. Based on the test sample set and the trained LSTM-Transformer hybrid model, the simulation results of water quality parameters are obtained.

[0008] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for simulating river water quality based on the LSTM-Transformer hybrid model.

[0009] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for simulating river water quality based on the LSTM-Transformer hybrid model.

[0010] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for simulating river water quality based on the LSTM-Transformer hybrid model.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects: This application preprocesses and filters features from multi-parameter water quality monitoring data, constructs an LSTM-Transformer hybrid model including an LSTM module, a Transformer module, and a fully connected layer module for training and simulation, effectively solving the limitations of a single model in water quality time series simulation. Simulation results of water quality parameters are obtained based on the test sample set and the trained LSTM-Transformer hybrid model. The LSTM module fully preserves the temporal order, effectively mitigating the information decay and memory confusion problems of traditional LSTM models as the processing sequence grows. Simultaneously, the Transformer module implements global attention, compensating for the local detail capture bias caused by the weakened temporal order bias in a single Transformer model. The organic integration of these two modules achieves complementary advantages, further mining the deep information in the time series data, thereby improving the accuracy and reliability of river water quality parameter simulation. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a river water quality simulation method based on an LSTM-Transformer hybrid model, provided in one embodiment of this application; Figure 2 This is a schematic diagram showing the geographical location of a water quality monitoring station selected in one embodiment of this application; Figure 3 A schematic diagram of an LSTM-Transformer hybrid model constructed according to an embodiment of this application; Figure 4 A schematic diagram of the input-output structure of an LSTM-Transformer hybrid model constructed according to an embodiment of this application; Figure 5 A comparison of dissolved oxygen simulation results between the LSTM-Transformer hybrid model and the baseline model provided in an embodiment of this application; Figure 6 A scatter plot of dissolved oxygen simulation from an LSTM model provided in an embodiment of this application; Figure 7 Dissolved oxygen simulation scatter plot of a Transformer model provided in an embodiment of this application; Figure 8Scatter plot of dissolved oxygen simulation for an LSTM-Transformer hybrid model provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] To make the objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] like Figure 1 As shown in the figure, this application provides a method for simulating river water quality based on an LSTM-Transformer hybrid model, including: S1: Obtain multi-parameter water quality monitoring data from the target watershed stations and preprocess the data to obtain a dataset.

[0017] S2, Filter the model input features and model output features based on the dataset.

[0018] S3. Extract the model input sequence and model output sequence based on the model input features and the model output features, and normalize the model input sequence and model output sequence to obtain the normalized model input sequence and the normalized model output sequence.

[0019] S4, the normalized model input sequence and the normalized model output sequence are divided into a training sample set, a validation sample set and a test sample set.

[0020] S5. Construct an LSTM-Transformer hybrid model that includes an LSTM module, a Transformer module, and a fully connected layer module, and perform iterative training using the training sample set and the validation sample set to obtain the trained LSTM-Transformer hybrid model.

[0021] S6. Based on the test sample set and the trained LSTM-Transformer hybrid model, the simulation results of water quality parameters are obtained.

[0022] This application constructs an LSTM-Transformer hybrid model by integrating the structural and functional complementary advantages of LSTM and Transformer models, and applies this hybrid model to the field of water quality simulation. Specifically, the LSTM-Transformer hybrid model organically combines the long short-term memory capability of LSTM with the self-attention mechanism of Transformer models. It utilizes LSTM to initially capture the dependencies between time-series data, and integrates the Transformer model to achieve global attention, thereby effectively mitigating the problems of information decay and memory confusion in single LSTM models and the local capture bias of Transformer models, thus enabling deeper mining of deeper information in time-series data. Using multi-parameter water quality monitoring data of the target watershed as input, and through optimized training of model parameters, the model achieves efficient and accurate simulation of river water quality changes.

[0023] In an exemplary embodiment, S1 includes: S11, acquire multi-parameter water quality monitoring data of the target watershed stations.

[0024] S12, use linear interpolation to fill in missing values ​​in the multi-parameter water quality monitoring data to obtain a dataset.

[0025] In practical applications, step 11 involves acquiring multi-parameter water quality monitoring data from the target watershed stations.

[0026] The multi-parameter water quality monitoring data comes from the China National Environmental Monitoring Centre. The collected multi-parameter water quality monitoring data includes dissolved oxygen, water temperature, pH value, permanganate index, ammonia nitrogen, total phosphorus, total nitrogen, conductivity, and turbidity.

[0027] Step 12, preprocessing of multi-parameter water quality monitoring data.

[0028] The multi-parameter water quality monitoring data obtained in step 11 is preprocessed to obtain a dataset. Water quality monitoring station equipment requires regular maintenance and repair, and the acquired multi-parameter water quality monitoring data often contains missing and outlier values. Linear interpolation is used to impute missing values ​​in the multi-parameter water quality monitoring data obtained in step 11 to obtain a complete dataset.

[0029] Linear interpolation is shown in formula (1).

[0030] (1) in, x It is the index of water quality parameter data points used for linear interpolation; b for x The index of a recently known water quality parameter data point. a for xThe index of the most recent known water quality parameter data point; yes x The most recent known water quality parameter data point, yes x The next most recent known water quality parameter data point.

[0031] In one exemplary embodiment, S2 includes: S21. Typical water quality parameters are selected as the output features of the model.

[0032] S22, use Pearson correlation coefficient analysis to calculate the correlation coefficient between the remaining water quality parameters and the model output characteristics.

[0033] S23. Based on the ranking of the absolute values ​​of the correlation coefficients, the input features of the model are selected.

[0034] In practical applications, water quality parameters are often correlated to varying degrees. Selecting water quality parameters that are highly correlated with the model output features as model inputs can help optimize the simulation process and improve simulation accuracy.

[0035] Step 21: Define the output target of the model and select typical water quality parameters as the output features of the model.

[0036] Step 22: Feature filtering is performed on the dataset obtained in step 12. Pearson correlation coefficient analysis is used to calculate and evaluate the correlation strength between the remaining water quality parameters and the selected model output features. The calculation formula for Pearson correlation coefficient analysis is shown in (2).

[0037] (2) in, r represents the Pearson correlation coefficient, with a value range of [-1, 1]; This is the location index; m is the total number of water quality parameters. It is the i-th observation value of the first water quality parameter. It is the i-th observation value of the second water quality parameter; It is the mean of all observed values ​​for the first water quality parameter. It is the mean of all observed values ​​for the second water quality parameter.

[0038] Step 23: Rank the correlation coefficients by absolute value to obtain the model input features. r A larger absolute value usually indicates a stronger correlation in the calculation, which helps with filtering. r The water quality parameters with the highest absolute values ​​are used as input features for the model.

[0039] In an exemplary embodiment, step S3 normalizes the model input sequence and the model output sequence to obtain normalized model input sequence and normalized model output sequence, including: S31: The model input sequence and the model output sequence are standardized using the min-max normalization method, mapping all values ​​in the model input sequence and the model output sequence to the [0,1] interval, to obtain the normalized model input sequence and the normalized model output sequence.

[0040] In practical applications, based on the selected model input and output features, the sequences containing these features are extracted from the acquired dataset and used as the model input and output sequences. Considering the differences in units and magnitudes of various water quality parameters, direct training may lead to model instability, thereby weakening its generalization ability.

[0041] Step 31 uses the min-max normalization method to standardize the model input and output sequences, obtaining normalized model input and output sequences to eliminate the influence of dimensions on the training process. The normalization process is represented by formula (3): (3) in, These are the normalized values ​​for each water quality parameter. X These are the water quality parameter values ​​before normalization. It is the maximum value of the water quality parameter. This is the minimum value of the water quality parameter. After normalization, all values ​​in the model input and output sequences are mapped to the [0,1] interval.

[0042] In one exemplary embodiment, S4 includes: S41: The normalized model input sequence and the normalized model output sequence are divided by a time-by-time sliding segmentation method to generate several sample sets.

[0043] S42: Divide the sample set into a training sample set and a test sample set, and divide the test sample set from the training sample set.

[0044] In practical applications, step 41 involves using a time-sliding segmentation method to divide the normalized model input and output sequences obtained in S3, generating several sample sets.

[0045] Step 42: Divide the generated sample set into a training sample set and a test sample set according to a predetermined ratio, and then further divide the training sample set into a validation sample set according to a predetermined ratio. The training sample set is used to train the model parameters, and the validation sample set is used to verify the model's training effectiveness during the training process. The test sample set is used to test the trained model to verify and evaluate its simulated prediction performance.

[0046] In one exemplary embodiment, the LSTM-Transformer hybrid model includes a forward propagation process: S51, the LSTM module is used to extract and update the state of the training sample set to generate the LSTM hidden state corresponding to each time step.

[0047] S52, the LSTM hidden state is input into the Transformer module, and after the position order is assigned by position encoding, the feature interaction information is extracted by multi-head self-attention mechanism and feedforward neural network to generate the Transformer hidden state.

[0048] S53, Select the hidden state of the Transformer at the last time step, and map it through the fully linked layer module to obtain the simulation results of water quality parameters.

[0049] In practical applications, the LSTM-Transformer hybrid model consists of three modules: an LSTM module, a Transformer module, and a fully connected layer module. Specifically: First, the LSTM module's temporal processing capabilities are used to initially capture the temporal dependencies between water quality parameters in the training sample set; the LSTM module output is the LSTM hidden state. Second, a Transformer module is constructed, using the LSTM hidden state output as input. The Transformer module further extracts deeper feature interactions between the LSTM hidden states; the Transformer module output is the Transformer hidden state. Finally, the Transformer hidden state at the last time step is mapped through the fully connected layer module to obtain the simulation results of the water quality parameters. Specifically: Step 51: Initially capture the time dependence between water quality parameters in the training sample set based on the LSTM module.

[0050] This invention utilizes an LSTM module to extract and update the state of the training sample set at each time step to initially capture the dependencies between water quality parameters and generate corresponding LSTM hidden states for each time step. Compared to the traditional approach of directly using the mapping result of the last hidden state of the LSTM as the model output, this invention uses all LSTM hidden states from all time steps as the output, thus fully preserving the temporal order and interaction between features of the LSTM hidden states. This effectively alleviates the information decay and memory confusion problems that occur with the growth of the LSTM processing sequence at the output level.

[0051] Step 52: Based on the Transformer module, further extract deeper feature interaction information between the LSTM hidden states.

[0052] The Transformer module consists of two parts: a positional encoder and an encoder. The encoder mainly comprises a multi-head attention mechanism and a feedforward neural network. The multi-head self-attention mechanism runs multiple independent attention heads in parallel, with each attention head independently performing the complete self-attention mechanism computation. The feedforward neural network consists of two fully connected layers and an activation function. Layer normalization is applied after the multi-head attention mechanism and the feedforward neural network to stabilize the input distribution, accelerate convergence, and alleviate gradient problems.

[0053] The Transformer module takes the LSTM hidden states obtained in step 51 as input, with the input dimension matching the dimension of the LSTM hidden states. First, a positional encoding composed of a combination of sine and cosine functions is used, leveraging its periodicity to assign different positional orders to the LSTM hidden states. Then, the positionally ordered LSTM hidden states are input to the encoder, which utilizes a multi-head self-attention mechanism and a feedforward neural network to delve deeper into the feature interactions between the LSTM hidden states. Compared to the Transformer model that directly uses the raw data as input, this method not only alleviates the potential bias in capturing local features at the data level but also further enhances the model's representational ability. The Transformer module outputs the Transformer hidden states for all time steps. The information transfer within the Transformer module is represented as follows: (4) (5) (6) in, h For the number of attention heads, Attention Indicates the first h The attention output calculated by each attention head; It is the query matrix for each attention head. It is the key matrix for each attention head. It is the value matrix for each attention head; For the attention dimension; The operation normalizes each row of the matrix so that the sum of each row is 1; It is the final output of the multi-head attention mechanism; Indicates the first 1 The output of each attention head, Indicates the first 2 The output of each attention head, head h Indicates the first h The output of each attention head; Indicates feature concatenation operation; To output the weight matrix; FFN This represents the output of the feedforward neural network; x This represents the input vector of the feedforward neural network; This is the weight matrix of the first fully connected layer. It is the weight matrix of the first fully connected layer; It is the bias vector of the first fully connected layer. It is the bias vector of the second fully connected layer; ReLU This represents the activation function.

[0054] Step 53: The fully linked layer module maps and outputs the simulation results of water quality parameters.

[0055] Under the self-attention mechanism of the Transformer module, the Transformer hidden state output by the Transformer module at each time step is a weighted sum of the values ​​for the entire time step. The Transformer hidden state at the last time step not only aggregates the global dependency information for the entire time step but also obtains the last-position inductive bias for the entire time step through positional encoding. Therefore, the Transformer hidden state at the last time step is taken and mapped to the simulation results of water quality parameters through a fully connected layer module.

[0056] In an exemplary embodiment, step S5 involves iterative training using the training sample set and the validation sample set to obtain a trained LSTM-Transformer hybrid model, and further includes: S54: Set the hyperparameter search range within the LSTM module and the Transformer module.

[0057] S55: Set hyperparameter combinations for model training; the hyperparameter combinations include optimization algorithm, loss function, learning rate, weight decay, batch size, and number of iterations. A grid search method is used to iteratively train the hyperparameter combinations within the search range to obtain the trained LSTM-Transformer hybrid model.

[0058] In practical applications, based on the constructed LSTM-Transformer hybrid model, the search range of hyperparameters within the model is set, and the key configurations required for training are defined for model training; specifically: Step 61: Set the search range for hyperparameters within the model.

[0059] Based on the LSTM-Transformer hybrid model built using S5, the hyperparameter search range within the hybrid model was set. For the LSTM module, two hyperparameters were used for searching: the number of hidden layer neurons and the number of network layers. For the Transformer module, three hyperparameters were used for searching: the number of encoder layers, the attention head, and the hidden layer dimension of the feedforward neural network. A grid search method was used to traverse all combinations of hyperparameters within the model.

[0060] Step 62: Set the key configurations required for training and train the model.

[0061] Set up the key configurations required for training the hybrid model: optimization algorithm, loss function, learning rate, weight decay, batch size, and number of iterations. Based on this, iteratively train the LSTM-Transformer hybrid model using the training and validation sample sets from steps 41-42. A grid search method is used to traverse the hyperparameter combinations within the model to find the optimal configuration, ultimately obtaining the trained LSTM-Transformer hybrid model.

[0062] In one exemplary embodiment, after S6, the following is also included: S7: Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model by evaluating indicators, comparison charts of simulated and observed values, and scatter plots.

[0063] In practical applications, tests are conducted based on the trained LSTM-Transformer hybrid model. The simulation accuracy and application effect of the LSTM-Transformer hybrid model are evaluated by assessing the evaluation indicators, comparison charts of simulated and observed values, and scatter plots.

[0064] Step 71: Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model through evaluation metrics.

[0065] The test sample set generated in step 41 is used to drive the LSTM-Transformer hybrid model trained in step 62 to generate water quality parameter simulation results, and the baseline LSTM model and Transformer model are compared and analyzed. Both the LSTM and Transformer models are trained using the training and validation sample sets generated in steps 41-42 above, and tested with the LSTM-Transformer hybrid model on the same test sample set. The mean absolute error (MAE) is used. MAE ), root mean square error ( MSE ) and the coefficient of determination ( R 2 Three evaluation indicators quantify the simulation error of the model; Mean absolute error ( MAE ): (7) In equation (7), This represents the simulated value output by the model. Represents the observed value. n This represents the number of test samples calculated. MAE It is used to measure the magnitude of the average error between the simulated values ​​and the observed values ​​of a model. MAE The smaller the value, the higher the accuracy of the model.

[0066] Root mean square error (RMSE) MSE ): (8) The meanings of the parameters in equation (8) are the same as those in equation (8). MAE The calculation formulas are the same. MSE The average of the squared differences between simulated and observed values. MSE The smaller the value, the higher the accuracy of the model.

[0067] Coefficient of determination ( R 2 ): (9) In equation (9) This represents the mean of the observation sequence; the meanings of the other parameters are the same as... MAE The calculation formulas are the same. R 2 The value of generally varies between 0 and 1. R 2 =1 indicates that the simulated values ​​of the model perfectly match the observed values.

[0068] Step 72: Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model by comparing the simulated values ​​with the observed values ​​and using scatter plots.

[0069] Comparison plots and scatter plots of simulated and observed values ​​were created to compare the simulation results of the LSTM-Transformer hybrid model with those of the baseline model, analyzing the accuracy and application effectiveness of the LSTM-Transformer hybrid model's water quality parameter simulation results. In the comparison plots, the closer the model's simulated curve is to the actual observed curve, the higher the model's fitting accuracy. In the scatter plots, the diagonal line (y = x) represents the theoretically ideal simulation line. The closer the data points are to this line, the smaller the prediction error and the better the model's performance.

[0070] This application constructs a hybrid LSTM-Transformer model, combining the complementary advantages of LSTM and Transformer models, which excel in temporal correlation extraction. The hybrid model integrates the long short-term memory capability of LSTM with the global attention mechanism of Transformer. It utilizes the temporal processing capabilities of LSTM to initially capture the temporal dependencies between water quality parameters, and further extracts deeper feature interactions between LSTM hidden states based on Transformer modules. Trained using multi-parameter water quality monitoring data, the hybrid model can more effectively learn the complex nonlinear relationships between input features, achieving accurate simulation of river water quality parameters. Compared to single LSTM or Transformer models, this framework further improves the simulation accuracy of water quality parameters, providing scientific basis and technical support for river water quality monitoring and management decisions.

[0071] The technical solution of this application will be further illustrated below with specific examples.

[0072] The river generally flows from northwest to southeast. The upper reaches are fed by multiple mountain streams and tributaries, experiencing relatively little human impact and pollution, resulting in generally good water quality. The middle and lower reaches, flowing from north to south through several counties, are densely populated with towns and cities, and are a hub of agricultural and industrial activity, thus facing significant ecological and environmental pressures. Therefore, in this embodiment, a water quality monitoring station in the middle and lower reaches of this river is selected as a case study to simulate dissolved oxygen levels in the river using an LSTM-Transformer hybrid model. The geographical location of this monitoring station is shown below. Figure 2 The main steps are as follows: Step 1: Obtain multi-parameter water quality monitoring data, preprocess it, and obtain the dataset. Specifically: Step 1.1: Obtain multi-parameter water quality monitoring data from the target watershed stations.

[0073] The multi-parameter water quality monitoring data comes from the China National Environmental Monitoring Centre. The collected multi-parameter water quality monitoring data includes dissolved oxygen, water temperature, pH value, permanganate index, ammonia nitrogen, total phosphorus, total nitrogen, conductivity, and turbidity.

[0074] Step 1.2, preprocessing of multi-parameter water quality monitoring data.

[0075] The multi-parameter water quality monitoring data obtained in step 1.1 is preprocessed to obtain a dataset. Water quality monitoring station equipment needs to be maintained and repaired regularly, and the obtained multi-parameter water quality monitoring data often contains missing values ​​and outliers. The missing values ​​of the multi-parameter water quality monitoring data obtained in step 1.1 are filled in using linear interpolation to obtain a complete dataset. The linear interpolation is shown in formula (1).

[0076] In this embodiment, after data acquisition and preprocessing in step 1, a certain water quality monitoring station contains a total of 4264 multi-parameter water quality monitoring data, spanning from January 2021 to December 2022, with a sampling interval of 4 hours. Each record covers nine water quality parameters: dissolved oxygen, water temperature, pH value, permanganate index, ammonia nitrogen, total phosphorus, total nitrogen, conductivity, and turbidity.

[0077] Step 2: Based on the preprocessed dataset from Step 1, filter the model's input and output features; specifically: Water quality parameters are often correlated to varying degrees. Selecting water quality parameters that are highly correlated with the model output features as model inputs can help optimize the simulation process and improve simulation accuracy.

[0078] Step 2.1: First, clarify the output target of the model and select typical water quality parameters as the output features of the model.

[0079] Step 2.2 involves feature filtering of the dataset obtained in Step 1, and using Pearson correlation coefficient analysis to calculate and evaluate the correlation strength between the remaining water quality parameters and the selected model output features. The calculation formula for Pearson correlation coefficient analysis is shown in (2).

[0080] Dissolved oxygen is a key indicator for measuring the ecological health of rivers. The dissolved oxygen level in rivers reflects the balance between oxygen production and consumption in aquatic ecosystems, directly demonstrating the self-purification capacity of water bodies. Accurate simulation and prediction of river dissolved oxygen concentration is of great significance for the monitoring and management of river water environments. This embodiment selects dissolved oxygen as the model output feature and uses the Pearson correlation coefficient method to calculate the correlation strength between other water quality parameters and dissolved oxygen.

[0081] Step 2.3: Based on the ranking of the absolute values ​​of the correlation coefficients, select the model input features. The larger the absolute value of the r value, the stronger the correlation is usually. Select the water quality parameters with the highest absolute values ​​of r values ​​as the model input features.

[0082] For the calculation in step 2.2 r The absolute values ​​of the values ​​are ranked, and the final model input features are determined to be five parameters: water temperature, pH value, total nitrogen, conductivity, and turbidity.

[0083] Step 3: Based on the model input and output features selected in Step 2, extract the corresponding model input and output sequences, and normalize the extracted model input and output sequences; specifically: Based on the model input and output features selected in step 2, the corresponding model input and output sequences are extracted from the dataset obtained in step 1. Considering the differences in units and magnitudes of different water quality parameters, direct training may lead to model instability, thereby weakening its generalization ability. Therefore, the min-max normalization method is used to standardize the model input and output sequences to obtain normalized model input and output sequences, so as to eliminate the influence of the units on the training process. The normalization process is represented by formula (3).

[0084] Step 4: Based on the normalized model input and output sequences obtained in Step 3, generate training, validation, and test sample sets; specifically: Step 4.1: Divide the normalized model input and output sequences obtained in Step 3 into several sample sets by using a time-sliding segmentation method.

[0085] Step 4.2 involves dividing the generated sample set into a training sample set and a test sample set according to a predetermined ratio. Then, a validation sample set is further divided from the training sample set according to a set ratio. The training sample set is used to train the model parameters, while the validation sample set is used to verify the model's training effectiveness during the training process. Finally, the trained model is tested using the test sample set to verify and evaluate its simulated prediction performance.

[0086] In this embodiment, a time-step sliding segmentation method is used to generate the sample set, with a sliding window length of 120 time steps. The structure of the generated samples is as follows: Input t =[ ], for t Five input variables at time 1, the simulation target is tDissolved oxygen at any given time. The last 10% of all generated samples were used as the test sample set, and the first 90% of the generated samples were randomly shuffled and divided into training and validation sample sets in an 8:2 ratio. The final ratio of the generated training, validation, and test sample sets was 7.8:1.2:1. Table 1 shows the total number of sample sets generated at this site and the number of sample sets divided.

[0087] Table 1: Total number of generated sample sets and number of sample sets partitioned

[0088] Step 5: Based on the training, validation and test sample sets obtained in Step 4, construct the LSTM-Transformer hybrid model required for model training.

[0089] The LSTM-Transformer hybrid model consists of three modules: an LSTM module, a Transformer module, and a fully connected layer module. Specifically: First, the LSTM module's temporal processing capabilities are used to initially capture the temporal dependencies between water quality parameters in the training sample set; the LSTM module output is the LSTM hidden state. Second, a Transformer module is constructed, using the LSTM hidden state output as input. The Transformer module further extracts deeper feature interactions between the LSTM hidden states; the Transformer module output is the Transformer hidden state. Finally, the Transformer hidden state at the last time step is mapped through the fully connected layer module to obtain the simulation results of the water quality parameters. Specifically: Step 5.1: Initially capture the time dependence between water quality parameters in the training sample set based on the LSTM module.

[0090] This invention utilizes an LSTM module to extract and update the state of the training sample set at each time step to initially capture the dependencies between water quality parameters and generate corresponding LSTM hidden states for each time step. Compared to the traditional approach of directly using the mapping result of the last hidden state of the LSTM as the model output, this invention uses all LSTM hidden states from all time steps as the output, thus fully preserving the temporal order and interaction between features of the LSTM hidden states. This effectively alleviates the information decay and memory confusion problems that occur with the growth of the LSTM processing sequence at the output level.

[0091] Step 5.2: Based on the Transformer module, further extract deeper feature interaction information between the LSTM hidden states.

[0092] The Transformer module consists of two parts: a positional encoder and an encoder. The encoder mainly comprises a multi-head attention mechanism and a feedforward neural network. The multi-head self-attention mechanism runs multiple independent attention heads in parallel, with each attention head independently performing the complete self-attention mechanism computation. The feedforward neural network consists of two fully connected layers and an activation function. Layer normalization is applied after the multi-head attention mechanism and the feedforward neural network to stabilize the input distribution, accelerate convergence, and alleviate gradient problems.

[0093] The Transformer module takes the LSTM hidden states obtained in step 5.1 as input, with the input dimension consistent with the dimension of the LSTM hidden states. First, a positional encoding composed of a combination of sine and cosine functions is used to assign different positional orders to the LSTM hidden states by utilizing their periodicity. Then, the LSTM hidden states with assigned positional orders are input into the encoder, which uses a multi-head self-attention mechanism and a feedforward neural network to delve deeper into the feature interaction information between the LSTM hidden states. Compared to the Transformer model that directly uses the original data as input, this method not only alleviates the potential bias problem of the Transformer in capturing local features at the data level, but also further improves the model's representational ability. The output of the Transformer module is the Transformer hidden states at all time steps. The information transmission within the Transformer module is as shown in equations (4)-(6).

[0094] Step 5.3: The fully linked layer module maps and outputs the simulation results of water quality parameters.

[0095] Under the self-attention mechanism of the Transformer module, the Transformer hidden state output at each time step is a weighted summation of the entire time step. The Transformer hidden state at the last time step not only aggregates the global dependency information of the entire time step but also obtains the last-position inductive bias through positional encoding. Therefore, the Transformer hidden state at the last time step is taken and mapped to the simulation results of water quality parameters through a fully connected layer module. The constructed LSTM-Transformer hybrid model is as follows: Figure 3 As shown.

[0096] Step 6: Based on the LSTM-Transformer hybrid model constructed in Step 5, set the search range of hyperparameters within the model and configure the key settings required for training; specifically: Step 6.1: Set the search range for hyperparameters within the model.

[0097] Based on the LSTM-Transformer hybrid model constructed in step 5, the search range for hyperparameters within the model is set. The LSTM module searches for two hyperparameters: the number of hidden layer neurons and the number of network layers. The Transformer module searches for three hyperparameters: the number of encoder layers, the attention head, and the hidden layer dimension of the feedforward neural network. A grid search method is used to traverse all combinations of hyperparameters within the model.

[0098] Step 6.2: Set the key configurations required for training and train the model.

[0099] Set the key configurations required for model training: optimization algorithm, loss function, learning rate, weight decay, batch size, and number of iterations. Based on this, iteratively train the LSTM-Transformer hybrid model using the training and validation sample sets from step 4. Employ a grid search method to traverse the hyperparameter combinations within the model to find the optimal configuration, ultimately obtaining the trained LSTM-Transformer hybrid model.

[0100] For the LSTM module in this embodiment, a single-layer LSTM structure was used, and the number of hidden layer neurons was tested within the range of 2 to 64. The Transformer module also adopted a single-layer encoder structure and set two attention heads to achieve a balance between model capacity and training stability. Following empirical design principles, the hidden layer dimension of the feedforward neural network was set to 2 to 4 times the input dimension, and the test range was 8 to 256. For details on the input and output structure of the LSTM-Transformer hybrid model, please refer to... Figure 4 .

[0101] This embodiment uses the Adam optimization algorithm for model training, with an initial learning rate of 0.001 and a weight decay coefficient of 0.0001 introduced to prevent overfitting. The root mean square error (RMSE) is selected. MSE The loss function is set to 16 / 32, and the number of iterations is set to 150. During training, the loss value is continuously monitored, and the model parameters corresponding to the minimum loss on the validation sample set are saved as the optimal configuration.

[0102] Step 7: Test the LSTM-Transformer hybrid model trained in Step 6. Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model by evaluating the evaluation index, the comparison chart of simulated values ​​and observed values ​​and the scatter plot.

[0103] Step 7.1: Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model through evaluation metrics.

[0104] The test sample set generated in step 4 is used to drive the LSTM-Transformer hybrid model trained in step 6 to generate water quality parameter simulation results, and the baseline LSTM model and Transformer model are compared and analyzed. Both the LSTM and Transformer models are trained using the training and validation sample sets generated in step 4, and tested with the LSTM-Transformer hybrid model on the same test sample set. The mean absolute error (MAE) is used as the statistical metric. MAE ), root mean square error ( MSE ) and the coefficient of determination ( R 2 The simulation error of the three evaluation indicators is quantified as shown in equations (7) to (8).

[0105] Step 7.2: Evaluate the simulation accuracy and application effect of the LSTM-Transformer hybrid model by comparing the simulated values ​​with the observed values ​​and using scatter plots.

[0106] Comparison plots and scatter plots of simulated and observed values ​​were created to compare the simulation results of the LSTM-Transformer hybrid model with those of the baseline model, analyzing the accuracy and application effectiveness of the LSTM-Transformer hybrid model's water quality parameter simulation results. In the comparison plots, the closer the model's simulated curve is to the actual observed curve, the higher the model's fitting accuracy. In the scatter plots, the diagonal line (y = x) represents the theoretically ideal simulation line. The closer the data points are to this line, the smaller the prediction error and the better the model's performance.

[0107] In this embodiment, the evaluation metrics of the LSTM-Transformer hybrid model and the single benchmark model on the test sample set of this site are shown in Table 2. As can be seen from Table 2, the three models... R 2 All metrics are greater than 0.70, indicating that the model construction is reasonable and possesses good interpretability. Compared to the baseline model, the LSTM-Transformer hybrid model exhibits the best simulation results. Specifically, its MAE Compared to LSTM and Transformer models, the performance was reduced by 0.0524 and 0.0646 respectively. MSE They decreased by 0.0277 and 0.0596 respectively. R 2 These figures represent increases of 0.0723 and 0.0494, respectively.

[0108] Table 2: Dissolved oxygen simulation results for the test sample set MAE, MSE and RMSE

[0109] The simulation results of the LSTM-Transformer hybrid model and the single benchmark model on the test sample set at this site are compared, for example... Figure 5 As shown, the LSTM-Transformer hybrid model has better curve fitting performance than other models. Specifically, at multiple peaks, the simulated dissolved oxygen values ​​are more closely matched with the observed values, and the overall dissolved oxygen concentration change curve is more consistent with the actual observed values.

[0110] Figure 6 , Figure 7 , Figure 8 Scatter plots showing the simulation results for LSTM, Transformer, and LSTM-Transformer hybrid models, respectively. Figures 6-8 The scatter plot shown demonstrates that the LSTM-Transformer hybrid model exhibits a more accurate distribution pattern compared to the baseline model. More data points are densely clustered near the diagonal of y=x, exhibiting the lowest dispersion and strongest convergence. This simulation result validates the accuracy and efficiency of the LSTM-Transformer hybrid model.

[0111] The above results demonstrate that the proposed LSTM-Transformer hybrid model-based river water quality simulation method, used for dissolved oxygen simulation at a water quality monitoring station, achieves the lowest loss index and highest fitting accuracy on the test sample set. The simulated dissolved oxygen curve most closely matches the observed values, resulting in the best overall simulation results. This example verifies that the invented LSTM-Transformer hybrid model can meet the simulation requirements for water quality parameters.

[0112] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores data to be processed. The I / O interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal via a network connection. When the computer program is executed by the processor, it implements the above-described methods.

[0113] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0114] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0116] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0117] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0118] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for simulating river water quality based on an LSTM-Transformer hybrid model, characterized in that, include: Acquire multi-parameter water quality monitoring data from stations in the target watershed and preprocess them to obtain a dataset; Filter the model input features and model output features based on the dataset; Based on the model input features and the model output features, the model input sequence and the model output sequence are extracted, and the model input sequence and the model output sequence are normalized to obtain the normalized model input sequence and the normalized model output sequence. The normalized model input sequence and the normalized model output sequence are divided into a training sample set, a validation sample set, and a test sample set; Construct an LSTM-Transformer hybrid model that includes an LSTM module, a Transformer module, and a fully connected layer module, and perform iterative training using the training sample set and the validation sample set to obtain a trained LSTM-Transformer hybrid model. Based on the test sample set and the trained LSTM-Transformer hybrid model, the simulation results of water quality parameters are obtained.

2. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, Multi-parameter water quality monitoring data from stations in the target watershed were acquired and preprocessed to obtain a dataset, including: Acquire multi-parameter water quality monitoring data from stations in the target watershed; The missing values ​​of the multi-parameter water quality monitoring data were imputed using linear interpolation to obtain the dataset.

3. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, Based on the dataset, the model input features and model output features are filtered, including: Typical water quality parameters are selected as features for model output. The correlation coefficient between the remaining water quality parameters and the model output characteristics was calculated using the Pearson correlation coefficient analysis method. The input features of the model are selected based on the ranking of the absolute values ​​of the correlation coefficients.

4. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, The model input sequence and model output sequence are normalized to obtain the normalized model input sequence and the normalized model output sequence, including: The model input sequence and the model output sequence are standardized using the min-max normalization method, which maps all values ​​in the model input sequence and the model output sequence to the interval [0,1], resulting in the normalized model input sequence and the normalized model output sequence.

5. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, The normalized model input sequence and the normalized model output sequence are divided into a training sample set, a validation sample set, and a test sample set, including: The normalized model input sequence and the normalized model output sequence are divided by a time-sliding segmentation method to generate several sample sets; The sample set is divided into a training sample set and a test sample set, and a validation sample set is extracted from the training sample set.

6. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, The LSTM-Transformer hybrid model includes a forward propagation process: The LSTM module is used to extract and update the state of the training sample set, generating the LSTM hidden state corresponding to each time step; The LSTM hidden state is input into the Transformer module. After the position order is assigned by position encoding, the feature interaction information is extracted using a multi-head self-attention mechanism and a feedforward neural network to generate the Transformer hidden state. The hidden state of the Transformer at the last time step is selected and mapped through the fully linked layer module to obtain the simulation results of water quality parameters; Iterative training is performed using the training sample set and the validation sample set to obtain a trained LSTM-Transformer hybrid model, specifically including: Set the hyperparameter search range within the LSTM module and the Transformer module; The model is trained by setting a combination of hyperparameters; the combination of hyperparameters includes optimization algorithm, loss function, learning rate, weight decay, batch size and number of iterations; the hyperparameters within the search range are traversed by a grid search method for iterative training to obtain the trained LSTM-Transformer hybrid model.

7. The river water quality simulation method based on the LSTM-Transformer hybrid model according to claim 1, characterized in that, After obtaining the simulation results of water quality parameters based on the test sample set and the trained LSTM-Transformer hybrid model, the process also includes: The simulation accuracy and application effect of the LSTM-Transformer hybrid model are evaluated by assessing the evaluation indicators, comparing simulated values ​​with observed values, and using scatter plots.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the river water quality simulation method based on the LSTM-Transformer hybrid model as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the river water quality simulation method based on the LSTM-Transformer hybrid model as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the river water quality simulation method based on the LSTM-Transformer hybrid model as described in any one of claims 1-7.

Images