Large-scale watershed water quality simulation method and system based on CNN-Transformer

Abstract

The present invention discloses a large-scale basin water quality simulation method and system based on CNN-Transformer, including: obtaining meteorological and hydrological driving data, water quality data and basin attribute data of multiple basins; preprocessing the data, dividing the processed data into training set and test set according to a certain ratio; embedding the three-dimensional time series data of the training set; encoding and linear transformation of the data; training the model, obtaining the final CNN-Transformer water quality simulation model according to the evaluation index of the training result, and inputting the data in the test set into the model to obtain the simulated value of the basin water quality. The present invention combines the Tansformer model with the convolutional neural network to better handle some water quality simulation problems with non-stationary and nonlinear relationships, and better capture the interdependence between multivariate variables.

Description

Technical Field

[0001] The present invention belongs to the technical field of water quality simulation, and specifically relates to a large-scale watershed water quality simulation method and system based on CNN-Transformer. Background Art

[0002] Watershed water quality simulation is a method of evaluating and predicting water quality by establishing a mathematical model to simulate the hydrological process, pollutant transport and transformation process of water bodies in a watershed. Watershed water quality indicators include temperature, dissolved oxygen, nitrogen compounds and organic pollutants. Watershed water quality indicators directly affect the survival of organisms and ecological balance in water bodies. For example, the reduction of dissolved oxygen in water bodies will force aquatic animals to change their breathing patterns or reduce their activity levels, which will lead to delayed development, some reproductive problems (such as increased egg mortality and defects) and deformity problems. In addition, industrial, agricultural and urbanization activities have caused a large number of pollutants to enter water bodies, such as nutrients (nitrogen, phosphorus), heavy metals, organic pollutants, etc. These pollutants not only threaten the ecological health of water bodies, but also have a negative impact on human survival. Watershed water quality simulation can assess the health of water quality, thereby maintaining the stability of the ecosystem, which is of great significance to human sustainable development.

[0003] Common water quality simulation models include QUAL2E, WASP, SWAT, etc. These traditional water quality simulation models calculate water quality indicators based on some physical and chemical formulas and a large number of related parameters. However, these models have some problems. For example, the model database depends on the properties and structure of the river basin in a specific area, which may not be well adapted to the water environment simulation of river basins in other areas, resulting in low simulation accuracy and other problems. In addition, these models rely on a large number of physical and chemical parameters. How to obtain and process these parameters is also an issue that needs to be carefully considered in realistic water environment simulation research. Traditional methods are usually based on statistical analysis and assumptions of time series, such as stationarity and linear relationships. It is difficult to deal with non-stationary and nonlinear water quality simulation and prediction problems, and it is also difficult to capture the interdependence between complex multivariate variables.

[0004] In recent years, the Transformer model based on the self-attention mechanism has achieved the top level in the field of natural language processing. Studies have shown that it is also highly efficient in processing time series problems. Convolutional neural networks (CNNs) have achieved great success in the field of image recognition and processing, and have the ability to efficiently extract features. This invention combines convolutional neural networks with the Transformer model and applies them to the field of watershed water environment simulation. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a large-scale river basin water quality simulation method and system based on CNN-Transformer, so as to solve the problem that it is difficult to simulate the water quality indicators of multiple river basins at the same time in the prior art, the difficulty in obtaining a large number of parameters and the difficulty in calibrating parameters in traditional models, and the difficulty in capturing the complex relationship between multivariate variables in long-distance water quality time series problems.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A large-scale watershed water quality simulation method based on CNN-Transformer of the present invention comprises the following steps:

[0008] 1) Obtain meteorological and hydrological driving data, water quality data and basin attribute data for multiple basins over a large area within a certain period of time;

[0009] 2) Preprocess the data obtained in step 1), including deleting duplicate time series data, deleting erroneous records, deleting outliers and standardizing data, to obtain preprocessed meteorological and hydrological driving data, water quality data and watershed attribute data; then convert the date of the data into time characteristic data; divide the processed meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data into training sets and test sets according to a certain ratio;

[0010] 3) Embed the three-dimensional time series data of the training set in step 2);

[0011] 4) Input the data processed in step 3) into the encoder, and then perform linear transformation on the data processed by the encoder;

[0012] 5) Perform training according to the set number of iterations and the corresponding loss function, repeat steps 3) and 4) to obtain the initial CNN-Transformer water quality simulation model; use different combinations of hyperparameters for training to obtain multiple initial CNN-Transformer water quality simulation models, and select the CNN-Transformer water quality simulation model with the best performance on the test set as the final CNN-Transformer water quality simulation model according to the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, input the data in the test set into the final CNN-Transformer water quality simulation model, and then inversely standardize the output of the model to obtain the simulated value of the water quality of the basin.

[0013] Furthermore, the meteorological and hydrological driving data are time series data, including precipitation, snow water equivalent, surface air pressure, maximum temperature, minimum temperature, daily average temperature and wind speed component.

[0014] Furthermore, the water quality data is time series data, including dissolved oxygen and water temperature; the water quality data is non-daily continuous data, and its observation values ​​exist only in a limited number of dates.

[0015] Furthermore, the watershed attribute data are divided into five groups, including topography, climate, hydrology, land use and soil; specific data features include: coordinates, altitude, slope, annual snow cover percentage, long-term average annual and monthly temperature, precipitation, flow, base flow index, land use percentage of agriculture, bare land, forest, glacier, lake and city, soil depth, and soil component percentages of sand, silt, clay and organic carbon.

[0016] Furthermore, the data standardization process specifically includes: Z-score Standardization converts the data into a distribution with a mean of 0 and a standard deviation of 1, and ensures that the data has a similar scale and distribution; the calculation formula is as follows:

[0017] ;

[0018] in, x is the original data, is the average value of the original data, is the standard deviation of the original data, is the processed data.

[0019] Furthermore, the specific process of converting the date of the data into time feature data is: by judging whether the current date is a specific date of the week, the days of the week are mapped to the range of [-0.5, 0.5], and the formula is expressed as follows:

[0020] ;

[0021] in, Represents the time feature data after mapping the days of the week, Indicates a specific day of the week. Indicates the total number of days in a week;

[0022] By judging whether the current date is a specific date in a month, the number of days in a month is mapped to the range of [-0.5, 0.5]. The formula is as follows:

[0023] ;

[0024] in, Represents the time feature data after mapping the days in a month. Indicates a specific date in January. Indicates the total number of days in a month;

[0025] By judging whether the current date is a specific date in a year, the number of days in a year is mapped to the range of [-0.5, 0.5]. The formula is as follows:

[0026] ;

[0027] in, Represents the time feature data after mapping the days of the year. Indicates a specific date of the year. Represents the total number of days in a year.

[0028] Furthermore, the meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data obtained after preprocessing in step 2) are all three-dimensional time series data, wherein the first dimension represents different watersheds, the second dimension represents time, and the third dimension represents features and attributes, thereby being able to train data from multiple watersheds at one time and capture the synergistic effects between multiple watersheds.

[0029] Furthermore, the specific process of data embedding in step 3) is: position encoding, time encoding and linear transformation of the input data.

[0030] Furthermore, the step 3) specifically includes:

[0031] 31) Perform position encoding on the meteorological and hydrological driving data and watershed attribute data obtained after processing in step 2) to obtain a position encoding vector. The specific formula is as follows:

[0032] ;

[0033] in, POS Indicates the relative position of the encoded data in all time series, represents the dimension of the time series vector, d represents the dimension of the position encoding vector;

[0034] 32) Temporally encode the temporal feature data obtained in step 2) and transform the temporal feature data into dimensional time encoding vector;

[0035] 33) The meteorological and hydrological driving data and watershed attribute data are transformed into d dimensional vector, and add it to the position encoding vector and the time encoding vector obtained in step 31) and step 32) to obtain a data embedding vector, and pass the data embedding vector as input to the dropout layer, randomly setting the output of some neurons to zero to reduce the complex co-adaptation relationship between neurons and improve the generalization ability.

[0036] Furthermore, the encoder consists of multiple encoder layers, each encoder layer includes an attention layer, a residual link and layer normalization and a CNN layer.

[0037] Furthermore, the step 4) specifically includes:

[0038] 41) Input the processed data in step 3) into the attention layer of the encoder to calculate the attention score; project the data to obtain the query tensor , the key tensor Sum value tensor , the calculation formula is as follows:

[0039] ;

[0040] in, X For input data, are parameters that can be learned by the model, i =1,2,..., H , H Indicates the number of attention heads;

[0041] Calculate the attention score for each attention head scores i , the formula is as follows:

[0042] ;

[0043] The attention weight is calculated using the softmax function for the attention score of each attention head. A i , the formula is as follows:

[0044] ;

[0045] Weight each value tensor using the attention weights V i Get the output of each attention head, the formula is as follows:

[0046] ;

[0047] in, is the output of each attention head;

[0048] The outputs of all attention heads are concatenated and linearly transformed to obtain the final multi-head attention output, as follows:

[0049] ;

[0050] in, is the multi-head attention output, is the concatenation function, are parameters available for training;

[0051] 42) Perform residual connection on the multi-head attention output obtained in step 41) and then perform layer normalization;

[0052] 43) The data obtained in step 42) is passed through the CNN layer to capture features: through a convolution kernel size of 1 and the number of input channels d , the number of output channels is The one-dimensional convolution layer is used for feature extraction, and the data is then processed by the activation function. The dropout layer is used to prevent data overfitting. The activation function uses the ReLU function or the GELU function. The output is passed through a convolution kernel with a size of 1 and an input channel number of , the number of output channels is d The one-dimensional convolution layer and dropout layer are processed to transform the data from Dimensions are remapped to d Dimensionality to capture features of different scales in sequence data and enhance the representation ability of the model;

[0053] 44) adding the output of the two convolutional layers in step 43) to the output after layer normalization in step 42), and normalizing again to obtain the final encoder layer output;

[0054] 45) Pass the output result obtained in step 44) to the next encoder layer, repeat steps 41) to 44) and repeat Second-rate, is the number of encoder layers, after The output of the encoder is obtained by processing the encoder layers;

[0055] 46) Perform a linear transformation on the output of the encoder in step 45) from Dimensions are mapped to Dimension, where Indicates the number of water quality indicators that need to be simulated.

[0056] Furthermore, the loss function selected in step 5) is the standardized mean square error, and the calculation formula is:

[0057] ;

[0058] in, represents the standardized mean square error, Represents the analog value, represents the observed value, Represents the standard deviation of the observations.

[0059] Furthermore, according to the set number of iterations, step 3)-step 4) are repeated. When the last iteration is completed, the training is stopped and the model structure is saved to obtain the initial CNN-Transformer water quality simulation model.

[0060] Furthermore, the hyperparameters include hidden layer size, training set batch size, number of iterations, and preheating length of model training; using different hyperparameter combinations, repeating steps 3) and 4) for training, and obtaining multiple initial CNN-Transformer water quality simulation models.

[0061] Furthermore, the evaluation indicators selected in step 5) are Nash efficiency coefficient and root mean square error, and the formulas are as follows:

[0062] ;

[0063] in, NSE represents the Nash efficiency coefficient, RMSE represents the root mean square error, represents the simulated value of dissolved oxygen, represents the observed value of dissolved oxygen, Indicates the number of dissolved oxygen observations; the test set data is input into multiple initial CNN-Transformer water quality simulation models to obtain the simulated value of dissolved oxygen, and the simulated value and observed value of dissolved oxygen are calculated NSE or RMSE ,choose NSE or RMSE The best performing model is selected and its model structure is saved to obtain the final CNN-Transformer water quality simulation model.

[0064] The present invention also provides a large-scale river basin water quality simulation system based on CNN-Transformer, comprising: a data acquisition module, a data processing module, a data embedding module, a data encoding module and a simulation module;

[0065] The data acquisition module is used to obtain meteorological and hydrological driving data, water quality data and basin attribute data of multiple basins in a large range within a certain period of time;

[0066] The data processing module is used to pre-process the acquired data to obtain pre-processed meteorological and hydrological driving data, water quality data and watershed attribute data; then convert the date of the data into time characteristic data; and divide the processed meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data into a training set and a test set according to a certain ratio;

[0067] A data embedding module is used to embed the three-dimensional time series data of the training set;

[0068] A data encoding module, used to input the embedded data into an encoder and then perform a linear transformation on the data processed by the encoder;

[0069] The simulation module is used to perform training according to the set number of iterations and the corresponding loss function to obtain the initial CNN-Transformer water quality simulation model; multiple initial CNN-Transformer water quality simulation models are obtained by training with different combinations of hyperparameters; according to the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, the CNN-Transformer water quality simulation model with the best performance on the test set is selected as the final CNN-Transformer water quality simulation model, the data in the test set is input into the final CNN-Transformer water quality simulation model, and the output of the model is then inversely standardized to obtain the simulated value of the water quality of the basin.

[0070] Beneficial effects of the present invention:

[0071] 1. The present invention adopts a deep learning algorithm to solve the problem that traditional water quality simulation models require the calibration of a large number of parameters, thereby improving the efficiency of simulation and reducing the difficulty of simulation.

[0072] 2. The water quality simulation model of the present invention combines the Tansformer model with the convolutional neural network, which can better handle some water quality simulation problems with non-stationary and nonlinear relationships, and can better capture the interdependence between multivariate variables.

[0073] 3. The water quality simulation model of the present invention can input data from multiple river basins at one time. At the same time, the time scale of the data is large. Therefore, it can reflect the spatial and temporal changes of river basin water quality on a large scale, capture the synergistic effect between adjacent river basins, and improve the simulation effect of the model.

[0074] 4. The present invention can simulate water quality indicators of multiple river basins at the same time, and performs well in different river basins and is transferable. The present invention solves the problem that many traditional models cannot establish reliable water quality simulation models in areas where water quality data are sparse and missing, and has great reference value for water quality simulation research in these areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 Flow chart of the method of the present invention.

[0076] Figure 2 It is a time series diagram of the simulated and observed dissolved oxygen values ​​in the watershed of the present invention. DETAILED DESCRIPTION

[0077] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0078] Reference Figure 1 As shown, a large-scale watershed water quality simulation method based on CNN-Transformer of the present invention comprises the following steps:

[0079] 1) Obtain meteorological and hydrological driving data, water quality data and basin attribute data for multiple basins over a large area within a certain period of time;

[0080] The meteorological and hydrological driving data are time series data, including precipitation, snow water equivalent, surface air pressure, maximum temperature, minimum temperature, daily average temperature and wind speed component.

[0081] The water quality data are time series data, including dissolved oxygen and water temperature; the water quality data are non-daily continuous data, and they have observation values ​​only on a limited number of dates.

[0082] The watershed attribute data are divided into five groups, including topography, climate, hydrology, land use and soil; specific data features include: coordinates, altitude, slope, annual snow cover percentage, long-term average annual and monthly temperature, precipitation, flow, base flow index, land use percentage of agriculture, bare land, forest, glacier, lake and city, soil depth, and soil component percentage of sand, silt, clay and organic carbon.

[0083] 2) Preprocess the data obtained in step 1), including deleting duplicate time series data, deleting erroneous records, deleting outliers and standardizing data, to obtain preprocessed meteorological and hydrological driving data, water quality data and watershed attribute data; then convert the date of the data into time characteristic data; divide the processed meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data into training sets and test sets according to a certain ratio;

[0084] Data standardization processing specifically includes: Z-score Standardization converts the data into a distribution with a mean of 0 and a standard deviation of 1, and ensures that the data has a similar scale and distribution; the calculation formula is as follows:

[0085] ;

[0086] in, x is the original data, is the average value of the original data, is the standard deviation of the original data, is the processed data.

[0087] The specific process of converting the date of the data into time feature data is: by judging whether the current date is a specific date of the week, the number of days of the week (the first day is 0, the second day is 1, and so on) is mapped to the range of [-0.5, 0.5]. The formula is as follows:

[0088] ;

[0089] in, Represents the time feature data after mapping the days of the week, Indicates a specific day of the week. Indicates the total number of days in a week;

[0090] By judging whether the current date is a specific date in a month, the number of days in a month is mapped to the range of [-0.5, 0.5]. The formula is as follows:

[0091] ;

[0092] in, Represents the time feature data after mapping the days in a month. Indicates a specific date in January. Indicates the total number of days in a month;

[0093] By judging whether the current date is a specific date in a year, the number of days in a year is mapped to the range of [-0.5, 0.5]. The formula is as follows:

[0094] ;

[0095] in, Represents the time feature data after mapping the days of the year. Indicates a specific date of the year. Represents the total number of days in a year.

[0096] The meteorological and hydrological driving data, water quality data, watershed attribute data, and time characteristic data obtained after preprocessing are all three-dimensional time series data, in which the first dimension represents different watersheds, the second dimension represents time, and the third dimension represents features and attributes. This makes it possible to train data from multiple watersheds at one time and capture the synergistic effects between multiple watersheds.

[0097] 3) Embed the three-dimensional time series data of the training set in step 2);

[0098] The specific process of data embedding is: position encoding, time encoding and linear transformation of input data; specifically including:

[0099] 31) Perform position encoding on the meteorological and hydrological driving data and watershed attribute data obtained after processing in step 2) to obtain a position encoding vector. The specific formula is as follows:

[0100] ;

[0101] in, POS Indicates the relative position of the encoded data in all time series, represents the dimension of the time series vector, d represents the dimension of the position encoding vector;

[0102] 32) Temporally encode the temporal feature data obtained in step 2) and transform the temporal feature data into dimensional time encoding vector;

[0103] 33) The meteorological and hydrological driving data and watershed attribute data are transformed into d dimensional vector, and add it to the position encoding vector and the time encoding vector obtained in step 31) and step 32) to obtain a data embedding vector, and pass the data embedding vector as input to the dropout layer, randomly setting the output of some neurons to zero to reduce the complex co-adaptation relationship between neurons and improve the generalization ability.

[0104] 4) Input the data processed in step 3) into the encoder, and then perform linear transformation on the data processed by the encoder;

[0105] The encoder is composed of multiple encoder layers, each of which includes an attention layer, a residual link and a layer normalization and a CNN layer; specifically, it includes:

[0106] 41) Input the processed data in step 3) into the attention layer of the encoder to calculate the attention score; project the data to obtain the query tensor , the key tensor Sum value tensor , the calculation formula is as follows:

[0107] ;

[0108] in, X For input data, are parameters that can be learned by the model, i =1,2,..., H , H Indicates the number of attention heads;

[0109] Calculate the attention score for each attention head scores i , the formula is as follows:

[0110] ;

[0111] The attention weight is calculated using the softmax function for the attention score of each attention head. A i , the formula is as follows:

[0112] ;

[0113] Weight each value tensor using the attention weights V i Get the output of each attention head, the formula is as follows:

[0114] ;

[0115] in, is the output of each attention head;

[0116] The outputs of all attention heads are concatenated and linearly transformed to obtain the final multi-head attention output, as follows:

[0117] ;

[0118] in, is the multi-head attention output, is the concatenation function, are parameters available for training;

[0119] 42) Perform residual connection on the multi-head attention output obtained in step 41) and then perform layer normalization;

[0120] 43) The data obtained in step 42) is passed through the CNN layer to capture features: through a convolution kernel size of 1 and the number of input channels d , the number of output channels is The one-dimensional convolution layer is used for feature extraction, and the data is then processed by the activation function. The dropout layer is used to prevent data overfitting. The activation function uses the ReLU function or the GELU function. The output is passed through a convolution kernel with a size of 1 and an input channel number of , the number of output channels is d The one-dimensional convolution layer and dropout layer are processed to transform the data from Dimensions are remapped to d Dimensionality to capture features of different scales in sequence data and enhance the representation ability of the model;

[0121] 44) adding the output of the two convolutional layers in step 43) to the output after layer normalization in step 42), and normalizing again to obtain the final encoder layer output;

[0122] 45) Pass the output result obtained in step 44) to the next encoder layer, repeat steps 41) to 44) and repeat Second-rate, is the number of encoder layers, after The output of the encoder is obtained by processing the encoder layers;

[0123] 46) Perform a linear transformation on the output of the encoder in step 45) from Dimensions are mapped to Dimension, where Indicates the number of water quality indicators that need to be simulated.

[0124] 5) Perform training according to the set number of iterations and the corresponding loss function, repeat steps 3) and 4), and obtain the initial CNN-Transformer water quality simulation model; use different combinations of hyperparameters to train to obtain multiple initial CNN-Transformer water quality simulation models, and select the CNN-Transformer water quality simulation model with the best performance on the test set as the final CNN-Transformer water quality simulation model according to the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, input the data in the test set into the final CNN-Transformer water quality simulation model, and then inversely standardize the output of the model to obtain the simulated value of the water quality of the basin;

[0125] The selected loss function is the standardized mean square error, and the calculation formula is:

[0126] ;

[0127] in, represents the standardized mean square error, Represents the analog value, represents the observed value, Represents the standard deviation of the observations.

[0128] Repeat steps 3) to 4) according to the set number of iterations. When the last iteration is completed, stop training and save the model structure to obtain the initial CNN-Transformer water quality simulation model.

[0129] The hyperparameters include hidden layer size, training set batch size, number of iterations, and preheating length of model training; using different hyperparameter combinations, repeating steps 3) and 4) for training, and obtaining multiple initial CNN-Transformer water quality simulation models.

[0130] The selected evaluation indicators are Nash efficiency coefficient and root mean square error, and the formulas are as follows:

[0131] ;

[0132] in, NSE represents the Nash efficiency coefficient, RMSE represents the root mean square error, represents the simulated value of dissolved oxygen, represents the observed value of dissolved oxygen, Indicates the number of dissolved oxygen observations; the test set data is input into multiple initial CNN-Transformer water quality simulation models to obtain the simulated value of dissolved oxygen, and the simulated value and observed value of dissolved oxygen are calculated NSE or RMSE ,choose NSE or RMSE The best performing model is selected and its model structure is saved to obtain the final CNN-Transformer water quality simulation model.

[0133] The present invention is trained with data from a total of 796 river basins in the United States and Central Europe, spanning the period from 1981 to 2019. Figure 2 The following is a time series diagram of the observed and simulated dissolved oxygen values ​​in three of the basins. The gray solid line is the simulated value of dissolved oxygen, the black solid point is the observed value of dissolved oxygen, and the Nash efficiency coefficient and root mean square error of each basin are listed at the top. The horizontal axis is time, and the vertical axis is dissolved oxygen content. Figure 2 As shown, the model can capture the periodic changes of dissolved oxygen and simulate the dissolved oxygen content more accurately, showing excellent performance.

[0134] The present invention also provides a large-scale river basin water quality simulation system based on CNN-Transformer, comprising: a data acquisition module, a data processing module, a data embedding module, a data encoding module and a simulation module;

[0135] The data acquisition module is used to obtain meteorological and hydrological driving data, water quality data and basin attribute data of multiple basins in a large range within a certain period of time;

[0136] The data processing module is used to pre-process the acquired data to obtain pre-processed meteorological and hydrological driving data, water quality data and watershed attribute data; then convert the date of the data into time characteristic data; and divide the processed meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data into a training set and a test set according to a certain ratio;

[0137] A data embedding module is used to embed the three-dimensional time series data of the training set;

[0138] A data encoding module, used to input the embedded data into an encoder and then perform a linear transformation on the data processed by the encoder;

[0139] The simulation module is used to perform training according to the set number of iterations and the corresponding loss function to obtain the initial CNN-Transformer water quality simulation model; multiple initial CNN-Transformer water quality simulation models are obtained by training with different combinations of hyperparameters; according to the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, the CNN-Transformer water quality simulation model with the best performance on the test set is selected as the final CNN-Transformer water quality simulation model, the data in the test set is input into the final CNN-Transformer water quality simulation model, and the output of the model is then inversely standardized to obtain the simulated value of the water quality of the basin.

[0140] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A large-scale watershed water quality simulation method based on CNN-Transformer, characterized in that: Here are the steps: 1) Obtain meteorological and hydrological driving data, water quality data, and basin attribute data for multiple basins over a large area within a certain period of time; 2) Preprocessing the data obtained in step 1) to obtain preprocessed meteorological and hydrological driving data, water quality data and watershed attribute data; and then converting the date of the data into time characteristic data; Divide the processed data into training set and test set according to a certain ratio; 3) Embedding the three-dimensional time series data of the training set in step 2); 4) inputting the data processed in step 3) into the encoder, and then performing a linear transformation on the data processed by the encoder; 5) Perform training according to the set number of iterations and the corresponding loss function, repeat steps 3) and 4) to obtain the initial CNN-Transformer water quality simulation model; use different combinations of hyperparameters to train to obtain multiple initial CNN-Transformer water quality simulation models, and select the CNN-Transformer water quality simulation model with the best performance on the test set as the final CNN-Transformer water quality simulation model based on the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, and input the data in the test set into the final CNN-Transformer water quality simulation model to obtain the simulated value of the water quality of the basin; The meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data obtained after preprocessing in step 2) are all three-dimensional time series data, wherein the first dimension represents different watersheds, the second dimension represents time, and the third dimension represents features and attributes, so that data of multiple watersheds can be trained at one time to capture the synergistic effect between multiple watersheds; The encoder is composed of a plurality of encoder layers, each of which includes an attention layer, a residual link and a layer normalization and a CNN layer; The step 4) specifically includes: 41) Input the processed data in step 3) into the attention layer of the encoder to calculate the attention score; project the data to obtain the query tensor Q i , the key tensor K i Sum value tensor V i , the calculation formula is as follows: Q i =W i Q ·X K i =W iK ·X V i =W i V ·X; Among them, X is the input data, W i Q , W i K , W i V are parameters that can be learned by the model, i = 1, 2, ..., H, H represents the number of attention heads; Calculate the attention scores of each attention head i , the formula is as follows: scores i =Q i ·K i T ; The attention weight A is calculated using the softmax function for the attention score of each attention head. i , the formula is as follows: Weight each value tensor V using the attention weight i Get the output of each attention head, the formula is as follows: head i =A i ·V i ; Among them, head i is the output of each attention head; The outputs of all attention heads are concatenated and linearly transformed to obtain the final multi-head attention output, as follows: MultiHead=Concat(head1,head2,...,head H )W O ; Among them, MultiHead is the multi-head attention output, Concat is the concatenation function, and W O are parameters available for training; 42) Perform residual connection on the multi-head attention output obtained in step 41), and then perform layer normalization; 43) The data obtained in step 42) is passed through the CNN layer to capture features: through a convolution kernel size of 1, the number of input channels is d, and the number of output channels is d ff The one-dimensional convolution layer is used for feature extraction, and the data is then processed by the activation function. The dropout layer is used to prevent data overfitting. The activation function uses the ReLU function or the GELU function. The output is passed through a convolution kernel with a size of 1 and an input channel number of d. ff , a one-dimensional convolutional layer and a dropout layer with an output channel number of d are used to process the data from d ff Remap the dimension to d dimension to capture the features of different scales in the sequence data and enhance the representation ability of the model; 44) adding the output of the two convolutional layers in step 43) to the output after layer normalization in step 42), and normalizing again to obtain the final encoder layer output; 45) The output result obtained in step 44) is passed to the next encoder layer, and steps 41) to 44) are repeated N times. x times, N x is the number of encoder layers, after N x The output of the encoder is obtained by processing the encoder layers; 46) Perform a linear transformation on the output of the encoder in step 45) to map it from d dimensions to d out Dimension, where d out Indicates the number of water quality indicators that need to be simulated.

2. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The data standardization process in step 2) specifically includes: performing Z-score standardization on the data to convert the data into a distribution with a mean of 0 and a standard deviation of 1, and ensuring that the data has a similar scale and distribution; the calculation formula is as follows: Among them, x is the original data, is the average value of the original data, σ x is the standard deviation of the original data, and z is the processed data.

3. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The specific process of converting the date of the data into time feature data is: by judging whether the current date is a specific date of the week, the days of the week are mapped to the range of [-0.5, 0.5], and the formula is expressed as follows: Among them, f w Represents the time feature data after mapping the days of the week, t w Indicates a specific day of the week, n w Indicates the total number of days in a week; By judging whether the current date is a specific date in a month, the number of days in a month is mapped to the range of [-0.5, 0.5]. The formula is as follows: Among them, f m Represents the time feature data after mapping the days in a month, t m Indicates a specific date in January, n m Indicates the total number of days in a month; By judging whether the current date is a specific date in a year, the number of days in a year is mapped to the range of [-0.5, 0.5]. The formula is as follows: Among them, f y Represents the time feature data after mapping the days of the year, t y Indicates a specific date in a year, n y Represents the total number of days in a year.

4. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The specific process of data embedding in step 3) is: position encoding, time encoding and linear transformation of input data.

5. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 4 is characterized in that: The step 3) specifically includes: 31) Position encoding is performed on the meteorological and hydrological driving data and the watershed attribute data obtained after processing in step 2) to obtain a position encoding vector. The specific formula is as follows: Among them, pos represents the relative position of the encoded data in all time series, j represents the dimension of the time series vector, and d represents the dimension of the position encoding vector; 32) performing time coding on the time feature data processed in step 2), and converting the time feature data into a d-dimensional time coding vector through a linear transformation layer; 33) The meteorological and hydrological driving data and the watershed attribute data are converted into a d-dimensional vector through a linear transformation layer, and the vector is added to the position encoding vector and the time encoding vector obtained in step 31) and step 32) to obtain a data embedding vector, and the data embedding vector is passed as input to the dropout layer, and the output of a part of the neurons is randomly set to zero to reduce the complex co-adaptation relationship between neurons.

6. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The encoder consists of multiple encoder layers, each of which includes an attention layer, residual links and layer normalization, and a CNN layer.

7. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The loss function selected in step 5) is the standardized mean square error, and the calculation formula is: Among them, NMSE represents the standardized mean square error, y m represents the simulated value, y0 represents the observed value, and σ0 represents the standard deviation of the observed value.

8. The large-scale watershed water quality simulation method based on CNN-Transformer according to claim 1 is characterized in that: The evaluation indicators selected in step 5) are Nash efficiency coefficient and root mean square error, and the formulas are as follows: Among them, NSE represents the Nash efficiency coefficient, RMSE represents the root mean square error, and y m Represents the simulated value of dissolved oxygen, y0 represents the observed value of dissolved oxygen, and n represents the number of dissolved oxygen observations; the data of the test set are input into multiple initial CNN-Transformer water quality simulation models to obtain the simulated value of dissolved oxygen, and the NSE or RMSE is calculated according to the simulated value and observed value of dissolved oxygen. The model with the best NSE or RMSE performance is selected, and its model structure is saved to obtain the final CNN-Transformer water quality simulation model.

9. A large-scale watershed water quality simulation system based on CNN-Transformer, characterized in that: include: Data acquisition module, data processing module, data embedding module, data encoding module and simulation module; The data acquisition module is used to obtain meteorological and hydrological driving data, water quality data and basin attribute data of multiple basins in a large area within a certain period of time; The data processing module is used to pre-process the acquired data to obtain pre-processed meteorological and hydrological driving data, water quality data and watershed attribute data; and then convert the date of the data into time characteristic data; The processed meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data are divided into training sets and test sets according to a certain ratio; A data embedding module is used to embed the three-dimensional time series data of the training set; A data encoding module, used to input the embedded data into an encoder and then perform a linear transformation on the data processed by the encoder; The simulation module is used to perform training according to the set number of iterations and the corresponding loss function to obtain an initial CNN-Transformer water quality simulation model; multiple initial CNN-Transformer water quality simulation models are obtained by training with different combinations of hyperparameters; according to the evaluation indicators of the initial CNN-Transformer water quality simulation model on the test set, the CNN-Transformer water quality simulation model with the best performance on the test set is selected as the final CNN-Transformer water quality simulation model; the data in the test set is input into the final CNN-Transformer water quality simulation model, and the output of the model is then inversely standardized to obtain the simulated value of the water quality of the basin; The meteorological and hydrological driving data, water quality data, watershed attribute data and time characteristic data obtained after the preprocessing are all three-dimensional time series data, wherein the first dimension represents different watersheds, the second dimension represents time, and the third dimension represents features and attributes, so that the data of multiple watersheds can be trained at one time to capture the synergistic effect between multiple watersheds; The encoder is composed of a plurality of encoder layers, each of which includes an attention layer, a residual link and a layer normalization and a CNN layer; (41) Input the processed data into the attention layer of the encoder to calculate the attention score; project the data to obtain the query tensor Q i , the key tensor K i Sum value tensor V i , the calculation formula is as follows: Q i =W i Q ·X K i =W i K ·X V i =W i V ·X; Among them, X is the input data, W i Q , W i K , W i V are parameters that can be learned by the model, i = 1, 2, ..., H, H represents the number of attention heads; Calculate the attention scores of each attention head i , the formula is as follows: scores i =Q i ·K i T ; The attention weight A is calculated using the softmax function for the attention score of each attention head. i , the formula is as follows: Weight each value tensor V using the attention weight i Get the output of each attention head, the formula is as follows: head i =A i ·V i ; Among them, head i is the output of each attention head; The outputs of all attention heads are concatenated and linearly transformed to obtain the final multi-head attention output, as follows: MultiHead=Concat(head1,head2,...,headH)W O ; Among them, MultiHead is the multi-head attention output, Concat is the concatenation function, and W O are parameters available for training; (42) Perform residual connection on the multi-head attention output obtained in step (41), and then perform layer normalization; (43) The data obtained in step (42) is passed through the CNN layer to capture features: through a convolution kernel size of 1, the number of input channels is d, and the number of output channels is d ff The one-dimensional convolution layer is used for feature extraction, and the data is then processed by the activation function. The dropout layer is used to prevent data overfitting. The activation function uses the ReLU function or the GELU function. The output is passed through a convolution kernel with a size of 1 and an input channel number of d. ff , a one-dimensional convolutional layer and a dropout layer with an output channel number of d are used to process the data from d ff Remap the dimension to d dimension to capture the features of different scales in the sequence data and enhance the representation ability of the model; (44) Add the output of the two convolutional layers in step (43) to the output after layer normalization in step (42), and normalize again to obtain the final encoder layer output; (45) Pass the output result obtained in step (44) to the next encoder layer, repeat steps (41) to (44), and repeat N x times, N x is the number of encoder layers, after N x The output of the encoder is obtained by processing the encoder layers; (46) Perform a linear transformation on the output of the encoder in step (45), mapping it from d dimensions to d out Dimension, where d out Indicates the number of water quality indicators that need to be simulated.

Images