Rainfall runoff pollution multi-dimensional data dynamic interpolation method based on self-attention mechanism and physical constraint

Abstract

The invention relates to a rainfall runoff pollution multi-dimensional data dynamic interpolation method based on a self-attention mechanism and physical constraints, and relates to a missing data interpolation method based on artificial intelligence. Comprising the steps of basic data collection, data preprocessing, missing data review, feature engineering, SAITS model construction, reconstruction and mask filling joint training strategy, optimal interpolation result output, and verification of the reliability of an interpolation result based on physical constraints and statistical parameters. According to the method, the problem that the spatial-temporal heterogeneity is difficult to consider in the existing rainfall related data interpolation by using simple mean interpolation or singly using a spatial position is solved, and meanwhile, an accurate interpolation mode is provided for the interpolation of rainfall events and corresponding runoff pollution missing data; the distribution of rainfall events and corresponding pollution data in time and space ranges is natural, and the credibility of an interpolation result is ensured.

Description

Technical Field

[0001] This invention relates to a dynamic interpolation method for multidimensional rainfall-runoff pollution data based on a self-attention mechanism and physical constraints, belonging to the technical field of rainfall-runoff data interpolation. Background Technology

[0002] In the field of urban surface runoff pollution, the lack of relevant data makes it difficult to conduct precise research on key driving factors of runoff pollution and to quantify the contribution of relevant influencing factors to pollution. In addition, due to varying degrees of missing data in the dataset, the incompleteness of historical rainfall event data makes it impossible to analyze the interannual changes in runoff pollution during the urbanization process, and on a spatial scale, it masks the pollution characteristics of different urban areas. In the construction of runoff pollution load accounting models, the lack of data makes it difficult to accurately quantify model parameters, making it impossible to accurately simulate and predict pollution loads, and consequently making it difficult for urban managers to accurately implement governance plans and emergency response measures when extreme events occur.

[0003] In the study of rainfall-runoff pollution in the Yangtze River Basin, missing data is a core challenge hindering the refined analysis of pollution characteristics. Current methods primarily rely on traditional approaches to handle missing data: For time-dimensional data imputation, linear interpolation and ARIMA models are the main methods. Linear interpolation assumes that pollutant concentrations change linearly over time, but ignores the sudden, impulsive characteristics of extreme rainfall events such as storm runoff; while ARIMA models are time-series predictions based on stationarity assumptions, failing to capture the nonlinear trends resulting from the interplay of climate change and human activities. For spatial data imputation, Kriging interpolation and spatial mean substitution (reference stations) are the main methods. Kriging interpolation relies on empirical variability functions to fit spatial correlations, resulting in a large spatial range and significant differences, making it difficult to standardize semivariogram models.

[0004] Traditional methods for imputing missing data have limitations in addressing the multi-level spatial structure of different cities, functional zones, and underlying surfaces within a river basin, making it difficult to simultaneously incorporate both temporal and spatial dimensions. Specific problems include: (1) Cross-scale information fragmentation: Hydrological conditions vary greatly across different regions in the upper, middle, and lower reaches of the Yangtze River. Traditional methods use globally unified imputation parameters, leading to deviations in the imputation of pollution concentrations during rainfall events; (2) Regional failure of small sample data: There are large gaps between some less studied areas and surrounding urban areas, resulting in sparse spatial distribution. Kriging-based spatial interpolation methods will produce large errors; (3) Dynamic disconnection and missing data: The impact of extreme climate and human activities is not considered, and the imputed data sequence cannot reflect the true characteristics of rainfall-runoff pollution.

[0005] Based on this, the present invention proposes a dynamic interpolation method for multidimensional rainfall-runoff pollution data based on self-attention mechanism and physical constraints. Summary of the Invention

[0006] To address the aforementioned problems, this invention discloses a dynamic interpolation method for multidimensional data of rainfall-runoff pollution based on a self-attention mechanism and physical constraints. The specific technical solution is as follows:

[0007] A data restoration technology for dynamic monitoring of multidimensional rainfall-runoff pollution based on self-attention mechanism and physical constraints includes the following steps:

[0008] Step 1: Collect basic data: Basic data includes meteorological and rainfall data, runoff pollution data, and human and social data of the study area. Collecting basic data forms a basic dataset X;

[0009] Step 2: Preprocessing of basic data: Perform preliminary processing on the basic dataset X obtained in Step 1 to obtain dataset X. * ; Identify the pre-processed dataset X * Data missing types; test dataset X * The normality;

[0010] Step 3: Rainfall event segmentation: Based on the dataset X processed in Step 2 * The timing and location of secondary rainfall events in the middle of the day are identified using unique thermal coding to distinguish different regions and underlying surface types;

[0011] Step 4: Dataset X * Missing data handling: Using the dataset X obtained in step 2 * Before training the SAITS model, the dataset X was... * The data in the dataset is randomly missing to obtain dataset X^;

[0012] Step 5: Based on two diagonal masking self-attention modules and a weighted combination module, and combined with the joint training loss function of reconstruction and mask filling, the training of the time series interpolation model SAITS based on the self-attention mechanism is realized;

[0013] Step 6: Impute missing data in the dataset X^ obtained in Step 4, and output the final imputation result;

[0014] Step 7: Based on the variation range of statistical parameters in statistics and the physical rules governing the occurrence of rainfall-runoff pollution, determine the rationality of the interpolation results;

[0015] Step 8: After deleting the missing data in the original dataset X, perform cluster analysis to obtain cluster result A; perform cluster analysis on the complete dataset after interpolation to obtain cluster result B; compare the two cluster results A and B to determine the reasonableness of the interpolation result.

[0016] Furthermore, the meteorological and precipitation data, runoff pollution data, and socio-cultural data in step 1 are specifically as follows:

[0017] Meteorological precipitation data include: rainfall amount per event, duration of each event, average rainfall intensity per event, pre-rain dry period of each event, annual average rainfall, annual average humidity, annual maximum temperature, and annual minimum temperature of the city where the study area is located;

[0018] Runoff pollution data include: average COD concentration, average TN concentration, average TP concentration, and average SS concentration for each rainfall event;

[0019] Human and social data include: the slope and impermeability of the underlying surface of the study area, the population, population density, GDP, GDP per capita, urban industrial added value, urban primary, secondary and tertiary industry added value, urban average slope, urban area, urban built-up area and urban green coverage rate.

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] ① Perform preliminary processing on the original basic dataset X collected in step 1, and delete rainfall events where both meteorological precipitation data and runoff pollution data are missing;

[0022] ② Use the Little test to identify the pre-processed dataset X. * Data missing types;

[0023] ③ Using SPSS software, the Kolmogorov-Smirnov test (KS test) and the Shapiro-Wilk test were used to test the dataset X. * The normality of the data was determined. Based on the results of the normality test, Spearman correlation analysis was performed on the data of annual average rainfall, annual average humidity, annual maximum temperature, annual minimum temperature, population size, population density, GDP output, GDP per capita, urban industrial added value, urban primary, secondary and tertiary industry added value, urban average slope, urban area, urban built-up area and urban built-up green coverage rate of the city in the study area, and indicators with correlation |r|≥0.7 were removed.

[0024] Furthermore, in step 4, the dataset X... * The data in the file is randomly missing. The specific steps are as follows:

[0025] In dataset X * Based on the existing missing values, a mask matrix is ​​generated by randomly masking 10% of the original values ​​using a mask:

[0026]

[0027] We label missing positions as 0 and non-missing positions as 1 to obtain the dataset X^, where M is the missing mask vector, T is the time step span of the dataset X^, t is a specific time point within it, and D is the dataset X^. * The data dimension range, where d is one of the dimensions, and X... * ={x1,x2,…,x t ,…x T}∈R T×D When the time is t Normalize each feature data in the obtained dataset X^ to obtain dataset Y, ensuring that the input values ​​of the SAITS model are stable.

[0028] Furthermore, step 5 uses a time series imputation model SAITS based on a self-attention mechanism to impute missing data. The time series imputation model SAITS includes two diagonally masked self-attention modules DMSA and a weighted combination module.

[0029] Furthermore, the specific operation of setting up the two diagonal masked self-attention modules (DMSA) in the time series interpolation model SAITS is as follows:

[0030] The first diagonal masking self-attention module (DMSA) consists of a diagonal masking self-attention module, a feedforward neural network, and a linear connection layer. The input of the first diagonal masking self-attention module (DMSA) is a dataset X^ that has been normalized, embedded with one-hot encoding, and combined with a missing mask. The output is the preliminary imputation data X1 of the first reconstruction.

[0031] The second diagonal masked self-attention module (DMSA) includes a diagonal masked self-attention module, a feedforward neural network, a linear connection layer, and a ReLU activation function. The input of the second diagonal masked self-attention module (DMSA) is the initial interpolated data X1 output by the first diagonal masked self-attention module (DMSA), and the output is the second reconstructed data X2.

[0032] Furthermore, the specific operation of setting the weighted combination module in the SAITS time series interpolation model is as follows:

[0033] Based on time dependence and the dynamic nature of missing information, and combining the missing information mask and the self-attention weights of the second diagonal masked self-attention module (DMSA), dynamic combined weights are generated using the Sigmoid activation function. These dynamic combined weights are then used to weight and calculate the third reconstructed data X3 by weighting the initial imputation data X1 output from the first diagonal masked self-attention module and the second reconstructed data X2 output from the second diagonal masked self-attention module (DMSA). Finally, the mask matrix is ​​used to... The dataset X^ is interpolated using the third reconstructed data X3, and the final interpolation result Xc is output.

[0034] Furthermore, the training process of the SAITS model in step 4 is as follows:

[0035] The joint training strategy for reconstruction and mask filling includes calculating the loss function for the reconstruction task (RT), the loss function for the mask filling task (MIT), and the overall loss function for joint training, thus completing the training part of the self-attention-based time series interpolation model SAITS.

[0036] Furthermore, the specific operation to obtain the optimization result output in step 6 is as follows: input the dataset X^ into the trained SAITS model, predict the missing values ​​of rainfall index and pollution index for each rainfall event in the dataset X^, and obtain the complete result after imputation.

[0037] The rainfall indicators for the aforementioned rainfall events include rainfall amount, rainfall duration, average rainfall intensity, and pre-rain dry period;

[0038] The pollution indicators include the average COD concentration, average TN concentration, average TP concentration, and average SS concentration for each rainfall event.

[0039] Furthermore, step 7, based on statistical principles, uses SPSS software to calculate the statistical parameters of the interpolated dataset, including the maximum, minimum, mean, standard deviation, variance, bias, kurtosis, and percentiles. The changes in the statistical parameters of the data before and after interpolation using the mean interpolation and the SAITS time series interpolation model based on the self-attention mechanism in steps 2-5 are compared to verify the rationality of the interpolation results from a statistical perspective.

[0040] The statistical parameters of the original data table and the complete data table after interpolation using the mean interpolation and the SAITS interpolation method based on the self-attention mechanism in steps 2-5 are compared to verify the rationality of the interpolation results before and after interpolation.

[0041] The beneficial effects of this invention are:

[0042] This invention addresses common problems with traditional methods for solving environmental issues such as rainfall-runoff pollution, including the difficulty in quantifying the combined impact of multiple environmental factors on rainfall-runoff pollution events. The invention employs a systematic and scientific approach. When collecting socio-cultural data for the study area, relevant statistical yearbook data for the corresponding years are retrieved based on the timing of rainfall events. The collected dataset is then preprocessed, including outlier removal (data missing both meteorological and rainfall data and runoff pollution data), dataset missing rate statistics, and feature engineering. By analyzing the correlation between socio-cultural data indicators and meteorological indicators such as annual average rainfall, annual average humidity, annual maximum temperature, and annual minimum temperature, highly autocorrelated indicators are removed to prevent overfitting. Finally, the preprocessed dataset is normalized for subsequent model training. Based on the existing missing data in the dataset, artificial missing data processing is performed by adding an artificial mask. The normalized data is then concatenated with the missing mask, mapped to a high-dimensional space, and one-hot encoded. The multi-head self-attention mechanism after the masking is calculated, and the output of the multi-head self-attention mechanism is nonlinearly transformed by a feedforward neural network (FNN). The processed multi-dimensional feature matrix is ​​then input into a trained prediction model to obtain the final interpolation result. The rationality of the interpolation result is verified based on the variation range of statistical parameters and the physical rules governing rainfall-runoff pollution.

[0043] The method of this invention overcomes the complexity of spatiotemporal interaction constraints through physical constraints, providing a precise and interpretable interpolation method for studying rainfall-runoff pollution at multiple scales in watersheds. It further lays the foundation for exploring the characteristics of rainfall-runoff pollution in different urban areas along the main stream of the watershed, and supports the construction of a smart watershed management platform.

[0044] The invention has achieved excellent results in research conducted in the Yangtze River Basin and has promising application prospects. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0046] Figure 1 This is a schematic diagram of the processing flow of the present invention.

[0047] Figure 2 This is a flowchart of the model of the present invention.

[0048] Figure 3 This is the statistical result of the missing rate of a single indicator in the dataset in this embodiment of the invention.

[0049] Figure 4This represents the overall missing rate of the dataset in this embodiment of the invention.

[0050] Figure 5 This is the result of the normality test of the dataset in this embodiment of the invention.

[0051] Figure 6 This is a schematic diagram of data missing and imputation modes in an embodiment of the present invention.

[0052] Figure 7 This is a schematic diagram of the observation reconstruction task and mask filling task in the SAITS model in an embodiment of the present invention.

[0053] Figure 8 This is the training result of the training set loss function in an embodiment of the present invention.

[0054] Figure 9 This is the training result of the test set loss function in this embodiment of the invention.

[0055] Figure 10 This is a comparison of the changes in statistical parameters before and after using the average value to imputate the missing dataset in this embodiment of the invention.

[0056] Figure 11 This is a comparison of the changes in statistical parameters before and after using the SAITS model to imputate missing datasets in an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0058] The technical solution of this invention is as follows:

[0059] Step 1: Basic data collection, including meteorological and precipitation data, runoff pollution data, and human and social data for the study area. Meteorological and precipitation data, such as rainfall amount, duration, average rainfall intensity, and pre-rain dry period for each rainfall event, are obtained from previously monitored rainfall events in existing literature. Human and social data are sourced from the official statistical website of the study area, provided by statistical yearbooks. This collection forms the basic dataset X.

[0060] Step 2: Basic Data Preprocessing. The original dataset X is pre-processed by deleting rainfall events with missing meteorological and runoff pollution data, resulting in dataset X. * Based on dataset X *This study investigates the missing data for eight indicators related to rainfall events: rainfall amount, rainfall duration, average rainfall intensity, pre-rain dry period, and corresponding average COD, TN, TP, and SS concentrations in runoff. It also assesses the overall missing data rate of the dataset. Furthermore, autocorrelation analysis is performed on the socio-cultural data and indicators such as annual average rainfall, annual average humidity, annual maximum temperature, and annual minimum temperature in this dataset. Indicators with a correlation coefficient |r| ≥ 0.7 are removed to reduce redundant information in the model application and avoid overfitting.

[0061] Step 3: Rainfall event segmentation. Based on the obtained dataset X * The timing and location of secondary rainfall events are identified using unique thermal coding to distinguish different regions and underlying surface types.

[0062] Step 4: Handling missing data in dataset X. * Before training the SAITS model, the dataset X was... * The data in the dataset is manually randomized to remove missing data, resulting in dataset X^.

[0063] Step 5: Based on the above processing results, use the self-attention mechanism-based time series missing data imputation method SAITS to impute the missing data and output the final imputation result.

[0064] Step 6: Based on the variation range of statistical parameters in statistics and the physical rules of rainfall runoff pollution occurrence, judge the rationality of the interpolation results. By comparing the two methods of mean interpolation and SAITS model interpolation, judge the advantages of the method of the present invention in interpolating the missing temporal and spatial dimensions of rainfall runoff pollution.

[0065] As an embodiment of the application of this invention in practical engineering, different functional areas and underlying surfaces of cities along the main stream of the Yangtze River are selected as the objects of implementation of this invention. The specific operation steps are as follows:

[0066] S1. Basic Data Collection: Based on literature review, the following cities were included: Panzhihua, Luzhou, Yibin, Chongqing, Yichang, Jingzhou, Jiujiang, Nanchang, Huanggang, Wuhan, Ezhou, Huangshi, Anqing, Hefei, Tongling, Chizhou, Wuhu, Yangzhou, Taizhou, Nanjing, Nantong, Zhenjiang, Changzhou, Suzhou, Wuxi, and Shanghai. The basic dataset X includes meteorological and rainfall data for the study area, including rainfall amount, duration, average rainfall intensity, pre-rainfall dry period, annual average rainfall, annual average humidity, annual maximum temperature, and annual minimum temperature for each rainfall event; rainfall runoff pollution data, including average COD, TN, TP, and SS concentrations for each rainfall event; and human and social data, including the slope and impermeability of the underlying surface of the study area, population size, population density, GDP, GDP per capita, urban industrial added value, added value of the primary, secondary, and tertiary industries, average urban slope, urban area, urban built-up area, and urban green coverage rate.

[0067] S2. Deletion of Abnormal Data Rows: Delete data rows in the base dataset X that are simultaneously missing meteorological precipitation data and runoff pollution data, resulting in dataset X. * .

[0068] S3. The dataset X obtained in step S2 above. * This study investigates the missing data for rainfall indicators (rainfall amount, rainfall duration, average rainfall intensity, and pre-rain dry period) and pollution indicators (average COD concentration, average TN concentration, average TP concentration, and average SS concentration for each rainfall event) and calculates the missing data for dataset X. * The overall missing rate, with specific results as follows: Figure 3 and Figure 4 As shown; using the Little MCAR test, determine the dataset X * The data missing type is random missing.

[0069] Table 1 Little's MCAR Test

[0070]

[0071] S4. The dataset X obtained in step S2 above. *The dataset contains characteristic indicators from various dimensions, including urban hydrological and meteorological indicators (annual average rainfall, annual maximum temperature, annual minimum temperature, annual average temperature, and annual average humidity) and urban human indicators (surface slope, surface impermeability, annual GDP per capita, urban population, urban industrial added value, primary industry added value, secondary industry added value, tertiary industry added value, urban area, urban built-up area, urban built-up area green coverage rate, and urban average slope). Using SPSS software, the normality of the dataset was tested using both the Kolmogorov-Smirnov test (KS test) and the Shapiro-Wilk test. The results of the normality test are as follows: Figure 5 As shown. According to the normality test results, since dataset X... * The data in the table do not conform to a normal distribution, so Spearman correlation coefficient was selected for correlation analysis. Indicators with high autocorrelation were removed, resulting in the indicator system shown in Table 2. The selected indicators all affect the generation of rainfall runoff pollution and, to some extent, the migration, transport, and spatial distribution characteristics of rainfall runoff pollutants.

[0072] Table 2 Indicators for Feature Engineering Selection

[0073] Selecting Indicators Indicator Code Rainfall amount of each rainfall event rainfall Rainfall intensity of each rainfall event intensity Rainfall duration of each rainfall event time Rainfall events before the dry season Dry_period Average COD concentration of each rainfall event COD Average TN concentration during rainfall events TN Average TP concentration in each rainfall event TP Average SS concentration of each rainfall event SS GDP per capita in cities GDP_ave Urban GDP GDP Urban industrial added value industrial added value Urban primary industry added value valueaddedoftheprimaryindustry Urban secondary industry added value addedvalueofthesecondaryindustry Urban tertiary industry added value addedvalueofthetertiaryindustry Water impermeability of the underlying surface impermeability Underlying surface slope Slope Urban annual rainfall year_rainfall average annual temperature in cities year_tempertature average urban slope i_ave Urban population population

[0074] S5. Using dataset X * Before training, perform training on dataset X. * The data in the image is manually and randomly missing. A diagram illustrating the missing data and imputation patterns is shown below. Figure 6 As shown. In dataset X * Based on the existing missing values, a mask matrix is ​​generated by randomly masking 10% of the original values ​​using a mask:

[0075]

[0076] Missing positions are marked as 0, and non-missing positions as 1, resulting in the dataset X^, where M is the missing mask vector, T is the time step span of the entire dataset, t is a specific moment within it, D is the dimensional range of the entire dataset, and d is a specific dimension within it, X*={x1,x2,…,x t ,…x T}∈R T×D When the time is t

[0077] S6. Standardize or normalize each feature data of the dataset after manual missing data processing in step S5 to ensure the stability of the model input values ​​and obtain the dataset X^.

[0078] S7. Construct the SAITS model, setting up two diagonal masking self-attention mechanism (DMSA) modules. The overall model framework structure diagram is as follows: Figure 2 As shown, the specific operation is as follows:

[0079] The first DMSA module normalizes the original data, embeds one-hot encoding, and links a missing mask as input. The input data undergoes layer-by-layer learning and processing through a diagonal masking self-attention module, a feedforward neural network, and linear connection layers. The output is the first reconstructed data X1. The second DMSA module takes the preliminary imputation data output from the first DMSA module as input. The input data undergoes layer-by-layer learning and processing through a diagonal masking self-attention module, a feedforward neural network, linear connection layers, and a ReLU activation function. The output is the second reconstructed data X2.

[0080] The multi-head attention formula for the diagonally masked self-attention module is as follows:

[0081]

[0082] Where: A is the self-attention weight, and h is the number of self-attention heads. It is the query transformation matrix of the i-th self-attention head. It is the key transformation matrix of the i-th self-attention head. It is the value transformation matrix of the i-th self-attention head. These are the parameters that project the input x onto the linear layers Q, K, and V, respectively. These are the output layer parameters in DiagMaskedMHA.

[0083] The feedforward neural network consists of two linear transformations, with a ReLU activation function between them. The specific equation is as follows:

[0084] FNN(x) = ReLU = (xW1 + b1)W2 + b2

[0085] Where x is the input to the linear layer, and W1 is the weight matrix of the first linear layer. W2 is the weight matrix of the second linear layer.

[0086] ① In the first diagonal masking self-attention module, the actual input feature vector and its missing mask vector It is synchronized as input. The specific formula is as follows:

[0087]

[0088] z = {FNN(DiagMaskedMHA(e))} N

[0089]

[0090]

[0091] Where: Projecting the input onto d model Dimension, resulting in e, where e is the input feature vector. Missing mask vector The combination of W and positional encoding serves as input for subsequent processing; e It is a weight matrix used to map the input feature vector and the actual mask vector to the model dimension. b e For bias terms, In the formula, N represents the number of stacked layers, which is 1 in this invention. Using a diagonal masking multi-head self-attention mechanism and stacked layers of a feedforward network, e is transformed into z, where z is the intermediate representation after processing by the diagonal masking self-attention module and the feedforward neural network; z is then transformed from d... model The dimension is reduced to D dimensions, and the result is... This is the output of the initial estimate of missing values ​​in the first diagonal masking self-attention module. Replace the corresponding value Missing values ​​in the data, to obtain To output the preliminary estimated missing values The complete feature vector after replacing the original missing values.

[0092] ② In the second diagonal masking self-attention module, take the values ​​from the first diagonal masking self-attention module. And continue learning.

[0093]

[0094] β={FFN(DiagMaskedMHA(α))} N

[0095]

[0096] Among them: and Synchronous input, projected from dimension D to dimension d model Dimension, yielding α, where α is the output after linear transformation and positional encoding; W α It is a weight matrix. b α It is a bias term. β is the intermediate representation after processing by the second multi-layer masked self-attention module and feedforward neural network. A nested diagonal masked self-attention module and feedforward neural network are performed once on α and β. Two linear projections are applied to β with ReLU activation to obtain the final output of the second diagonal masked self-attention module. W β and W γ It is the weight matrix, b β and b γ It is a bias term. b β ∈R D W γ ∈R D×D b γ ∈R D .

[0097] The weighted combination module in the S8.SAITS model is set up as follows: Based on time dependence and the dynamic nature of missing information, dynamic combination weights are generated using the sigmoid activation function, based on the self-attention weights and missing mask of the second diagonal masked self-attention mechanism (DMSA) module. These dynamic combination weights are then used to adjust the output of the first diagonal masked self-attention module. and the output of the second diagonal masking self-attention module The third reconstruction data was obtained by performing weighted calculations. Finally, the mask matrix is ​​used. Using reconstructed data Perform interpolation on the original data and output the final interpolation result.

[0098] The learnable parameters are dynamically adjusted through the following weighted combinations. and Weighting to form a better

[0099]

[0100]

[0101] in, It is derived from the average of the multi-head output weights A in the last layer of the second DMSA module. The average self-attention weights are then applied. and missing mask As a reference, to generate a model with learnable parameters W η (W η ∈R (T+D)×D ) and b η (b η ∈R D The combined weights η (η∈(0,1)) T×DBy combining weights η and form use Corresponding value replacement Missing values ​​in the data, i.e., the imputed data.

[0102] S9. Joint training of reconstruction task (RT) and mask filling task (MIT). A schematic diagram of the reconstruction task and mask filling task is shown below. Figure 7 As shown, the specific operation steps are as follows:

[0103] ① Based on the initial missing value imputation results output by the first diagonal masking self-attention mechanism module (DMSA) The output of the second diagonal masking self-attention mechanism module (DMSA) is refined to show the imputed missing values. The final interpolation result is generated by dynamically fusing the outputs of the two DMSA blocks based on self-attention weights.

[0104] ② Regarding the above three output results Calculate the mean absolute error (MAE) of all observations:

[0105]

[0106] Where n is the number of data points, y i It is the i-th actual value. yes The predicted value for the i-th data point.

[0107] ③ Calculate the mean absolute error (MAE) only for the positions covered by the artificial mask I:

[0108] MAE I =|y I -y|

[0109] Among them, y I y is the predicted value of the position covered by the artificial mask I, and y is the actual value of the corresponding position.

[0110] ④ Loss function of the learning objective: Jointly optimize the loss functions of the observation reconstruction task (ORT) and the mask imputation task (MIT). By balancing the weights of the loss functions of the reconstruction task and the mask imputation task, the overall loss function of the joint optimization training is guaranteed to converge faster. The overall loss function of the joint optimization training is:

[0111]

[0112] L = L ORT +L MIT

[0113] In the above formula, L ORT For the loss function of the reconstruction task, L MIT To mask the loss function of the task, l MAE is the mean absolute error loss between the missing values ​​and their respective imputed values; L is the total loss function.

[0114] The gradient of the total loss function with respect to the model parameters was calculated. An early stopping strategy based on patience value was adopted to avoid overfitting. In this experiment, the number of iterations was set to 1197 to complete the training part of the self-attention-based time series interpolation model SAITS. The specific training results of the loss function on the training and test sets are as follows. Figure 8 , 9 As shown.

[0115] S10. Based on the model trained in step S9, input the dataset X^ into the trained model, predict the missing values ​​of rainfall indicators (rainfall amount, rainfall duration, average rainfall intensity, pre-rain dry period) and pollution indicators (average COD concentration, average TN concentration, average TP concentration, average SS concentration) of the rainfall events in the dataset X^, and obtain the complete result Y after imputation of the dataset X^.

[0116] S11. Verify the rationality of the interpolation results based on statistical parameters. The missing data columns in the dataset are imputed using the average value and the time series interpolation method based on the self-attention mechanism described in this invention. The changes in statistical parameters before and after interpolation are compared to verify the rationality of the interpolation results from a statistical perspective. The specific steps are as follows:

[0117] ① For the final interpolated dataset Y obtained in step S10, use SPSS software to calculate the statistical parameters of dataset Y, including maximum, minimum, mean, standard deviation, variance, bias, kurtosis, and percentiles; use SPSS software to calculate the statistical parameters of dataset X. * The statistical parameters include maximum, minimum, mean, standard deviation, variance, bias, kurtosis, and percentiles; the dataset X... * The statistical parameters of the complete data table Y after interpolation were compared to verify the rationality of the interpolation results before and after. By comparing the changes in the statistical parameters of the two datasets, it can be seen that the interpolation results did not change the statistical regularity of the datasets. Therefore, it is considered that the method of using the SAITS model to interpolate missing data related to rainfall runoff pollution is reasonable and feasible.

[0118] ② The dataset was imputed using the mean value to fill in missing data. The changes in statistical parameters of the original dataset and the imputed dataset were calculated. The imputation effects of the two methods, imputation using the SAITS model and imputation using the mean value, were compared. It was found that the imputation method using the SAITS model for missing data related to rainfall runoff pollution was more effective.

[0119] S12. Combining the physical mechanisms of rainfall runoff pollution and the spatial differences in the distribution of different urban areas in the Yangtze River Basin, and based on constraints on physical variables such as rainfall amount and average rainfall intensity, the rationality of the interpolation results of missing data related to rainfall runoff pollution in different urban areas along the main stream of the Yangtze River is verified. The specific operation steps are as follows:

[0120] ① For the final interpolated dataset Y obtained in step S10, the 26 city data obtained are divided into upper, middle and lower reaches according to the geographical division of different cities. For the rainfall, rainfall duration, average rainfall intensity and pre-rain dry period corresponding to the rainfall runoff pollution events in the three different city categories of the upper, middle and lower reaches of the Yangtze River, the extreme values, average values ​​and median values ​​before and after interpolation are calculated to ensure that no extreme rainfall events appear in the missing data.

[0121] ② Missing values ​​in the original basic dataset X were removed. Cluster analysis was performed on the dataset after removal to obtain cluster result A, which reasonably classifies urban areas with similar runoff pollution events. Cluster analysis was performed on the complete dataset after interpolation to obtain cluster result B. Comparing the two cluster results A and B, they are the same. It is considered that the interpolation method has not changed the characteristics of rainfall runoff pollution in cities along the Yangtze River. The interpolation result is reasonable. Therefore, the method of using the SAITS model to interpolate missing data related to rainfall runoff pollution is reasonable and feasible.

[0122] For a comparison of the changes in statistical parameters before and after imputing missing datasets using the mean in the examples, see [link to example]. Figure 10 .

[0123] The comparison results of statistical parameter changes before and after using the SAITS model to impute missing datasets in the embodiments are shown in the figure. Figure 11 .

[0124] This invention can make full use of the time series and spatial information of rainfall events and corresponding runoff pollution to achieve reasonable interpolation of missing data related to rainfall runoff pollution at multiple spatial scales, such as different urban areas, functional zones, and underlying surfaces along the main stream of the Yangtze River.

[0125] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0126] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A rainfall runoff pollution multi-dimensional data dynamic monitoring data repair technology based on self-attention mechanism and physical constraints, characterized in that, The method comprises the following steps: Step 1: collecting basic data: the basic data includes meteorological rainfall data, runoff pollution data and human social data of the research area, and the basic data set X is formed by collecting the basic data; Step 2: Preprocessing of the base data: the base data set X obtained in step 1 is preliminarily processed to obtain a data set X * ; identify the data missing type of the preliminarily processed data set X * ; test the normality of the data set X * ; Step 3: Rainfall event classification: Based on the dataset X after step 2 * The time and location of the occurrence of the mid-field sub-rainfall event are encoded using one-hot encoding to distinguish different regions and underlying surface types. Step 4: Dataset X * Missing treatment: using the dataset X obtained in Step 2 * Before training the SAITS model, random missing treatment is performed on the data in dataset X * to obtain dataset X^; Step 5: based on two diagonal mask self-attention modules and a weighted combination module, a reconstruction and mask filling joint training loss function is combined to realize training of a time series interpolation model SAITS based on a self-attention mechanism; Step 6: the data set X obtained in step 4 is subjected to missing data interpolation, and the final interpolation result is output; Step 7: based on the variation amplitude of the statistical parameters in statistics and the physical rule constraint of the occurrence of rainfall runoff pollution, the rationality of the interpolation result is judged; Step 8: after deleting the missing data in the original data set X, the clustering result A is obtained by using clustering analysis; the clustering result B is obtained by performing clustering analysis on the complete data set after interpolation is completed; the rationality of the interpolation result is judged by comparing the two clustering results A and B.

2. The method of claim 1, wherein the method is characterized by: The meteorological rainfall data, runoff pollution data and human social data in step 1 are specifically as follows: The meteorological rainfall data includes: field rainfall, field rainfall duration, field rainfall average rainfall intensity, field rainfall dry period before rain, annual average rainfall of the city where the research area is located, annual average humidity, annual maximum temperature and annual minimum temperature; The runoff pollution data includes: field rainfall event average COD concentration, field rainfall event average TN concentration, field rainfall event average TP concentration and field rainfall event average SS concentration; The human social data includes: the slope and impermeability of the underlying surface where the research area is located, the population of the city, the population density, the GDP output value, the per capita GDP, the urban industrial added value, the urban first, second and third industry added value, the urban average slope, the urban area, the urban built-up area and the urban built-up area green coverage rate.

3. The method of claim 1, wherein the method is characterized by: The specific process of step 2 is as follows: ①The original basic data set X collected in step 1 is subjected to preliminary processing, and the field rainfall events in which the meteorological rainfall data and the runoff pollution data are all missing are deleted; ii. Identify the type of data missing in the dataset X after the initial processing using Little's test * ii. Identify the type of data missing in the dataset X after the initial processing using Little's test ③Based on SPSS software, the normality of data set X * was tested by Kolmogorov-Smirnov test (K-S test) and Shapiro-Wilk test. Based on the results of normality test, Spearman correlation analysis was performed on the data of annual average rainfall, annual average humidity, annual maximum temperature, annual minimum temperature, population, population density, GDP, GDP per capita, urban industrial added value, urban first, second and third industry added value, urban average slope, urban area, urban built-up area, urban built-up area green coverage rate of the city where the study area was located. The indicators with |r|≥0.7 were removed.

4. The method of claim 1, wherein the method is characterized by: The step 4 randomly missing processing is performed on the data in the data set X * , and the specific operation is as follows: In the dataset X * Based on the existing missing values, a mask matrix is generated by randomly masking 10% of the original values with a mask. The missing position is marked as 0, and the non-missing position is marked as 1, to obtain a data set X, wherein M is a missing mask vector, T is a time step span of the data set X, t is a certain time in T, D is a data dimension range of the data set X * , d is a certain dimension in D, X * ={x1,x2,…,x t ,…x T}∈R T×D , at time t Each feature data in the obtained data set X is normalized to obtain a data set Y, so as to ensure that the input value of the SAITS model is stable.

5. The method of claim 4, wherein the method is characterized by: The step 5 uses a time series interpolation model SAITS based on a self-attention mechanism to interpolate the missing data, and the time series interpolation model SAITS includes two diagonal mask self-attention modules DMSA and a weighted combination module.

6. The method of claim 5, wherein the method is characterized by: The specific operation of the two diagonal mask self-attention modules DMSA in the time series interpolation model SAITS is as follows: The first diagonal mask self-attention module DMSA includes a diagonal mask self-attention module, a feedforward neural network and a linear connection layer; the input of the first diagonal mask self-attention module DMSA is the data set X^ which is subjected to normalization processing, embedded one-hot encoding and combined with a missing mask, and the output is the first reconstructed preliminary interpolation data X1; The second diagonal mask self-attention module DMSA includes a diagonal mask self-attention module, a feedforward neural network, a linear connection layer and a ReLU activation function; the input of the second diagonal mask self-attention module DMSA is the preliminary interpolation data X1 output by the first diagonal mask self-attention module DMSA, and the output is the second reconstructed data X2.

7. The method of claim 5, wherein the method is characterized by: The specific operation of the weighted combination module in the time series interpolation model SAITS is as follows: According to the time dependence and the dynamic of missing information, the self-attention weights of the first diagonal mask self-attention module and the second diagonal mask self-attention module DMSA are combined through a Sigmoid activation function to generate dynamic combination weights, and the dynamic combination weights are used to weight the preliminary interpolation data X1 output by the first diagonal mask self-attention module and the second reconstruction data X2 output by the second diagonal mask self-attention module DMSA to obtain the third reconstruction data X3; finally, the third reconstruction data X3 is used to interpolate the data set X^ to output the final interpolation result Xc. The third reconstruction data X3 is used to interpolate the data set X^ to output the final interpolation result Xc.

8. The method of claim 1, wherein the method is characterized by: The training process of the SAITS model in step 4 is as follows: The joint training strategy of reconstruction and mask filling includes loss function calculation of the reconstruction task (RT), loss function calculation of the mask filling task (MIT), and overall loss function calculation of the joint training, which completes the training part of the time series interpolation model SAITS based on the self-attention mechanism.

9. The method of claim 1, wherein the method is characterized by: The specific operation of the step 6 optimization result output is as follows: input the data set X^ into the trained SAITS model, predict the missing values of the rainfall indicators and pollution indicators of the data set X^ about the game rainfall event, and obtain the complete result after interpolation; The game rainfall event rainfall indicators include rainfall, rainfall duration, average rain intensity, and dry period before rain; The pollution indicators include average COD concentration, average TN concentration, average TP concentration, and average SS concentration of the game rainfall event.

10. The method of claim 1, wherein the method is based on a self-attention mechanism and physical constraints for dynamic imputation of multi-dimensional data of rainfall runoff pollution. Step 7: Based on statistical principles, using SPSS software, calculate the statistical parameters of the interpolated data set, including maximum value, minimum value, average value, standard deviation, variance, bias, kurtosis, and percentile; compare the data statistical parameter change amplitude before and after using mean interpolation and using step 2-5 time series interpolation model SAITS based on self-attention mechanism, and verify the rationality of the interpolation result from a statistical point of view; Compare the statistical parameters of the original data table and the complete data table after interpolation using mean interpolation and using step 2-5 time series interpolation model SAITS based on self-attention mechanism, and verify the rationality of the interpolation result before and after.

Images