Daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network

Through multi-feature fusion and two-dimensional time convolution network, combined with adaptive Fourier decomposition and two-layer routing attention mechanism, the shortcomings of the existing runoff prediction model in multi-source data processing and non-stationary sequence feature extraction are solved, and the problem of the existing runoff prediction model is achieved. More efficient runoff prediction effects.

CN118643467BActive Publication Date: 2024-12-03ZHEJIANG UNIV CITY COLLEGE
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411102727.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-03
Estimated Expiration
2044-08-13

AI Technical Summary

Technical Problem

The existing runoff prediction model cannot fully extract the spatiotemporal characteristics of the basin when processing multi-source data, and the single neural network prediction model has high time and energy consumption in extracting non-stationary runoff sequence features, and is sensitive to data loss and error. .

Method used

Using a method based on multi-feature fusion and two-dimensional time convolution network, multi-source data is characterized by screening and fusion through adaptive Fourier decomposition and two-layer routing attention mechanisms to build expanded convolution and causal convolution kernels, combined with Residual connection and loss function MSAE are used for model training and prediction.

Benefits of technology

Effectively capture the internal pattern characteristics of the runoff sequence, improve the robustness and sensitivity of the model, alleviate the problem of gradient vanishing in the deep network, enhance the network's ability to capture spatiotemporal features, and improve the accuracy and reliability of runoff prediction. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118643467B_ABST
    Figure CN118643467B_ABST
Patent Text Reader

Abstract

The present invention relates to a daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, comprising the following steps: feature mining and feature fusion of hydrological and meteorological multi-source data of the target basin, constructing a multi-source fusion feature matrix as input of the spatiotemporal feature extraction stage; performing spatiotemporal feature extraction on the multi-source fusion features; constructing an overall network framework model, determining the training parameters of the network, and performing training and prediction. The beneficial effects of the present invention are: adopting a double-layer routing attention mechanism to perform feature fusion on internal pattern features and screening features, focusing on key information in multi-source features; introducing residual connections to enhance information flow and gradient propagation, alleviating the gradient vanishing problem in deep networks, and retaining original feature information while extracting key detail features; treating the fused multi-source feature sequence as a two-dimensional feature for dilated convolution, rather than treating it as a one-dimensional sequence of multiple channels, so as to enhance the network's ability to capture fine-grained information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer data processing, and in particular relates to a daily runoff prediction method based on multi-feature fusion and a two-dimensional temporal convolutional network. Background Art

[0002] An accurate and robust runoff prediction model is crucial for water resources management. It guides the development, scheduling and sustainable use of water resources and has a profound impact on various fields such as production, life, and ecological environment. In actual runoff prediction research, the basin flow, under the influence of various factors, presents complex properties such as trend, multi-source and randomness, which makes the existing runoff prediction model unable to fully extract the multi-source data characteristics of the basin. Therefore, in runoff prediction, constructing an effective prediction model to extract the spatiotemporal characteristics of multi-source data is the key to improving the accuracy and reliability of runoff prediction.

[0003] The research on hydrological forecasting has a history of more than 70 years since the 1950s. So far, according to the development history and modeling mechanism of hydrological forecasting models, hydrological forecasting models can be divided into three categories: empirical models, physical process driven models and data driven models.

[0004] Artificial intelligence models have excellent nonlinear mapping capabilities, powerful feature extraction technology and self-learning generalization capabilities. In terms of big data processing and feature extraction, neural network models have unique advantages. They are not limited by physical processes, can adapt well to complex time series data, and effectively solve the problems of multi-feature and multi-regular hydrological processes in hydrological forecasting. In recent years, with theoretical breakthroughs and technological advances, artificial intelligence models have achieved remarkable results in runoff prediction.

[0005] Although artificial intelligence models perform well in dealing with nonlinear relationships, they have the following shortcomings in extracting the characteristics of non-stationary runoff sequences: 1) It takes a lot of time and energy to conduct repeated training to find the optimal solution; 2) It is strongly correlated with the amount of training data. If there are problems such as missing data from some hydrological stations or incorrect data records, it will also cause deviations in the final prediction results.

[0006] In order to overcome the limitations of a single neural network prediction model, hybrid models combined with signal decomposition methods have become a hot topic in runoff prediction research. Decomposition techniques can decompose nonlinear and non-stationary sequences into multiple components to capture effective features such as trends and cycles. At present, many decomposition methods have been proposed and applied to the field of runoff prediction, such as empirical mode decomposition, ensemble empirical mode decomposition, empirical wavelet transform, and variational mode decomposition. Decomposition methods greatly improve the model prediction performance by mining the potential pattern characteristics inside the runoff and converting the non-stationary runoff sequence into a more stable form.

[0007] However, the existing hybrid models combined with signal decomposition methods often focus on the characteristic decomposition of runoff sequences, while ignoring the influence of other factors. The new methods proposed in recent years are still not thorough enough in the study of multi-source data, and the multi-source data prediction model requires scholars to devote their energy to in-depth research. Summary of the invention

[0008] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network.

[0009] This daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network includes the following steps:

[0010] S1. Perform feature mining and multi-source feature fusion on the hydrological data and meteorological multi-source data of the target basin, and convert the multi-source feature sequence into The adaptive Fourier decomposition sequence of runoff is spliced ​​and fused by channel to obtain multi-source fusion features , using double-layer routing attention to integrate multi-source features Dynamically allocate weights, use residual connections to add and normalize the multi-source fusion feature input and the double-layer routing attention output to obtain a multi-source fusion sequence ;

[0011] S2. Construct the basic dilated convolution kernel to fuse the multi-source sequence Perform dilated convolution as a two-dimensional feature, then construct a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution, then construct a spatial deconvolution kernel, and then construct a two-dimensional temporal convolution module;

[0012] S3. Construct the loss function MSAE to train and predict the prediction model.

[0013] As a preferred embodiment, in step S1, the correlation between the variables of the influencing factors of daily runoff is analyzed, and the maximum mutual information coefficient is used to measure the degree of correlation between two variables to perform feature screening on multi-source data. The closer the value of the maximum mutual information coefficient is to 1, the greater the correlation between the variables; the closer the value is to 0, the smaller the correlation between the variables; after feature screening, a multi-source feature sequence is obtained. .

[0014] Preferably, in step S1, an adaptive Fourier decomposition algorithm is used to decompose the nonlinear and non-stationary daily runoff time series. for IMF components , and get the adaptive Fourier decomposition sequence of runoff; multi-source feature sequence The adaptive Fourier decomposition sequence of runoff is spliced ​​and fused by channel to obtain multi-source fusion features :

[0015]

[0016] Using two-layer routing attention to fuse multi-source features Dynamically allocate weights and optimize feature fusion, and apply fine-grained tag-to-tag attention in the union of these routing areas; the two-layer routing attention filters out irrelevant information in the first layer of attention, and then further integrates the correlation information between each routing area in the second layer of attention.

[0017] Preferably, in step S1, the components of the dual-layer routing attention include region partitioning, region-to-region routing using a directed graph, and tag-to-tag attention:

[0018] For multi-source fusion features ,in represents a real number, represents the number of features, Indicates the length of time, express The number of channels; when performing regional division, firstly combine the multi-source fusion features Divide into non-overlapping regions, a single region contains feature matrix, That is ; Then, we obtain :

[0019]

[0020] The two-layer routing attention mechanism adopts the query-key-value (QKV) model, where Q, K, and V are respectively And the corresponding weight matrix Perform a linear transformation to obtain .

[0021] As a preferred method, in step S1, a directed graph is constructed by a connection matrix to find the participation relationship corresponding to different key-value pairs, and a strong correlation area is found for each given area; first, the number of and The average value of , , then calculate and The adjacency matrix of the correlation between regions is obtained, and only the top The index of the most relevant area is obtained to obtain the area-to-area routing index matrix ;use Calculate fine-grained tag-to-tag attention. After aggregating the K and V tensors in the routing region, apply fine-grained tag-to-tag attention on the KV tensor:

[0022]

[0023] Then and Using attention operation:

[0024]

[0025] in Indicates the use of deep convolution for parameterized operation; the multi-source fusion feature input and the double-layer routing attention output are added using residual connection and normalized to obtain the multi-source fusion sequence .

[0026] Preferably, in step S2, firstly, the multi-source fusion sequence Perform dilated convolution as a two-dimensional feature, and then construct a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution, keep the dilated causal convolution in TCN in the time dimension, use dilated convolution in the spatial dimension, and fuse multi-source sequences and convolution kernel In the output sequence element The mathematical model of the two-dimensional dilated causal convolution on the position is:

[0027]

[0028] In the formula, represents the spatial dimension position, Represents the time dimension position, Represents the width of the convolution kernel in spatial dimension, represents the width of the convolution kernel in the time dimension, represents the spatial dimension expansion coefficient, Represents the expansion coefficient of the time dimension; the receptive field of the two-dimensional expanded causal convolution operation The size is calculated as:

[0029]

[0030] in Indicates the number of network layers, Indicates The spatial dimension expansion coefficient of the layer, Indicates Dilation factor of the time dimension of the layer.

[0031] Preferably, in step S2, the method for constructing the spatial deconvolution kernel is: the width of the convolution kernel in the time dimension is 1; in the spatial dimension, the width of the convolution kernel is to restore the multi-source fusion feature. The spatial dimension of the target is set, and the specific spatial dimension is the convolution kernel width The calculation method is:

[0032]

[0033] in is the input spatial feature dimension of the two-dimensional dilated causal convolution, is the output spatial feature dimension of the two-dimensional dilated causal convolution; the spatial dimension of the output feature of the two-dimensional dilated causal convolution is restored to , for the 2D dilated causal convolution output and filters , in the output sequence element The calculation formula for positional deconvolution is:

[0034] 。

[0035] Preferably, in step S2, after constructing the spatial deconvolution kernel, the spatial deconvolution output features are weighted normalized, and Relu activation and Dropout operations are performed; a two-dimensional temporal convolution residual block is constructed by using a residual connection to connect the input of the two-dimensional dilated causal layer and the output after the Dropout operation. The size of the two-dimensional dilated causal convolution kernel and the deconvolution kernel is determined according to the size of the dilation factor and the number of network layers; a two-dimensional time convolution module is constructed, and the number and parameters of the residual blocks in the two-dimensional time convolution module are determined according to the size of the dilation factor and the number of network layers, and the outputs of each residual block are spliced ​​by channel using jump connections; the features of the last time point output by the two-dimensional time convolution module are intercepted to obtain the feature results at the next moment, and they are input into the linear layer to be mapped to the runoff prediction value.

[0036] Preferably, in step S3, a loss function MSAE is constructed based on MSE and MAE, and the parameters are adjusted. Control the balance between MSE and MAE in the loss function. The loss function MSAE formula is:

[0037]

[0038] in, Indicates i The measured value of daily runoff, Indicates i The predicted value of daily runoff, is the length of the runoff time series.

[0039] Preferably, in step S3, the overall model parameter update method adopts the Adam gradient descent method, and the model is trained using the training set; the trained model is used for prediction on the prediction set to obtain the runoff prediction value, and the prediction accuracy of the model is evaluated using evaluation indicators.

[0040] The beneficial effects of the present invention are:

[0041] 1) The present invention adopts an adaptive Fourier decomposition method to capture the internal pattern features of the runoff sequence, and at the same time performs feature screening on multi-source data through the maximum mutual information coefficient, and adopts a double-layer routing attention mechanism to fuse the internal pattern features and the screening features, focusing on the key information in the multi-source features.

[0042] 2) Based on the two-layer routing attention mechanism, the present invention introduces residual connections to enhance information flow and gradient propagation, alleviate the gradient vanishing problem in deep networks, and retain the original feature information while extracting key detail features.

[0043] 3) The present invention designs a two-dimensional dilated causal convolution kernel to capture global and local spatiotemporal feature information, and connects targeted deconvolution kernels in series to reconstruct information. The fused multi-source feature sequence is regarded as a two-dimensional feature for dilated convolution, rather than as a one-dimensional sequence of multiple channels, to enhance the network's ability to capture fine-grained information.

[0044] 4) The present invention constructs a loss function MSAE that takes into account both robustness and sensitivity and is easy to adjust flexibly, giving full play to the advantages of MAE and MSE, so that multi-feature fusion and two-dimensional temporal convolutional network can obtain a solution with better comprehensive performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a double-layer routing attention structure diagram of the present invention;

[0046] Figure 2 It is a two-dimensional dilated causal convolution structure diagram of the present invention;

[0047] Figure 3 It is a deconvolution structure diagram of the present invention;

[0048] Figure 4 It is a structural diagram of a two-dimensional temporal convolution residual block of the present invention;

[0049] Figure 5 This is a diagram of the multi-feature fusion and two-dimensional temporal convolutional network structure of the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0051] Embodiment 1

[0052] As an example, Figures 1 to 5 As shown in the figure, this daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network includes the following specific steps:

[0053] S1. Feature mining and feature fusion are performed on the multi-source hydrological and meteorological data of the target basin, and a multi-source fusion feature matrix is ​​constructed as the input of the spatiotemporal feature extraction stage, including:

[0054] Obtain the hydrological and meteorological data of the target basin hydrological station, and merge and save the hydrological and meteorological data according to the station through Python's Pandas and other data processing libraries;

[0055] After feature screening, the multi-source feature sequence is as follows:

[0056] (1)

[0057] in, Indicates The screening features are The value of the moment, Indicates the number of filtered features.

[0058] Adaptive Fourier decomposition algorithm is used to decompose nonlinear and non-stationary daily runoff time series for IMF components , in order to mine the internal patterns of daily runoff series. Different from the traditional Fourier decomposition, the adaptive Fourier decomposition algorithm uses a predefined cutoff frequency algorithm to improve the decomposition strategy of discrete Fourier transform. In order to further improve the decomposition effect, the adaptive Fourier decomposition algorithm uses a sparrow search algorithm to optimize the parameters of the cutoff frequency algorithm to adaptively obtain the optimal decomposition result of the improved Fourier decomposition method;

[0059] Multi-source feature sequences The adaptive Fourier decomposition sequence of runoff is spliced ​​and fused by channel to obtain multi-source fusion features , as shown in formula (2):

[0060] (2)

[0061] The multi-source feature sequence and the runoff adaptive Fourier decomposition sequence are fused, and the double-layer routing attention is used to dynamically assign weights to the multi-source feature representation to focus on key information. The multi-source feature input and the double-layer routing attention output are connected using residual connection.

[0062] S2. Extract spatiotemporal features from multi-source fusion features. The specific steps include:

[0063] S2.1. Construct a basic dilated convolution kernel and treat the fused multi-source feature sequence as a two-dimensional feature for dilated convolution, rather than treating it as a one-dimensional sequence of multiple channels.

[0064] S2.2. Construct a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution. Maintain the dilated causal convolution in TCN in the time dimension to ensure that the time series is not disturbed by future information. Since spatial features have no causal relationship, dilated convolution is used in the spatial dimension to fully extract the cross information, local information and global information of spatial features;

[0065] S2.3, construct a spatial deconvolution kernel, reconstruct the spatiotemporal feature information and restore the input details. The width of the convolution kernel in the time dimension is 1 to retain the advantages of causal convolution in the time dimension; in the spatial dimension, the width of the convolution kernel is set according to the goal of restoring the spatial dimension of the fused multi-source features;

[0066] S2.4, weight normalization is performed on the spatial deconvolution output features, and Relu (rectified linear unit) activation operation and Dropout (discarding) operation are performed;

[0067] S2.5, construct a two-dimensional temporal convolution residual block, such as Figure 4 As shown in the figure, the residual connection is used to connect the input of the two-dimensional dilated causal layer and the output after the Dropout operation to construct a two-dimensional time convolution residual block, so that the network can transmit the original information, solve the gradient disappearance problem in deep neural networks, and improve the training efficiency and generalization ability of the network;

[0068] S2.6. Determine the size of the two-dimensional dilated causal convolution kernel and the deconvolution kernel according to the size of the multi-source fusion features;

[0069] S2.7. Construct a two-dimensional temporal convolution module. Determine the number and parameters of residual blocks in the two-dimensional temporal convolution module according to the size of the expansion factor and the number of network layers, and splice the outputs of each residual block by channel using skip connections;

[0070] S2.8, intercept the features of the last time point output by the two-dimensional time convolution module, obtain the feature results of the next moment, and input them into the linear layer to map them to the runoff prediction value;

[0071] S3. Build the overall network framework model, determine the network training parameters, and conduct training and prediction, including:

[0072] S3.1, integrate steps S1 and S2 to construct a prediction model based on multi-feature fusion and two-dimensional temporal convolutional network;

[0073] S3.2, the training sample loss function in the model uses the MSAE function to optimize the model;

[0074] S3.3, the overall model parameter update method adopts Adam gradient descent method;

[0075] S3.4, using the training set to train the model in step S3.3;

[0076] S3.5. Use the trained model to predict the prediction set, obtain the runoff value, and evaluate it using evaluation indicators.

[0077] Embodiment 2

[0078] As another embodiment, this embodiment 2 proposes, on the basis of the embodiment 1, a more specific daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, wherein step S1 further includes:

[0079] When acquiring site data, data missing and data anomalies may occur due to a series of uncertain factors. In view of the high integrity requirements of the time series data used in the runoff prediction task, in this embodiment, a missing value filling method is adopted to handle missing values. In order to retain the time dependency of daily runoff data, the KNN method is used for data filling. In view of the fact that the outliers in the data set are consistent with the climate environment of the Weihe River Basin and contain hydrological and meteorological characteristic information, in this embodiment, these abnormal data are retained and regarded as characteristic data within the normal range, and are not classified as outliers for processing;

[0080] The polynomial regression model was used to perform linear regression analysis on the runoff of the target site to determine whether the daily runoff series had a good linear relationship;

[0081] Perform a stationarity analysis on the daily runoff data. A time series whose statistical properties do not change over time is considered a stationary time series. To determine whether a series is stationary, we can see whether its mean and variance change over time, and whether the autocorrelation coefficient is only related to the time interval and not to the time. In a stationary time series, the autocorrelation coefficient between random variables with a relatively close time distance is greater than the correlation coefficient between random variables with a relatively far distance. If in the autocorrelation graph, the larger the lag, the smaller the autocorrelation coefficient, and it decays rapidly to near 0, it means that the time series has short-term correlation and is a stationary time series;

[0082] The correlation analysis of influencing factors of daily runoff was conducted, and the maximum mutual information coefficient was used to measure the degree of correlation between two variables. The maximum mutual information coefficient can fully capture various interesting correlations without being restricted by specific function types, covering various functional relationships, including but not limited to linear, exponential and periodic functions. The closer the value of the maximum mutual information coefficient is to 1, the greater the correlation between the variables; the closer the value is to 0, the smaller the correlation between the variables.

[0083] It should be noted that the parts in this embodiment that are the same or similar to those in the first embodiment can be referenced to each other and will not be described in detail in this application.

[0084] Embodiment 3

[0085] As another embodiment, this embodiment 3 proposes, on the basis of embodiments 1 and 2, a more specific daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, wherein step S1 further includes:

[0086] Using two-layer routing attention to fuse multi-source features Dynamically assign weights to focus on key information. The core idea is to filter out most of the irrelevant key-value pairs at the coarse area level and retain only a small part of the routing area. Secondly, apply fine-grained label-to-label attention in the union of these routing areas. The two-layer routing attention can filter out irrelevant information in the first layer of attention, and then further integrate the correlation information between the routing areas in the second layer of attention, so that the model can learn and adapt to information of different modalities more flexibly. This method combined with two-layer routing attention helps to improve the model's ability to express multi-source hydrological datasets, optimize feature fusion and weight allocation, and thus more accurately reflect the complexity and diversity of hydrological processes.

[0087] The two-layer routing attention mainly consists of three components, namely, region partitioning, region-to-region routing using directed graphs, and tag-to-tag attention, e.g. Figure 1 shown.

[0088] Regional division:

[0089] For multi-source fusion features ,in refers to real numbers, represents the number of features, Indicates the length of time, express The number of channels. First, divide it into non-overlapping regions, a single region contains feature matrix, That is Then, we obtain , calculated as formula (3):

[0090] (3)

[0091] The two-layer routing attention mechanism adopts the query-key-value (QKV) model, where Q, K, and V are respectively With different weight matrices Perform a linear transformation to obtain .

[0092] Region-to-region routing using a directed graph:

[0093] By constructing a directed graph through the connection matrix, we can find the participation relationship corresponding to different key-value pairs and find the strong correlation area in each given area. First, we calculate the average value of Q and K in each area and get , , then calculate and The adjacency matrix of the correlation between regions is obtained, and only the top The index of the most relevant region is calculated as follows:

[0094] (4)

[0095] Marker-to-marker attention:

[0096] Leveraging the region-to-region routing index matrix , calculate the fine-grained token-to-token attention. However, since the routing area may be spatially scattered across the entire feature map, it is necessary to aggregate the K and V tensors in the routing area. Then, the fine-grained token-to-token attention is applied on these KV tensors, as shown in Equation (5):

[0097] (5)

[0098] Then , Use the attention operation as shown in formula (6), where Indicates the use of depthwise convolution for parameterized operations:

[0099] (6)

[0100] Use residual connection to add the multi-source fusion feature input and the double-layer routing attention output, and normalize them as shown in formula (7):

[0101] (7)

[0102] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 to 2 can be referenced to each other and will not be described in detail in this application.

[0103] Embodiment 4

[0104] As another embodiment, this embodiment 3 proposes, on the basis of embodiments 1 to 3, a more specific daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, in which step S2, steps S2.2 and S2.3 are specifically as follows:

[0105] S2.2, construct a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution, such as Figure 2 As shown in Figure 1. The dilated causal convolution in TCN is maintained in the time dimension to ensure that the time series is not disturbed by future information. Since spatial features have no causal relationship, dilated convolution is used in the spatial dimension to fully extract the cross information, local information, and global information of spatial features. For the fused sequence obtained after multi-feature fusion processing and convolution kernel , in the output sequence element The mathematical model of the two-dimensional dilated causal convolution at position is as follows:

[0106] (8)

[0107] In the formula, represents the spatial dimension position, Represents the time dimension position, Represents the width of the convolution kernel in the spatial dimension, Represents the width of the convolution kernel in the time dimension, represents the spatial dimension expansion coefficient, Represents the time dimension expansion coefficient. For the two-dimensional expansion causal convolution operation in Figure 2, its receptive field The size calculation method is as follows:

[0108] (9)

[0109] in Indicates the number of network layers, Indicates The spatial dimension expansion coefficient of the layer, Indicates The dilation coefficient of the temporal dimension of the layer. With freely variable spatiotemporal receptive fields, the 2D dilated causal convolution can flexibly capture local and global dependencies.

[0110] S2.3, construct a spatial deconvolution kernel, reconstruct the spatiotemporal feature information and restore the input details, such as Figure 3As shown. The width of the convolution kernel in the time dimension is 1 to retain the advantages of causal convolution in the time dimension; in the spatial dimension, the width of the convolution kernel is set according to the goal of restoring the spatial dimension of the fused multi-source features. The convolution kernel width is calculated as shown in formula (10):

[0111] (10)

[0112] in is the input spatial feature dimension of the two-dimensional dilated causal convolution, is the output spatial feature dimension of the two-dimensional dilated causal convolution. Through targeted deconvolution design, the spatial dimension of the output feature of the two-dimensional dilated causal convolution is restored to , reconstruct information and ensure the unity of space and time dimensions.

[0113] For the 2D dilated causal convolution output and filters , in the output sequence element The calculation formula for positional deconvolution is as follows:

[0114] (11)

[0115] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 to 3 can be referenced to each other and will not be described in detail in this application.

[0116] Embodiment 5

[0117] As another embodiment, this embodiment 3 proposes, on the basis of embodiments 1 to 3, a more specific daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, integrating steps S1 and S2 to construct a prediction model based on multi-feature fusion and two-dimensional temporal convolutional network (MFF-2DTCN), such as Figure 5 As shown;

[0118] Construct a loss function MSAE that takes into account both robustness and sensitivity and is easy to adjust flexibly. This loss function can be more sensitive like MSE when dealing with small errors, and more robust like MAE when dealing with large errors. By adjusting the parameters The balance between MSE and MAE in the loss function can be flexibly controlled. The loss function formula is shown in (12).

[0119] (12)

[0120] In the formula represents the measured value of runoff on the i-th day, represents the predicted value of runoff on the i-th day, and n is the length of the runoff time series.

[0121] The overall model parameter update method adopts the Adam gradient descent method. In this embodiment, the torch.optim.Adam method provided in the torch library is directly used.

[0122] Use the training set to train the integrated model in step S3.1. Provide the corresponding train and save methods in the model class in torch to train and save the model to obtain the optimal result.

[0123] The trained model is used to predict the prediction set to obtain the runoff prediction value, which is then evaluated using evaluation indicators.

[0124] In this embodiment, several evaluation indicators are used, including root mean square error (RMSE), mean absolute percentage error (MAPE) and Nash-Sutcliffe efficiency (NSE).

[0125] (13)

[0126] (14)

[0127] (15)

[0128] In the formula represents the measured value of runoff on the i-th day, represents the predicted value of runoff on the i-th day, It represents the average value of the measured daily runoff, and n is the length of the runoff time series.

[0129] RMSE is the square root of the ratio of the square of the deviation between the observed value and the true value to the number of observations. RMSE is used to measure the deviation between the observed value and the true value. RMSE is very sensitive to very large or very small errors in a set of measurements, so it can well reflect the precision of the measurement. MAPE is a relative measure that actually determines the error scale as a percentage unit rather than a variable unit. MAPE uses absolute values ​​to avoid the mutual cancellation of positive and negative errors, and is sensitive to relative errors and will not change due to the global scaling of the target variable. NSE is generally used to evaluate the quality of hydrological model simulation results, and its range is from negative infinity to 1. The closer NSE is to 1, the better the model quality and the higher the model credibility; the closer NSE is to 0, the more reliable the overall model results are, but the process simulation error is large; if NSE is much less than 0, the model is unreliable.

[0130] On this basis, in order to verify the effectiveness of the proposed model, this study used four existing advanced runoff prediction methods for comparative analysis with the method proposed in this application based on the hydrological and meteorological data of two hydrological stations. The four existing advanced runoff prediction methods are: long short-term memory network (LSTM), temporal convolutional network (TCN), adaptive Fourier decomposition and multi-scale temporal convolutional network (AFDM-MTCN) and temporal convolution and long short-term memory network (TCN-LSTM) model.

[0131] By comparing the performance differences of these five models in predicting the trend of hydrological runoff changes, the prediction ability of the proposed model is comprehensively evaluated. The performance index results are shown in Table 1:

[0132] Table 1 Performance indicators of five prediction methods at two hydrological stations

[0133]

[0134] Compared with the LSTM model, the RMSE of the TCN model is slightly lower than that of the LSTM model, and the MAPE and NSE are slightly higher than those of the LSTM model. It can be seen that the absolute accuracy of the TCN model is better than that of the LSTM model, and the relative accuracy of the TCN model is weaker than that of the LSTM model.

[0135] Compared with the LSTM and TCN models, it can be found that the RMSE and MAPE of the TCN-LSTM model are somewhat reduced, and the NSE is somewhat improved, which can prove that the TCN-LSTM network successfully integrates the advantages of TCN and LSTM. The performance of AFDM-MTCN and MFF-2DTCN is significantly better than that of LSTM, TCN and TCN-LSTM models. It can be seen that the internal potential model characteristics after adaptive Fourier decomposition play a huge role in runoff prediction.

[0136] Finally, compared with the AFDM-MTCN and MFF-2DTCN models, the performance of the MFF-2DTCN model proposed in this application has been improved to a certain extent, which proves the ability of the two-layer routing attention mechanism to capture key information and the ability of the two-dimensional temporal convolutional network to capture spatiotemporal features.

[0137] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 to 4 can be referenced to each other and will not be described in detail in this application.

[0138] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

Claims

1. A daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network, characterized in that: The following steps are involved: S1. Feature mining and multi-source feature fusion of hydrological data and meteorological multi-source data in the target basin are carried out, and the adaptive Fourier decomposition algorithm is used to decompose the nonlinear and non-stationary daily runoff time series. is an IMF component , and get the adaptive Fourier decomposition sequence of runoff; multi-source feature sequence The adaptive Fourier decomposition sequence of runoff is spliced ​​and fused by channel to obtain multi-source fusion features : ; Using two-layer routing attention to fuse multi-source features Dynamically allocate weights and optimize feature fusion, apply fine-grained tag-to-tag attention in the union of these routing areas; double-layer routing attention filters out irrelevant information in the first layer of attention, and then further integrates the correlation information between each routing area in the second layer of attention, and uses residual connection to add and normalize the multi-source fusion feature input and the double-layer routing attention output to obtain a multi-source fusion sequence ; S2. Construct the basic dilated convolution kernel to fuse the multi-source sequence As a two-dimensional feature, dilated convolution is performed, and then a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution is constructed. First, the multi-source fusion sequence Perform dilated convolution as a two-dimensional feature, and then construct a two-dimensional dilated causal convolution kernel based on dilated convolution and causal convolution, keep the dilated causal convolution in TCN in the time dimension, use dilated convolution in the spatial dimension, and fuse multi-source sequences and convolution kernel , ; In the output sequence element The mathematical model of the two-dimensional dilated causal convolution on the position is: ; In the formula, represents the spatial dimension position, Represents the time dimension position, Represents the width of the convolution kernel in spatial dimension, represents the width of the convolution kernel in the time dimension, represents the spatial dimension expansion coefficient, Represents the expansion coefficient of the time dimension; the receptive field of the two-dimensional expanded causal convolution operation The size is calculated as: ; in Indicates the number of network layers, Indicates The spatial dimension expansion coefficient of the layer, Indicates The time dimension dilation coefficient of the layer; Then construct the spatial deconvolution kernel and construct a two-dimensional temporal convolution module. The method of constructing the spatial deconvolution kernel is as follows: the convolution kernel width in the time dimension is 1; in the spatial dimension, the convolution kernel width is used to restore the multi-source fusion features. The spatial dimension of the target is set, and the specific spatial dimension is the convolution kernel width The calculation method is: ; in is the input spatial feature dimension of the two-dimensional dilated causal convolution, is the output spatial feature dimension of the two-dimensional dilated causal convolution; the spatial dimension of the output feature of the two-dimensional dilated causal convolution is restored to , for the 2D dilated causal convolution output and filters , ; In the output sequence element The calculation formula for positional deconvolution is: ; S3. Construct the loss function MSAE to train and predict the prediction model.

2. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 1 is characterized in that: In step S1, the correlation between the variables of the influencing factors of daily runoff is analyzed, and the maximum mutual information coefficient is used to measure the degree of correlation between two variables to perform feature screening on multi-source data. The closer the value of the maximum mutual information coefficient is to 1, the greater the correlation between the variables; the closer the value is to 0, the smaller the correlation between the variables; After feature screening, a multi-source feature sequence is obtained .

3. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 1 is characterized in that: In step S1, the components of the two-layer routing attention include region partitioning, region-to-region routing using directed graphs, and token-to-token attention: For multi-source fusion features ,in represents a real number, represents the number of features, Indicates the length of time, express The number of channels; when performing regional division, firstly combine the multi-source fusion features Divided into non-overlapping regions, a single region contains feature matrix, That is ; Then, we obtain : ; The two-layer routing attention mechanism adopts the query-key-value, or QKV, model, where Q, K, and V are respectively And the corresponding weight matrix Perform a linear transformation to obtain .

4. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 3 is characterized in that: In step S1, a directed graph is constructed by connecting the matrix to find the corresponding participation relationship of different key-value pairs, and the strong correlation area of ​​each given area is found; first, the strong correlation area of ​​each area is calculated. and The average value of , then calculate and The adjacency matrix of the correlation between regions is obtained, and only the top The index of the most relevant area is obtained to obtain the area-to-area routing index matrix ;use Calculate fine-grained tag-to-tag attention. After aggregating the K and V tensors in the routing region, apply fine-grained tag-to-tag attention on the KV tensor: ; Then and Using attention operation: ; in Indicates the use of deep convolution for parameterized operations; Use residual connection to add multi-source fusion feature input and double-layer routing attention output, and normalize them to obtain multi-source fusion sequence .

5. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 1 is characterized in that: In step S2, after the spatial deconvolution kernel is constructed, the spatial deconvolution output features are weighted normalized, and Relu activation and Dropout operations are performed; the input of the two-dimensional dilated causal layer and the output after the Dropout operation are connected by residual to construct a two-dimensional temporal convolution residual block. The size of the two-dimensional dilated causal convolution kernel and the deconvolution kernel is determined according to the size of the dilation factor and the number of network layers; a two-dimensional time convolution module is constructed, and the number and parameters of the residual blocks in the two-dimensional time convolution module are determined according to the size of the dilation factor and the number of network layers, and the outputs of each residual block are spliced ​​by channel using jump connections; the features of the last time point output by the two-dimensional time convolution module are intercepted to obtain the feature results at the next moment, and they are input into the linear layer to be mapped to the runoff prediction value.

6. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 1 is characterized in that: In step S3, the loss function MSAE is constructed based on MSE and MAE, and the parameters are adjusted Control the balance between MSE and MAE in the loss function. The loss function MSAE formula is: ; in, Indicates i The measured value of daily runoff, Indicates i The predicted value of daily runoff, is the length of the runoff time series.

7. The daily runoff prediction method based on multi-feature fusion and two-dimensional temporal convolutional network according to claim 1 is characterized in that: In step S3, the overall model parameter update method adopts the Adam gradient descent method, and the model is trained using the training set; the trained model is used for prediction on the prediction set to obtain the runoff prediction value, and the prediction accuracy of the model is evaluated using evaluation indicators.

Citation Information

Patent Citations

  • Face tampering detection method and system based on multi-source clues and mixed attention

    CN112818862A

  • Reservoir medium and long term monthly runoff prediction method based on improved time convolution network

    CN115409275A

  • Adaptive Fourier decomposition and multi-scale time convolution network daily runoff prediction method

    CN117217359A

  • Hybrid photovoltaic power prediction method and system based on multi-source data fusion

    US20220373984A1