A groundwater level prediction method based on time series convolution feature filtering neural network model

Abstract

The present invention relates to a method for predicting groundwater level based on a time series convolution feature filtering neural network model, comprising: collecting multi-sensor monitoring data related to groundwater level according to time series to obtain multi-feature variable time series data; performing data preprocessing and data segmentation on the multi-feature variable time series data to construct a multi-variable time series tensor of groundwater level, wherein the multi-variable time series tensor of groundwater level includes a training set, a validation set and a test set; establishing a time series convolution feature filtering neural network model; training and validating the time series convolution feature filtering neural network model based on the training set and the validation set to obtain an optimal prediction model; substituting the test set into the optimal prediction model to output multi-step prediction data. The present invention uses a time series convolution feature filtering neural network model to greatly improve the prediction accuracy of groundwater level for a long time in the future, and provide more reliable data support for decision-making on groundwater disasters.

Description

Technical Field

[0001] The present invention relates to the field of groundwater level technology, and in particular to a method for groundwater level prediction based on deep learning, and specifically to a method for groundwater level prediction based on a time series convolution feature filtering neural network model. Background Art

[0002] Groundwater is an important factor affecting the safety of underground projects. It may cause a variety of engineering disasters, such as soil liquefaction, settlement of underground tunnels, pipelines and other underground facilities, water leakage, landslides, foundation settlement and collapse. The groundwater level is affected by factors such as precipitation inflow and infiltration during drought, showing complex nonlinear, seasonal, trend, hysteresis and random characteristics. Compared with other time series prediction tasks, groundwater prediction is more challenging.

[0003] Traditional groundwater level prediction methods such as autoregressive models and linear regression models usually rely on statistical models or physical models. These methods are limited to a certain extent by data quality and model complexity. The prediction accuracy cannot adapt to complex engineering environments and cannot fully capture the nonlinear relationships and spatiotemporal dependencies in groundwater level data.

[0004] With the development of deep learning technology, especially the rise of neural network structures such as convolutional neural network (CNN) and long short-term memory network (LSTM), some new methods have emerged to solve the problem of groundwater level prediction. These methods use neural networks to automatically capture temporal patterns and feature extraction in data, and have shown good performance in some groundwater level prediction tasks.

[0005] However, groundwater level data usually exhibit long-term dependencies. Considering long-term predictions in the future increases the difficulty of groundwater level prediction. Recurrent neural networks such as long short-term memory networks (LSTM) are prone to gradient vanishing and gradient exploding problems when capturing long-term dependencies of groundwater levels, and cannot effectively perform long-term predictions.

[0006] In addition, groundwater level data may be affected by missing, noisy or outliers, which requires effective data processing methods. Since accurate prediction of groundwater level is crucial to engineering safety, more effective, accurate and stable groundwater level prediction methods are needed to improve prediction accuracy and meet the needs of engineering disaster prevention. Summary of the invention

[0007] In view of the shortcomings of the prior art, the main purpose of the present invention is to provide a groundwater level prediction method based on a time series convolutional feature filtering neural network model. The method is based on a time series convolutional filtering neural network and does not rely on a recurrent neural network structure, thereby effectively avoiding the above-mentioned problems.

[0008] The technical solution of the present invention is as follows:

[0009] A groundwater level prediction method based on a time series convolution feature filtering neural network model comprises the following steps:

[0010] Collect multi-sensor monitoring data related to groundwater level according to time series to obtain multi-feature variable time series data;

[0011] Performing data preprocessing and data segmentation on the multi-feature variable time series data to construct a groundwater level multivariate time series tensor, wherein the groundwater level multivariate time series tensor includes a training set, a validation set, and a test set;

[0012] Establish a time series convolution feature filtering neural network model; the time series convolution feature filtering neural network model includes a time series convolution feature filtering module, specifically including:

[0013] Establishing a hole causal convolution layer and an attention filter module, stacking the hole causal convolution layer and the attention filter module into multiple layers to perform multi-level feature extraction on the input data, and using the output result of the last layer of the attention filter module as the input tensor;

[0014] Training and validating the temporal convolutional feature filtering neural network model based on the training set and the validation set to obtain an optimal prediction model;

[0015] Substitute the test set into the optimal prediction model and output multi-step prediction data.

[0016] Preferably, the multi-sensor monitoring data related to groundwater level is collected in time series to obtain multi-feature variable time series data, specifically including:

[0017] Using multiple sensors to obtain data on groundwater level, rainfall, temperature, local reservoir discharge, and local river water level;

[0018] The groundwater level, rainfall, temperature, local reservoir discharge and local river water level data are arranged in time series to obtain multi-feature variable time series data, and organized into a data file format.

[0019] Preferably, the data preprocessing and data segmentation of the multi-feature variable time series data to construct a groundwater level multivariate time series tensor specifically includes:

[0020] Using a mean fitting method to replace or delete erroneous data of the multi-feature variable time series data;

[0021] The multi-feature variable time series data are formed into a multi-feature variable time series data matrix X, wherein each column of data represents a monitoring data feature, and each row of data represents a time series data set of a time interval;

[0022] Each column of the multi-feature variable time series data matrix X is i Perform normalization transformation to obtain the normalized sample matrix X * ;

[0023] Normalize the sample matrix X * Divided into training set, validation set and test set;

[0024] The training set, validation set and test set matrices are split into multivariate time series tensors according to three dimensions including batch size, time step and number of features.

[0025] Preferably, the temporal convolutional feature filtering neural network model also includes:

[0026] Input layer, inputting the multivariate time series tensor T as input data of the time series convolution feature filtering module;

[0027] Skip connection, connecting the output of each layer of attention filter module to the fully connected layer;

[0028] The fully connected layer takes the output of the last layer of attention filter module as input to generate a tensor T 1 ;

[0029] Residual connection, downsampling calculation is performed on the input data multivariate time series tensor T, and the downsampling result is weighted to obtain the tensor T 2 ;

[0030] Output layer, output T 1 and T 2 The sum of T 0 .

[0031] Preferably, the step of establishing a dilated causal convolutional layer includes:

[0032] Determine the expression of the dilated causal convolution:

[0033]

[0034] Where Z is the output original feature map, h[k] is the convolution kernel, K is the size of the convolution kernel, k is the convolution operation index, x is the input data of the previous neural network, n is the current time, and d is the void rate;

[0035] The dilated causal convolution performs zero padding during the convolution process. The specific formula is:

[0036]

[0037] Where p is the filling size, h[k] is the convolution kernel, and d is the void ratio;

[0038] After the convolution and padding operations, the ReLU function is activated. The expression of the ReLU function is:

[0039]

[0040] Where x is the input data of the previous neural network.

[0041] Preferably, the attention filtering module includes a compression module and a filtering module, specifically:

[0042] The compression module is used to reduce the dimension of the input data to obtain the description vector of each time node ;

[0043] The filtering module is used to restore the reduced dimension to the original size;

[0044] The activation function tanh is used to assign a weight to the description vector to determine the criticality of the description vector.

[0045] Preferably, the compression module uses an average pooling algorithm to reduce the size of the input data from N * 1 * TS compression is N * 1 * 1, to obtain the description vector x of the time node q , where the average pooling formula is:

[0046] ;

[0047] And / or, the operation of the filtering module is defined as:

[0048]

[0049] In the formula, sc is x q The score, F ex is the filtering module, W is the weight, , , TS is the time step, β is the dimensionality transformation rate, is the ReLU function;

[0050] The activation function tanh formula is:

[0051]

[0052] Where x is the input data of the previous neural network.

[0053] Preferably, the residual connection is a gated residual connection, and the connection process is:

[0054] Perform downsampling calculation on the input data multivariate time series tensor T;

[0055] Inputting the calculation result into the attention filtering module for judgment;

[0056] After judging the end, weight the downsampling results to obtain the tensor T 2 .

[0057] Preferably, obtaining the optimal prediction model includes:

[0058] Select the time step size according to the requirements, and adjust and evaluate it;

[0059] Train the temporal convolution feature filtering neural network model based on the training set, and use the back-propagation algorithm to adjust the model parameters to reduce the loss function;

[0060] Use Adam optimizer and appropriate learning rate to optimize the model;

[0061] Monitor model performance metrics, loss functions, and accuracy, and use validation sets to evaluate model performance to avoid overfitting;

[0062] When the loss function value on the validation set reaches the lowest point, the model parameters are saved to obtain the optimal prediction model.

[0063] Preferably, the multi-step prediction data is a three-dimensional tensor, and the tensor dimensions are batch size, time step, and prediction days.

[0064] The beneficial effects of the present invention relative to the prior art are:

[0065] The present invention proposes a groundwater level prediction method based on a time series convolution feature filtering neural network model. This method uses the hole causal convolution neural network and channel attention mechanism in deep learning to analyze and extract features from multi-sensor time series data. Compared with increasing the network depth or width to expand the receptive field, the hole convolution achieves the same goal through the expansion rate parameter, but does not require additional parameters. The attention filtering module assigns different weights to the output channels of the hole causal convolution layer, enhances important features, suppresses irrelevant features, optimizes model complexity and the number of parameters, and improves model efficiency. The model in this paper can reduce the computational burden while maintaining model performance, and has obvious advantages in computing speed. This technological innovation aims to cope with groundwater level monitoring, accelerate model training and improve prediction accuracy to cope with the complexity of predicting groundwater levels for a long time in the future, and can provide more reliable data support for decision-making related to groundwater disasters.

[0066] The present invention utilizes the void causal convolutional network, attention filtering mechanism and other technologies in deep learning to analyze and extract features of multi-sensor monitoring time series data, expand the receptive field of the neural network, capture the time dependence of the groundwater level, and realize multi-step prediction of the groundwater level.

[0067] The present invention establishes a groundwater level prediction model based on the time series convolution feature filtering method. The model uses the characteristics of the void causal convolution and attention filtering modules to enhance the model's ability to capture long-term data dependencies, exclude data information that may interfere with groundwater level prediction, and improve prediction accuracy. In comparison, recurrent neural network models such as LSTM and GRU require more computing resources to process time series data, and TCN models are easily affected by noise and outliers.

[0068] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easy to understand through the following description. In addition, the implementation of any embodiment of the present invention does not mean that multiple or all of the above-mentioned beneficial effects must be possessed or achieved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0070] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment in size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0071] Figure 1 It is a schematic diagram of the overall process of the neural network model based on temporal convolution feature filtering of the present invention;

[0072] Figure 2 Schematic diagram of the preprocessing steps of multi-feature variable time series data of the present invention;

[0073] Figure 3 This is a schematic diagram of the structure of a neural network model based on temporal convolutional feature filtering in the present invention;

[0074] Figure 4Schematic diagram of the dilated causal convolution structure of the present invention;

[0075] Figure 5 This is a schematic diagram of the attention filtering module of the present invention;

[0076] Figure 6a This is a schematic diagram of the model training results based on the DAR model of the present invention;

[0077] Figure 6b Schematic diagram of the loss function value of the present invention based on the DAR model for training;

[0078] Figure 6c This is a schematic diagram of the model training results based on the CNN-GRU model of the present invention;

[0079] Figure 6d This is a schematic diagram of the loss function value of the present invention based on the CNN-GRU model for training;

[0080] Figure 6e This is a schematic diagram of the model training results based on the GRU model of the present invention;

[0081] Figure 6f This is a schematic diagram of the loss function value of the present invention based on the GRU model for training;

[0082] Figure 6g This is a schematic diagram of the model training results based on the LSTM model of the present invention;

[0083] Figure 6h This is a schematic diagram of the loss function value of the present invention based on the LSTM model for training;

[0084] Figure 6i This is a schematic diagram of the model training results based on the TCN model of the present invention;

[0085] Figure 6j This is a schematic diagram of the loss function value of the present invention based on the TCN model for training. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with the embodiments and drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0087] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0088] It should be understood that the terms "include / comprise", "consist of..." or any other variations are intended to cover non-exclusive inclusion, so that a product, device, process or method that includes a series of elements includes not only those elements, but also may include other elements not explicitly listed when necessary, or also includes elements inherent to such product, device, process or method. In the absence of more restrictions, the elements defined by the sentence "include / comprise...", "consist of..." do not exclude the presence of other identical elements in the product, device, process or method that includes the elements.

[0089] It is also necessary to understand that terms such as "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside" and "outside" to indicate directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device, component or structure must have a specific direction, be constructed or operate in a specific direction, and should not be understood as a limitation on the present invention.

[0090] The implementation of the present invention is described in detail below in conjunction with preferred embodiments.

[0091] like Figure 1 As shown in FIG6 , the present invention provides a groundwater level prediction method based on a time series convolution feature filtering neural network model, see Figure 1 The flowchart shown includes the following steps: S10: collecting multi-sensor monitoring data related to groundwater level according to time series to obtain multi-feature variable time series data; S20: performing data preprocessing and data segmentation on the multi-feature variable time series data to construct a multi-variable time series tensor of groundwater level, wherein the multi-variable time series tensor of groundwater level includes a training set, a validation set and a test set; S30: establishing a time series convolution feature filtering neural network model; S40: training and validating the time series convolution feature filtering neural network model based on the training set and the validation set to obtain the optimal prediction model; S50: substituting the test set into the optimal prediction model to output multi-step prediction data.

[0092] The present invention uses a time series convolution feature filtering neural network model (DAR model), which can greatly improve the prediction accuracy of groundwater levels in the long term in the future and provide more reliable data support for groundwater disaster decision-making. More precisely, the idea of ​​the present invention is a groundwater level prediction method based on deep learning technology, which uses the void causal convolution neural network and attention filtering module in deep learning to extract and analyze the features of multi-sensor monitoring time series data. This technological innovation is designed to cope with groundwater level monitoring, save computing resource consumption, accelerate model training and prediction accuracy, and provide more reliable data support for groundwater disaster-related decision-making.

[0093] It should be noted that the so-called multi-step prediction means that by substituting the test set of the groundwater level multivariate time series tensor into the optimal prediction model, it is possible to predict and output data for multiple days. For example, if 30 days of monitoring data are known, the usual method uses 30 days of data to predict the 31st day of data, and can only predict 1 day of groundwater level data. The multi-step prediction method uses 30 days of data to predict data from 31 to 40 days, which is equivalent to predicting 10 days of groundwater level data. Of course, the method provided by the present invention can also predict more days of data when the complexity of the monitoring data is low and the accuracy of the final prediction result is within the required range.

[0094] In step S10, multi-sensor monitoring data related to groundwater level is collected in time series to obtain multi-feature variable time series data.

[0095] In this step, the multi-sensor monitoring data related to groundwater level referred to here refers to data including multiple features, including but not limited to the following features: groundwater level, rainfall, temperature, local reservoir discharge, local river water level, etc., data collected in time series.

[0096] In a specific embodiment, obtaining multi-feature variable time series data specifically includes:

[0097] Multiple sensors are used to obtain data on groundwater level, rainfall, temperature, local reservoir discharge, and local river water level. It should be understood that the measurement of each parameter requires the use of appropriate sensors to ensure the accuracy and reliability of the data. For example, groundwater level is usually measured using a water level gauge, rainfall is recorded by a rain gauge, temperature can be obtained through a temperature sensor, and reservoir discharge and river water level may require the use of devices such as flow meters and water level gauges.

[0098] Arrange the data of groundwater level, rainfall, temperature, local reservoir discharge and local river water level in time series to obtain multi-feature variable time series data, and organize them into data files. The collected data needs to be arranged in time series. This means that each data point needs to be marked with its corresponding timestamp so that it is clear when and where the data was collected during subsequent analysis and processing. Time series arrangement is the basis of time series data analysis, which makes the data show trends and laws that change over time.

[0099] It is easy to understand that according to the characteristics of neural networks, the more types of data features related to groundwater levels, the better the model prediction effect. These data are recorded in real time by daily monitoring instruments or sensors, including two dimensions: time and specific values, such as the number of millimeters of rainfall corresponding to a specific date. Of course, due to the real-time recording of instruments or sensors, the time dimension can be divided into smaller parts according to the needs of the model, such as every hour, every minute, etc. After the data is collected, the multi-sensor monitoring data is integrated to obtain multi-variable time series data and organized into data files for use.

[0100] In step S20, data preprocessing and data segmentation are performed on the multi-feature variable time series data to construct a groundwater level multivariate time series tensor, wherein the groundwater level multivariate time series tensor includes a training set, a validation set, and a test set. Figure 2 As shown, it specifically includes the following sub-steps:

[0101] In step S201, the multi-feature variable time series data is preprocessed. Due to the complexity of the data collection process, the collected multi-feature variable time series data often has missing values ​​and erroneous values. The erroneous data entering the model will affect the accuracy of the model, so the erroneous data needs to be preprocessed before modeling.

[0102] In some embodiments, the mean fitting method may be used to replace or delete the erroneous data. It is easy to understand that the mean fitting method is to replace the erroneous data with the mean of the relevant data according to the actual situation (a method of filling abnormal values).

[0103] In step S202, the multi-feature variable time series data, such as groundwater level, rainfall, temperature, local reservoir discharge, local river water level, etc., are integrated into a multi-feature variable time series data matrix X (i.e., a sample data matrix), where each column of data represents a monitoring data feature i, and X i Indicates that each row of data represents a time series data set with a time interval, represented by X k Indicates. ki Represents the data value of the kth time series data in the i-th column of the multi-feature variable time series data matrix X, where k is the time series number of each column of data.

[0104] In step S203, each column of the multi-feature variable time series data matrix X is i Perform normalization transformation to obtain the normalized sample matrix X * Due to the diversity of monitoring time series data features, the features of the data set have different value ranges. Therefore, before using the data, it is necessary to normalize the data into dimensionless scalars so that different features have the same measurement scale. At the same time, it can also eliminate the adverse effects caused by erroneous data, which is convenient for subsequent model training and prediction.

[0105] In some embodiments, the following formula is used to calculate each column of data X of the multi-feature variable time series data matrix X: i Perform normalization:

[0106]

[0107] In the formula, x ki represents the data value of the kth time series data in the i-th column of the multi-feature variable time series data matrix X, x ki * is the time series data x ki The normalized function value of * The kth time series data in the ith column of , where the value range of the ith column data is [x imin , x imax ], that is, x imin and x imax are the i-th column data X in the multi-feature variable time series data matrix X i The minimum and maximum values ​​of the normalized sample matrix X * Each column of data in X i * Table 1 is monitoring data, k is the time sequence number of each column of data.

[0108] It is easy to understand that the normalization transformation converts the time series data interval of multiple feature variables to [0, 1], so as to better adapt to the training of deep learning models and accelerate the convergence speed. When the evaluation criteria of sample data are different, it is necessary to dimensionalize them. Normalization can eliminate the impact of the dimension on the evaluation results and make different indicators comparable.

[0109] In step S204, the normalized sample matrix X * It is divided into training set, validation set and test set, of which 70% of the data is used as training set, 10% of the data is used as validation set, and 20% of the data is used as test set. In deep learning, training set, validation set and test set are three important parts of the data set, which are used to train, evaluate and test the performance of deep learning models.

[0110] The training set is a dataset used by deep learning models for training and learning, and is used to train the model's parameters. The validation set is a dataset used to evaluate the model's performance, and is used to adjust the model's parameters during the training process, improve the model's performance, and avoid overfitting or underfitting the model. The test set is a dataset used to evaluate the final performance of the model, which does not overlap with the training set and validation set, and is used to determine whether the model is accurate.

[0111] In step S205, the training set, validation set, and test set matrices are divided into a multivariate time series tensor T according to three dimensions (batch size, time step, number of features), which serves as the input tensor of the time series convolutional feature filtering neural network. The data is divided into these three dimensions to facilitate input into the neural network for training.

[0112] In some embodiments, batch size refers to the number of data samples that the neural network processes simultaneously in each training iteration. The batch size is set based on the size of the computer video memory used to train the neural network model, and the default value is 64. Time step refers to the number of time points contained in each sample in the time series. The time step can be customized to the best setting, such as using a time step of 30. This means that each input sample is a time series segment of length 30. The number of features refers to the data dimension at each time point. In multivariate time series, this usually corresponds to multiple variables observed simultaneously (such as groundwater level, rainfall, temperature, local reservoir discharge, and local river water level). The choice of the number of features depends entirely on the complexity of your dataset and problem. More features can provide richer information, but may also cause the model to overfit or require a more complex model structure to capture the relationship between these features.

[0113] In step S30, a temporal convolution feature filtering neural network model is established. Figure 3 , the temporal convolution feature filtering neural network model specifically includes:

[0114] Input layer, input multivariate time series tensor T, which may contain multiple time series, each of which contains observations of multiple variables at different time points.

[0115] The temporal convolutional feature filtering module includes establishing a void causal convolution layer and an attention filtering module. Specifically, the void causal convolution layer and the attention filtering module are stacked into multiple layers to perform multi-level feature extraction on the input data multivariate temporal tensor T, and the output result of the last layer of the attention filtering module is input as the input tensor into the fully connected layer.

[0116] Skip connections connect the output of each attention filter module to the fully connected layer. Skip connections connect the output of each attention filter module directly to the fully connected layer, which helps to preserve low-level detail information and alleviate the gradient vanishing problem in deep networks.

[0117] The fully connected layer,takes the output of the last layer of attention filter module as input,,generating tensor T1, which helps to map the extracted temporal features,to the target output space and generate tensor T1.

[0118] Residual connection, downsampling calculation is performed on the input data multivariate time series tensor T, and the downsampling result is weighted to obtain the tensor T 2 By introducing residual connections, the model can learn the residual between input and output, which helps improve the stability of training and the performance of deep networks.

[0119] Output layer, output T 1 and T 2 The sum of T 0 .

[0120] In some embodiments, a dilated causal convolution layer is established. Dilated causal convolution is a special convolution operation that better captures the causal relationship in the sequence by adding a dilation rate to the causal convolution model, expands the receptive field of the convolution kernel, and improves the model's generalization ability to the future. Figure 4 , including input layer, hidden layer and output layer, where the input layer is the first layer of the atrous causal convolutional network, responsible for receiving the original time series data. The data in the input layer is usually one-dimensional, with one or more feature values ​​corresponding to each time point. These feature values ​​are organized into a series of time series samples in the input layer for processing by subsequent layers. The hidden layer is the core part of the atrous causal convolutional network, consisting of multiple atrous causal convolutional layers. These layers are responsible for extracting the features of the input data and passing the features to the next layer through convolution operations. The output layer is the last layer of the atrous causal convolutional network, responsible for converting the features extracted by the hidden layer into the final prediction results. The output layer usually contains a series of neurons, which perform weighted summation based on the features passed by the hidden layer and produce the final output through an activation function (such as softmax).

[0121] In a specific embodiment, establishing a dilated causal convolutional layer includes:

[0122] Determine the expression of the dilated causal convolution:

[0123]

[0124] In the formula, Z is the output original feature map, h[k] is the convolution kernel, K is the size of the convolution kernel, k is the convolution operation index, x is the input data of the previous neural network, n is the current time, and d is the void rate; this expression indicates that each element of the output tensor is obtained by multiplying and adding the convolution kernel and the input tensor at a certain position.

[0125] The dilated causal convolution performs zero padding during the convolution process. The specific formula is:

[0126]

[0127] Where p (padding) is the padding size. Padding refers to filling zero values ​​around the edges of the input data to keep the input and output sizes consistent when performing operations such as convolution or pooling. h[k] is the convolution kernel, and d is the dilation rate.

[0128] After the convolution and padding operations, the ReLU function is activated. The expression of the ReLU function is:

[0129]

[0130] Where x is the input data of the previous neural network.

[0131] In some embodiments, an attention filtering module is established, such as Figure 5 As shown, the attention filtering module includes a compression module and a filtering module. The implementation process of the attention filtering module is: the compression module reduces the dimension of the input data, and the filtering module restores the dimension to the original size, determines the criticality of the features describing the vector, and assigns the weight of the vector.

[0132] Specifically, the compression module uses the average pooling algorithm to reduce the size of the input data from N * 1 * TS compression is N * 1 * 1, thus obtaining the description vector x of the time node q , where the average pooling formula is:

[0133]

[0134] Where TS is the time step length, t is the current time step, and d is the number of input channels.

[0135] The operations of the filter module are defined as follows:

[0136]

[0137] In the formula, x q is a description vector, which represents the input data of the previous neural network, and sc is x q The score, F ex is the filtering module, W is the weight, , , TS is the time step, β is the dimension transformation rate, and σ is the ReLU function;

[0138] First, the input data is the channel description vector xq , through two fully connected layers x q The importance of each channel is predicted, and the Relu function is added in the two fully connected layers to enhance the nonlinear ability of the filtering module. Finally, the output of the filtering module is mapped to [-1, 1] through the Tanh function.

[0139] The Tanh function formula is as follows:

[0140]

[0141] If the weight calculated by the activation function tanh is a negative value, it means that the filtering mechanism believes that these channels have a negative impact on the prediction results, and the model will reduce the weight of these channels to reduce the impact of the above information on the overall results. If the weight calculated by the activation function tanh is a positive value, the greater the weight, the more important the channel.

[0142] In some embodiments, the hole causal convolution layer and the attention filter module are stacked into multiple layers, and the input tensor of the hole causal convolution layer in each middle layer is the output tensor of the attention filter module in the previous layer. The output result of the attention filter module in the last layer is input as the input tensor into the fully connected layer, and the function of the fully connected layer is to adjust the tensor size so that the output result conforms to the prediction data format.

[0143] In some embodiments, the residual connection is preferably a gated residual connection, and the function of the gated residual connection is to prevent the gradient from vanishing or exploding due to the excessive depth of the neural network layer. The connection process of the gated residual connection is: downsampling the input data multivariate time series tensor T, and the downsampling operation refers to performing a one-dimensional convolution calculation on T with a convolution kernel size of 1, and the calculation result is input into the attention filtering module for judgment, and the necessity of the information entering the residual connection is judged. After the judgment is completed, the weight assignment of the downsampling result is obtained to obtain the tensor T 2 . T 2 Wait for the output result T of the fully connected layer at the end of the network 1 Add together to get the final output result T of the DAR model 0 , which is the 30-day groundwater level prediction result.

[0144] In some embodiments, the final output of the neural network model is a three-dimensional tensor T 0 , the dimensions are: batch size, time step, number of prediction days, the specific dimensions are [64, 30, 30].

[0145] In step S40, the time series convolution feature filtering neural network model is trained and verified based on the training set and the validation set to obtain the optimal prediction model. In this step, the training set of the groundwater level multivariate time series tensor is used to train the model, and the model performance indicators, loss function and accuracy are monitored. The validation set of the groundwater level multivariate time series tensor is used to verify the loss function value and the early stopping method to evaluate the model performance to avoid overfitting.

[0146] When setting hyperparameters, pay attention to the following hyperparameters: time step size, batch size, number of features, number of channels in convolution modules, number of days for prediction, learning rate, and patience value in early stopping method. Among them, the specific size of the time step should be selected according to the problem requirements, and needs to be adjusted and evaluated; the batch size defaults to 64, the number of features defaults to 5, the number of channels in the convolution module is 64 and 32 for the two layers respectively, the number of days for prediction is 30 days, the learning rate defaults to 0.0001, and the patience value is used to monitor the number of rounds in which the loss function value of the validation set does not decrease continuously during the model training process. When the number of rounds in which the validation set loss does not decrease continuously exceeds the patience value, the training process will be terminated. Setting the patience value helps to stop training when the model reaches the best performance to avoid overfitting and save computing resources.

[0147] In some embodiments, step S40 specifically includes the following sub-steps:

[0148] In step S401, the specific size of the time step should be selected according to the problem requirements, that is, the time step size is selected according to the requirements, and is adjusted and evaluated;

[0149] In step S402, after preprocessing the multi-feature variable time series data, the time series convolution feature filtering neural network model is trained based on the training set. The back propagation algorithm is applied in the training process, which is suitable for learning multi-layer neural networks. The back propagation algorithm is based on the gradient descent method. It calculates the gradient of the loss function to the model parameters and updates the parameters in the opposite direction of the gradient to minimize the loss function. The information processing capability of the back propagation algorithm comes from the combination of multi-layer nonlinear activation functions in the neural network, which enables the neural network to learn complex functional relationships and has a strong function reproduction ability.

[0150] Continuously adjust the model parameters to reduce the loss function. This embodiment uses the MSE loss function, and the calculation formula of the MSE loss function is:

[0151]

[0152] in, is the model's predicted value for the i-th sample, y i is the actual target value of the i-th sample, and N is the number of samples.

[0153] The loss function calculates the gradient of the loss function to the model parameters, and propagates the gradient from the output layer to the input layer through the back propagation algorithm to update the parameters of each layer. Generally, the smaller the loss function, the better. In this step, the model parameters need to be adjusted to continuously reduce the loss function.

[0154] In step S403, the model is optimized using the Adam optimizer and a suitable learning rate. The Adam optimizer is an adaptive optimization algorithm that can adjust the learning rate based on historical gradient information. In actual use, the learning rate can be adjusted according to the data size and data complexity. In this embodiment, the learning rate is 0.0005.

[0155] In step S404, the model performance indicators, loss function and accuracy are monitored, and the validation set is used to evaluate the model performance to avoid overfitting. The model performance is measured by the mean square error MSE and the coefficient of determination R 2 Value measurement, multiple training to obtain MSE and determination coefficient R 2 The model at the optimal value. The loss function is a calculation function used to measure the difference between the model's predicted value and the true value. It is a non-negative real-valued function. The smaller the loss function, the better the robustness of the model. In the process of parameter adjustment, the model is solved and evaluated by minimizing the loss function. The accuracy of the model refers to the number of all correctly predicted samples in the model / the total number of observed samples for a given test set. In the process of parameter adjustment, the accuracy of the model should be continuously improved as much as possible.

[0156] In step S405, when the loss function value on the validation set reaches the lowest point, the model parameters are saved to obtain the optimal model.

[0157] By executing the above steps, the optimal prediction model file is obtained to predict the groundwater level.

[0158] In step S50, the test set of the groundwater level multivariate time series tensor is substituted into the optimal prediction model, and the model calculates and outputs multi-step prediction data.

[0159] When predicting groundwater levels, the input data is consistent with the training model input data, and the input is a three-dimensional tensor with the dimensions of batch size, time step, and number of feature channels. The prediction output result is a three-dimensional tensor with the dimensions of batch size, time step, and number of prediction days.

[0160] The present invention is a groundwater level prediction based on a time series convolution feature filtering neural network model. The multi-sensor time series data is analyzed and feature extracted using deep learning techniques such as hole causal convolution and channel attention mechanism to achieve multi-step prediction of groundwater level. This technological innovation aims to cope with the complexity of groundwater level monitoring and prediction and provide more reliable data support for decision making.

[0161] This embodiment takes the groundwater level data of Petrignano, Italy as an example. The example data features include five features: groundwater level height, rainfall, temperature, local reservoir discharge, and local river water level height in Petrignano, Italy from 2009 to 2016. The prediction time is the groundwater level in the next 30 days.

[0162] In this embodiment, four common neural network models are selected, including long short-term neural network (LSTM), gated recurrent unit (GRU), convolutional gated recurrent unit (CNN-GRU), temporal convolutional network (TCN) and temporal convolutional feature filtering neural network (DAR) model for comparison. The model training adopts multiple repeated experiments and a random seed method to reduce the influence of randomness. All five models are trained 40 times with different initialization seeds. After actual testing, the model training results have been stable after 40 repeated experiments.

[0163] Figure 6a The DAR model prediction results of the present invention are compared with the true values. Figure 6c Compare the prediction results of the CNN-GRU model with the actual values. Figure 6e Compare the GRU model prediction results with the true values. Figure 6g Compare the prediction results of the LSTM model with the actual values. Figure 6i This is the comparison between the TCN model prediction results and the true value. In the figure, x is the number of test set samples, and y is the groundwater level. Figure 6a The model mean square error MSE is 0.205, the mean absolute percentage error MAPE is 0.006, and the determination coefficient R 2 The prediction results of other models are shown in Table 1. By comparison, the DAR model is the best among the five models.

[0164] Table 1 The optimal prediction results of the 40-shot random seed model

[0165]

[0166] Figure 6b Schematic diagram of the loss function value for training the DAR model of the present invention, Figure 6d Schematic diagram of the loss function value for training the CNN-GRU model. Figure 6f Schematic diagram of the loss function value for training the GRU model. Figure 6h Schematic diagram of the loss function value for training the LSTM model. Figure 6jSchematic diagram of the loss function value for training the TCN model. In the figure, x is the number of training cycles, and y is the loss function value. The loss function values (training set loss, validation set loss, test set loss) are basically stable around the optimal value in 10 training cycles, while the loss function value of the trained LSTM model is shown in Figure 6h , and it only reaches stability after 40 cycles. It can be seen that the loss function of the DAR model converges quickly and stably during the training process, saving a large amount of computing resources.

[0167] Table 2 Training time of the model

[0168]

[0169] In terms of training time, as shown in Table 2 above, the loss function of the model in this paper drops the fastest, approaching the extreme value near the 10th step and dropping to the lowest position at the 53rd step. The fast dropping speed of the loss function means that the model training process is stable and consumes little computing resources. The training times of the 5 models from small to large are DAR < TCN < CNN - GRU < GRU < LSTM. The DAR model has the shortest training time and has significant advantages in saving computing resources and improving the operation speed.

[0170] Through the groundwater level prediction method based on the DAR model of the present invention, the error of the prediction model is reduced, the accuracy of the model prediction is improved, and it is applicable to complex prediction environments and computational device and computational time limitations, realizing accurate prediction of the groundwater level in the future for a long time.

[0171] Although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present invention. Certain features described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, the various features described in the context of a single implementation can also be implemented separately or in any suitable sub - combination in multiple implementations.

[0172] The embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the disclosed embodiments.

[0173] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A groundwater level prediction method based on a time series convolution feature filtering neural network model, characterized in that: The steps include: Collect multi-sensor monitoring data related to groundwater level according to time series to obtain multi-feature variable time series data; Performing data preprocessing and data segmentation on the multi-feature variable time series data to construct a groundwater level multivariate time series tensor, wherein the groundwater level multivariate time series tensor includes a training set, a validation set, and a test set; A temporal convolution feature filtering neural network model is established, wherein the temporal convolution feature filtering neural network model includes a temporal convolution feature filtering module, specifically including: Establishing a hole causal convolution layer and an attention filter module, stacking the hole causal convolution layer and the attention filter module into multiple layers to perform multi-level feature extraction on the input data, and using the output result of the last layer of the attention filter module as the input tensor; Training and validating the temporal convolutional feature filtering neural network model based on the training set and the validation set to obtain an optimal prediction model; Substituting the test set into the optimal prediction model, and outputting multi-step prediction data; wherein The temporal convolution feature filtering neural network model also includes: Input layer, inputting the multivariate time series tensor T as input data of the time series convolution feature filtering module; Skip connection, connecting the output of each layer of attention filter module to the fully connected layer; The fully connected layer takes the output of the last layer of attention filter module as input to generate tensor T1; The gated residual connection performs downsampling calculation on the multivariate time series tensor T of the input data, and assigns weights to the downsampling results to obtain tensor T2; Output layer, outputs T0, the sum of T1 and T2; and The attention filtering module includes a compression module and a filtering module, wherein the compression module reduces the dimension of the input data to obtain a description vector x for each time node q , the filtering module restores the reduced dimension to the original size, and uses the activation function tanh to assign a weight to the description vector to determine the criticality of the description vector; and The compression module uses an average pooling algorithm to reduce the size of the input data from N * 1 * TS compression is N * 1 * 1, to obtain the description vector x of the time node q , where the average pooling formula is: ； Where TS is the time step, t is the current time step, and d is the number of input channels; The operation of the filtering module is defined as: ； In the formula, sc is x q The score, F ex is the filtering module, W is the weight, , , TS is the time step, β is the dimension transformation rate, and σ is the ReLU function; The activation function tanh formula is: ； Where x is the input data of the previous neural network.

2. The method for predicting groundwater level according to claim 1, characterized in that: The method collects multi-sensor monitoring data related to groundwater level according to time series to obtain multi-feature variable time series data, specifically including: Using multiple sensors to obtain data on groundwater level, rainfall, temperature, local reservoir discharge, and local river water level; The groundwater level, rainfall, temperature, local reservoir discharge and local river water level data are arranged in time series to obtain multi-feature variable time series data, and organized into a data file format.

3. The method for predicting groundwater level according to claim 1, characterized in that: The data preprocessing and data segmentation of the multi-feature variable time series data to construct a groundwater level multivariate time series tensor specifically includes: Using a mean fitting method to replace or delete erroneous data of the multi-feature variable time series data; The multi-feature variable time series data are formed into a multi-feature variable time series data matrix X, wherein each column of data represents a monitoring data feature, and each row of data represents a time series data set of a time interval; Each column of the multi-feature variable time series data matrix X is i Perform normalization transformation to obtain the normalized sample matrix X * ; Normalize the sample matrix X * Divided into training set, validation set and test set; The training set, validation set and test set matrices are split into multivariate time series tensors according to three dimensions including batch size, time step and number of features.

4. The method for predicting groundwater level according to claim 1, characterized in that: The establishment of the hole causal convolutional layer includes: Determine the expression of the dilated causal convolution: ； Where Z is the output original feature map, h[k] is the convolution kernel, K is the size of the convolution kernel, k is the convolution operation index, x is the input data of the previous neural network, n is the current time, and di is the void rate; The dilated causal convolution performs zero padding during the convolution process. The specific formula is: ； Where p is the filling size, h[k] is the convolution kernel, and di is the void ratio; After the convolution and padding operations, the ReLU function is activated. The expression of the ReLU function is: ； Where x is the input data of the previous neural network.

5. The method for predicting groundwater level according to claim 1, characterized in that: The connection process of the gated residual connection is: Perform downsampling calculation on the multivariate time series tensor T of the input data; Inputting the calculation result into the attention filtering module for judgment; After judging the end, weights are assigned to the downsampling results to obtain tensor T2.

6. The method for predicting groundwater level according to claim 1, characterized in that: The obtaining of the optimal prediction model comprises: Select the time step size according to the requirements, and adjust and evaluate it; Train the temporal convolution feature filtering neural network model based on the training set, and use the back-propagation algorithm to adjust the model parameters to reduce the loss function; Use Adam optimizer and appropriate learning rate to optimize the model; Monitor model performance metrics, loss functions, and accuracy, and use validation sets to evaluate model performance to avoid overfitting; When the loss function value on the validation set reaches the lowest point, the model parameters are saved to obtain the optimal prediction model.

7. The method for predicting groundwater level according to claim 1, characterized in that: The multi-step prediction data is a three-dimensional tensor, and the tensor dimensions are batch size, time step, and prediction days.

Images