A method for estuary water level time series prediction based on improved Informer deep learning framework

Abstract

The present invention discloses a method for predicting estuary water level time series based on an improved Informer deep learning framework. The method comprises: obtaining water level time series data at a location to be predicted; decomposing the water level time series data at the location to be predicted into a number of characteristic parameters; for each characteristic parameter, using it as data input, and outputting the predicted characteristic parameter through a pre-trained PSO-Informer neural network model corresponding to each characteristic parameter; the PSO-Informer neural network model adjusts the model hyperparameters through a PSO algorithm during training; and adding all predicted characteristic parameters to obtain a predicted water level time series.

Description

Technical Field

[0001] The present invention relates to the technical field of estuary water level time series prediction, and in particular to a estuary water level time series prediction method based on an improved Informer deep learning framework. Background Art

[0002] Estuaries are densely populated, with developed agriculture, transportation, and urban agglomerations. Of the world's 32 largest cities, 22 are located near estuaries. However, estuaries are extremely vulnerable to coastal flooding, especially in recent years due to rising sea levels caused by global climate change. Therefore, accurate prediction of estuary water levels plays a vital role in protecting human life and property.

[0003] For estuary areas, when the astronomical tide caused by the tidal force of celestial bodies propagates to the estuary, it will be affected by the change in water depth and the width of the estuary. In addition, the water flow upstream of the estuary causes a nonlinear effect on the astronomical tide, making it difficult to predict the water level. Therefore, due to the complexity of dynamics and topography in this area, it is more difficult to predict the tidal level. In the past, many studies have predicted water levels based on traditional physical models to analyze the interaction between ocean dynamics and river dynamics. However, due to its expensive computational cost and the need for runoff data of the downstream rivers upstream of the estuary, there are certain difficulties in the actual prediction process. Summary of the invention

[0004] In view of this, the purpose of the present invention is to propose a estuary water level time series prediction method based on an improved Informer deep learning framework, which can solve at least one technical problem mentioned in the background technology.

[0005] According to one aspect of the present invention, a method for predicting estuary water level time series based on an improved Informer deep learning framework is provided.

[0006] Obtain the water level time series data at the location to be predicted;

[0007] Decompose the water level time series data at the location to be predicted into several characteristic parameters;

[0008] For each characteristic parameter, it is used as data input, and the predicted characteristic parameter is output through the pre-trained PSO-Informer neural network model corresponding to each characteristic parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training; the PSO-Informer neural network model includes: n groups of neural network subunits, each subunit includes an Informer network and a PSO adjustment unit; wherein:

[0009] The informer network is configured to take each feature parameter as the input of data and output the predicted feature parameter through each feature parameter;

[0010] A PSO adjustment unit, configured to adjust model hyperparameters by using a PSO algorithm during training;

[0011] All predicted characteristic parameters are added together to obtain the predicted water level time series.

[0012] In the above technical scheme, for the existing estuary area water level prediction model based on machine learning, the machine learning methods used are mainstream time series prediction methods such as RNN, LSTM and Transformer. Although the above algorithms have good performance in time series prediction, they still have shortcomings. For example, recurrent neural networks such as RNN and LSTM use a step-by-step method to process sequence data, which is inefficient in capturing long-range dependencies in the sequence, so they can only effectively capture short-term time dependencies, and the prediction effect for longer time series is lower than that of Transformer. Since Transformer uses a self-attention mechanism to perform comprehensive attention calculations on the entire input sequence, it is better than recurrent neural networks for predicting long time series, but as the sequence length increases, the computing resource requirements and memory consumption will increase significantly. Based on Transformer, Informer reduces unnecessary calculations by improving the attention mechanism and hierarchical architecture design, especially in long sequence tasks, which can greatly improve computing efficiency, reduce training and reasoning time, and ensure the same prediction effect as Transformer. At the same time, the selection of Informer hyperparameters plays an important role in the training effect of Informer. Manually setting hyperparameters is time-consuming and labor-intensive, and finding the optimal hyperparameters is not accurate enough. Therefore, PSO is added to the Informer framework to optimize hyperparameters. The PSO optimization algorithm has the characteristic of relatively simple parameter adjustment. Therefore, in the application, it is easier to tune the parameters according to the characteristics of the problem to obtain better performance. In summary, the present invention proposes a estuary time series prediction method based on an improved Informer deep learning framework.

[0013] In some embodiments, the water level time series data at the location to be predicted is decomposed into several characteristic parameters. Specifically, an adaptive noise-assisted method of complete set empirical mode decomposition is used to decompose the water level time series data at the location to be predicted into several intrinsic mode functions.

[0014] In the above technical solution, compared with the traditional segmentation method, decomposing the time series into several modes can reduce the difficulty of training the model. Because the signal complexity of the decomposed intrinsic mode functions is smaller than that of the original signal, it is conducive to the training of the model; and training the model separately according to the intrinsic mode functions of different frequencies can better capture the time series information at different scales. Variants of classical modal decomposition include EEMD, CEEMD, etc. In this case, CEEMDAN is used. Compared with EEMD and CEEMD, its advantage is that it reduces modal mixing and improves the accuracy of decomposition. The advantage of using CEEMDAN in this case is mainly that the residual error between the decomposed modes and the original signal is very small after adding them together (see Figure 5 ), so the model training can be directly performed on several intrinsic mode functions without considering the influence of residuals. If the residual is large after the mode separation using the EEMD and CEEMD methods, it may be necessary to train a model for the residual separately. However, the training of the residual is difficult, and the insufficient accuracy of the trained model may introduce new errors.

[0015] In some embodiments, all predicted characteristic parameters are added together to obtain a predicted water level time series, specifically:

[0016] By adding the intrinsic mode functions (IMFs) predicted by all models, the predicted water level time series can be obtained.

[0017] In the above technical solution, the residual after CEEMDAN separation is very small, so there is no need to superimpose the residual part when adding the IMFs results.

[0018] In some embodiments, the model hyperparameters are adjusted by the PSO algorithm during the prediction process. Specifically, the feature dimension, sparse attention mechanism factor, and dropout three hyperparameters in the Informer framework are selected as hyperparameters optimized using PSO.

[0019] In the above technical solution, the reason for selecting the above three hyperparameters is that for the Informer framework, these three parameters play an important role in the effect of model training. Feature dimension and sparse attention factor represent the complexity of the training model, which plays a role in balancing model accuracy and training cost, and dropout parameter is an important parameter to prevent overfitting in machine learning. In summary, the selection of these three parameters largely determines the effect of the model, and the selection of the number of optimized parameters also takes into account the time required for optimization using the PSO algorithm. If too many parameters are optimized, the time cost will also need to be greatly increased.

[0020] According to another aspect of the present invention, a training method for a PSO-Informer neural network is provided, wherein the neural network comprises n groups of neural network subunits, each subunit comprising an Informer network and a PSO adjustment unit, and the training method comprises the following steps:

[0021] Get sample water level time series data;

[0022] Decompose the sample water level time series data into n characteristic parameters respectively, and obtain the true labels of the characteristic parameters corresponding to the sample water level time series data;

[0023] For each characteristic parameter of the sample water level time series data, use it as data input; initialize the hyperparameters of the Informer network and the particle position of the PSO adjustment unit in the neural network subunit corresponding to each characteristic parameter; the Informer network is trained according to the position and characteristic parameters of the initialized particles, and the prediction label corresponding to each characteristic parameter is obtained;

[0024] For each group of neural network subunits, a first loss value is calculated based on the predicted label of the neural network subunit and the corresponding true label; based on the first loss value, the PSO adjustment unit adjusts the particle position and travel speed to optimize the hyperparameters of the Informer network, and uses the hyperparameters and the particle position and travel speed as the initial parameters for the next round of iteration;

[0025] Repeat the above steps to train the neural network.

[0026] In the above technical scheme, in order to better use the above method, the present application proposes a training method for a PSO-Informer neural network. The steps of the training method correspond to the various steps of the above method. The specific principles have been described above and will not be repeated here.

[0027] In some embodiments, the hyperparameters include feature dimension, sparse attention mechanism factor, and dropout.

[0028] According to another aspect of the present invention, a estuary water level time series prediction device based on an improved Informer deep learning framework is provided, based on the above method; comprising:

[0029] An acquisition module, used to acquire water level time series data at a location to be predicted;

[0030] A decomposition module is used to decompose the water level time series data at the location to be predicted into several characteristic parameters;

[0031] The prediction module is used to take each feature parameter as data input and output the predicted feature parameter through the pre-trained PSO-Informer neural network model corresponding to each feature parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training;

[0032] The output module is used to add all predicted characteristic parameters to obtain the predicted water level time series.

[0033] In the above technical scheme, in order to better use the above method, the present application proposes an estuary water level time series prediction device based on an improved Informer deep learning framework. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0034] According to another aspect of the present invention, a device for predicting estuary water level time series based on an improved Informer deep learning framework is provided, comprising:

[0035] at least one processor and a memory communicatively coupled to the at least one processor;

[0036] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above method.

[0037] In the above technical solution, in order to better run and process the method, the above method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0038] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program, wherein the computer program implements the above method when executed by a processor.

[0039] In the above technical solution, in order to better run and use the method, the above method is stored in a computer-readable storage medium, and the above method is implemented by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a flow chart of an embodiment of a method for predicting estuary water level time series based on an improved Informer deep learning framework of the present invention;

[0042] Figure 2 It is a flow chart of an embodiment of a training method of a PSO-Informer neural network of the present invention;

[0043] Figure 3 It is a structural schematic diagram of an embodiment of a neural network of the present invention;

[0044] Figure 4 It is a schematic diagram of input and output when using the model for prediction;

[0045] Figure 5 In the embodiment, the water level time series is decomposed into several intrinsic mode functions IMFs using CEEMDAN and then recombined to compare with the original water level time series;

[0046] Figure 6 It shows a comparison chart between all predicted samples and actual values ​​outputted in the embodiment, in which the five trained models are evaluated using the test set;

[0047] Figure 7 In the embodiment, the test set is used to evaluate the five trained models, and the IMFs values ​​of the 24th step in all the output prediction samples are combined into a comparison chart of the time series of the intrinsic mode function IMFs and the actual intrinsic mode function IMFs time series.

[0048] Figure 8 In the embodiment, the five trained models are evaluated using the test set, and the error graph generated by comparing the intrinsic mode function IMFs values ​​of different prediction steps in all prediction samples with the actual values ​​is selected;

[0049] Fig. 9 In the embodiment, the five trained models are evaluated using the test set, and the error statistical histogram of all predicted samples is shown;

[0050] Fig.10 In the embodiment, the five trained models are evaluated using the test set, and all the predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series and the comparison chart of the measured water level time series;

[0051] Fig.11In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the 24th step of all predicted samples is selected to form a comparison chart of the water level time series and the measured water level time prediction series;

[0052] Fig.12 In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the water level time series with different prediction steps in all prediction samples are selected to compare with the actual values ​​to generate an error graph;

[0053] Fig.13 In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the error statistical histogram generated by all predicted samples is obtained.

[0054] Fig.14 It is a structural schematic diagram of an embodiment of an estuary water level time series prediction device based on an improved Informer deep learning framework of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0056] The present invention provides a method for predicting estuary water level time series based on an improved Informer deep learning framework, which can solve at least one technical problem mentioned in the background technology.

[0057] Embodiment 1

[0058] See also Figure 1 , a method for estuary water level time series prediction based on improved Informer deep learning framework,

[0059] S1. Obtaining water level time series data at the location to be predicted;

[0060] S2, decomposing the water level time series data at the location to be predicted into several characteristic parameters;

[0061] In this embodiment, S2, decompose the water level time series data at the position to be predicted into several characteristic parameters, specifically: use the adaptive noise-assisted method of complete set empirical mode decomposition to decompose the water level time series data at the position to be predicted into several intrinsic mode functions. Compared with the traditional segmentation method, decomposing the time series into several modes can reduce the difficulty of training the model. Because the signal complexity of the decomposed intrinsic mode functions is smaller than that of the original signal, it is conducive to the training of the model; and training the model separately according to the intrinsic mode functions of different frequencies can better capture the time series information at different scales. Variants of classical modal decomposition include EEMD, CEEMD, etc. In this case, CEEMDAN is used. Compared with EEMD and CEEMD, the advantage of using CEEMDAN is that it reduces modal mixing and improves the accuracy of decomposition. The advantage of using CEEMDAN in this case is mainly that the residual error between the decomposed modes and the original signal is very small (see Figure 5 ), so the model training can be directly performed on several intrinsic mode functions without considering the influence of residuals. If the residual is large after the mode separation using the EEMD and CEEMD methods, it may be necessary to train a model for the residual separately. However, the training of the residual is difficult, and the insufficient accuracy of the trained model may introduce new errors.

[0062] S3. For each feature parameter, use it as data input, and output the predicted feature parameter through the pre-trained PSO-Informer neural network model corresponding to each feature parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training; the PSO-Informer neural network model includes: n groups of neural network subunits, each subunit includes an Informer network and a PSO adjustment unit; wherein:

[0063] The informer network is configured to take each feature parameter as the input of data and output the predicted feature parameter through each feature parameter;

[0064] A PSO adjustment unit, configured to adjust model hyperparameters by using a PSO algorithm during training;

[0065] In this embodiment, the model hyperparameters are adjusted by the PSO algorithm during the prediction process. Specifically, the feature dimension, sparse attention mechanism factor, and dropout three hyperparameters in the Informer framework are selected as hyperparameters optimized using PSO. It should be noted that the number of model trainings is determined by the number of intrinsic mode functions decomposed by the time series. The number of models trained is determined by the number of intrinsic mode functions decomposed. The reason for selecting the above three hyperparameters is that, for the Informer framework, these three parameters play an important role in the effect of model training. The feature dimension and sparse attention factor represent the complexity of the training model, which play a role in balancing the accuracy of the model and the training cost, and the dropout parameter is an important parameter to prevent overfitting in machine learning. In summary, the selection of these three parameters largely determines the effect of the model, and the selection of the number of optimized parameters also takes into account the time required for optimization using the PSO algorithm. If there are too many optimized parameters, the time cost also needs to be greatly increased.

[0066] S4. Add all predicted characteristic parameters to obtain the predicted water level time series.

[0067] In this embodiment, all predicted characteristic parameters are added to obtain the predicted water level time series. Specifically, the predicted water level time series is obtained by adding the intrinsic mode function IMFs results predicted by all models. It should be noted that the residual after CEEMDAN separation is very small, so there is no need to superimpose the residual part when adding the IMFs results.

[0068] Embodiment 2

[0069] See also Figure 2 A training method for a PSO-Informer neural network, wherein the neural network comprises n groups of neural network subunits, each subunit comprising an Informer network and a PSO adjustment unit, and the training method comprises the following steps:

[0070] A1. Obtain sample water level time series data;

[0071] A2. Decompose the sample water level time series data into n characteristic parameters, and obtain the true labels of the characteristic parameters corresponding to the sample water level time series data;

[0072] A3. For each characteristic parameter of the sample water level time series data, use it as the data input; initialize the hyperparameters of the Informer network and the particle position of the PSO adjustment unit in the neural network subunit corresponding to each characteristic parameter; the Informer network is trained according to the position of the initialized particles and the characteristic parameters, and obtains the prediction label corresponding to each characteristic parameter; the hyperparameters include feature dimension, sparse attention mechanism factor, and dropout.

[0073] A4. For each group of neural network subunits, a first loss value is calculated according to the predicted label of the neural network subunit and the corresponding true label; based on the first loss value, the PSO adjustment unit adjusts the particle position and the travel speed to optimize the hyperparameters of the Informer network, and uses the hyperparameters and the particle position and the travel speed as the initial parameters for the next round of iteration;

[0074] A5. Repeat the above steps to train the neural network.

[0075] Embodiment 3

[0076] See also Figure 3 , a neural network, comprising: n groups of neural network subunits, each subunit comprising an Informer network and a PSO adjustment unit; wherein:

[0077] The informer network is configured to take each feature parameter as the input of data and output the predicted feature parameter through each feature parameter;

[0078] The PSO adjustment unit is configured to adjust the model hyperparameters through the PSO algorithm during training.

[0079] In order to better explain the embodiments of the present invention, the content schemes disclosed in Embodiments 1 to 3 will be specifically described one by one. Figure 3 , as follows:

[0080] Step S1: Obtain the water level time series data at the location to be predicted, which needs to include time information and water level information;

[0081] Step S2: Use CEEMDAN (Complete Ensemble Empirical Mode Decomposition with Adaptive Noise) to decompose the water level time series into several intrinsic mode functions (IMFs) of different complexity and simpler than the original time series. The decomposition is performed using the CEEMDAN class in the open source Python library PyEMD.

[0082] The specific process of decomposition is as follows:

[0083] 1) Original water level time series X Add different white noise to form multiple noisy signals .

[0084] 2) Find each noisy signal The local maximum and minimum of are obtained and the upper and lower envelopes are constructed using cubic spline interpolation.

[0085] 3) Calculate each noisy signal The average value of the upper and lower envelopes .

[0086] 4) Calculate each noisy signal The average value of the corresponding envelope The difference, that is .

[0087] 5) Judgment Whether the difference between the number of zero crossing points and the number of extreme value points within the data range does not exceed 1, and the local average value of the envelope at any time is 0. If the conditions are met, then The noisy signal The first eigenmode function of If not satisfied, Repeat steps 2) to 4) until the requirement of 5) is met.

[0088] 6) Each noisy signal of After averaging, we get the first eigenmode function , and X minus Get the residual , and then the residual Add different white noise to form multiple noisy residual signals , and then repeat steps 2) to 6) for multiple noisy residual signals until there are no more Can be extracted.

[0089] Step S3: Decompose the several Input them into the PSO-Informer framework to train different models for each IMFs, and The sequence is divided into training set, validation set and test set, where the PSO-INFORMER framework is as follows:

[0090] Particle Swarm Optimization (PSO) is an optimization algorithm based on swarm intelligence. It finds the optimal solution to a problem by simulating the movement and mutual influence of multiple particles (representing candidate solutions) in the search space. PSO has attracted widespread attention for its simplicity, ease of implementation, and strong global search capabilities. Its basic principles are as follows:

[0091] 1) Initialize the number of search particles, number of iterations, and inertia weight w And the acceleration coefficient c 1 and c 2.

[0092] 2) Initialize the position of each particle and calculate the fitness value. The particle position represents a point in the parameter space formed by the hyperparameters to be optimized.

[0093] 3) Initialize the optimal position of the individual p best,i And the global optimal position g best , the individual optimal position is the optimal position searched by a single particle, and the global optimal position is the optimal position searched by all particles.

[0094] 4) According to the individual optimal position p best,i And the global optimal position g best Update each particle's velocity and position as follows:

[0095]

[0096] in, is the velocity of a particle at the next moment, is the velocity of a particle at the current moment, is the position of a particle at the next moment, is the position of a particle at the current moment, is the inertia weight, and is the acceleration coefficient, and It is a random number uniformly distributed between [0, 1], which increases the randomness of the algorithm and the search diversity.

[0097] 5) Calculate the fitness of the updated particle position and update the individual optimal position p best,i and the global optimal position g best .

[0098] 6) Repeat steps 4) to 5) until the iteration ends and the global optimal position is g best This is the optimal hyperparameter combination.

[0099] Informer is a deep learning model for time series prediction, which is specially designed to complete long time series prediction tasks. It is based on the Transformer architecture, but proposes an algorithm called ProbSparse Self-Attention to reduce the amount of calculation and memory consumption. It solves the problem of Transformer's computational efficiency and memory consumption when processing long time series data.

[0100] The Informer architecture consists of an embedding layer, an encoder, a decoder, and a fully connected layer. The encoder consists of multiple layers of one-dimensional convolutional layers, a maximum pooling layer, and a multi-head probabilistic sparse attention layer. The decoder consists of a masked multi-head probabilistic sparse attention mechanism and a cross attention mechanism group. The operating mechanism of Informer is as follows:

[0101] 1) The input time series is processed by the embedding layer, which divides the time series into the time series for prediction and the target time series, and then expands them into high-dimensional vectors. The time series for prediction is processed into a three-dimensional vector with the shape of (batch size, sequence length, model dimension). The batch size represents the number of samples used for training at one time, the sequence length represents the sequence length required to predict the target time series, and the model dimension represents the complexity of the model. The larger the dimension, the more complex the trained model. The target time series is processed into a three-dimensional vector with the shape of (batch size, target sequence length, model dimension), which is used to calculate the prediction error with the predicted value.

[0102] 2) The three-dimensional vector used for prediction is input into the encoding layer, and is processed by the multi-layer one-dimensional convolution layer, the maximum pooling layer, and the multi-head probabilistic sparse attention mechanism inside the encoding layer before being output. In the multi-head probabilistic sparse attention mechanism, the input three-dimensional sequence generates several query vectors (Query), key vectors (Key), and value vectors (Value), and the attention weight is calculated through the query vector, key vector, and value vector; the method is to randomly extract several key vectors and all query vectors, and use formula (2) to screen out several query vectors that meet the standards, and use formula (3) to calculate the attention weight of the screened query vectors and all key vectors.

[0103]

[0104] in, In informer neural networks, this term is the sparsity score. Represents the key vector and the query vector. In the standard self-attention mechanism, the query vector (query) is dot-producted with all key vectors (key) to obtain the attention score A(Q,K,V). However, this global calculation will increase quadratically with the increase of sequence length. Therefore, in order to optimize this process, Informer uses a sparsity scoring mechanism to select only the key vectors that are most relevant to the query vector, thereby reducing the amount of calculation. In informer, this stands for attention weight, which is used to measure the similarity or correlation between elements in a sequence. It determines which other elements' information the current element should pay attention to, and assigns different weights to these elements to comprehensively consider their influence. This process helps capture local and global dependencies in a sequence. For example, in long time series data, attention weights help the model identify which time steps are more important for the prediction of the next time step. For example, in weather forecasting, the weather conditions at certain specific moments may have an important impact on future weather, and attention weights can help the model capture this relationship.

[0105] and Represents the query vector and key vector, and Represents any two elements in the query vector Q and the key vector K; the query vector and the key vector are two vectors used to calculate the relationship between elements in the input sequence. They are high-dimensional vectors generated by the informer from the input sequence, and the correlation between different elements in the sequence is calculated by multiplying the two vectors. Since multiplying the key vector and the query vector takes a lot of computing power for long time series, the informer uses formula 3 to filter and query vectors. Most relevant key vector , thereby reducing the calculation of attention weight Calculation amount. is the model dimension of the input sequence, is the key sequence length, and formula (2) measures the query vector by calculating the difference between the maximum attention score and the average attention score. With key vector The concentration of attention distribution between them, and each query vector The calculated values ​​are arranged from large to small, and the query vectors corresponding to the first several large values ​​are selected. The selected query vectors are used to calculate the attention weight using formula (3), where Q, K, and V are the query vector, key vector, and value vector matrices, and the Softmax function is as shown in formula (4).

[0106]

[0107] in, Represents the calculated query vector With key vector The attention score of , which normalizes the attention score.

[0108] In the one-dimensional convolution layer and the maximum pooling layer, the size of the attention weight vector is reduced, which improves the computational efficiency.

[0109] 3) The three-dimensional vector as the prediction target and the result output from the encoder are input to the decoder, and then output after the masked multi-head probabilistic sparse attention mechanism and cross-attention mechanism in the decoder. The three-dimensional vector as the prediction target is processed by the masked self-attention mechanism to ensure that the decoding process can only use the data of the current and previous time steps to prevent future information leakage. The result of the three-dimensional vector as the prediction target processed by the masked self-attention mechanism and the high-dimensional features output from the encoder are cross-attention calculated in the cross-attention layer, thereby obtaining relevant information from the encoder output.

[0110] 4) The decoder output is input to the fully connected layer, which generates a prediction value and calculates the error with the target time series. The gradient is then calculated through back propagation to update the parameters.

[0111] 5) Repeat steps 1) to 4) until the iteration is complete and the model training is completed.

[0112] Step 4: Initialize the Informer and PSO algorithms, specify the initial hyperparameters of the Informer, and specify the number of search particles, number of iterations, and inertia weight of the PSO. w And the acceleration coefficient c 1 and c 2 and initialize the particle position. The feature dimension, sparse attention mechanism factor, and dropout in the Informer framework are selected as the hyperparameters optimized by PSO.

[0113] Step 5: Use Informer to train several models based on the training set according to the position of each particle, and then use the validation set to verify the error between the model prediction result and the actual result, and pass the error to the PSO framework to adjust the particle position and travel speed according to the error to optimize the hyperparameters.

[0114] Step 6: PSO inputs the adjusted hyperparameters into the Informer framework and uses the training set to train several models again. After that, the trained model is used to calculate the error using the validation set and then passed into PSO to adjust the particle position and speed.

[0115] Step 7: Repeat steps 5 and 6 until the iteration is completed to obtain the final model. Use the test set to test the model results and output the test error.

[0116] Step 8: Add the intrinsic mode functions (IMFs) predicted by all models to obtain the predicted water level time series.

[0117] The water level time series data set predicted in the embodiment is derived from the measured data of the Bayuquan water level observation station from April 2021 to August 2023, with a time resolution of 10 minutes. After dividing the water level time series into 5 intrinsic mode functions using CEEMDAN, 70% of the data are divided as training sets for training, 10% of the data set is used as a validation set for PSO to optimize hyperparameters, and 20% of the data set is used as a test set to evaluate the model effect, and finally 5 models are obtained. The model predicts 24 consecutive intrinsic mode function values ​​after inputting 96 consecutive actual intrinsic mode function values. The 24 consecutively predicted intrinsic mode function values ​​are 1 prediction sample. After that, 1 actual intrinsic mode function value is added to continue predicting the next 24 consecutive intrinsic mode function values. After the prediction is completed, all predicted intrinsic mode function values ​​are added in place to obtain the predicted water level time series. The specific prediction method can be seen. Figure 2 .

[0118] Figure 4 This is a schematic diagram of input and output when using the model for prediction; this diagram is an explanation of the data input and output when testing the model in the embodiment, that is, after inputting the intrinsic mode function values ​​of several time series to predict a sequence, the predicted time series results are added with a true value to predict the next time series. Figure 4 The process of predicting input and output by the trained model is shown in the figure. Now let's explain the meaning of N, i, k: N represents the length of the measured sequence input during prediction; K represents the length of the predicted sequence output during prediction; i represents the number of the sequence element, such as 1, 2, 3, ... N. For example, the trained model needs to input the water level values ​​of the past 96 time steps to predict the water level values ​​of the next 24 steps, then N=96, k=24; during the input process, I input the values ​​of the measured time series 1, 2, 3...95, 96, and output the values ​​of 97, 98...120 time steps; then when continuing to predict, I input the measured data numbered 2, 3, 4...96, 97, and predict the time series values ​​of 98, 99, 100...121 steps.

[0119] Figure 5 In the embodiment, the water level time series is decomposed into several intrinsic mode functions IMFs by using CEEMDAN and then recombined, and the comparison chart is with the original water level time series; the blue solid line which is decomposed into several intrinsic mode functions IMFs by CEEMDAN and then recombined and the orange solid line of the actual value is blocked because the error value is very small.

[0120] Figure 6 In the embodiment, the five trained models are evaluated using the test set, and the comparison chart of all the predicted samples and the actual values ​​is output; RMSE stands for root mean square error. Figure 6 In , the predicted IMFs are compared with the actual IMFs and the RMS error is calculated.

[0121] Figure 7 In the embodiment, the test set is used to evaluate the five trained models, and the IMFs values ​​of the 24th step in all the output prediction samples are combined into a comparison chart of the time series of the intrinsic mode function IMFs and the actual intrinsic mode function IMFs time series. Figure 7 The WLV in the formula stands for water level. Figure 7 It means taking the 24th step value from each prediction sample to form the predicted water level time series, comparing it with the measured water level series and calculating the root mean square error RMSE.

[0122] Figure 8 In the embodiment, the five trained models are evaluated using the test set, and the error graph generated by comparing the intrinsic mode function IMFs values ​​of different prediction steps in all prediction samples with the actual values ​​is selected;

[0123] Fig. 9 In the embodiment, the five trained models are evaluated using the test set, and the error statistical histogram of all predicted samples is shown;

[0124] Fig.10 In the embodiment, the five trained models are evaluated using the test set, and all the predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series and the comparison chart of the measured water level time series;

[0125] Fig.11 In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the 24th step of all predicted samples is selected to form a comparison chart of the water level time series and the measured water level time prediction series;

[0126] Fig.12 In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the water level time series with different prediction steps in all prediction samples are selected to compare with the actual values ​​to generate an error graph;

[0127] Fig.13 In the embodiment, the five trained models are evaluated using the test set, and all predicted intrinsic mode function IMFs values ​​are added in place to obtain the predicted water level time series, and the error statistical histogram generated by all predicted samples is obtained.

[0128] Figures 4 to 13From the error of decomposing the original time series data by CEEMDAN, to the error of model testing prediction obtained by training the decomposed IMFs using the test set. It shows that whether it is the decomposition of the original time series by CEEMDAN, the prediction of each IMFs using the trained model, or the addition of all predicted IMFs to get the predicted water level time series, the error is small under certain conditions by comparing with the test set. Figure 5 , the RMSE of the original water level time series after decomposition using CEEMDAN and then added to the original water level time series is only 4.08e-9. Figure 6 For the comparison between the predicted values ​​and the actual values ​​of the models trained with different IMFs when using the test set for error evaluation, the maximum RMSE error does not exceed 0.085m. Figure 7 , Figure 8 They are the errors of several models trained for IMFs under prediction of different step lengths, indicating that after continuous prediction of 24 time steps, the maximum RMSE does not exceed 0.15m. Fig. 9 It is a classification comparison of the RMSE calculated using several model prediction values ​​and measured values. Basically, 90% of the RMSE obtained by all model predictions is less than 0.1m. Figure 10 to Figure 13 The error analysis between the predicted water level value obtained by adding all the predicted IMFs and the measured value is also satisfactory under certain conditions.

[0129] Embodiment 4

[0130] See also Fig.14 , a estuary water level time series prediction device based on an improved Informer deep learning framework, based on the above method; comprising:

[0131] An acquisition module, used to acquire water level time series data at a location to be predicted;

[0132] A decomposition module is used to decompose the water level time series data at the location to be predicted into several characteristic parameters;

[0133] The prediction module is used to take each feature parameter as data input and output the predicted feature parameter through the pre-trained PSO-Informer neural network model corresponding to each feature parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training;

[0134] The output module is used to add all predicted characteristic parameters to obtain the predicted water level time series.

[0135] In the above technical scheme, in order to better use the method described in one of the above embodiments, the present application proposes an estuary water level time series prediction device based on an improved Informer deep learning framework. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0136] Embodiment 5

[0137] An estuary water level time series prediction device based on an improved Informer deep learning framework, comprising:

[0138] at least one processor and a memory communicatively coupled to the at least one processor;

[0139] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in one or two embodiments.

[0140] In the above technical solution, in order to better run and process the method described in Embodiment 1 or Embodiment 2, the above method is stored in a memory, and a processor is used to execute the stored method. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0141] Embodiment 6

[0142] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method described in one or two embodiments is implemented.

[0143] In the above technical solution, in order to better run and use the method described in Embodiment 1 or Embodiment 2, the above method is stored in a computer-readable storage medium, and a processor is used to implement the method described in Embodiment 1 or Embodiment 2. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0144] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for predicting estuary water level time series based on an improved Informer deep learning framework, characterized in that: The method comprises the following steps: Obtain the water level time series data at the location to be predicted; Decompose the water level time series data at the location to be predicted into n characteristic parameters; For each characteristic parameter, take it as data input, and output n predicted characteristic parameters through the pre-trained PSO-Informer neural network model corresponding to each characteristic parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training; the PSO-Informer neural network model includes: n groups of neural network sub-units, each group of neural network sub-units processes a feature parameter, and each sub-unit contains an Informer network and a PSO adjustment unit; where: The informer network is configured to take each feature parameter as the input of data and output the predicted feature parameter through each feature parameter; A PSO adjustment unit, configured to adjust model hyperparameters by using a PSO algorithm during training; Add the predicted n characteristic parameters to obtain the predicted water level time series; The water level time series data at the location to be predicted is decomposed into a number of characteristic parameters. Specifically, an adaptive noise-assisted method of complete set empirical mode decomposition is used to decompose the water level time series data at the location to be predicted into n intrinsic mode functions.

2. The method for predicting estuary water level time series based on the improved Informer deep learning framework according to claim 1, characterized in that: Add up all predicted characteristic parameters to obtain the predicted water level time series, specifically: By adding the intrinsic mode functions (IMFs) predicted by all models, the predicted water level time series can be obtained.

3. The method for predicting estuary water level time series based on the improved Informer deep learning framework according to claim 1, characterized in that: During the prediction process, the PSO algorithm is used to adjust the model hyperparameters. Specifically, the feature dimension, sparse attention mechanism factor, and dropout hyperparameters in the Informer framework are selected as hyperparameters optimized using PSO.

4. A PSO-Informer neural network training method, characterized in that: Based on the method described in any one of claims 1 to 3, the neural network includes n groups of neural network subunits, each subunit includes an Informer network and a PSO adjustment unit, and the training method includes the following steps: Get sample water level time series data; Decompose the sample water level time series data into n characteristic parameters respectively, and obtain the true labels of the characteristic parameters corresponding to the sample water level time series data; For each characteristic parameter of the sample water level time series data, use it as data input; initialize the hyperparameters of the Informer network and the particle position of the PSO adjustment unit in the neural network subunit corresponding to each characteristic parameter; the Informer network is trained according to the position and characteristic parameters of the initialized particles, and the prediction label corresponding to each characteristic parameter is obtained; For each group of neural network subunits, a first loss value is calculated based on the predicted label of the neural network subunit and the corresponding true label; based on the first loss value, the PSO adjustment unit adjusts the particle position and travel speed to optimize the hyperparameters of the Informer network, and uses the hyperparameters and the particle position and travel speed as the initial parameters for the next round of iteration; Repeat the above steps to train the neural network.

5. A PSO-Informer neural network training method as claimed in claim 4, characterized in that: The hyper parameters include feature dimension, sparse attention mechanism factor, and dropout.

6. A estuary water level time series prediction device based on an improved Informer deep learning framework, characterized in that: A method according to any one of claims 1 to 5; comprising: An acquisition module, used to acquire water level time series data at a location to be predicted; A decomposition module is used to decompose the water level time series data at the location to be predicted into several characteristic parameters; The prediction module is used to take each feature parameter as data input and output the predicted feature parameter through the pre-trained PSO-Informer neural network model corresponding to each feature parameter; the PSO-Informer neural network model adjusts the model hyperparameters through the PSO algorithm during training; The output module is used to add all predicted characteristic parameters to obtain the predicted water level time series.

7. A river estuary water level time series prediction device based on an improved Informer deep learning framework, characterized in that: include: at least one processor and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images