Sea fog visibility prediction method based on SLSTM-SVM model

Abstract

The invention discloses a sea fog visibility prediction method based on an SLSTM-SVM model, and aims to overcome the defects of an existing sea fog visibility prediction method, the sea fog visibility prediction method comprises the following steps: firstly carrying out smooth stabilization processing on an LSTM network to obtain an SLSTM, and then replacing a softmax layer in the SLSTM with an SVM classifier to construct a model. Four types of hydrological data such as rainfall and water level and five types of meteorological data such as air temperature and air pressure form a sequence input model, an SVM classifier predicts and outputs a value through SLSTM layer smoothing and stabilization processing and data deep mining, parameters are updated through back propagation, and iterative training is carried out until errors reach the standard. Meanwhile, an autocorrelation function is adopted to evaluate data stability, and a Z-score method is utilized to normalize the data. The method can quickly converge the network weight to a globally optimal solution, is high in prediction accuracy, and has great application potential in long-term sea fog visibility prediction.

Description

Technical Field

[0001] The invention relates to the technical field of sea fog visibility prediction, and more specifically to a sea fog visibility prediction method based on an SLSTM-SVM model. Background Art

[0002] Sea fog visibility is essentially time series data, so its prediction problem can be transformed into a time series prediction modeling problem. Existing research mainly focuses on two categories: parametric model methods and non-parametric model methods. The former method mainly models and predicts the time series of sea fog visibility data based on mathematical theoretical knowledge such as statistics and probability distribution. This method models the time series of sea fog visibility data through limited parameters and does not depend on the size of the data set.

[0003] In recent years, with the rapid development of big data acquisition technology and artificial intelligence technology, deep learning has gradually become a popular direction for non-parametric prediction models and has been favored by more researchers. At present, most sea fog visibility predictions still use shallow learning methods. For example, the support vector regression (SVR) model can better solve the learning problem of small sample sea fog visibility data, but cannot rely on comparative experiments or exhaustive search to obtain model parameters for large sample sea fog visibility data, which seriously affects the learning ability and generalization ability. On the other hand, shallow learning methods are easy to capture the temporal correlation of sea fog visibility, but it is not easy to capture the spatial characteristics of sea fog visibility. The multi-layer architecture makes sea fog visibility dependent and heterogeneous in the temporal and spatial dimensions. In order to improve the performance of sea fog visibility prediction, it is necessary to capture the characteristics of the temporal and spatial dimensions at the same time. Therefore, how to use the powerful learning ability of deep learning to accurately predict large-scale and highly complex sea fog visibility data has become one of the urgent problems to be solved in the marine field. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a sea fog visibility prediction method based on the SLSTM-SVM model, which can quickly converge the network weights to a global optimal solution, can be effectively applied to sea fog visibility prediction, and further improve the accuracy of sea fog visibility prediction results.

[0005] To achieve the above object, the present invention provides the following technical solution: a sea fog visibility prediction method based on the SLSTM-SVM model, comprising the following steps:

[0006] Step 1: Smooth and stabilize the LSTM network to obtain a neural network model SLSTM based on the stable LSTM;

[0007] Step 2: Modify the softmax layer in SLSTM to SVM classifier to obtain a neural network model based on SLSTM-SVM;

[0008] Step 3: In the SLSTM-SVM model, the four types of hydrological data under sea fog: precipitation, water level, water temperature, flow velocity, and the five types of meteorological data: temperature, air pressure, humidity, light, wind speed, a total of 9 types of data, are combined to form the sea fog visibility sequence {X = x1, x2, ..., x n};

[0009] Step 4: Use the sequence data as the input of the SLSTM model. Smooth and stabilize the input data by stacking m layers of SLSTM models, and perform deep mining of the data through the native LSTM part of the SLSTM model to determine the information that needs to be retained by subsequent units.

[0010] Step 5: Modify the softmax layer of the SLSTM network, use the SVM classifier in the softmax layer to make effective predictions and directly output the predicted values, evaluate the model error prediction values, and use back propagation to reversely update the weight parameters and bias parameters of the SLSTM network;

[0011] Step 6: Iterate the process of step 5 above to repeatedly train the SLSTM-SVM model until the overall network error is reduced to a certain range, output the sequence and complete the prediction.

[0012] In summary, the present invention has the following beneficial effects:

[0013] 1. The present invention can accurately calculate the visibility value of the sea fog area through data processing and analysis models. The LSTM network is smoothly optimized and fused with the SVM classifier to obtain an SLSTM-SVM neural network model. The input data is mapped to a high-dimensional space using a nonlinear function to obtain a global optimal solution. The value predicted by this method is basically consistent with the actual sea fog visibility, and has good nonlinear learning ability for the sea fog visibility time series data, and has great application potential in long-term sea fog visibility prediction.

[0014] 2. The present invention proposes a sea fog visibility prediction method based on the SLSTM-SVM model, and uses the data autocorrelation analysis method to preprocess the data sequence, further improving the reliability of the model. The model can effectively predict sea fog visibility. Compared with several existing sea fog visibility prediction methods, this method has a higher degree of fitting, a smaller error, and a more accurate prediction, which has certain advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is the basic structure of LSTM model.

[0016] Figure 2 This is the SLSTM algorithm implementation process.

[0017] Figure 3 It is the structure diagram of the SVM model.

[0018] Figure 4 SLSTM-SVM model framework. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0020] According to the characteristics of the sea fog visibility data time series, this embodiment performs a stable optimization process on the basic LSTM model to obtain the SLSTM model. In order to more efficiently obtain the global optimal solution and improve its running speed and classification ability, the softmax layer in the SLSTM model is modified to an SVM classifier for fitting calculation to obtain the SLSTM-SVM model.

[0021] Among them, the LSTM network is smoothed and stabilized to obtain a neural network model SLSTM based on stable LSTM.

[0022] LSTM neural network model:

[0023] The LSTM neural network model is an improvement on the recurrent neural network (RNN) algorithm. A cell is added to the original algorithm to increase the long-term memory function so that the information no longer decays, thereby overcoming the problem of gradient disappearance in the RNN network. In the LSTM network, the added unit is usually composed of three threshold structures: forget gate, input gate and output gate, and a state vector transmission line. Among them, the state vector transmission line is responsible for long-term memory, and the three thresholds are responsible for the selection of short-term memory. The forget gate determines how to retain the historical information of the memory module at the current moment. The input gate determines the transmission of input layer information to the hidden layer memory module. The output gate determines the storage and output of module information. When information enters the LSTM network, it is judged whether the information is useful according to the set rules, and the information judged to be useful is retained, and the rest of the information is forgotten through the forget gate. The basic structure of the LSTM model is as follows Figure 1 shown.

[0024] The forward propagation process in the LSTM neural network can be described by the following three steps.

[0025] 1. Update the forget gate. The information allowed to pass through the cell is determined by the S-type neural network layer. t-1 and x t Input into the Sigmoid function and output a vector f with a value of 0 to 1 t , represents the retention ratio of each part of information transmission, where 1 means all information is transmitted and 0 means all information is discarded. Output vector f t With the input value h t-1and x t The functional relationship between them is:

[0026] f t =σ(W f [h t-1 ,x t ]+b f ) (Formula 1)

[0027] In formula 1, W f and b f They represent the weight parameter and bias parameter of the forget gate respectively, and σ represents the Sigmoid function.

[0028] 2. Update the cell state value. This process determines what information needs to be updated and updates the initial state value with the value generated by the tanh layer. The intermediate vector i t and With the input vector h t-1 and x t Related, described as follows:

[0029] i t =σ(W i [h t-1 ,x t ]+b i ) (Formula 2)

[0030]

[0031] Next, state C t-1 Update to status C t , that is, multiply by the information transfer vector f t The retention ratio is calculated, and then information is discarded according to the ratio. t The update equation is:

[0032]

[0033] 3. Determine the output value. First, you need to determine the information part to be output through the Sigmoid layer. Then process the state vector after the tanh layer and multiply it with the output weight of the Sigmoid layer to get the final result. The intermediate vector O t , input vector h t-1 , output value h t The relationship is as follows:

[0034] O t =σ(W o [h t-1 ,x t ]+b O ) (Formula 5)

[0035] ht =O t ×tanhC t (Formula 6)

[0036] Where W o and b O are the weight parameters and bias parameters of the output gate respectively.

[0037] The LSTM network also includes a back-propagation process, starting from the current time t, each time the error term is calculated and propagated to the upper layer of the network. The gradient of each weight is calculated according to the corresponding error term generated, and the parameters are iteratively updated using the gradient descent method.

[0038] SLSTM neural network model based on stationary LSTM:

[0039] Since sea fog visibility has the characteristic of dynamic change over time, LSTM is suitable for the analysis and prediction of sea fog visibility data. The sea fog visibility data time series is a random time series, which usually exhibits non-stationary characteristics. Therefore, it is necessary to evaluate the stationarity of the original sea fog visibility observation sequence. Here, the autocorrelation function (ACF) is used to obtain the estimated value, see formula 7.

[0040]

[0041] By drawing the ACF graph of the estimated value, the state of the time series is judged to determine the period. When the autocorrelation function value exceeds the 95% confidence interval and there is an obvious tailing effect, the sequence can be judged as a non-stationary sequence, otherwise the sequence is a stable sequence.

[0042] When the sea fog visibility sequence is a non-stationary sequence, its period difference can be calculated based on the original sea fog visibility sequence. Get the stationary difference series The present invention uses a stationary difference sequence as the output data of the sample set to reconstruct the sample set and proposes a neural network model (SLSTM) based on stationary LSTM. SLSTM is used to predict the results and restore the results to the original sea fog visibility. The SLSTM algorithm implementation process is as follows: Figure 2 As shown. It can be seen that the algorithm is divided into two processes: forward propagation and backward propagation. Forward propagation mainly calculates the training results of the input training samples, and backward propagation is to reversely update the network weight parameters and bias parameters. The SLSTM algorithm implemented by the present invention adds a data smoothing and stabilization process before the forward propagation process, and adds a data sequence recovery process after outputting the prediction results.

[0043] Modify the softmax layer in SLSTM to SVM classifier to obtain the neural network model based on SLSTM-SVM:

[0044] SVM Classifier:

[0045] Considering that the softmax layer of the SLSTM network is prone to numerical overflow during numerical calculation due to the large output value of the output node, numerical overflow may also occur when calculating the loss function, and the model training time is long and the efficiency is low. In order to ensure the stability of numerical calculation, the SVM algorithm is used for fitting calculation. By modifying the softmax layer of the SLSTM network, the SVM classifier is used in the softmax layer to make predictions and directly output the predicted values, the accuracy of the prediction results is further improved.

[0046] SVM is a computing module composed of kernel functions, which implicitly maps data to high-dimensional space through kernel functions to increase the computing power of the model. The main features of SVM are as follows:

[0047] 1. SVM can eventually be transformed into a quadratic programming problem to find the global optimal solution and avoid the problem of local optimality.

[0048] 2. The SVM topology is determined by the support vector, which effectively solves the problem that traditional neural networks need to repeatedly try out the network structure.

[0049] 3. SVM uses nonlinear transformation to map input data to a high-dimensional feature space and constructs a linear regression function in the high-dimensional feature space, so that the model has better generalization ability and avoids the problem of dimensionality disaster.

[0050] First, input sample x i Using the nonlinear function φ(x i ), mapping from low-dimensional space X to high-dimensional feature space H. This method can transform low-dimensional nonlinear problems into high-dimensional linear regression problems, which can be implemented using the SVM kernel function:

[0051] K(x i ,y j )=φ(x i )·φ(y j ) (Formula 8)

[0052] Assume that its sample data can be expressed as:

[0053] T={(x1,y1),(x2,y2),(x3,y3),...(x n ,y n ))}∈(X,Y) n (Formula 9)

[0054] The linear regression model can be obtained:

[0055]

[0056] According to the principle of structural risk minimization, the variable parameter ω,b in formula (10) is processed as follows:

[0057]

[0058] Among them, R[f] is the empirical risk, C(e i ) is the loss function, λω 2 For confidence risk.

[0059] In order to deal with the risk that cannot be assessed accurately, a slack variable ξ is introduced i , are all greater than 0, i∈[1,n]:

[0060]

[0061] In formula (12), ξ i , represents the relaxation factor, in the objective function Can improve generalization ability, It can reduce the empirical risk of the objective function. Introduce Lagrangian function to solve the quadratic programming problem:

[0062]

[0063] in, C is the penalty factor and ε is an insensitive parameter.

[0064] The Lagrangian function extreme value must meet the following conditions:

[0065]

[0066] The Lagrange dual problem can be obtained as follows:

[0067] Constraints: Solving formula (20) yields Substitute it into formula (13) and combine it with formula (10) to get the nonlinear regression function:

[0069]

[0070] Where K(x i ,x j ) is the kernel function. Since the output has only one parameter, visibility, j = 1, that is:

[0071]

[0072] At this point, SVM transforms the non-linear problem into a high-dimensional space through the inner product function transformation, and performs linear regression in the high-dimensional space. The output of SVM is a linear function combination of the intermediate layer nodes. Each intermediate node has a support vector corresponding to it, and the corresponding Lagrange multiplier is the weight. The SVM model structure is as follows: Figure 3 shown.

[0073] A neural network model based on SLSTM-SVM:

[0074] The study of the influence of various parameters on sea fog visibility found that only some hydrological and meteorological elements have a greater impact on the visibility value. For local areas of the input space, only a few connection weights affect the output, which is more similar to a local approximation network. Therefore, under the premise of smooth processing of the LSTM neural network model, a new method combining the SVM Gaussian (RBF) kernel function with the SLSTM neural network is used to more efficiently obtain the global optimal solution by mapping the input data to a high-dimensional space using a nonlinear function. The training process still uses k-fold cross validation, with a k value of 10, and the Gaussian radial basis kernel function used is:

[0075] The final function model is:

[0076]

[0077] In the SLSTM-SVM model, 4 types of hydrological data (precipitation, water level, water temperature, flow velocity) and 5 types of meteorological data (temperature, air pressure, humidity, light, wind speed) under sea fog are firstly used to form a total of 9 types of data to form the sea fog visibility input sequence {X = x1, x2, ..., x n}. The data is then used as the input of the SLSTM model. By stacking m layers of SLSTM models, the input data is smoothed and stabilized, and the native LSTM part of the SLSTM model is used to deeply mine the data to determine the information that needs to be retained for subsequent units. SLSTM contains 1 smoothing layer, 1 tanh layer, and 3 Sigmoid functions. By connecting the input of the previous layer of data and the output of the current data, the interaction between the SLSTM layers of each layer is achieved. Finally, the softmax layer of the SLSTM network is modified, and the SVM classifier is used in the softmax layer to make effective predictions and directly output the predicted values. By evaluating the model error prediction value, the back propagation is used to reversely update the weight parameters and bias parameters of the SLSTM network. Iterate the above process to repeatedly train the SLSTM-SVM model until the overall error of the network is reduced to a certain range, output the sequence and complete the prediction. The SLSTM-SVM model framework is as follows Figure 4 shown.

[0078] By calibrating and analyzing the data, the curve of the comprehensive correlation error and the cumulative distribution function shows that the above 9 types of data (4 types of hydrological data and 5 types of meteorological data) have a multidimensional impact on sea fog visibility, and the model can be trained and predicted. The prediction process first standardizes the data, and uses the Z-score standard data normalization method to obtain the mapping result value of each value. The data normalized by Z-score conforms to the standard normal distribution, which can eliminate the dimensional influence between the 9 types of data, remove the data with excessive correlation errors in the existing data, and achieve data cleaning.

[0079] The present invention can accurately calculate the visibility value of the sea fog area through data processing and analysis model. The LSTM network is smoothly optimized and fused with the SVM classifier to obtain an SLSTM-SVM neural network model. The input data is mapped to a high-dimensional space using a nonlinear function to obtain a global optimal solution. The value predicted by this method is basically consistent with the actual sea fog visibility, and has good nonlinear learning ability for the sea fog visibility time series data, and has great application potential in long-term sea fog visibility prediction.

[0080] The present invention uses the method of data autocorrelation analysis to preprocess the data sequence, further improving the reliability of the model. The model can effectively predict sea fog visibility. Compared with several existing sea fog visibility prediction methods, this method has a higher degree of fitting, a smaller error, and a more accurate prediction, which has certain advantages.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the design concept of the present invention should be included in the protection scope of the present invention.

Claims

1. A sea fog visibility prediction method based on SLSTM-SVM model, characterized by: The steps include: Step 1: Smooth and stabilize the LSTM network to obtain a neural network model SLSTM based on the stable LSTM; Step 2: Modify the softmax layer in SLSTM to SVM classifier to obtain a neural network model based on SLSTM-SVM; Step 3: In the SLSTM-SVM model, the four types of hydrological data under sea fog: precipitation, water level, water temperature, flow velocity, and the five types of meteorological data: temperature, air pressure, humidity, light, wind speed, a total of 9 types of data, are combined to form the sea fog visibility sequence {X = x1, x2, ..., x n }; Step 4: Use the sequence data as the input of the SLSTM model, smooth and stabilize the input data by stacking m layers of SLSTM models, and perform deep mining of the data through the native LSTM part of the SLSTM model to determine the information that needs to be retained by subsequent units; Step 5: Modify the softmax layer of the SLSTM network, use the SVM classifier in the softmax layer to make effective predictions and directly output the predicted values, evaluate the model error prediction values, and use back propagation to reversely update the weight parameters and bias parameters of the SLSTM network; Step 6: Iterate the process of step 5 above to repeatedly train the SLSTM-SVM model until the overall network error is reduced to a certain range, output the sequence and complete the prediction.

2. The sea fog visibility prediction method based on the SLSTM-SVM model according to claim 1, characterized in that: The sea fog visibility series is a random time series, and the autocorrelation function (ACF) is used to obtain the estimated value for its stability. The formula is as follows: And draw the ACF diagram of the estimated value to judge the state of the time series to determine the period; when the autocorrelation function value exceeds the 95% confidence interval and there is an obvious tailing effect, it can be judged that the series is a non-stationary series, otherwise the series is a stable series.

3. The sea fog visibility prediction method based on the SLSTM-SVM model according to claim 2 is characterized in that: When the sea fog visibility sequence is a non-stationary sequence, the period difference ▽ is calculated based on the original sea fog visibility sequence. T X t , and obtain the stationary difference sequence {▽TXt,t=1,2,3,…}.

4. The sea fog visibility prediction method based on the SLSTM-SVM model according to claim 1, characterized in that: The SVM Gaussian (RBF) kernel function is combined with the SLSTM neural network to map the input data to a high-dimensional space using a nonlinear function to more efficiently obtain the global optimal solution. The training process uses k-fold cross validation, with the k value of 10, and the Gaussian radial basis kernel function used is: The final function model is:

5. The sea fog visibility prediction method based on the SLSTM-SVM model according to claim 1, characterized in that: The SLSTM includes 1 smoothing layer, 1 tanh layer and 3 Sigmoid functions, and the interaction between the SLSTM layers of each layer is realized by connecting the input of the previous layer data and the output of the current data.

6. The method for predicting sea fog visibility based on the SLSTM-SVM model according to claim 1, characterized in that: The influence relationship between the 9 types of data composed of the 4 types of hydrological data and the 5 types of meteorological data and the sea fog visibility is multidimensional, and the model can be used for data training and prediction. The prediction process first standardizes the data, and adopts the Z-score standard data normalization method to obtain the mapping result values ​​of each numerical value. The data normalized by Z-score conforms to the standard normal distribution, which can eliminate the dimensional influence between the 9 types of data, remove the data with excessively large correlated errors in the existing data, and realize data cleaning.

Images