A prediction method based on the PSO-ADGM-SVR combined model

Abstract

This invention belongs to the field of data mining technology and relates to a prediction method based on a PSO-ADGM-SVR combined model. The method includes: collecting the most recent slope displacement data and preprocessing it; inputting the preprocessed data into an optimized PSO-ADGM model to obtain PSO-ADGM prediction results; inputting the preprocessed data into an optimized PSO-SVR model to obtain PSO-SVR prediction results; calculating a weight vector, and using the weight vector to weight and combine the PSO-ADGM and PSO-SVR prediction results to obtain the slope displacement predicted by the combined model. ADGM can uncover the trend characteristics of time-series data, while SVR can reflect the random characteristics of time-series data. The two are complementary; therefore, this invention couples the ADGM and SVR models to obtain the PSO-ADGM-SVR combined model, improving prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of data mining technology, specifically relating to a prediction method based on a PSO-ADGM-SVR combined model. Background Technology

[0002] Time series forecasting is widely used in various fields. Time series data, among many types of data, is an important and complex data type, referring to data arranged chronologically and exhibiting strong temporal correlation. Time series forecasting is the process of using historical data to predict future values, and it has wide applications in multiple fields. The key steps involved in time series forecasting are: Data Collection: Collecting time-related data sequences, typically a series of data points arranged in chronological order, including temperature records, sensor data, etc.; Data Preprocessing: To ensure data quality and the accuracy of the forecast results, missing values ​​in the data sequence need to be processed, and outliers need to be checked and handled; Data Analysis: Performing trend analysis, seasonality analysis, and periodicity analysis, calculating the autocorrelation function (ACF) and partial autocorrelation function (PACF) to provide a basis for selecting a forecasting model; Model Selection: Selecting a forecasting model based on the characteristics of the data; Dataset Splitting: Dividing the dataset into training and test sets for evaluation after model training; Model Training: Using the training set to train the selected time series forecasting model. Model evaluation: The predictive performance of the model is evaluated using test set data. Commonly used evaluation metrics include mean squared error (MSE) and mean absolute error (MAE).

[0003] Commonly used time series forecasting models include: grey model, ARIMA model, regression forecasting method, machine learning method, etc. Each of these forecasting models has its own advantages and disadvantages and applicable scope, and their forms and methods are fixed and deterministic.

[0004] However, traditional grey models have low prediction accuracy, and while the above models have good prediction effects for a specific type of data, their generalization ability is insufficient, making it difficult to effectively solve problems caused by complexity and uncertainty in actual engineering. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention employs a prediction method based on a PSO-ADGM-SVR combined model. The PSO-ADGM-SVR combined model includes a PSO-ADGM model and a PSO-SVR model; the prediction method includes:

[0006] S1. Collect the most recent historical slope displacement data and preprocess it;

[0007] S2. Input the preprocessed data into the optimized PSO-ADGM model to obtain the PSO-ADGM prediction results;

[0008] S3. Input the preprocessed data into the optimized PSO-SVR model to obtain the PSO-SVR prediction results;

[0009] S4. Calculate the weight vector and use the weight vector to weight and combine the PSO-ADGM prediction results and PSO-SVR prediction results to obtain the slope displacement predicted by the combined model; where ADGM is the nearest neighbor cumulative discrete grey model, SVR is the support vector regression machine, and PSO is the particle swarm algorithm.

[0010] The optimization process of the PSO-ADGM model includes: obtaining slope displacement training samples, constructing the ADGM(1,1) model, calculating the predicted value expression of the ADGM(1,1) model, determining the parameters of the ADGM(1,1) model based on the predicted value expression of the ADGM(1,1) model, optimizing the parameters of the ADGM(1,1) model using the POS algorithm based on the slope displacement training samples, obtaining the optimal parameter values, and using the optimal parameter values ​​as the parameters of the ADGM(1,1) model to obtain the optimized PSO-ADGM model.

[0011] Constructing the ADGM(1,1) model includes: constructing the original sequence X (0) A new sequence X is generated by accumulating the nearest neighbors of the original sequence. (1) Based on the new sequence, construct the nearest-neighbor cumulative discrete grey model ADGM(1,1): x (1) (k+1)=β1x (1) (k)+β2; where x (1) (k) represents the kth data point in the new sequence, and β1 and β2 are parameters.

[0012] The expression for calculating the predicted values ​​of the ADGM(1,1) model includes:

[0013] According to sequence X (1) The ADGM(1,1) model is solved using the least squares method to obtain estimates of the values ​​of parameters β1 and β2. These estimates are then incorporated into the ADGM(1,1) model for recursion, yielding the time response expression of the ADGM(1,1) model:

[0014]

[0015] By performing cumulative subtraction on the time response expression of the ADGM(1,1) model, we obtain the predicted value expression of the ADGM(1,1) model:

[0016]

[0017] Where, x (1) (k) represents the k-th data in the new sequence, x (0) (k+1) represents the (k+1)th data point in the original sequence. This is the k-th predicted value after cumulative subtraction and restoration. For the (k+1)th data point of the predicted new sequence, Let β1 be the parameter of the k-th dimension, and n be the length of the original sequence.

[0018] The optimization process of the PSO-SVR model includes: obtaining slope displacement training samples, obtaining the original SVR model, and determining the parameters of the original SVR model; optimizing the parameters of the original SVR model using the POS algorithm based on the obtained slope displacement training samples, obtaining the optimal parameter values, and using the optimal parameter values ​​as the parameters of the original SVR model to obtain the optimized PSO-SVR model.

[0019] The kernel function of the SVR model is the Gaussian radial basis function.

[0020] The calculation of the weight vector includes:

[0021] S41. Obtain historical slope displacement data and preprocess it;

[0022] S42. Input the preprocessed data into the optimized PSO-ADGM model to obtain the PSO-ADGM prediction results;

[0023] S43. Input the preprocessed data into the optimized PSO-SVR model to obtain the PSO-SVR prediction results;

[0024] S44. Construct an objective function based on the PSO-ADGM prediction results and the PSO-SVR prediction results, solve the objective function, and obtain the weight vector.

[0025] The objective function is:

[0026] minJ=minKEK T

[0027]

[0028] K = [k1, k2]

[0029] Where K is the weight vector, k1 and k2 are the weights of the PSO-ADGM model and the PSO-SVR model, respectively, E is the error matrix, and R is the unit vector.

[0030] The error matrix is:

[0031]

[0032] in, This represents the true label y and the prediction result of the i-th model. The error.

[0033] Solving the objective function involves using the Lagrange multiplier method to obtain the optimal solution for the weight vector.

[0034]

[0035] Where K0 is the optimal solution for the weight vector.

[0036] Beneficial effects:

[0037] 1. The Adjacent Accumulation Grey Prediction Model (ADGM) can uncover the trend characteristics of time series data, while the Support Vector Regression (SVR) model can reflect the randomness characteristics of time series data. The two are complementary to a certain extent. Therefore, this invention selects ADGM and SVR models as single prediction models and couples them using weights to obtain the PSO-ADGM-SVR combined model, which improves the prediction accuracy of the model. 2. Based on the traditional grey prediction model, this invention improves the traditional grey sequence generation method by using the adjacent accumulation grey sequence operator according to the principle of prioritizing new information. This makes the model more adaptable to the development law of things themselves and improves the prediction accuracy. Attached Figure Description

[0038] Figure 1 A flowchart illustrating a prediction method based on a PSO-ADGM-SVR combined model provided in an embodiment of the present invention;

[0039] Figure 2 A flowchart illustrating the optimization process of the PSO-ADGM model provided in this embodiment of the invention;

[0040] Figure 3 A flowchart illustrating the optimization process of the PSO-SVR model provided in this embodiment of the invention;

[0041] Figure 4 A schematic diagram illustrating the prediction results of the PSO-ADGM model provided in an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the prediction results of the PSO-SVR model provided in an embodiment of the present invention;

[0043] Figure 6 A schematic diagram illustrating the prediction results of the PSO-ADGM-SVR combined model provided in an embodiment of the present invention;

[0044] Figure 7 A comparison chart of prediction results from different models provided in embodiments of the present invention;

[0045] Figure 8 A comparison chart of evaluation metrics for different models provided in embodiments of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, the PSO-ADGM-SVR combined model includes: the PSO-ADGM model and the PSO-SVR model; the prediction methods include:

[0048] S1. Collect the most recent historical slope displacement data and perform preprocessing;

[0049] S2. Input the preprocessed data into the optimized PSO-ADGM model to obtain the PSO-ADGM prediction results;

[0050] like Figure 2 As shown, the optimization process of the PSO-ADGM model includes: obtaining a slope displacement dataset and dividing the dataset into a training set and a validation set; constructing an ADGM(1,1) model, calculating the predicted value expression of the ADGM(1,1) model, determining the parameters of the ADGM(1,1) model based on the predicted value expression of the ADGM(1,1) model, optimizing the parameters of the ADGM(1,1) model using the POS algorithm based on the training set to obtain the optimal parameter values, using the optimal parameter values ​​as the parameters of the ADGM(1,1) model to obtain the optimized PSO-ADGM model; and validating the prediction results of the optimized PSO-ADGM model using the validation set.

[0051] The nearest-neighbor gray prediction model ADGM(1,1) is a time series prediction method suitable for small samples and incomplete information. It can predict future development trends. Based on the traditional gray prediction model, it improves the traditional gray sequence generation method by using the nearest-neighbor gray sequence operator according to the principle of prioritizing new information.

[0052] Constructing the PSO-ADGM model includes: the original sequence being X (0) =(x (0) (1),x (0) (2),…,x (0) (n)), using the original sequence X (0) The nearest neighbor accumulation is performed to obtain the nearest neighbor accumulation generation sequence X. (1)=(x (1) (1),x (1) (2),…,x (1) (n)), the implementation method of neighbor accumulation is shown in formula (1). In the formula, γ is defined as the neighbor accumulation coefficient, which is used to adjust the weight of historical information in the gray sequence generation process, and its value range is (-1,1).

[0053]

[0054] The grey model generated by neighbor accumulation is called the neighbor accumulation discrete grey model ADGM(1,1):

[0055] x (1) (k+1)=β1x (1) (k)+β2 (2)

[0056] Generate sequence X based on nearest neighbor accumulation. (1) The least squares method is used to solve the nearest-neighbor cumulative discrete grey model ADGM(1,1) to estimate the values ​​of parameters β1 and β2:

[0057] [β1,β2] T = (B T B) -1 B T Y (3)

[0058]

[0059] After obtaining the estimated values ​​of β1 and β2, substitute them into (2) for recursion to obtain the time response expression of the nearest-neighbor cumulative discrete model:

[0060]

[0061] By performing cumulative subtraction on the time response expression of the nearest-neighbor cumulative discrete model, the predicted value expression of the ADGM(1,1) model is obtained:

[0062]

[0063] Where, x (1) (k) represents the k-th data in the new sequence, x (0) (k+1) represents the (k+1)th data point in the original sequence. This is the k-th predicted value obtained by cumulatively subtracting the original sequence. For the (k+1)th data point of the predicted new sequence, Let β1 be the parameter of the k-th dimension, and n be the length of the original sequence.

[0064] The fitness function of the POS algorithm is the average relative error between the predicted result and the true result of the training set. With the goal of minimizing the average relative error, the optimal value of the nearest neighbor accumulation coefficient γ of ADGM(1,1) is determined. In the PSO-ADGM model, the optimal parameter of the model is γ = 0.0149.

[0065] S3. Input the preprocessed data into the optimized PSO-SVR model to obtain the PSO-SVR prediction results;

[0066] like Figure 3 As shown, the optimization process of the PSO-SVR model includes: obtaining slope displacement training samples, obtaining the original SVR model, and determining the parameters of the original SVR model; optimizing the parameters of the original SVR model using the POS algorithm based on the slope displacement data, obtaining the optimal parameter values, and using the optimal parameter values ​​as the parameters of the original SVR model to obtain the optimized PSO-SVR model.

[0067] Training samples D = {(x1,y1),(x2,y2),…,(x n ,y n )},y i ∈R, where n is the number of training samples, x i Historical slope displacement, including slope displacement data collected over multiple time periods, y i It is the slope displacement collected in a subsequent time period.

[0068] For nonlinear regression problems, Support Vector Regression (SVR) can be used to map the input data to a high-dimensional space using a kernel function, and then use a linear regression algorithm to find an optimal hyperplane, transforming the nonlinear regression problem into a linear regression problem and obtaining the final regression function.

[0069]

[0070] Where, α i For Lagrange multipliers, Let K(x) be the Lagrange multiplier for the dual problem, b be an undetermined constant, and K(x) be the multiplier for the dual problem. i The kernel function is defined as follows: Different kernel functions will have different effects on the prediction results. This invention selects the Gaussian radial basis function (RBF), which has a simple structure, fewer unknown parameters, and certain generalizability.

[0071] K(x i ,x j )=exp(-‖x i -x j || 2 / σ 2 (10)

[0072] Wherein, σ is the kernel function parameter, and its value can have a significant impact on the final prediction result. The value of the kernel function parameter is usually determined by trial and error, empirical selection, cross-validation and other methods. Subjective factors involved in this process can affect the accuracy of SVR.

[0073] In the SVR model, the penalty factor C controls the model's fitting accuracy. Adjusting the value of C can effectively prevent overfitting and underfitting. Considering the strong global search capability, fast convergence speed, and ease of implementation of the particle swarm optimization algorithm, this invention utilizes this algorithm to optimize the kernel function parameter σ and the penalty factor C. The optimal parameter for C is 581.3069, and the optimal parameter for σ is 3.3785.

[0074] Specifically, it includes:

[0075] Step 1: Determine the parameters of the SVR model, including the penalty factor C and the kernel function parameter σ.

[0076] Step 2: Construct the fitness function for the particle swarm optimization algorithm. In SVR prediction, the mean square error (MSE) or root mean square error (RMSE) of the predicted and actual results is usually used as the fitness function.

[0077] Step 3: Initialize the position and velocity of the particle swarm; each particle represents a set of parameters, with position representing the value of the parameter and velocity representing the direction and rate of change of the parameter; during initialization, a set of particles can be randomly generated and their positions and velocities can be randomly assigned within the parameter range.

[0078] Step 4: Calculate the fitness function value of each particle based on the training samples. Based on the fitness function value, obtain the position of the local optimal particle and the position of the global optimal particle for each particle. The local optimal particle is the particle corresponding to the historical maximum fitness function value of the particle, and the global optimal particle is the particle with the largest fitness function value in the particle swarm.

[0079] Step 5: Update the velocity and position of each particle based on the position of the local optimal particle and the position of the global optimal particle;

[0080] Step 6: Repeat steps 4 and 5 until the maximum number of iterations is reached or the fitness function value meets the set threshold to obtain the position of the globally optimal particle. Use the position of the globally optimal particle as the optimal parameter of the SVR model.

[0081] The process of optimizing the neighbor accumulation coefficient γ of the neighbor accumulation grey prediction model using the POS algorithm is similar to the process described above.

[0082] S4. Calculate the weight vector, and use the weight vector to weight and combine the PSO-ADGM prediction results and PSO-SVR prediction results to obtain the combined model prediction result:

[0083]

[0084] in, The prediction results for PSO-ADGM This is the prediction result for PSO-SVR.

[0085] The reasonable allocation of weights for each individual model is one of the key factors affecting the prediction accuracy of the combined model. Commonly used weighting methods include: arithmetic mean method, binomial coefficient method, and reciprocal mean square error method. These methods are simple in form but lack applicability. Therefore, this invention adopts the optimal weighted combination method based on minimum error.

[0086] The calculation of the weight vector includes: acquiring historical slope displacement data and preprocessing it; inputting the preprocessed data into the optimized PSO-ADGM model and PSO-SVR model to obtain the PSO-ADGM prediction results and PSO-SVR prediction results, respectively; constructing an objective function based on the PSO-ADGM prediction results and PSO-SVR prediction results, solving the objective function, and obtaining the optimal weight vector; wherein, in one embodiment, the historical slope displacement data of the first 28 periods are used as training samples to predict the slope displacement data of the last 5 periods, and the optimal weight vector obtained is: k1 = 0.6072, k2 = 0.3928.

[0087] Optimal weights refer to the process of normalizing the weights, constructing an objective function, and solving it with the constraint of minimizing error or maximizing accuracy to obtain the values ​​of each weight in the combined model.

[0088]

[0089] This invention uses the sum of squared errors between predicted and measured values ​​as the objective function, and finds the optimal weights by minimizing this function. Assume the final prediction result is obtained by combining m individual prediction models, and the prediction result of the i-th individual model at time t is... y t Given the actual data at time t, the prediction result of the combined model at this time is:

[0090]

[0091] The prediction error of the combined model is:

[0092]

[0093] Among them, e itLet be the prediction error of the i-th model at time t.

[0094] Sum of squared errors:

[0095]

[0096] The error matrix of the system can then be defined as:

[0097]

[0098] K = [k1,k2,…,k m (16)

[0099] Therefore, the objective function is:

[0100] minJ=minKEK T (17)

[0101]

[0102] The optimal solution for the weight vector obtained using the Lagrange multiplier method is:

[0103]

[0104] Figure 4 and Figure 5 These are the prediction results from two single models. Figure 6 These are the prediction results from the PSO-ADGM-SVR combined prediction model. Figure 7 This is a comparison chart of the prediction results from different models. For example... Figure 8 As shown, this invention utilizes the mean relative error (MRE), root mean square error (RMSE), and coefficient of determination (R²). 2 The predictive performance of the model is evaluated; among them, the lower the values ​​of the mean relative error (MRE) and root mean square error (RMSE), the better the predictive performance, and the coefficient of determination R... 2 The closer the value is to 1, the better the model's predictive performance; Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 It can be seen that the PSO-ADGM-SVR combined prediction model outperforms the single model.

[0105] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A prediction method based on the PSO-ADGM-SVR combined model, characterized in that, The PSO-ADGM-SVR combined model includes: the PSO-ADGM model and the PSO-SVR model; the prediction methods include: S1. Collect the most recent historical slope displacement data and preprocess it; S2. Input the preprocessed data into the optimized PSO-ADGM model to obtain the PSO-ADGM prediction results; The optimization process of the PSO-ADGM model includes: obtaining slope displacement training samples, constructing the ADGM(1,1) model, calculating the predicted value expression of the ADGM(1,1) model, determining the parameters of the ADGM(1,1) model based on the predicted value expression of the ADGM(1,1) model, optimizing the parameters of the ADGM(1,1) model using the POS algorithm based on the slope displacement training samples, obtaining the optimal parameter values, and using the optimal parameter values ​​as the parameters of the ADGM(1,1) model to obtain the optimized PSO-ADGM model. Constructing the ADGM(1,1) model includes: constructing the original sequence A new sequence is generated by accumulating the nearest neighbors in the original sequence. Based on the new sequence, construct the nearest-neighbor cumulative discrete grey model ADGM(1,1): ;in, For the k-th data in the new sequence, and For parameters; The expression for calculating the predicted values ​​of the ADGM(1,1) model includes: According to the sequence Solve the ADGM(1,1) model using the least squares method to obtain the parameters. and The estimation of the value of the parameter and The estimated values ​​are added to the ADGM(1,1) model for recursion, resulting in the time response expression of the ADGM(1,1) model: By performing cumulative subtraction on the time response expression of the ADGM(1,1) model, we obtain the predicted value expression of the ADGM(1,1) model: in, For the k-th data in the new sequence, This is the (k+1)th data point in the original sequence. This is the k-th predicted value after cumulative subtraction and restoration. For the (k+1)th data point of the predicted new sequence, The parameter of the k-th dimension , The length of the original sequence; S3. Input the preprocessed data into the optimized PSO-SVR model to obtain the PSO-SVR prediction results; S4. Calculate the weight vector and use the weight vector to weight and combine the PSO-ADGM prediction results and PSO-SVR prediction results to obtain the slope displacement predicted by the combined model; where ADGM is the nearest neighbor cumulative discrete grey model, SVR is the support vector regression machine, and PSO is the particle swarm algorithm.

2. The prediction method based on the PSO-ADGM-SVR combined model according to claim 1, characterized in that, The optimization process of the PSO-SVR model includes: obtaining slope displacement training samples, obtaining the original SVR model, and determining the parameters of the original SVR model; optimizing the parameters of the original SVR model using the POS algorithm based on the obtained slope displacement training samples, obtaining the optimal parameter values, and using the optimal parameter values ​​as the parameters of the original SVR model to obtain the optimized PSO-SVR model.

3. The prediction method based on the PSO-ADGM-SVR combined model according to claim 2, characterized in that, The kernel function of the SVR model is the Gaussian radial basis function.

4. The prediction method based on the PSO-ADGM-SVR combined model according to claim 1, characterized in that, The calculation of the weight vector includes: S41. Obtain historical slope displacement data and preprocess it; S42. Input the preprocessed data into the optimized PSO-ADGM model to obtain the PSO-ADGM prediction results; S43. Input the preprocessed data into the optimized PSO-SVR model to obtain the PSO-SVR prediction results; S44. Construct an objective function based on the PSO-ADGM prediction results and the PSO-SVR prediction results, solve the objective function, and obtain the weight vector.

5. The prediction method based on the PSO-ADGM-SVR combined model according to claim 4, characterized in that, The objective function is: in, For the weight vector, These are the weights for the PSO-ADGM model and the PSO-SVR model, respectively. Here is the error matrix. It is a unit vector.

6. The prediction method based on the PSO-ADGM-SVR combined model according to claim 5, characterized in that, The error matrix is: in, , indicating the true label The prediction results of the i-th model The error.

7. The prediction method based on the PSO-ADGM-SVR combined model according to claim 6, characterized in that, Solving the objective function involves using the Lagrange multiplier method to obtain the optimal solution for the weight vector. in, This is the optimal solution for the weight vector.

Images