A feature optimization method for time series data prediction

Abstract

The application discloses a feature optimization method for time series data prediction, and comprises the following steps: 1, obtaining a time series data set to be predicted; 2, using a correlation analysis method to perform correlation calculation on feature parameters of the time series data set, and calculating correlation coefficients between the feature parameters of the data set; 3, obtaining correlation coefficients of other feature parameters and a prediction target feature parameter according to step 2, and selecting a plurality of feature parameter subsets according to different correlation coefficient thresholds; 4, inputting the feature parameter subsets into a pre-trained long short-term memory network model respectively, outputting prediction values of the prediction target feature parameter a s , and obtaining an optimal feature subset corresponding to the prediction target feature parameter a s according to errors of the prediction values and true values corresponding to each feature parameter subset, wherein the long short-term memory network model is trained by using feature parameter subsets corresponding to a training time series data set. The application optimizes high-dimensional data input, eliminates features invalid for prediction, and establishes an optimal feature subset.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data analysis and prediction, and relates to a feature optimization method for time series data prediction, in particular to a deep prediction model feature optimization method based on correlation analysis. BACKGROUND

[0002] With the continuous progress of science and technology, the importance of data is increasingly prominent. Mining the implicit information in time series data and conducting data analysis has great practical significance. Predicting the monitored time series data has become an important research content. Nowadays, with the gradual increase of data complexity and irregularity in the industrial field, it is difficult for traditional statistical models to achieve ideal prediction results. With the development of deep learning technology, neural network models are widely used in data processing. Among many neural network models, the recurrent neural network (RNN) has much better performance than other neural networks in processing time series data because it introduces the concept of "memory" into the neural network. Among its many variants, the long short-term memory neural network (LSTM) solves the problems of short-term memory and gradient explosion and disappearance of traditional recurrent neural networks, and can effectively process long-time data. The recurrent neural network taking LSTM as an example has provided more applications and innovations in prediction in various fields in recent years, and is also an effective method for time series data prediction.

[0003] In the face of time series data, an important task is how to select or extract important features from the complex data set for research and problem solving. This is because the data features in different application scenarios have different effectiveness, and even there are invalid and redundant features. Directly using all features for prediction not only consumes additional storage space and increases model training cost, but also reduces prediction accuracy due to high model complexity. Feature selection can obtain a low-dimensional representation of the original feature under the premise of preserving its main information. Data-driven prediction methods generally include important parameter features, which have a very important influence on the performance of model prediction, so the parameter optimization method of the model is usually used in combination with the modeling method. After feature optimization of high-dimensional time series data, the main features in the data can be obtained, so that the model can obtain better prediction accuracy. SUMMARY

[0004] In view of the above prior art, the technical problem to be solved by the present application is to provide a feature optimization method for time series data prediction, which optimizes high-dimensional data input, eliminates features that are invalid for prediction, and establishes an optimal feature subset.

[0005] To solve the above technical problems, a feature optimization method for time series data prediction according to the present application comprises:

[0006] Step 1, obtaining a time series data set to be predicted;

[0007] Step 2, using a correlation analysis method to analyze the characteristic parameters a1, a2…a n of the time series data set, performing correlation calculation to calculate the correlation coefficients between the characteristic parameters of the data set;

[0008] Step 3, according to the correlation coefficients of the other characteristic parameters and the predicted target characteristic parameter a s obtained in step 2, selecting a plurality of characteristic parameter subsets Sub λ according to different correlation coefficient thresholds {λ0<λ<1};

[0009] Step 4, inputting the predicted data corresponding to the characteristic subsets Sub λ of step 3 into a pre-trained long short-term memory network model respectively, outputting the predicted values y s of the predicted target characteristic parameter a λ , and obtaining the optimal characteristic subset of the predicted target characteristic parameter a λ according to the errors between different predicted values y s and the true values; the long short-term memory network model is obtained according to the operations of steps 2 and 3 to obtain a plurality of characteristic subsets Sub λ , and then the training data corresponding to Sub λ is input into the network trained.

[0010] Further, the long short-term memory network model is a multi-input and single-output LSTM model; the training method of the long short-term memory network model comprises:

[0011] (1) initializing the batch size batchsize, the training iteration number epoch, the time series length sl, and the learning rate α of the input training data; when the validation set loss val_loss does not decrease continuously for θ times of iteration, the learning rate is changed to α×β, but the minimum is not less than k, wherein θ, β, and k are hyperparameters, and 0<β<1;

[0012] (2) setting the loss function MAE:

[0013]

[0014] wherein y i represents the true value of the sample, f(x i ) represents the predicted value of the sample, and m represents the total number of sample points;

[0015] (3) selecting a plurality of characteristic subsets Sub λThe corresponding training data are trained as the input of the network, and the weight parameters of the LSTM are updated according to the loss function gradient descent algorithm until the network training is completed.

[0016] Further, the correlation analysis method is a Spearman rank correlation method, a grey correlation analysis method or a maximum information coefficient method.

[0017] The present application has the following advantages: the present application provides a feature optimization method for time series data prediction, the correlation between the input features is analyzed, the feature subsets with high correlation with the prediction target are filtered and selected according to the correlation coefficient index, and a prediction model based on deep learning is established to verify the prediction performance of different feature subsets, the feature optimization effects of different correlation methods are compared according to the experimental result evaluation index, and the optimal feature subset based on the correlation method is found.

[0018] Compared with the prior art, the present application is based on the demand for feature optimization of high-dimensional input data, and the input parameters are filtered and selected by using the correlation analysis method, and the optimal feature subset for the prediction target is established. (1) For the problem of invalid parameters in time series data prediction, three different correlation analysis methods are proposed for feature optimization, irrelevant features are removed in descending order of correlation coefficient, and multiple feature subsets of different dimensions are selected according to the threshold value, so as to establish the optimal feature subset with the purpose of reducing prediction error. (2) In the aspect of evaluating the feature subset, the feature optimization result is verified by combining the prediction experiment, the selected feature subset is input into the model for prediction, the optimal feature subset corresponding to each correlation method is found according to the experimental result evaluation index, the feature optimization effects of different correlation methods are compared, and the optimal model for parameter prediction is established. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The present application is a feature optimization method for time series data prediction;

[0020] Figure 2 The present application is a feature optimization method for time series data prediction;

[0021] Figure 3 The present application is a feature optimization method for time series data prediction;

[0022] Figure 4(a) is a relative error result graph of the prediction result and the real data using the MIC method;

[0023] Figure 4(b) is an absolute error result graph of the prediction result and the real data using the MIC method;

[0024] Figure 5A comparison chart of prediction curves optimized for the three correlation methods of the present application. DETAILED DESCRIPTION

[0025] The present application is further described below in conjunction with the accompanying drawings and examples.

[0026] The present application is based on feature selection of time series data prediction, fully considers the problem of high input dimension and many invalid features in the trend tracking prediction process of industrial system failure data, and proposes a feature optimization method for LSTM prediction model based on correlation analysis. The feature subset screening based on correlation measurement is used as a preprocessing step for establishing an optimal prediction model, the performance of the model is used to evaluate the advantages and disadvantages of the feature subset, three different correlation analysis methods of Spearman, gray correlation analysis and maximum information coefficient (MIC) are used for feature optimization, irrelevant features are removed, and multiple feature subsets of different dimensions are screened according to the threshold setting. An LSTM prediction model is established for prediction experiment verification, the effects of different correlation methods are compared, the optimal feature subset corresponding to the current prediction target is found according to the experimental result error, and the improvement of the proposed feature optimization method on the prediction accuracy is verified.

[0027] In combination Figure 1 , the present application comprises the following steps:

[0028] Step 1: Prepare the training data set: Take the time series data set A as the original data for parameter prediction, divide the training set and test set for the data set A, and standardize the entire data.

[0029] Step 2: Correlation analysis: select any one of the three correlation analysis methods of Spearman rank correlation (S), gray correlation analysis (G), and maximum information coefficient (M) to calculate the correlation between the feature parameters a1, a2…a n in the data set A.

[0030] Step 3: Subset screening: in the data set A, for the prediction target parameter a s , according to the correlation coefficient table obtained by the correlation analysis method in step 2, the correlation between other parameters and a s is obtained and the coefficient threshold is selected, different threshold values {λ|0<λ<1} are set, and different correlation threshold subsets Sub λ are obtained to prepare for subsequent prediction experiments.

[0031] Step 4: Construct the prediction model and train:

[0032] Step 4.1: Construct a multi-input single-hidden layer long short-term memory network (LSTM) model, the internal structure and gating mechanism of the LSTM neuron satisfyFigure 2 .

[0033] Step 4.2: training the LSTM prediction model, the plurality of feature subsets Sub λ The corresponding training data are respectively taken as the input of the network, the data are processed into a plurality of time sequence subsets with a length of sl, the normalized plurality of time sequence subsets with the length of sl are randomly shuffled, and the weight parameters of the LSTM are updated according to the loss function gradient descent algorithm until the network training is completed.

[0034] Step 5: for the test data set, the correlation threshold subset Sub λ The corresponding data are input into the LSTM model after training, the prediction value and the prediction parameter curve are obtained through the single-step prediction of the network, the error between the prediction value and the true value is calculated, and the optimal feature subset for the prediction target is obtained by comparing the error.

[0035] The following gives an embodiment in combination with specific application and parameters:

[0036] Embodiment

[0037] The present application comprises the following steps:

[0038] Step one: taking the industrial process fault data as the original data set A for parameter prediction, A has q kinds of fault severity, which are A1, A2…A q The training set A of the experiment is formed by splicing the data with different severity. train The data are normalized and standardized for pretreatment, and an unknown severity A is reserved as the test set of the experiment. test

[0039] Step two: selecting any one of the three correlation analysis methods of Spearman rank correlation (S), grey correlation analysis (G) and maximum information coefficient (M) to calculate the correlation of the feature parameters a1, a2…a n of the data set A, to obtain a two-dimensional matrix of correlation coefficients. The correlation between each parameter and the prediction target parameter a s (1≤s≤n) is calculated according to the following steps. s The calculation process of the three methods is as follows:

[0040] (1) Spearman rank correlation: a non-parametric index for evaluating the correlation between variables by using rank variables, and the calculation method is as follows:

[0041]

[0042] Wherein, d k represents two feature parameters a​x , a y , a x , a y , a x , a y , a k , a x , a y , a x , a y , a x , a y , a i , a i , a i , a i , a i , a i , a i , a i , a i , a i , a x , a y , a x , a y , a x , a y , a

[0043] (2) Grey correlation analysis: the basic idea is to determine the geometric shape of the reference series and the comparison series to judge whether the contact is close. The calculation method is as follows:

[0044]

[0045] Wherein, a0 is the prediction target parameter, i.e. the reference series under this algorithm, a0=(a0(1), a0(2), …, a0(m)), m represents the total number of sample points, and the comparison series is a i =(a i (1),a i (2),…,a i (m)), i=1, 2, …, n, a′0 and a′ i represent the dimensionless series of a0 and a i , a i (k) represents the kth sample point of series a i , a i ′(k) represents the kth sample point of series a′ i , 1≤k≤m, min() is the minimum value function, max() is the maximum value function, and p is the resolution coefficient, 0

[0046] (3) Maximum information coefficient (MIC): MIC measures the correlation degree between two random variables according to the joint probability density. For two-dimensional joint features a x and a y , the sample set is denoted as D={(x, y)|x∈a x , y∈a y}. By dividing the value range of a x and a y into u and v different intervals, the sample space is discretized into a u×v grid G, and the mutual information is further estimated:

[0047]

[0048] The maximum mutual information on different scale grids G is further calculated after standardization, and R is obtained s _MIC:

[0049]

[0050] where D| G represents the probability distribution introduced when the sample set D is divided using the grid G, log2() represents the logarithm operation with base 2, p(x) and p(y) are the empirical marginal probability densities of a x and a y , respectively, and p(x, y) is the empirical joint probability density of a x and a y ; min() is the minimum function, max() is the maximum function, and B(m) is a function of the number of samples.

[0051] Step three: screening of the feature subset for the predicted target parameter a s According to the correlation coefficient table obtained by the above three methods, the correlation size of other parameters with a s is selected by a coefficient threshold, irrelevant features are removed by adopting a descending order sorting method of correlation coefficients, and different threshold values {λ | 0 < λ < 1} are set to obtain multiple different feature subsets Sub λ , which are used as input features for subsequent prediction experiments.

[0052] Step four: construction and training of the LSTM model:

[0053] Construction of the long short-term memory network (LSTM) model: a multi-input single hidden layer LSTM model is built, and the number of LSTM hidden layer neurons c is set. The output layer contains one neuron.

[0054] Training of the LSTM model: the training data is processed into several time series subsets with a length of sl, the normalized several subsets with a length of sl are randomly shuffled, and 80% of the data set is used as the training data set and the remaining 20% is used as the validation data set. The training data set and the validation data set are input into the constructed LSTM model for training, and the network weight parameters are updated according to the Adam optimization algorithm.

[0055] The specific training includes the following steps:

[0056] (1) Initialize the batch size of input training data, the number of training iterations, the time series length sl, the learning rate a. When the val_loss does not decrease for consecutive iterations of 0, the learning rate is changed to a x b, but the minimum is not less than k, where 0 < b < 1.

[0057] (2) Set the loss function MAE, the calculation formula is as follows:

[0058]

[0059] Where, y i represents the true value of the sample, f(x i ) represents the predicted value of the sample, and m represents the total number of sample points.

[0060] (3) The multiple feature subsets Sub λ selected in step three are used as the input of the network for training, and the weight parameters of the LSTM are updated according to the gradient descent algorithm of the loss function until the network training is completed.

[0061] Step five: input the test data set A test to the LSTM model after training using the correlation threshold subset Sub λ selected in step three, and through the single-step prediction of the network, the predicted value and the predicted parameter curve can be obtained, and the error between the predicted value and the true value is calculated. By comparing the average absolute error of different feature subsets, the feature set with the smallest error value is obtained as the optimal subset of the current prediction target.

[0062] An embodiment is given in combination with specific parameters. The data in this example comes from the simulation data set of the thermal-hydraulic system of a nuclear power plant. The experimental condition is the leakage of the pressurizer water space. The sampling interval of this group of data is 1s, including 24 feature parameters V01-V24. The main parameter V17 of this working condition, the steam generator steam outlet flow, is taken as the prediction target parameter. According to three correlation analysis methods, multiple subsets of different dimensions after optimization are input into the network model for prediction experiment. Before training, the data is normalized and standardized.

[0063] Analysis of the results of the feature optimization and prediction experiment of the thermal-hydraulic system of a nuclear power plant:

[0064] The experimental data set selected the water space leakage of the regulator under the working condition as the fault data of the nuclear system, which can be divided into different levels according to its severity (the value is set from 0.0 to 1.0, and 1.0 is the most serious fault level), and V17 is taken as the prediction target parameter. According to the experimental steps, first of all, the maximum information coefficient (MIC) method in the three correlation methods is used to analyze the correlation and subset screening of the target parameter. According to the step-by-step increase of the threshold value and the step-by-step decrease of the parameter, three feature subsets are selected in addition to all working condition parameters and only the prediction target parameter.

[0065] The working condition data set in the data set except for the severity of 0.55 is selected to splice to form the training data set, and the severity of 0.55 is used as the test data set for experiment. Table 1 is the prediction result evaluation index comparison of using the MIC correlation method to select the feature subset of the parameter V17. The error indicators used include mean percentage error, mean square error, mean absolute error, and mean square root error. According to the average absolute error value, the optimal feature subset is Sub4.

[0066] According to this process, the optimal feature subset under three different correlation methods can be obtained. Table 2 is the optimal feature subset prediction result comparison of parameter V17 under three methods optimization. Figure 3 The comparison chart of the prediction result of the feature parameter V17 using the MIC method optimal subset and the real data is shown in Table 2. Figures 4(a)-4(b) The relative error and absolute error absolute value diagram of the prediction result of each sample point of the time series data and the real data is shown in Table 2. Figure 5 The prediction curve comparison chart of V17 based on the optimal subset obtained by optimizing three correlation methods is shown in Table 2.

[0067] Table 1 parameter V17 subset screening and experimental result index

[0068]

[0069] Table 2 V17 three correlation method optimal subset experimental result index

[0070]

Claims

1. A feature optimization method for time series data prediction, characterized in that: include: Step 1: Take the industrial process fault data as the original data set A for parameter prediction. There are q fault severity levels in A, namely A1, A2…A q ; The training set A of the experiment is formed by splicing data of different severity train , normalize and standardize the data, and reserve the unknown severity A test As a test set for the experiment; Step 2: Use the correlation analysis method to analyze the characteristic parameters a1, a2…a of the time series data set. n Perform correlation calculation to calculate the correlation coefficient between each characteristic parameter of the data set; Step 3: According to step 2, other characteristic parameters and predicted target characteristic parameters a are obtained. s The correlation coefficient of , according to different correlation coefficient thresholds {λ|0<λ<1}, select several feature parameter subsets Sub λ ; Step 4: Subset the feature set described in step 3 λ The corresponding data to be predicted are input into the pre-trained long short-term memory network model, and the predicted target feature parameter a is output. s The predicted value y λ , according to different predicted values ​​y λ The error with the true value is used to obtain the predicted target feature parameter a s The corresponding optimal feature subset; the long short-term memory network model is to obtain several feature subsets Sub according to steps 2 and 3. λ , then Sub λ The corresponding training data are input into the network training respectively; Step 5: Test dataset A test The input is fed into the trained LSTM model, and the network's single-step prediction can generate a predicted value for the unknown severity data, and calculate the error between the predicted value and the true value. Step 6: Find the optimal feature subset corresponding to each correlation method based on the experimental results evaluation indicators, compare the feature optimization effects of different correlation methods, and establish the optimal model for parameter prediction.

2. The feature optimization method for time series data prediction according to claim 1, characterized in that: The long short-term memory network model is a multi-input, single-output LSTM model; the training method of the long short-term memory network model includes: (1) Initialize the batch size of the input training data, the number of training iterations epoch, the time series length sl, and the learning rate α; when the validation set loss val_loss does not decrease for θ consecutive iterations, the learning rate is changed to α×β, but the minimum is not less than k, where θ, β, and k are hyperparameters, 0<β<1; (2) Set the loss function MAE: Among them, y i represents the true value of the sample, f(x i ) represents the predicted value of the sample, and m represents the total number of sample points; (3) Several feature subsets obtained by step 3 are filtered λ The corresponding training data are used as the input of the network, and the weight parameters of LSTM are updated according to the loss function gradient descent algorithm until the network training is completed.

3. The feature optimization method for time series data prediction according to claim 1, characterized in that: The correlation analysis method is Spearman rank correlation method, grey correlation analysis method or maximum information coefficient method.

Images