Online Prediction Method for Time Series Based on Rule Evolution of Kernel Recursive Maximum Correlation Entropy

Abstract

The present invention belongs to the field of time series prediction, and provides an online prediction method for time series of kernel recursive maximum correlation entropy based on rule evolution. First, the present invention uses a normalization method and phase space reconstruction to preprocess the acquired data, fully mining the useful information in the data; then, it uses a dual rule of compatibility metric and arousal index to realize the autonomous learning and evolution of the rule base, weakening the adverse effects of outliers or complex noises; afterwards, it combines the kernel recursive maximum cross-correlation entropy method and the sparsification strategy to update the model parameters, forming a compact dictionary to reduce the computational complexity while further enhancing the model's dynamic tracking ability for time series and improving the prediction accuracy; finally, test data is used to perform output prediction on the trained model to verify the efficiency of the model. The present invention can perform structural evolution for unknown complex environments, has strong autonomy and robustness, and can achieve a balance between high prediction accuracy and low computational complexity.

Description

Technical Field

[0001] The present invention belongs to the field of time series prediction, and relates to an online prediction method for time series based on rule evolution of kernel recursive maximum correntropy. Background Art

[0002] A time series is a set of data arranged in chronological order, which widely exists in many fields such as nature, industrial production, and financial technology. With the rapid development of sensors and storage devices, the data scale and update speed of time series are also constantly increasing. In the face of an environment with data explosion and complex noise, it is required that the online prediction model built can not only mine the hidden information of the data, but also show strong performance in aspects such as the nonlinearity, non-stationarity, and non-Gaussianity of the time series. Compared with other online prediction models, the kernel adaptive filter has the advantages of strong generalization ability, simple iterative update, and low computational complexity, and can effectively handle complex nonlinear prediction problems.

[0003] Although the kernel adaptive filter is widely used in the field of time series prediction, there are still the following deficiencies: (1) Poor ability to capture the time-varying characteristics of dynamic systems. Although Wu et al. proposed the kernel recursive maximum correntropy method by replacing the traditional mean square error criterion with the correntropy criterion in the paper "Wu Z, Shi J, Zhang X, et al. Kernelrecursive maximum correntropy[J]. Signal Processing, 2015, 117: 11 - 16.", which improved the performance of the model in the environment of outliers or non-Gaussian noise, due to the model only adjusting the parameters without structural adaptive evolution, the effect of the model tracking the time-varying characteristics of the time series is poor, resulting in low prediction accuracy. (2) High computational complexity of loading a complete dictionary. When processing data in each iteration, the model needs to increase the corresponding kernel space to store new data, and the size of the complete dictionary depends on the size of the data samples, which poses challenges in terms of computing time and memory, making it difficult for the model to be applicable to time series with a large data scale.

[0004] Therefore, the present invention takes time series with a large scale and time-varying characteristics as the research object, and proposes an online prediction method for time series based on rule evolution of kernel recursive maximum correntropy to achieve the structural adaptive evolution and parameter adaptive update of the model, thereby reducing the computational complexity and improving the prediction accuracy of the time series. The present invention is funded by the National Natural Science Foundation Project (62173063). Summary of the Invention

[0005] The present invention provides an online prediction method for time series based on rule evolution of kernel recursive maximum correntropy to solve the problems of poor online prediction performance and too high computational cost in the prior art.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A kernel recursive maximum correlation entropy time series online prediction method based on rule evolution, the specific steps are as follows:

[0008] Step 1: Collect time series data from the real world and normalize it.

[0009] First, we construct a sample {(x(n), d(n)), n = 1, 2, ...} for the prediction problem, where x(n) represents the model input vector composed of the t-dimensional feature input x(n), that is, x(n) = [x1(n), x2(n), ..., x t (n)], d(n) represents the prediction target, and n represents the time; secondly, considering the dimensional difference of the multi-dimensional features in the input vector, the normalization method is used to process the data, thereby reducing the adverse effect of the data on the accurate prediction of the model. The calculation formula is as follows:

[0010]

[0011] Among them, x(n) and x′(n) are the values of the input data before and after normalization, respectively. min and x max are the minimum and maximum values of the input data respectively; similarly, the predicted target is normalized, and the normalized value of the predicted target is d′(n).

[0012] Step 2: Reconstruct the phase space of the normalized time series data. The purpose is to deeply explore the useful information (correlation and dynamic characteristics, etc.) in the time series. Therefore, the time series data is reconstructed in phase space. The reconstructed input vector It is expressed as:

[0013]

[0014] Among them, τ t (n) and m t (n) are the delay time and embedding dimension of the t-th feature sequence in the normalized input vector x′(n), T is the transpose, and the prediction target is

[0015] Step 3: Divide the data set reconstructed in step 2, and set and initialize the parameters.

[0016] 80% of the reconstructed time series data are selected as the training set, and the total amount of the training set is recorded as N, and the rest is used as the test set.

[0017] Set model parameters, including the kernel width σ′ in the correlation entropy, the Gaussian kernel size σ, the clustering center learning rate η, the arousal exponent learning rate β, the regularization parameter γ, and the arousal threshold and the distance threshold δ.

[0018] At the first moment, the first training data is substituted into the model to create the first rule, form the rule base, and initialize the key parameters of the model, including the arousal exponent a1(1)=0, the clustering center dictionary and the expansion coefficient κ<·,·> represents the Gaussian kernel function, and the kernel functions of variables b and b * are defined as:

[0019]

[0020] where σ is the Gaussian kernel size.

[0021] Step 4: According to the training data set and parameter settings, start the iterative training of the model from the second moment. Specifically

[0022] Step 4.1: Load 1 training data at each moment, and calculate the compatibility measure and arousal exponent under each rule in the rule base;

[0023] The model defines rules using the compatibility measure and arousal exponent.

[0024] The compatibility measure ρ i is used to measure the correlation between the input vector and the clustering center R under the i-th rule. When the compatibility measure value reaches the maximum, it indicates that the similarity between the current input sample and the rule is the greatest, and at this time, this rule is the most compatible rule. The input vector and the clustering center R of the i-th rule i (n) The compatibility measure between them is as follows:

[0025]

[0026] where ρ i (n)∈[0,1], t represents the total number of features contained in the input vector, r represents the correlation dependence between two observed variables, and is calculated as:

[0027]

[0028] where and respectively represent the average values of the l-th feature input at the n-th moment and the clustering center R.

[0029] The arousal exponent ai can be used as a supplementary compatibility metric ρ i to create new rules to reduce the negative impact of outliers. The calculation formula for the wake-up exponent a under the i-th rule at the n-th moment is expressed as:

[0030] a i (n) = (1 - β)a i (n - 1) + β(1 - ρ i (n)) (6)

[0031] where β represents the learning rate of the wake-up exponent.

[0032] Step 4.2: Compare the minimum value of the wake-up exponent a i (n) with the wake-up threshold to determine whether to create a new rule. Among them, the range of the wake-up threshold is 0 - 1, and it is specifically divided into the following two situations:

[0033] Situation 1: When the minimum wake-up exponent is greater than the wake-up threshold, that is create a new rule. At this time, the number of rules in the rule base increases and is updated to r = r + 1, and initialize the parameters in this new rule, including the clustering center dictionary expansion coefficient

[0034] Situation 2: When the minimum wake-up exponent is less than or equal to the wake-up threshold that is include this input vector (i.e., the current time series) into the most compatible rule in the rule base, and recursively update the clustering center. The calculation formula is as follows:

[0035]

[0036] where η ∈ [0, 1] represents the learning rate of the clustering center.

[0037] After that, in the subsequent parameter update of the model structure, select the relevant entropy criterion to replace the mean square error criterion in the traditional kernel recursive least squares method as the cost function, and use the kernel recursive maximum correlation entropy method to improve the prediction performance of the model in the presence of non-Gaussian noise or outliers. The optimization objective based on the relevant entropy is defined as:

[0038]

[0039] where Ω is the filter weight, represents the current input at the j-th moment Through the input vector after nonlinear mapping, γ represents the regularization parameter, and ||·|| represents the L2 norm. Under the i-th rule at the n-th time, the gradient descent method is used to obtain the intermediate variable h i (n), z i (n) and λ i (n), that is, the intermediate variable is calculated according to formula (9), and the calculation formula is:

[0040]

[0041] Among them, c ik (n) is the dictionary set in the i-th rule at the n-th time, k is the number of input vectors contained in the dictionary set, Q i (n-1) is the matrix variable of the i-th rule at the n-1th moment.

[0042] Since the kernel recursive maximum correlation entropy method calculates parameters based on a set of historical data, each set will form a dictionary, namely C i (n) = [c i1 (n),…,c in′ (n)], where n′ is the number of vectors stored in the dictionary set. If the dictionary contains all the input data, the computational burden will increase significantly. Therefore, in order to reduce the computational complexity, the novelty criterion is used for sparse processing so that only relevant input vectors are retained in the dictionary to form a compact dictionary. The distance calculation formula is defined as Where c ii* =[c1,…,c m ] represents the i-th rule in the dictionary set c. * An input vector is then compared with a distance threshold to determine whether the data is included in the dictionary, where the threshold range is 0-1. It is divided into the following two parts:

[0043] (1) When When the current input sample Included in the dictionary, At this time, the matrix variable Q i (n) and expansion coefficient θ i (n) The update formula is as follows:

[0044]

[0045] in, e i (n) is the prediction error, expressed as

[0046] (2) When When the current data is excluded from the dictionary, the dictionary remains unchanged. At this time, the matrix variable Q i (n) and expansion coefficient θi (n) The calculation formula is as follows:

[0047]

[0048] where

[0049] Step 4.3: Load the training data and determine whether the training is completed;

[0050] If the current time is less than the total amount of the training set, that is, n < N, return to Step 4.1 to enter the next time iteration update; otherwise, the model training is completed, export the rule base, and enter the next step.

[0051] Step 5: Use the model trained in Step 4 to select the most compatible rule in the rule base to perform output prediction on the test data set, and then perform denormalization calculation on the sample data. Finally, use evaluation metrics to measure the prediction accuracy of the model, including root mean square error (RMSE) and symmetric mean absolute percentage error (SMAPE), which are defined as follows:

[0052]

[0053] where d(j) and are the true value and the predicted value respectively, and n is the number of samples.

[0054] Compared with the prior art, the present invention has the following obvious advantages:

[0055] (1) The present invention can perform structure evolution for unknown complex environments, with strong autonomy and robustness. Specifically, the present invention constructs a rule base from scratch according to the input data through compatibility measurement and wake-up index, adaptively updates the structure, and fully extracts the useful information contained in the data, ensuring the quality of the rules;

[0056] (2) When dealing with data containing outliers or non-Gaussian noise, the present invention adopts the kernel recursive maximum correlation entropy method to effectively suppress the adverse effects brought by them, realize the real-time capture of the dynamic changes of time series, and produce accurate prediction results;

[0057] (3) In addition, the use of the adaptive and sparsification method of the rule base in the present invention can ensure the compactness of the dictionary structure, achieving a balance between high prediction accuracy and low computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of the time series online prediction model of the present invention.

[0059] Figure 2(a) is the prediction curve of the Beijing PM2.5 time series of the present invention.

[0060] Figure 2(b) shows the error curve of the present invention for the Beijing PM2.5 time series. Detailed implementation manners

[0061] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0062] The hardware device used in the present invention includes a PC machine.

[0063] As Figure 1 shown, for the online prediction method of the kernel recursive maximum correlation entropy time series based on rule evolution, the specific implementation measures are as follows:

[0064] Step 1: Obtain a total of 8760 groups of PM2.5 pollutants and 4-dimensional meteorological data of the Capital International Airport in Beijing from January 1, 2019 to December 31, 2019, including air temperature, air pressure, dew point and wind speed. The data is collected once per hour. At this time, the input vector of the model is x(n) = [x1(n), x2(n), …, x5(n)], and a single-step prediction is performed on the Beijing PM2.5 pollutants, and the prediction target is determined as d(n) = x1(n + 1); the data set {(x(n), d(n)), n = 1, 2, …} is normalized according to formula (1) to construct an input-output sample set (x′(n), d′(n)), so as to avoid poor model prediction effects caused by large differences in the dimensions of multi-dimensional data sets.

[0065] Step 2: To fully extract the hidden information in the time series, according to the phase space reconstruction theory, the delay time and embedding dimension are determined to be τ = 1 and m = 20 respectively, and 5000 groups of data after reconstruction are selected. At this time, the input vector and prediction target are respectively

[0066]

[0067] Step 3: 80% of the reconstructed Beijing time series is used as the training set. At this time, the total amount of data in the training set is 4000, and the remaining data is used as the test set. At the first moment, the first training data enters the model, creates the first rule, and forms a rule base. The model parameters are set as shown in Table 1; the key parameters are initialized, including the wake-up index a1(1) = 0, the clustering center dictionary and the expansion coefficient

[0068] Table 1 Model parameter settings

[0069]

[0070] Step 4: Use the sequentially arriving training set to evolve the rule base from the second moment and iteratively train the model. Specifically

[0071] Step 4.1: Calculate the compatibility measure and arousal index between the training data at the current moment and each rule in the rule base according to Formula (4) and Formula (6) respectively;

[0072] For the 5-dimensional time series data of Beijing, the input vector at the nth moment and the clustering center R of the ith rule i (n) has the following compatibility measure:

[0073]

[0074] Among them, the correlation dependence r between the input at the current moment and the clustering center R of the ith rule i (n) is calculated according to Formula (5).

[0075] Step 4.2: Compare the minimum arousal index a imin (n) and the arousal threshold Discuss whether to add new rules, which is specifically divided into the following two cases:

[0076] Case 1: When a imin (n) > 1e-5, generate a new rule. At this time, the number of rules in the rule base increases, that is, r = r + 1, and initialize the key parameters under this rule, including the clustering center dictionary and the expansion coefficient

[0077] Case 2: When a imin (n) ≤ 1e-5, store the current time series in the most compatible rule base and update the clustering center R i (n), which is expressed as

[0078]

[0079] Due to the complexity of the external environmental noise, the collected Beijing time series data has non-Gaussian characteristics, resulting in an unsatisfactory final prediction effect of the model. To solve the above problems, an optimization objective is established according to Formula (8). Essentially, the correlation entropy criterion is used to replace the traditional mean square error criterion, and the model parameters are updated based on the kernel recursive maximum correlation entropy method to weaken the adverse effects of non-Gaussian noise or outliers. Then, the intermediate variables are calculated according to Formula (9). In addition, the dimension of the kernel matrix in the kernel recursive maximum correlation entropy method depends on the size of the data volume. When dealing with large-scale data, it is difficult to avoid a large amount of calculation and storage burden. Therefore, a sparsification method is adopted to make the model only save the relevant input vectors to form a compact dictionary. By comparing with the relationship between the distance threshold δ, analyze whether the current data is added to the dictionary, which is specifically divided into the following two parts:

[0080] (1) When it is the case, the current input data is added to the dictionary, and the matrix variable Q i (n) and the expansion coefficient θ i (n) are updated according to formula (10);

[0081] (2) When it is the case, the current input data is excluded, the dictionary remains unchanged, and the matrix variable Q i (n) and the expansion coefficient θ i (n) are updated according to formula (11);

[0082] Step 4.3: Determine whether the training data set has been loaded completely, that is, n < 4000?

[0083] If n < 4000, it means that the training is not over, continue with the next moment iterative update, and return to Step 4.1; otherwise, the model training is completed, and enter the next step.

[0084] Step 5: Use the test set to test on the trained model, calculate the predicted values of Beijing PM2.5 pollutants, and perform anti-normalization processing on them. Draw the predicted curve and error curve of the Beijing PM2.5 pollutants of the present invention, as shown in Figure 2. It can be seen from the figure that the present invention can effectively track the change trend of the time series, and the predicted error value is small, indicating the accuracy of the model prediction. Finally, calculate RMSE and SMAPE according to formula (12), as shown in Table 2. Continuing to compare with the adaptive normalization sparse quantization kernel recursive least squares method (ANS-QKRLS) and the quantization kernel recursive generalized maximum correlation entropy method (QKRGMC), the evaluation indexes obtained by the present invention are all the best, further verifying the effectiveness of the present invention for predicting Beijing PM2.5.

[0085] Table 2 Prediction results of various methods

[0086]

[0087] Finally, it should be noted that: the above examples are only used to illustrate the implementation mode of the present invention. It should be understood that the examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, for those skilled in the art, without departing from the concept of the present invention, several deformations and improvements can still be made. The modifications of various equivalent forms of the present invention all fall within the scope defined by the appended claims of this application, and all belong to the protection scope of the present invention.

Claims

1. An online prediction method for kernel recursive maximum correlation entropy time series based on rule evolution, characterized in that It includes the following steps: Step 1: Collect time series data and normalize it; the data is airport meteorological data, including temperature, air pressure, dew point, and wind speed; Step 2: Perform phase space reconstruction on the normalized time series data to deeply mine useful information in the time series; Step 3: Divide the data set reconstructed in Step 2 and set and initialize the parameters; Step 4: According to the training data set and parameter settings, start iterative training of the model from the second moment; Step 5: Use the model trained in Step 4 to select the most compatible rule in the rule base to perform output prediction on the test data set, then perform denormalization calculation on the sample data, and finally use evaluation indicators to measure the prediction accuracy of the model; The specific content of Step 4 is as follows: Step 4.1: Load 1 training data at each moment and calculate its compatibility metric and arousal index under each rule in the rule base; The model defines rules using compatibility metrics and arousal indices; The compatibility metric ρ i is used to measure the correlation between the input vector and the cluster center R under the i-th rule; when the compatibility metric value reaches the maximum, it indicates that the similarity degree between the current input sample and the rule is the largest, and at this time this rule is the most compatible rule; the input vector and the cluster center R of the i-th rule i (n) The compatibility metric between them is shown as follows: where ρ i (n) ∈ [0, 1], t represents the total number of features included in the input vector, and r represents the correlation dependence between two observed variables, calculated as: Among them, and respectively represent the average value of the l-th feature input at the n-th moment and the clustering center R; The wake-up index a described above i can be used as a supplement to the compatibility metric ρ i to create a new rule to reduce the negative impact of outliers; the calculation formula of the wake-up index a under the i-th rule at the n-th moment is expressed as: a i y(n) = (1 - β)a i y(n - 1)+β(1 - ρ i (n)) (6) Among them, β represents the learning rate of the arousal index; Step 4.2: Compare the minimum value of the wake-up index a i (n) with the wake-up threshold to determine whether to create a new rule; Step 4.3: Load the training data and determine whether the training is completed; If the current moment is less than the total amount of the training set, that is, n < N, then return to Step 4.1 and enter the next moment for iterative update; otherwise, the model training is completed, export the rule base, and enter the next step.

2. The online prediction method for kernel recursive maximum correlation entropy time series based on rule evolution according to claim 1, characterized in that The specific content of Step 1 is as follows: Construct a sample {(x(n), d(n)), n = 1, 2, …} for the prediction problem, where x(n) represents the model input vector composed of t-dimensional feature inputs x(n), that is, x(n) = [x1(n), x2(n), …, x t (n)], d(n) represents the prediction target, and n represents the time instant; secondly, considering the dimensional differences of the multi-dimensional features in the input vector, use the normalization method to process the data; Perform normalization processing on the prediction target, and the normalized value of the prediction target is d′(n).

3. The online prediction method of kernel recursive maximum correlation entropy time series based on rule evolution according to claim 2, wherein, In Step 1, the calculation formula for normalization processing is as follows: where x(n) and x′(n) are the values of the input data before and after normalization respectively, and x min and x max are the minimum and maximum values of the input data respectively.

4. A method for online prediction of a kernel recursive maximum correlation entropy time series based on rule evolution according to claim 1, characterized in that, The specific content of Step 2 is as follows: Perform phase space reconstruction on the normalized time series data, and the reconstructed input vector is expressed as: Among them, τ t (n) and m t (n) are respectively the delay time and embedding dimension of the t-th feature sequence in the normalized input vector x′(n), T is the transpose, and the prediction target at this time is 5. A method for online prediction of a kernel recursive maximum correlation entropy time series based on rule evolution according to claim 1, characterized in that The specific content of Step 3 is as follows: Select 80% of the reconstructed time series data as the training set, and record the total amount of the training set as N, and the rest as the test set; Set model parameters, including the kernel width σ′ in the correlation entropy, the Gaussian kernel size σ, the learning rate η of the cluster center, the learning rate β of the arousal index, the regularization parameter γ, and the arousal threshold and the distance threshold δ; At the first moment, the first training data is substituted into the model to create the first rule, form the rule base, and perform initialization settings on the key parameters of the model, including the wake-up index a1(1) = 0, the clustering center dictionary and the expansion coefficient κ<·,·> represents the Gaussian kernel function, and the kernel functions of variables b and b * are defined as: Among them, σ is the Gaussian kernel size.

6. The online prediction method for time series of kernel recursive maximum correlation entropy based on rule evolution according to claim 1, wherein The specific content of Step 4.2 is as follows: The described wake-up threshold ranges from 0 to 1 and is specifically divided into the following two cases: Case 1: When the minimum arousal index is greater than the arousal threshold, i.e., create a new rule. At this time, the number of rules in the rule base increases, and is updated to r = r + 1, and initialize the parameters in this new rule, including the cluster center dictionary expansion coefficient Case 2: When the minimum arousal index is less than or equal to the arousal threshold That is at this time, this input vector (i.e., the current time series) is incorporated into the most compatible rule in the rule base, and the cluster center is recursively updated. The calculation formula is as follows: Among them, η ∈ [0,1] represents the learning rate of the cluster center; After that, in the subsequent parameter update of the model structure, select the correlation entropy criterion to replace the mean square error criterion in the traditional kernel recursive least squares method as the cost function, and use the kernel recursive maximum correlation entropy method to improve the prediction performance of the model in the presence of non-Gaussian noise or outliers; the optimization objective based on correlation entropy is defined as: where Ω is the filter weight, denotes the current input at the j-th moment the input vector after non-linear mapping, γ represents the regularization parameter, ||·|| represents the L2 norm; at the n-th moment under the i-th rule, using the gradient descent method, the intermediate variables h i (n), z i (n) and λ i (n), that is, calculate the intermediate variables according to formula (9), and the calculation formula is: where c ik (n) is the dictionary set in the i-th rule at the n-th moment, k is the number of input vectors included in the dictionary set, Q i (n - 1) is the matrix variable of the i-th rule at the (n - 1)-th moment; Since the kernel recursive maximum correlation entropy method calculates parameters based on a set of historical data, each set will form a dictionary, i.e., C i (n) = [c i1 (n), …, c in′ (n)], where n′ is the number stored in the dictionary set; if the dictionary contains all the input data, it will lead to a substantial increase in the computational burden; therefore, to reduce the computational complexity, a novelty criterion is adopted for sparsification processing to keep only the relevant input vectors in the dictionary, forming a compact dictionary; the distance calculation formula is defined as where represents the i-th input vector in the dictionary set c under the i-th rule * ; then, it is compared with a distance threshold to determine whether the data is included in the dictionary, where the threshold range is 0 - 1; it is specifically divided into the following two parts: (1) When the current input sample is incorporated into the dictionary, at this time, the matrix variable Q i (n) and the expansion coefficient θ i (n) update formulas are as follows: Among them, e i (n) is the prediction error, expressed as (2) When occurs, the current data is excluded from the dictionary, and the dictionary remains unchanged. At this time, the matrix variable Q i (n) and the expansion coefficient θ i (n) are calculated in the form of: Among them 7. An online prediction method for kernel recursive maximum correlation entropy time series based on rule evolution according to claim 1, characterized in that, In Step 5, evaluation indicators are used to measure the prediction accuracy of the model, including root mean square error RMSE and symmetric mean absolute percentage error SMAPE.

Images