A mid-term prediction method for solar 10.7cm radio flux based on stacking ensemble learning

Abstract

The present invention provides a medium-term forecast method for solar 10.7cm radio flux based on Stacking ensemble learning. First, linear regression and multi-layer perceptron are used as benchmark models to forecast F10.7, and then the forecast results of multiple models are integrated by using Stacking learning strategy. The forecast value of the benchmark model is fused by Lasso minimum angle regression method to obtain the final forecast result. The present invention effectively avoids the shortcomings of each single model and obtains a forecast result with higher accuracy and stronger robustness. The Lasso minimum angle regression method is used as a meta-model to automatically ignore the features with low contribution to highlight the most important features, and to avoid the occurrence of overfitting problems to a certain extent, so that the forecast result is better than other meta-models.

Description

Technical Field

[0001] The invention belongs to the field of space environment parameter prediction and is suitable for medium-term prediction of solar 10.7cm radio flux. Background Art

[0002] Solar activity and its near-Earth space environment have a wide range of impacts on human living environment. In recent years, with the continuous progress of human science and technology and the continuous improvement of information level, the coupling between various technical systems has become increasingly stronger, and solar activity has caused abnormalities or failures in spacecraft, power systems, communication equipment, radar and navigation equipment. Therefore, the prediction of solar activity has gradually developed into a hot topic in the field of space environment research.

[0003] The solar 10.7cm radio flux (hereinafter referred to as F10.7) is closely related to solar activity. It is a typical parameter for measuring solar activity and an important index for predicting the impact of solar activity on the earth's environment (Wang Xin. Medium-term forecast of daily values ​​of solar 10.7cm radio flux based on deep learning [J]. Journal of Spacecraft Measurement and Control. 2017, 36 (2): 118-122.). F10.7 refers to the solar radio emission with a wavelength of 10.7cm, which belongs to the solar slow-varying radio in the centimeter band. It is the total radio radiation flux value of all source regions on the solar surface. It is a commonly used index for comprehensively measuring the intensity of extreme ultraviolet radiation in the solar chromosphere, transition zone and corona (Liu Siqing, Zhong Qiuzhen, Wen Jing, et al. Research on medium-term forecast model of solar 10.7cm radio flux (I). Journal of Space Science, 2010, 30 (1): 1-8). In atmospheric model research, the F10.7 index is often used as a characterization of solar activity. Therefore, the forecast of F10.7 index has become an important research topic in the forecast of space environmental parameters, especially the forecast of atmospheric density.

[0004] The difficulty of F10.7 index forecasting lies in the improvement of forecasting accuracy and the expansion of application scenarios. For a long time, domestic and foreign scholars have carried out a variety of research on technologies and methods for F10.7 forecasting, such as Fourier analysis, curve fitting, meteorological method, precursor method, neural network method and artificial intelligence method (Jiao Yan. Comparison of F10.7cm radio flux forecasting methods based on McNish-Lincoln linear regression technology [J]. Space Environment. 2004, 1: 56-63; Zhang Qin. Study on the dynamic behavior and predictability of the monthly average variation of 10.7cm radio flux [J]. Astrophysical Journal. 1993, 13 (1): 67-72; Zhong Qiuzhen, Liu Siqing, He Juanxiong, et al. Application of singular spectrum analysis in the medium-term prediction of solar 10.7cm radio flux [J]. Journal of Space Science. 2005, 25 (3): 199-204). Various technologies and methods have their own advantages and disadvantages, but most of them have their own limitations, and generally have the disadvantages of insufficient accuracy and low generalization. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a medium-term forecast method for solar 10.7cm radio flux based on Stacking ensemble learning. First, linear regression and multi-layer perceptron are used as benchmark models to forecast F10.7, and then the forecast results of multiple models are integrated using the Stacking learning strategy. The forecast values ​​of the benchmark models are fused using the Lasso minimum angle regression method to obtain the final forecast result. The present invention can effectively integrate the advantages of different benchmark models, and its forecast performance is significantly better than that of a single benchmark model.

[0006] The technical solution adopted by the present invention to solve the technical problem comprises the following steps:

[0007] 1) Using the K-fold cross-validation strategy on the original training set, the linear regression model is trained, denoted as CV-LReg; using the K-fold cross-validation strategy on the original training set, the multi-layer perceptron model is trained, denoted as CV-MLP;

[0008] 2) Selecting the Lasso minimum angle regression model as the meta-model, constructing a training set to train it; the input of the training set includes three parts, the first part is the predicted value of CV-LReg on the training set, the second part is the predicted value of CV-MLP on the training set, and the third part is the average value of all input features of the original training set, and the output is consistent with the output of the original training set;

[0009] 3) Input the historical value of F10.7, and forecast the F10.7 data for a period of time in the future based on the trained model to obtain the forecast result.

[0010] The K-fold cross validation is a 4-fold cross validation.

[0011] The steps of training the model by the 4-fold cross-validation strategy are as follows: first, the training data set is randomly divided into 4 groups; one group of data is taken out as the test set in turn, and the remaining 3 groups of data are used as training sets to train a sub-model to obtain a combination of 4 sub-models; if the sub-model is a linear regression model, the combination is recorded as CV-LReg; if the sub-model is a multi-layer perceptron model, the combination is recorded as CV-MLP; the prediction value of the combined model is the average value of the predictions of the 4 sub-models.

[0012] The Lasso minimum angle regression model assumes X as the independent variable and Y as the dependent variable. The standardized value of the observed data obtained by sampling n times is (X, Y), where X is an n×p matrix, n>p, and the i-th observation value of X is X i =(X i1 ,X i2 ,...,X ip ) TIndependent of each observation, i∈[1,2,...,n]; Y is an n×1 matrix, Y=(Y 1 ,Y 2 ,...,Y n ) T ; Then the linear regression model of Y on X is y i =α i =∑β j x ij +ε i , where ε i ~N(0,σ 2 ), For standardized data Therefore, α=0, and further arrangement yields Y=βX+ε; let the constraint normal form expression be st∑ j |β|≤t, where t≥0 is a harmonic parameter. The Lasso process continuously adjusts the value of t to reduce the overall regression coefficient and compress the coefficient of the insignificant variable until it reaches 0.

[0013] The Lasso minimum angle regression meta-model assumes is the solution of the unconstrained least squares problem, then let the constraint normal form expression be The cross-validation method is used to determine the parameter s and define the statistic CV(s) = ∑(y i -∑β j (s)x ij ) 2 , the corresponding s when CV takes the minimum value is the final parameter.

[0014] The beneficial effects of the present invention are: the prediction results of multiple single prediction models are integrated by using the ensemble learning idea, which effectively avoids the shortcomings of each single model and obtains a prediction result with higher accuracy and stronger robustness; in step 2), the Lasso minimum angle regression method is used as a meta-model, which can automatically ignore the features with low contribution to highlight the most important features, avoid the occurrence of overfitting problems to a certain extent, and make the prediction results better than other meta-models. The present invention proposes a new model construction method, which provides a new idea for space environment forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a workflow diagram of the Stacking ensemble learning prediction method of the present invention.

[0016] Figure 2The mean absolute percentage error (MAPE) of the linear regression model (LReg), multi-layer perceptron model (MLP), 4-fold cross-validation linear regression model (CV-LReg), 4-fold cross-validation multi-layer perceptron model (CV-MLP) and stacking ensemble learning model for 1-27 days forecasting is shown, and the average value of all test dates is shown. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with the accompanying drawings and embodiments. The present invention includes but is not limited to the following embodiments.

[0018] In actual space environment forecasting, users usually have historical data for a period of time. The present invention designs an F10.7 forecasting method based on Stacking ensemble learning, the purpose of which is to learn rules from historical data and use the rules to forecast the F10.7 data for a certain time period (point) in the future.

[0019] In order to improve the prediction accuracy of F10.7, the technical solution adopted by the present invention comprises the following steps:

[0020] Step 1.1 (Training base model 1): Use the 4-fold cross-validation strategy to train the linear regression model on the original training set, denoted as CV-LReg;

[0021] Step 1.2 (training base model 2): ​​train the multi-layer perceptron model using a 4-fold cross-validation strategy on the original training set, denoted as CV-MLP;

[0022] Step 2.1 (Constructing the meta-model training data set): The input of the meta-model training set consists of three parts (the first part is the predicted value of CV-LReg on the training set, the second part is the predicted value of CV-MLP on the training set, and the third part is the average value of all input features of the original training set). The output is consistent with the output of the original training set. The input and output together constitute the training set of the meta-model;

[0023] Step 2.2 (Training meta-model): Select the Lasso Least angle regression model (Lasso-Lar) as the meta-model (Meta-Learner) and train it on the newly constructed training set;

[0024] Step 3: Input the historical value of F10.7, and forecast the F10.7 data for a period of time in the future based on the trained model (base model + meta-model) to obtain the forecast result.

[0025] The above steps adopt the idea of ​​Stacking ensemble learning, so the method is recorded as Stacking. Its workflow is as follows Figure 1 shown.

[0026] The specific contents of the above steps that need further explanation are as follows:

[0027] 1. Use the 4-fold cross-validation strategy to train the linear regression model and the multi-layer perceptron model (Step 1)

[0028] In order to effectively avoid the overfitting problem, the present invention adopts the K-fold cross validation method (the K value is set to 4 in this paper) to train the linear regression model and the multi-layer perceptron model. The F10.7 daily observation data from 1974-01-01 to 2007-12-31 is used as the training set.

[0029] The steps of training the model with the 4-fold cross-validation strategy are as follows: first, randomly divide the training data set into 4 groups; take out one group of data as the test set in turn, and use the remaining 3 groups of data as the training set to train a sub-model, so that a combination of 4 sub-models can be obtained. If the sub-model is a linear regression model, the combination is recorded as CV-LReg; if the sub-model is a multi-layer perceptron model, the combination is recorded as CV-MLP. In the model use stage, the prediction value of the combined model is the average of the predictions of the 4 sub-models.

[0030] 2. Train the Lasso minimum angle regression meta-model (Step 2)

[0031] Lasso Least Angle Regression (LassoLAR), also known as L1 regularization of linear regression, is a machine learning algorithm for variable extraction (Wang Mei, Sun Yingqi, Song Kaoping, et al. Analysis of factors affecting water injection volume in dense well network based on Lasso-LAR [J]. Mathematics in Practice and Knowledge. 2016, 46(8): 124-131.). Its principle is: let X be the independent variable and Y be the dependent variable. The standardized value of the observed data obtained by sampling n times is (X, Y), where X is an n×p matrix (n>p), and the i-th observation value of X is X i =(X i1 ,X i2 ,...,X ip ) T , i∈[1,2,...,n] is independent of each observation; Y is an n×1 matrix, Y=(Y 1 ,Y 2 ,...,Y n ) T . Then the linear regression model of Y on X is:

[0032] y i =α i =∑β j x ij +εi (3)

[0033] where ε i ~N(0,σ 2 ), For standardized data Therefore, α = 0. Further rearranging equation (3) yields:

[0034] Y=βX+ε (4)

[0035] According to the definition of Lasso, to screen out significant variables, we need to add certain constraints on the basis of formula (4) (Tang Jianrong, Deng Lin. Analysis of carbon footprint influencing factors based on LARS-Lasso method [J]. Soft Science. 2014, 28(9): 124-128.), and let the constraint paradigm expression be:

[0036]

[0037] In formula (5), t≥0 is a harmonic parameter. The Lasso process continuously adjusts the value of t to reduce the overall regression coefficient and compress the coefficient of the insignificant variable until it reaches 0.

[0038] It should be noted that, assuming is the solution of the unconstrained least squares problem, then when When , the constraint is meaningless, so divide both sides of the constraint in equation (5) by make s∈[0,1], then:

[0039]

[0040] The s in formula (6) is the standard parameter for selecting the final model. Lasso is essentially a quadratic programming problem. The least angle regression (LARS) algorithm proposed by Bradley Efron (Efron B, Hastie T, Johnstone I, et al. Least angle regression [J]. Annals of Statistics. 2004, 32 (2): 407-499.) can solve the Lasso calculation problem very well. The LARS method is slightly changed to limit only one variable to enter or exclude the model each time, so as to obtain the solution path of Lasso LAR, and the best model is selected by determining that formula (6) is parameter s. The cross-validation method (CV) proposed by Efron and Tibshirani (Efron B, Tibshirani R. An Introduction to the Bootstrap [M]. London: Chapman and Hall. 1993.) is used to determine the parameter s to define the statistic:

[0041] CV(s)=∑(y i -∑β j (s)x ij ) 2 (7)

[0042] The s corresponding to the minimum value of CV is the final parameter (Ke Zhenglin. Application of Lasso and related methods in multivariate linear regression models [D]. Beijing: Beijing Jiaotong University, 2011.).

[0043] In order to quantitatively evaluate the performance of the method (Stacking) of the present invention, the F10.7 daily observation data from 1974-01-01 to 2020-03-30 were selected for experiments, where the data from 1974-01-01 to 2007-12-31 were used as the training set, and the data from 2007-12-01 to 2020-03-30 were used as the test set. The input feature quantity and the forecast time step were selected as 54 and 27 respectively. Four forecast methods were selected for experimental comparison with Stacking, and the four methods are:

[0044] -Use the linear regression model alone, denoted as LReg;

[0045] -Use the multi-layer perceptron model alone, denoted as MLP;

[0046] -The linear regression model trained using 4-fold cross validation is denoted as CV-LReg;

[0047] -Multilayer perceptron model trained using 4-fold cross validation, denoted as CV-MLP.

[0048] The model prediction performance is evaluated by the mean absolute percentage error (MAPE) between the true value and the predicted value, which is defined as follows:

[0049]

[0050] Among them, n is the number of data points, and i represent the predicted value and the true value respectively.

[0051] Figure 2 The comparison results of the forecast values ​​of 5 methods 1-27 days in advance are shown. In the experiment, it is found that in the forecast step of 1-4 days, the performance of CV-LReg and LReg methods is similar to that of Stacking method, and both are better than MLP and CV-MLP methods. However, after 5 days, the forecast performance of CV-LReg and LReg methods continues to deteriorate, and the MAPE of the forecast value on the 27th day has reached 9%. The forecast performance of MLP and CV-MLP methods is highly volatile, with the MLP method being the most volatile, and the MAPE of the forecast value has been maintained at a relatively high level; the forecast performance of CV-MLP method continues to deteriorate after a forecast step of 15 days. The MAPE value of Stacking ensemble learning method in the forecast step of 1-27 is basically at a lower position among the five methods, indicating that the forecast performance of Stacking ensemble learning method is better than the other four methods as a whole. It is worth mentioning that the performance of CV-LReg is better than CV-MLP for 1-4 days forecast, and CV-MLP is better than CV-LReg for 5-27 days forecast, and the Stacking method can selectively integrate the advantages of both: that is, the Stacking integration algorithm is more inclined to CV-LReg for 1-4 days forecast, and more inclined to CV-MLP for 5-27 days forecast. This is consistent with the expectation of the present invention.

Claims

1. A mid-term prediction method for solar 10.7cm radio flux based on Stacking ensemble learning, characterized in that: The following steps are involved: 1) Using the K-fold cross-validation strategy on the original training set, the linear regression model is trained, denoted as CV-LReg; using the K-fold cross-validation strategy on the original training set, the multi-layer perceptron model is trained, denoted as CV-MLP; 2) Selecting the Lasso minimum angle regression model as the meta-model, constructing a training set to train it; the input of the training set includes three parts, the first part is the predicted value of CV-LReg on the training set, the second part is the predicted value of CV-MLP on the training set, and the third part is the average value of all input features of the original training set, and the output is consistent with the output of the original training set; The Lasso minimum angle regression model assumes X as the independent variable and Y as the dependent variable. The standardized value of the observed data obtained by sampling n times is (X, Y), where X is an n×p matrix, n>p, and the i-th observation value of X is X i =(X i1 ,X i2 ,...,X ip ) T Independent of each observation, i∈[1,2,...,n]; Y is an n×1 matrix, Y=(Y1,Y2,...,Y n ) T ; Then the linear regression model of Y on X is y i =α i =∑β j x ij +ε i , where ε i ~N(0,σ 2 ), For standardized data Therefore, α=0, and further arrangement yields Y=βX+ε; let the constraint normal form expression be st∑ j |β|≤t, where t≥0 is a harmonic parameter. The Lasso process is to continuously adjust the value of t to make the overall regression coefficient smaller and compress the coefficient of the insignificant variable until it is 0; The Lasso minimum angle regression meta-model assumes is the solution of the unconstrained least squares problem, then let the constraint normal form expression be The cross-validation method is used to determine the parameter s and define the statistic CV(s) = ∑(y i -∑β j (s)x ij ) 2 , the corresponding s when CV takes the minimum value is the final parameter; 3) Input the historical value of F10.7, and forecast the F10.7 data for a period of time in the future based on the trained model to obtain the forecast result.

2. The method for medium-term prediction of solar 10.7 cm radio flux based on Stacking ensemble learning according to claim 1 is characterized in that: The K-fold cross validation is a 4-fold cross validation.

3. The mid-term prediction method for solar 10.7 cm radio flux based on Stacking ensemble learning according to claim 2 is characterized in that: The steps of training the model by the 4-fold cross-validation strategy are as follows: first, the training data set is randomly divided into 4 groups; one group of data is taken out as the test set in turn, and the remaining 3 groups of data are used as training sets to train a sub-model to obtain a combination of 4 sub-models; if the sub-model is a linear regression model, the combination is recorded as CV-LReg; if the sub-model is a multi-layer perceptron model, the combination is recorded as CV-MLP; the prediction value of the combined model is the average value of the predictions of the 4 sub-models.

Images