Coronal index prediction method based on Transform model
Through the Coronal Index prediction method based on the Transformer model, the difficulty of overfitting and long-term dependency capture of traditional timing models when processing high-dimensional and high-complexity data is solved, and more accurate and efficient coronal index prediction is achieved to adapt to different Characteristics of time scale and data volume.
Patent Information
- Application Number
- CN202411059656.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-12-10
AI Technical Summary
Traditional timing prediction models such as LSTMs are difficult to capture long-term dependencies when processing high-dimensional and high-complex time series data, which are easy to overfit, and are sensitive to hyperparameters and sequence lengths, resulting in poor coronary index prediction performance, especially in Limited data volume or poor performance when multivariate sequence processing.
Using the coronal index prediction method based on the Transformer model, the Transformer model and model training are constructed through data preparation, feature extraction, and the self-attention mechanism is used to process sequences in parallel, extract multivariable and spatial-temporal features, and reduce the complexity of hyperparameter parameter adjustment. Adapt to different time scales.
The Transformer model can more effectively capture global dependencies, improve the accuracy and generalization ability of corona index prediction, reduce the risk of overfitting, adapt to input sequences of different lengths, simplify model parameter adjustment, and improve computing efficiency and prediction speed.
Smart Images
Figure CN119106256A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital information processing, and in particular to a corona index prediction method based on a Transformer model. Background Art
[0002] With the continuous development of science and technology, human beings have a deeper understanding of the universe. The corona is the outermost layer of the sun's atmosphere, with a temperature of over 1 million degrees.
[0003] The corona is a natural phenomenon, which refers to the outermost layer of the solar atmosphere (the inner part is the chromosphere and photosphere), with a thickness of more than several million kilometers. The corona is the outermost layer of the solar atmosphere, extending from the edge of the chromosphere to several solar radii or even further. It is divided into the inner corona, the middle corona and the outer corona. The inner corona only extends to about 1.3 solar radii from the solar surface; the outer corona can reach several solar radii or even further. The corona is composed of very thin and completely ionized plasma, which is mainly protons, highly ionized ions and high-speed free electrons. The temperature of the corona is hundreds of times that of the solar surface.
[0004] The models used in traditional methods of preventing the corona index are usually time series prediction models, such as the long short-term memory network model (LSTM model). In some cases, especially when there are very long-term dependencies in the time series, the long short-term memory network model may still face problems that are difficult to capture, resulting in the model's insufficient ability to predict the trend of changes in the corona index in the distant future; in traditional time series prediction models, there is a risk of overfitting when processing high-dimensional and high-complexity time series data; it is sensitive to hyperparameters and sequence length; LSTM models usually require a large amount of historical data for training in order to capture patterns and trends in time series. If the amount of available data is limited, the model may not be able to fully learn the features in the data, resulting in poor prediction performance; if the corona index prediction problem involves multiple related variables, the LSTM model may face difficulties in processing multivariate sequences, and additional feature engineering and model design are required to effectively integrate different input information. Summary of the invention
[0005] The purpose of the present invention is to provide a corona index prediction method based on the Transformer model to solve the problems raised in the above background technology.
[0006] To achieve the above purpose, a corona index prediction method based on the Transformer model includes the following steps:
[0007] S1. Data preparation: Acquire and prepare time series data related to solar activity;
[0008] S2, feature extraction: extract features from the prepared time series data;
[0009] S3. Build Transformer model: Design and build Transformer model according to the complexity of prediction task and data characteristics;
[0010] S4. Training model: Use the prepared dataset to train the constructed Transformer model;
[0011] S5. Model evaluation: After training is completed, use the validation set or test set to evaluate the performance of the model.
[0012] Preferably, the time series data related to solar activity in step S1 includes historical data of the solar coronal index, sunspot number and solar wind speed, and the data are collected through solar observatories, solar satellites and astronomical observation organizations.
[0013] Preferably, the feature extraction in step S2 includes:
[0014] Spatial-temporal information conversion: converting the original spatial-temporal data into a form suitable for model processing;
[0015] Multivariate feature extraction: Solar activity is affected by many factors, and multivariate features need to be extracted, including the number of sunspots, solar wind speed, and solar radiation. For each variable, statistical features, frequency domain features, and time domain features are extracted;
[0016] Spatial correlation feature extraction: Feature extraction includes modeling spatial correlation, using spatial filters or graph convolutional neural networks to extract correlation features between different regions of the solar surface;
[0017] Dynamic feature modeling: Solar activity has dynamic changes. We extract the trend, periodicity and seasonality characteristics of the time series and use this information to model the evolution of solar activity.
[0018] Spatial-temporal feature interaction: Graph convolutional neural networks or spatial-temporal attention mechanisms are used in neural networks to extract spatial and temporal feature interactions to capture spatiotemporal relationships.
[0019] Preferably, the Transformer model in step S3 includes an encoder and a decoder, wherein the encoder is used to encode the input sequence into a hidden representation, and the decoder is used to generate a prediction result, and the number of layers, the number of hidden units and other hyperparameters of the model need to be adjusted and optimized according to the data.
[0020] Preferably, in the model evaluation in step S5, mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) is used to evaluate the accuracy, generalization ability and robustness of the model;
[0021] Mean Squared Error (MSE): Mean squared error is the average of the squares of the differences between the predicted values and the true values. It measures the average deviation between the model's predicted values and the true values. The smaller the MSE value, the better the model's fitting effect.
[0022] Root Mean Squared Error (RMSE): Root Mean Squared Error is the square root of the mean square error. It measures the average deviation between the model prediction value and the true value and has the same dimension as the original data. The smaller the RMSE value, the better the prediction effect of the model.
[0023] Mean Absolute Error (MAE): Mean absolute error is the average of the absolute values of the differences between the predicted values and the true values. MAE measures the average degree of deviation between the model's predicted values and the true values. Unlike MSE and RMSE, it does not consider the square of the error and can better reflect the actual gap between the predicted values and the true values.
[0024] Preferably, the step S1 includes collecting the acquired time series data related to solar activity, normalizing the collected data, and segmenting the processed data into a training set, a validation set, and a test set.
[0025] Preferably, the normalization processing method in step S1 is:
[0026] The collected time series data related to solar activity are processed by the [-1,1] normalization algorithm. The specific formula is:
[0027]
[0028] Among them, Xsca is the list after normalization, Xsca is the mean, Xmax is the maximum value, and in order not to lose the signal data of negative values, the normalization range used is [-1,1].
[0029] Preferably, the method of training the constructed Transformer model in step S4 is:
[0030] Through the Adamw optimization algorithm and MSELoss, the learning rate is continuously updated, the model of every 10 iterations is saved, and tested; the specific process is as follows:
[0031] 50% of the solar activity-related time series data set is selected as the training set, 40% of the data is selected as the test set, and stored in a 40000*1 matrix. The rest of the data is stored in the same form as the test set.
[0032] The training set is continuously put into the model training, and the epoch is set to 100 and the batch_size is set to 100;
[0033] Save the training model for every 10 epochs of iteration, along with its mean squared error.
[0034] Preferably, the step S4 specifically comprises:
[0035] a. Extract the test set data and input it into the 10 saved models for prediction testing;
[0036] b. Output the predicted waveform and its mean square error respectively;
[0037] c. By comparing the mean square error and the prediction graph, find the best prediction result and output it through an image.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] 1. The self-attention mechanism of the Transformer model enables the model to process input sequences in parallel, making more efficient use of computing resources during training and inference, and speeding up model training and prediction.
[0040] 2. The self-attention mechanism of the Transformer model allows the model to directly capture the global dependencies in the sequence, and can better handle long-distance dependencies, which is conducive to the prediction of long-term trends such as the corona index.
[0041] 3. The Transformer model can consider the information of each position in the input sequence at the same time, is not affected by the length of the sequence, can better capture the local and global patterns in the time series, and helps to improve the accuracy of the corona index prediction.
[0042] 4. The Transformer model has relatively few hyperparameters, including the number of layers of the encoder and decoder, the number of attention heads, etc., which reduces the complexity of parameter adjustment and helps to find the appropriate model structure and parameter settings more quickly.
[0043] 5. The Transformer model usually has better generalization ability when processing longer time series. Due to the characteristics of its self-attention mechanism, it can better adapt to input sequences of different lengths and more effectively cope with the corona index prediction task at different time scales.
[0044] 6. It can intuitively visualize the degree of attention paid by the model to the input sequence during the prediction process, which helps to understand the basis of the model's prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a schematic diagram of the steps of the method of the present invention. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] See also Figure 1 The present invention provides a technical solution: a corona index prediction method based on a Transformer model, comprising the following steps:
[0048] S1. Data preparation: Acquire and prepare time series data related to solar activity;
[0049] It should be noted that the time series data related to solar activity include historical data of the solar coronal index, sunspot number and solar wind speed, which are collected through solar observatories, solar satellites and astronomical observation organizations;
[0050] After the data collection is completed, the collected data is normalized and the processed data is segmented into a training set, a validation set, and a test set;
[0051] The method for normalizing the collected data is:
[0052] The collected time series data related to solar activity are processed by the [-1,1] normalization algorithm. The specific formula is:
[0053]
[0054] Among them, Xsca is the list after normalization, Xsca is the mean, Xmax is the maximum value, and in order not to lose the signal data of negative values, the normalization range used is [-1,1]
[0055] S2, feature extraction: extract features from the prepared time series data;
[0056] It should be noted that feature extraction includes:
[0057] Spatial-temporal information conversion: converting the original spatial-temporal data into a form suitable for model processing;
[0058] Multivariate feature extraction: Solar activity is affected by many factors, and multivariate features need to be extracted, including the number of sunspots, solar wind speed, and solar radiation. For each variable, statistical features, frequency domain features, and time domain features are extracted;
[0059] Spatial correlation feature extraction: Feature extraction includes modeling spatial correlation, using spatial filters or graph convolutional neural networks to extract correlation features between different regions of the solar surface;
[0060] Dynamic feature modeling: Solar activity has dynamic changes. We extract the trend, periodicity and seasonality characteristics of the time series and use this information to model the evolution of solar activity.
[0061] Spatial-temporal feature interaction: Graph convolutional neural networks or spatial-temporal attention mechanisms are used in neural networks to extract spatial and temporal feature interactions to capture spatiotemporal relationships;
[0062] Generate the relationship vector of two features through the RM (Relation Module) relationship module:
[0063] Purpose: To design an RM model to obtain the relationship between two features through calculation;
[0064] step:
[0065] Input: two features fi, f2;
[0066] 1. Calculate the difference between two features;
[0067] f diff =θ(f1)-Φ(f2),
[0068] Where θ(f1)=ReLU(BN(W θ f1))、φ(f2)=ReLU(BN(W φ f2));
[0069] Parameter weight W in the fully connected layer θ ,
[0070] ② Based on diference, calculate more compact relationvector,
[0071] r1,2=RM(f1,f2)=ReLU(BN(Wf diff )),in,
[0072] Output: r1,2.
[0073] S3. Build Transformer model: Design and build Transformer model according to the complexity of prediction task and data characteristics;
[0074] It should be noted that the Transformer model consists of an encoder and a decoder, where the encoder is used to encode the input sequence into a hidden representation and the decoder is used to generate prediction results. The number of layers, number of hidden units and other hyperparameters of the model need to be adjusted and optimized according to the data.
[0075] S4. Training model: Use the prepared dataset to train the constructed Transformer model;
[0076] It should be noted that when training the constructed Transformer model, the specific steps are:
[0077] a. Extract the test set data and input it into the 10 saved models for prediction testing;
[0078] b. Output the predicted waveform and its mean square error respectively;
[0079] c. By comparing the mean square error and the prediction graph, find the best prediction result and output it through an image.
[0080] The way to train the constructed Transformer model is:
[0081] Through the Adamw optimization algorithm and MSELoss, the learning rate is continuously updated, the model of every 10 iterations is saved, and tested; the specific process is as follows:
[0082] 50% of the solar activity-related time series data set is selected as the training set, 40% of the data is selected as the test set, and stored in a 40000*1 matrix. The rest of the data is stored in the same form as the test set.
[0083] The training set is continuously put into the model training, and the epoch is set to 100 and the batch_size is set to 100;
[0084] Save the training model for every 10 epochs of iteration, along with its mean squared error.
[0085] S5. Model evaluation: After training is completed, use the validation set or test set to evaluate the performance of the model.
[0086] It should be noted that the validation set or test set is used to evaluate the performance of the model, and the mean square error, root mean square error or mean absolute error indicators are used to evaluate the accuracy, generalization ability and robustness of the model;
[0087] Taking the root mean square error as an example: The calculation formula of RMSE is:
[0088] [RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2}],
[0089] Among them, (n) represents the number of data, (y_i) is the actual observation value of the (i)th corona index, and (\hat{y}_i) is the (i)th corona index value predicted by the model based on the input sequence. The smaller the RMSE index, the better the prediction effect of the model.
[0090] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A corona index prediction method based on Transformer model, characterized in that: The following steps are involved: S1. Data preparation: Acquire and prepare time series data related to solar activity; S2, feature extraction: extract features from the prepared time series data; S3. Build Transformer model: Design and build Transformer model according to the complexity of prediction task and data characteristics; S4. Training model: Use the prepared dataset to train the constructed Transformer model; S5. Model evaluation: After training is completed, use the validation set or test set to evaluate the performance of the model.
2. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: The time series data related to solar activity in step S1 include historical data of the solar coronal index, the number of sunspots and the solar wind speed, and the data are collected through solar observatories, solar satellites and astronomical observation organizations.
3. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: The feature extraction in step S2 includes: Spatial-temporal information conversion: converting the original spatial-temporal data into a form suitable for model processing; Multivariate feature extraction: Solar activity is affected by many factors, and multivariate features need to be extracted, including the number of sunspots, solar wind speed, and solar radiation. For each variable, statistical features, frequency domain features, and time domain features are extracted; Spatial correlation feature extraction: Feature extraction includes modeling spatial correlation, using spatial filters or graph convolutional neural networks to extract correlation features between different regions of the solar surface; Dynamic feature modeling: Solar activity has dynamic changes. We extract the trend, periodicity and seasonality characteristics of the time series and use this information to model the evolution of solar activity. Spatial-temporal feature interaction: Graph convolutional neural networks or spatial-temporal attention mechanisms are used in neural networks to extract spatial and temporal feature interactions to capture spatiotemporal relationships.
4. A corona index prediction method based on Transformer model according to claim 1, characterized in that: The Transformer model in step S3 includes an encoder and a decoder, wherein the encoder is used to encode the input sequence into a hidden representation, and the decoder is used to generate a prediction result. The number of layers, the number of hidden units and other hyperparameters of the model need to be adjusted and optimized according to the data.
5. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: In the model evaluation in step S5, mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) is used to evaluate the accuracy, generalization ability and robustness of the model; Mean Squared Error (MSE): Mean squared error is the average of the squares of the differences between the predicted values and the true values. It measures the average deviation between the model's predicted values and the true values. The smaller the MSE value, the better the model's fitting effect. Root Mean Squared Error (RMSE): Root Mean Squared Error is the square root of the mean square error. It measures the average deviation between the model prediction value and the true value and has the same dimension as the original data. The smaller the RMSE value, the better the prediction effect of the model. Mean Absolute Error (MAE): Mean absolute error is the average of the absolute values of the differences between the predicted values and the true values. MAE measures the average degree of deviation between the model's predicted values and the true values. Unlike MSE and RMSE, it does not consider the square of the error and can better reflect the actual gap between the predicted values and the true values.
6. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: The step S1 includes collecting the acquired time series data related to solar activity, normalizing the collected data, and segmenting the processed data into a training set, a validation set, and a test set.
7. A method for predicting the corona index based on a Transformer model according to claim 6, characterized in that: The normalization processing method in step S1 is: The collected time series data related to solar activity are processed by the [-1,1] normalization algorithm. The specific formula is: Among them, Xsca is the list after normalization, Xsca is the mean, Xmax is the maximum value, and in order not to lose the signal data of negative values, the normalization range used is [-1,1].
8. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: The method of training the constructed Transformer model in step S4 is: Through the Adamw optimization algorithm and MSELoss, the learning rate is continuously updated, the model of every 10 iterations is saved, and tested; the specific process is as follows: ①. Select 50% of the solar activity-related time series data set as the training set, 40% of the data as the test set, and store them in a 40000*1 matrix. The rest of the data is stored in the same form as the test set. ②. Continuously put the training set into the model training, and set the epoch to 100 and the batch_size to 100; ③. Save the training model for every 10 cycles of iteration, as well as its mean square error.
9. The method for predicting the corona index based on the Transformer model according to claim 1, characterized in that: The specific steps of step S4 are as follows: a. Extract the test set data and input it into the 10 saved models for prediction testing; b. Output the predicted waveform and its mean square error respectively; c. By comparing the mean square error and the prediction graph, find the best prediction result and output it through an image.