A high-energy electron flux prediction method based on a deep learning fusion model

Abstract

This paper presents a high-energy electron flux prediction method based on a deep learning fusion model, relating to the field of space physics. A dataset is acquired from historical high-energy electron flux observations, and then processed through data processing, including calibration, noise reduction, redundancy handling, and imbalance-based processing. A CNN-LSTM-AM deep learning fusion model is constructed, consisting of CNN convolutional layers, LSTM recurrent network layers, and an attention mechanism AM layer. Redundancy and imbalance-based processing are added to the data to improve prediction accuracy and robustness. A CNN-LSTM-AM deep learning fusion model with optimized learning rate is specifically designed to effectively meet the requirements for high-energy electron flux prediction.

Description

Technical Field

[0001] This invention relates to the field of space physics technology, specifically a method for predicting high-energy electron flux based on a deep learning fusion model. Background Technology

[0002] Geosynchronous orbit, a crucial operational region for spacecraft, hosts thousands of satellites. The safe and stable operation of these satellites directly impacts the normal functioning of critical infrastructure such as modern communications and meteorological monitoring. During magnetospheric substorm recovery, a sustained surge in high-energy electron flux occurs in geosynchronous orbit, a phenomenon known as high-energy electron flux enhancement (HEEF). Due to the extremely strong penetrating power of high-energy electrons, HEEF can lead to deep charging effects on spacecraft, posing a serious threat to the safe operation of near-Earth satellites. Therefore, establishing reliable high-energy electron flux forecasting models can effectively help satellites take protective measures, thereby minimizing the harm caused by extreme space weather to satellites and even human life.

[0003] Current high-energy electron flux prediction techniques can be mainly divided into traditional prediction methods based on physical mechanisms and data-driven prediction methods. In traditional methods, empirical statistical models rely excessively on expert knowledge bases, and their subjective parameterization process leads to significant uncertainty in the prediction results. While deep learning technology has shown great potential in the field of artificial intelligence, it still faces challenges, specifically the following key issues:

[0004] (1) The problem of handling data sample complexity: Most existing forecast models fail to fully consider the complexity of high-energy electron flux datasets. Specifically, problems such as feature redundancy and class imbalance that are common in the samples lack systematic analysis and processing. This deficiency in feature engineering will directly affect the generalization performance and prediction stability of the model. Studies have shown that modeling methods that ignore the inherent characteristics of the data may lead to an increased risk of overfitting and reduce the applicability of the model under different spatial environmental conditions;

[0005] (2) Problem of single forecast model: Existing forecast models generally use a single model to predict high-energy electron flux, but high-energy electron flux data itself has many uncertainties. Therefore, a single model cannot integrate the complexity of high-energy electron flux data. It is necessary to consider using a multi-model fusion method to make more accurate predictions of high-energy electron flux based on the characteristics of high-energy electron flux samples.

[0006] (3) Performance optimization of forecast models: Although traditional machine learning methods have significant advantages in computational efficiency, their prediction accuracy often falls short of practical application requirements. In contrast, deep learning-based forecast models have made significant progress in prediction accuracy through hierarchical feature extraction and representation learning. However, due to limitations in model architecture design and training strategies, the performance of existing deep learning methods still has room for optimization, especially in handling the nonlinear dynamic characteristics of high-energy electron flux, requiring further algorithm improvements to enhance prediction accuracy.

[0007] In summary, the ability to predict high-energy electron flux is closely related to the development level of the space weather forecasting system and is a crucial technical support for ensuring the safe operation of space infrastructure. Based on this practical need, it is imperative to construct a high-precision high-energy electron flux forecasting model to reduce the impact of the space radiation environment on spacecraft, which is of great significance for critical decisions such as mission planning and orbit design. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a high-energy electron flux prediction method based on a deep learning fusion model. This method adds redundancy and imbalance processing to the data, improving the accuracy and robustness of the prediction. Furthermore, it specifically designs a CNN-LSTM-AM deep learning fusion model with optimized learning rate, effectively meeting the requirements for high-energy electron flux prediction.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a high-energy electron flux prediction method based on a deep learning fusion model, comprising the following steps:

[0010] Step 1: High-energy electron flux data acquisition and processing

[0011] Data sets are acquired based on historical high-energy electron flux observation data as required, and then data processing is performed, including calibration, noise reduction, redundancy reduction, and imbalance-like processing.

[0012] Redundancy handling employs principal component analysis to perform feature dimensionality reduction on high-energy electron flux data, including the following steps:

[0013] (1) Data standardization: The high-energy electron flux samples are centralized;

[0014] (2) Covariance matrix calculation: Analyze the linear relationship between features;

[0015] (3) Eigenvalue decomposition: Solving for the eigenvalues ​​and eigenvectors of the covariance matrix;

[0016] (4) Principal component selection: Sort by eigenvalue size and select the top eigenvalues. The eigenvectors corresponding to the largest eigenvalues ​​are used as principal components;

[0017] (5) Data projection: Mapping the original data to the selected principal component space;

[0018] Imbalanced processing employs a partitioned SMOTH optimization method to equalize the high-energy electron flux sample, including the following steps:

[0019] Define class imbalance: ,in, Indicates the number of low-energy electron samples. Indicates the number of high-energy electron samples. A larger value indicates a more severe imbalance in the dataset;

[0020] First, the sample space is divided into multiple molecular regions. Within each molecular region, a density-based spatial clustering algorithm is used for local density estimation and noise filtering, while each sample is assigned a class label. Second, based on the clustering results, the molecular regions are further subdivided into multiple atomic regions, and adaptive SMOTE resampling based on the target value is implemented within each atomic region until class balance is achieved. Finally, global sample balance is achieved by iteratively traversing all atomic and molecular regions.

[0021] Step 2: High-energy electron flux prediction method based on deep learning fusion model

[0022] A CNN-LSTM-AM deep learning fusion model is constructed, consisting of CNN convolutional layers, LSTM recurrent network layers, and an attention mechanism AM layer, as detailed below:

[0023] The CNN convolutional layer organizes the processed multidimensional feature vectors into a two-dimensional feature moment according to the time step. A feature transformation module is designed, including a Flatten layer to flatten the high-dimensional feature vectors into one-dimensional feature vectors and a ReLU activation function to enhance feature representation through nonlinear transformation. The function expression is as follows:

[0024]

[0025] For any input Its output Hard zero truncation is implemented in the negative domain, while the identity mapping is maintained in the positive domain. A max-pooling strategy is used to construct the pooling layer. Downsampling reduces the spatial dimension of the feature vector while retaining the most significant feature responses. Its mathematical expression is:

[0026]

[0027] In the formula, As input features, For the corresponding output features, Indicates stride length. Represents the pooling size. The depth of the feature vector;

[0028] The LSTM recurrent network layer serves as a temporal feature extraction module, performing operations at each time step. The LSTM cell updates its internal state using the following set of formulas:

[0029]

[0030] in, It is the sigmoid activation function. This represents element-wise multiplication. and These represent the weight matrix and the bias term, respectively. Represents the Gate of Oblivion Indicates the input gate. Indicates the output gate. It is the hyperbolic tangent function, which transforms any real number Mapped to the continuous interval [-1, 1].

[0031] Furthermore, in step one, the calibration process calibrates the data samples based on a linear regression algorithm. In the initial stage of the calibration process, key features are extracted from the dataset and represented as vectors. The extracted feature values ​​are combined with the corresponding true calibration values ​​to form a training sample set. ,in Indicates the first The feature vector of each sample For its corresponding true value, a calibration function is obtained by fitting it using a linear regression algorithm based on the training samples. , represented as: In the formula, For the weight vector, As a bias term, minimize the predicted value Compared with the true value The mean square error between them is used to determine the optimal value. and , represented as: In the formula, This represents the mean square error.

[0032] Furthermore, in step one, the noise reduction process employs a moving average method for data smoothing, given a length of... The original data sequence Select window size as The smoothed sequence pass Calculate and select window size A balance must be struck between noise suppression and data fidelity.

[0033] Furthermore, the redundancy processing in step one specifically includes:

[0034] Covariance is defined as follows:

[0035]

[0036] The covariance matrix is ​​defined as follows:

[0037]

[0038] In obtaining the covariance matrix Then, its eigenvalues ​​and eigenvectors are solved through eigenvalue decomposition. and the corresponding feature vector Satisfies the characteristic equation: eigenvectors Mathematical definition of: ,in For the identity matrix, each eigenvalue There is a unique corresponding feature vector , eigenvalue Arrange from largest to smallest, select the first... The largest eigenvalues ​​and their corresponding eigenvectors: ;

[0039] Given contains A dataset of samples After a linear transformation of the projection matrix, the original high-dimensional feature space is mapped to... The principal component space is represented by the mapping process as follows:

[0040]

[0041] Based on the maximum variance theory, the optimal... 3D feature representation, from the covariance matrix Before the election The eigenvectors corresponding to the largest eigenvalues.

[0042] Furthermore, in step two, a learning rate optimization method is introduced to optimize performance based on the CNN-LSTM-AM deep learning fusion model. The learning rate optimization method adopts an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler, and the specific process is as follows:

[0043] set up Based on the learning rate, The preset minimum learning rate threshold is based on the maximum number of iterations. and minimum batch size The formula for calculating the total number of system iterations under these two constraints is as follows:

[0044]

[0045] After initialization, the algorithm enters the iterative optimization phase. For the ... In this iteration, the corresponding cosine annealing learning rate scheduling coefficient is calculated using the following formula:

[0046]

[0047] Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and at the end of the training cycle. In the t-th iteration, the update mechanism of the learning rate is as follows:

[0048]

[0049] In each training cycle After completion, the base learning rate Adaptive updates are performed according to the following formula:

[0050]

[0051] The improved adaptive learning rate expression is derived as follows:

[0052]

[0053] In the formula, and These are the first-order moment estimates and second-order moment estimates after bias correction, respectively.

[0054] Compared with existing technologies, the beneficial effects of this invention are as follows: In the data processing process, in addition to conventional calibration and noise reduction, this invention performs redundancy processing on the feature dimensionality reduction technique based on principal component analysis and class imbalance processing based on partition-SMOTH optimization. This has a significant effect on reducing the feature dimensionality of the data and improving prediction performance, effectively reducing the class imbalance characteristics of the data, improving the accuracy and robustness of the forecast model, and designing CNN convolutional layers to address the data format differences, LSTM recurrent network layers to address the differences in data time scales, and an attention mechanism AM layer for dynamic allocation of feature weights, forming a CNN-LSTM-AM deep learning fusion model. The model is then optimized for performance using an improved Adam algorithm with a dynamic learning rate based on cosine annealing, effectively meeting the needs of high-energy electron flux enhancement event prediction tasks and providing reliable technical support for space weather forecasting. Attached Figure Description

[0055] Figure 1 This is a technical roadmap diagram of the prediction method of the present invention;

[0056] Figure 2 This is a performance evaluation data graph of different hyperparameter optimization methods for the SVM model in this invention;

[0057] Figure 3 This is a structural diagram of the CNN-LSTM-AM deep learning fusion model in this invention;

[0058] Figure 4 This is a structural diagram of the feature transformation module of the CNN convolutional layer in this invention;

[0059] Figure 5 This is a structural diagram of the LSTM recurrent network layer in this invention;

[0060] Figure 6 This is a structural diagram of the attention mechanism AM layer in this invention;

[0061] Figure 7 This is a performance comparison chart of the deep learning fusion model and the shallow model of the present invention in the embodiment. Detailed Implementation

[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0063] This invention aims to improve the reliability and accuracy of prediction models, and the specific technical approach combines... Figure 1 As shown, a systematic data processing workflow was first constructed. After completing routine data calibration and noise reduction, feature redundancy and class imbalance were addressed. Then, a high-energy electron flux prediction method based on a shallow model was designed, improving the SVM-based high-energy electron flux prediction model and optimizing it using a hyperparameter optimization method based on particle swarm optimization. Next, based on the analysis of the characteristics of high-energy electron flux samples, a high-energy electron flux prediction method based on a deep learning fusion model was proposed. To further improve model performance, a learning rate optimization method was introduced, and the model performance was analyzed and tested. Finally, the performance of the shallow learning model and the deep learning fusion model was compared and tested, verifying the advancement and feasibility of the deep learning fusion model.

[0064] S1. High-energy electron flux data acquisition and processing

[0065] First, the dataset is acquired, followed by data processing. In addition to calibration and noise reduction, this invention also addresses data redundancy and class imbalance. The data samples are calibrated using a linear regression algorithm, denoised using a moving average method, redundancy is addressed using principal component analysis (PCA) for feature reduction, and class imbalance is addressed using a partitioned SMOTH optimization method. This systematic data processing approach ensures the performance of subsequent models.

[0066] The dataset is obtained from historical high-energy electron flux observation data according to requirements, and the data is processed, including calibration, noise reduction, redundancy reduction, and imbalance-like processing, as shown below:

[0067] 1.1 Calibration Process

[0068] A calibration model based on a linear regression algorithm is constructed. In the initial stage of the calibration process, key features are extracted from the dataset and represented as vectors. The extracted feature values ​​are combined with the corresponding true calibration values ​​to form a training sample set. ,in Indicates the first The feature vector of each sample For its corresponding true value, a calibration function is obtained by fitting it using a linear regression algorithm based on the training samples. , represented as:

[0069]

[0070] In the formula, For the weight vector, This is a bias term.

[0071] By minimizing the predicted value Compared with the true value The mean square error between them is used to determine the optimal value. and , represented as:

[0072]

[0073] In the formula, Mean squared error is used to quantify the overall deviation between the model's predicted values ​​and the actual values.

[0074] 1.2 Noise Reduction Processing

[0075] The moving average method is used to smooth the data and reduce noise in high-energy electron flux data, given a length of... The original data sequence Select window size as ( < The smoothed sequence is then... Calculated using the following formula:

[0076]

[0077] By selecting the appropriate window size This invention achieves a balance between noise suppression and data fidelity, and the window size... The value is 50.

[0078] 1.3 Redundancy handling

[0079] Principal component analysis (PCA) is used to perform feature dimensionality reduction on high-energy electron flux data. The core idea is to map the original high-dimensional feature space to a low-dimensional subspace through orthogonal transformation, reducing dimensionality while preserving data information to the maximum extent. The main steps include:

[0080] (1) Data standardization: The high-energy electron flux samples are centralized;

[0081] (2) Covariance matrix calculation: Analyze the linear relationship between features;

[0082] (3) Eigenvalue decomposition: Solving for the eigenvalues ​​and eigenvectors of the covariance matrix;

[0083] (4) Principal component selection: Sort by eigenvalue size and select the top eigenvalues. The eigenvectors corresponding to the largest eigenvalues ​​are used as principal components;

[0084] (5) Data projection: Mapping the original data to the selected principal component space.

[0085] Wherein: covariance quantifies the linear correlation between various feature dimensions of high-energy electron flux samples, and the formula is defined as follows:

[0086]

[0087] The formula for the covariance matrix is ​​defined as follows:

[0088]

[0089] covariance matrix elements Reflects characteristics and The linear correlation between any two features and ,when When, it indicates and There is a positive correlation, meaning that an increase in one feature is often accompanied by an increase in another feature; when When, it means and They are negatively correlated; an increase in one characteristic is usually accompanied by a decrease in the other. When, explain and Statistically, they are independent of each other. The absolute value of the covariance directly reflects the strength of the correlation between features: the larger the absolute value, the stronger the correlation; the smaller the absolute value, the weaker the correlation.

[0090] In obtaining the covariance matrix Then, its eigenvalues ​​and eigenvectors are solved through eigenvalue decomposition. and the corresponding feature vector Satisfies the characteristic equation: eigenvectors Mathematical definition of: .

[0091] identity matrix This is the core part of the characteristic equation, where each eigenvalue... There is a unique corresponding feature vector This ensures the determinism of the principal component directions. The eigenvalues... Arrange from largest to smallest, select the first... The largest eigenvalues ​​and their corresponding eigenvectors: .

[0092] Given contains A dataset of samples After a linear transformation of the projection matrix, the original high-dimensional feature space is mapped to... The principal component space is represented by the mapping process as follows:

[0093]

[0094] Based on the maximum variance theory, the optimal feature selection process is used to determine the optimal feature. 3D feature representation, from the covariance matrix Before the election The eigenvectors corresponding to the largest eigenvalues ​​constitute the eigenvectors of the eigenvalues. The foundation of 3D projection space.

[0095] 1.4 Handling Class Imbalance

[0096] Class imbalance is a common problem in machine learning, specifically referring to a significant difference in the number of samples from different classes in a dataset. This phenomenon is particularly pronounced in high-energy electron flux prediction tasks, where the number of high-energy electron (>2MeV) samples is relatively small. To quantify the degree of this imbalance, class imbalance is defined as follows: ,in, Indicates the number of low-energy electron samples. Indicates the number of high-energy electron samples. A larger value indicates a more severe imbalance in the dataset.

[0097] The core mechanism of Synthetic Minority Oversampling Technique (SMOTE) is to perform linear interpolation in the feature space of minority class samples to generate representative synthetic samples, thereby expanding the minority class sample size and achieving class distribution balance. However, this algorithm still has limitations in the precise control of the number of synthetic samples, and cannot achieve fine-grained control over the number of generated samples. To overcome the limitations of traditional SMOTE methods, an improved oversampling algorithm based on density estimation (DS-SMOTE) has been proposed. This algorithm significantly improves the feature representation ability of imbalanced datasets by introducing sample distribution density features, and shows superior performance in terms of model training efficiency and classification accuracy. However, both of these methods tend to generate excessive minority class samples, which not only increases computational complexity but may also lead to model overfitting, thereby impairing its generalization performance. In addition, the neglect of intra-class imbalance in existing methods further restricts their application in complex scenarios such as high-energy electron flux forecasting.

[0098] To address the aforementioned issues, a SMOTH optimization method based on variable-scale sample partitioning (partition-SMOTH) is proposed. The core of this method lies in the following steps: First, the sample space is divided into multiple molecular regions. Within each molecular region, a density-based spatial clustering algorithm is used for local density estimation and noise filtering, while simultaneously assigning a class label to each sample. Second, based on the clustering results, the molecular regions are further subdivided into multiple atomic regions, and adaptive SMOTE resampling based on the target value is implemented within each atomic region until class balance is achieved. Finally, global sample balance is achieved by iteratively traversing all atomic and molecular regions. This hierarchical processing strategy not only preserves the physical characteristics of the data but also achieves unified modeling from the data level to the physical level, significantly improving the robustness and interpretability of the algorithm.

[0099] S2. High-energy electron flux prediction method based on shallow model

[0100] In the field of high-energy electron flux prediction, a typical shallow learning prediction model is a prediction model based on support vector machine (SVM) combined with hyperparameter optimization method. This invention introduces a hyperparameter optimization algorithm based on particle swarm optimization to optimize the SVM model, and tests the optimization performance of the SVM model through different performance evaluation indicators.

[0101] SVM, as a classic supervised learning algorithm, has demonstrated unique advantages in predicting high-energy electron flux enhancement events. In the algorithm implementation, the choice of kernel function is crucial; commonly used mapping functions include linear kernel functions, polynomial kernel functions, and radial basis function kernel functions. The SVM algorithm can be solved using quadratic programming, as shown in the following formula:

[0102]

[0103] In the formula, and The normal vector representing the hyperplane. This indicates the bias term.

[0104] For sample points ( , The distance to the hyperplane is expressed as follows:

[0105]

[0106] The core idea of ​​SVM is to construct an optimal decision boundary that satisfies the optimization criterion of maximizing class margin, and its mathematical expression is as follows:

[0107]

[0108] In the formula The categories of each sample point are determined by express, Indicates the number of samples. Representative vector and The inner product of . This optimization problem can be transformed into its dual problem, calculated as follows:

[0109]

[0110] The constraints are satisfied:

[0111]

[0112] In the formula Represents the Lagrange multipliers. This represents the penalty constant used to control model complexity.

[0113] Finally, the classification hyperplane is represented as:

[0114]

[0115] In the formula Let represent the optimal solution to the dual problem. This represents the class of the support vectors on the boundary.

[0116] To improve the performance of the SVM model due to the complexity of the samples, a hyperparameter optimization algorithm is introduced to fine-tune the parameters of the SVM model based on data processing, thereby improving the prediction performance.

[0117] In SVM classifiers, key hyperparameter configurations mainly include the penalty constant. and kernel function. Penalty constant. This is a regularization parameter, and its value, as an important factor adjusting model complexity, directly affects the generalization ability of the classifier; while the choice of kernel function determines the mapping method of the feature space, achieving nonlinear classification by transforming the original feature space to a high-dimensional space. This is achieved through systematic adjustment... The optimal balance between model complexity and classification accuracy can be found through the selection of hyperparameters and kernel functions. Therefore, careful selection of hyperparameters is necessary to ensure the performance and accuracy of machine learning models.

[0118] Conventional hyperparameter optimization methods generally employ grid search, whose core idea is to find the hyperparameter configuration that optimizes model performance by traversing all possible combinations in a predefined hyperparameter space.

[0119] The implementation steps of the grid search method are as follows:

[0120] (1) Define the hyperparameter space: Define a set of candidate values ​​for each hyperparameter.

[0121] (2) Generate parameter grid: Combine the candidate values ​​of each hyperparameter into a multidimensional grid.

[0122] (3) Traverse the grid points: For each combination of hyperparameters in the grid, train the model and evaluate its performance. Cross-validation is usually used to evaluate the generalization performance of the model.

[0123] (4) Select the optimal combination: Select the hyperparameter combination that performs best on the validation set as the final result.

[0124] The mathematical form of the grid search method is: In the formula, Hyperparameter combination The corresponding model performance metrics.

[0125] This invention introduces a hyperparameter optimization method based on the Particle Swarm Optimization (PSO) algorithm, the specific implementation mechanism of which is as follows:

[0126] During the particle state update process, the first... The velocity vector of each particle The evolution is influenced by the inertial component maintaining the original motion trend of the particle, the cognitive component guiding the particle to approach the individual's historical optimal position, and the social component driving the particle to converge towards the group's historical optimal position. This multi-factor collaborative update mechanism can be represented by the following mathematical model:

[0127]

[0128] Inertia weight in the formula The degree of influence of controlling historical speed, cognitive coefficient social coefficient Adjust the relative importance of individual experience and group information in the search process respectively.

[0129] In the hyperparameter optimization framework, the set of hyperparameters to be optimized is modeled as a high-dimensional search space. Each particle is defined as a candidate solution in this space, and its position vector encodes a specific configuration of the hyperparameters, where each dimension corresponds to a value of a hyperparameter. This mapping relationship is formally represented as:

[0130]

[0131] During the hyperparameter optimization process, each Indicates the first The particle in the first The specific values ​​for each hyperparameter dimension are determined. A hyperparameter optimization strategy based on particle swarm optimization is employed to find the optimal solution in the parameter space through an iterative search mechanism. Its advantage lies in the fact that through information sharing and collaborative exploration among particles, the algorithm can perform a global search in the high-dimensional parameter space, effectively avoiding getting trapped in local optima. Furthermore, the parallel nature of the algorithm makes its computational efficiency significantly higher than that of the traditional naive network method.

[0132] Next, we analyze the performance of two hyperparameter optimization methods. In binary classification tasks, model performance evaluation typically relies on the confusion matrix and its derived indices. The confusion matrix consists of four key elements: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). TP represents instances where the model correctly predicts the presence of high-energy electron flux enhancement events, TN reflects instances where the model accurately identifies events without enhancement. Conversely, FP represents misclassification where the model misclassifies normal events as enhancement events, and FN indicates that the model fails to detect actual enhancement events. From a model evaluation perspective, TP and TN reflect the model's correct prediction ability; higher values ​​indicate better classification performance. FP and FN represent the model's misclassification rate; lower values ​​are better. By analyzing the proportional relationships of these indices, we can comprehensively evaluate the model's prediction accuracy and reliability across different categories.

[0133] Based on the confusion matrix analysis framework, this invention employs accuracy (ACC), recall, precision, and... The coefficients are used as performance evaluation indicators to quantify the predictive performance of the SVM model under different hyperparameter optimization methods. A controlled variable method is employed. First, in the data processing stage, all experimental groups use the same data processing method. Then, the effects of hyperparameter optimization methods are compared. Different optimization methods are set to optimize the SVM model, including: zero-based optimization, grid search method, and PSO method. Specific analysis results are combined with… Figure 2 As shown in the comparative analysis, the SVM model optimized by the PSO method exhibits significant performance across all evaluation dimensions.

[0134] S3. High-energy electron flux prediction method based on deep learning fusion model

[0135] Based on the characteristics of high-energy electron flux data samples, a deep learning fusion model design is proposed, mainly consisting of three core modules: a CNN convolutional layer designed to address the data format differences, an LSTM recurrent network layer designed to address the differences in data time scales, and an attention mechanism (AM) layer for dynamic allocation of feature weights, forming a CNN-LSTM-AM-based deep learning fusion model. Furthermore, to further optimize model performance, an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler is introduced to optimize the deep learning fusion model.

[0136] Constructing a CNN-LSTM-AM deep learning fusion model, its basic structure combines Figure 3 As shown, it includes, in sequence: CNN convolutional layers, LSTM recurrent network layers, and attention mechanism AM layers, with the specific design as follows:

[0137] CNN convolutional layers process multidimensional feature vectors obtained from time-series datasets, organize them according to time steps to construct a two-dimensional feature moment, and combine them with... Figure 4 As shown, to adapt to this structured feature representation, a feature transformation module was designed. This module contains two core components: the Flatten layer is used to flatten the high-dimensional feature vector into a one-dimensional feature vector, and the ReLU activation function enhances the feature expressive power through nonlinear transformation. Its mathematical expression is:

[0138]

[0139] For any input Its output This piecewise linear function has unique nonlinear properties: it implements hard zero truncation in the negative domain (x<0) while maintaining the identity mapping in the positive domain (x≥0).

[0140] After completing the design of the CNN convolutional layers, a max pooling strategy is used to construct pooling layers. This reduces the spatial dimension of the feature vectors through downsampling while retaining the most significant feature responses. The mathematical expression for this is:

[0141]

[0142] In the formula, As input features, For the corresponding output features, Indicates stride length. Represents the pooling size. The depth (number of channels) of the feature vector.

[0143] LSTM recurrent network layers serve as temporal feature extraction modules, combined with Figure 5 As shown, at each time step The LSTM cell updates its internal state using the following set of formulas:

[0144]

[0145] in, It is the sigmoid activation function. This represents element-wise multiplication. and These formulas, representing the weight matrix and bias term respectively, together constitute the core computational flow of LSTM, enabling fine-grained control over temporal information: forget gate. The decision on how much historical information to retain is made through the input gate. Controls the storage and output of new information. Adjust the output intensity of the current state. It is the hyperbolic tangent function. This function will convert any real number Mapped to the continuous interval [−1,1].

[0146] By fusing CNN convolutional layers with LSTM recurrent network layers, automatic extraction and learning of input data features are achieved, thereby significantly reducing the workload of traditional feature engineering. This fusion model not only improves forecast accuracy but also provides a more effective technical approach for high-energy electron flux forecasting.

[0147] The attention mechanism AM layer is used to further improve the model's feature extraction capabilities, combined with Figure 6 As shown, the attention mechanism, as a highly efficient deep learning technique, can automatically identify and learn the intrinsic correlations between data by simulating the selective attention mechanism in human cognition. This characteristic gives it a significant advantage in high-energy electron flux enhancement forecasting tasks. This invention combines the self-attention mechanism with a CNN-LSTM fusion model, enabling the model to adaptively extract key features based on the dynamic characteristics of high-energy electron flux data, significantly improving the model's adaptability to samples. The self-attention mechanism's ability to model long-range dependencies effectively solves the limitations of traditional neural networks in capturing long-distance temporal dependencies. The model provides a visual explanation of feature weights, enhancing the interpretability and user-friendliness of the prediction results. Thanks to the parallelizability of the attention mechanism, the model training and inference efficiency are greatly improved, providing computational efficiency guarantees for processing large-scale high-energy electron flux datasets. This fusion model not only improves prediction accuracy but also provides a more reliable and efficient technical solution for high-energy electron flux forecasting tasks.

[0148] Building upon this, the performance of the CNN-LSTM-AM deep learning fusion model is optimized using a learning rate optimization method. The traditional Adam optimization algorithm dynamically adjusts the learning rate of each parameter by calculating the first and second moments of the gradient, thereby accelerating convergence while avoiding getting trapped in local optima. However, while this algorithm significantly improves the model's learning efficiency, it still has some limitations, although its convergence efficiency is slightly reduced. Therefore, this invention introduces an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler, the specific process of which is as follows:

[0149] set up Based on the learning rate, This is the preset minimum learning rate threshold. Based on the maximum number of iterations. and minimum batch size The formula for calculating the total number of system iterations under these two constraints is as follows:

[0150]

[0151] After initialization, the algorithm enters the iterative optimization phase. For the... In this iteration, the corresponding cosine annealing learning rate scheduling coefficient is calculated using the following formula:

[0152]

[0153] Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and at the end of the training cycle. Specifically, in the t-th iteration, the learning rate update mechanism is as follows:

[0154]

[0155] In each training cycle After completion, the base learning rate Adaptive updates are performed according to the following formula:

[0156]

[0157] Substituting the basic learning rate update strategy from the above formula into the model parameter update formula of the Adam optimization algorithm, the improved adaptive learning rate expression can be derived as follows:

[0158]

[0159] In the formula, and These are the first-order moment estimate and the second-order moment estimate after bias correction, respectively. From the above theoretical derivation, it can be concluded that the improved algorithm architecture significantly enhances the flexibility of the adaptive learning rate by introducing a dynamic adjustment mechanism. This improvement strategy enables the improved Adam algorithm based on the dynamic learning rate scheduler to effectively solve several inherent problems of the traditional Adam algorithm.

[0160] In summary, the specific research results and innovative points of this invention can be summarized as follows:

[0161] (1) The high-energy electron flux data were systematically processed. In terms of redundancy processing, feature dimensionality reduction technology based on principal component analysis was used to process the high-energy electron flux data. Experimental results show that this method has a significant effect on reducing the feature dimension of the data and improving the prediction performance. In terms of class imbalance processing, the SMOTE optimization method based on variable-scale high-energy electron flux sample partitioning was used to process the high-energy electron flux data for class imbalance. Comparative experiments show that this method effectively reduces the class imbalance characteristics of the data and improves the accuracy and robustness of the prediction model.

[0162] (2) Based on the current research status of high-energy electron flux forecasting, the typical shallow forecasting model-SVM model in high-energy electron flux forecasting tasks was improved. The hyperparameter optimization method based on particle swarm optimization was used to optimize the SVM forecasting model. Through controlled variable experiments and comparative analysis, the significant advantages of this optimization method in improving model performance were confirmed. It effectively solved the problem of time-consuming and laborious process in traditional manual parameter tuning, significantly improved parameter configuration efficiency, and improved model performance.

[0163] (3) Based on the study of the characteristics of high-energy electron flux data samples, a high-energy electron flux forecasting method based on a deep fusion model is proposed. A CNN convolutional layer is designed to address the differences in data format, and an LSTM recurrent network layer is designed to address the differences in data time scale. Furthermore, an attention mechanism AM layer for dynamic allocation of feature weights is introduced. For the learning rate optimization problem, an improved Adam algorithm with a dynamic learning rate based on cosine annealing is adopted. Experimental results show that this optimization method effectively meets the needs of deep learning models in the high-energy electron flux enhancement event forecasting task, providing reliable technical support for space weather forecasting.

[0164] Furthermore, performance comparison tests were conducted on conventional algorithms (MLP, CNN, LSTM, GRU) and shallow models with the CNN-LSTM-AM deep learning fusion model through examples to verify the effectiveness of the deep learning fusion model of the present invention for high-energy electron flux prediction.

[0165] Example

[0166] This embodiment compares the performance of the CNN-LSTM-AM deep learning fusion model with shallow learning prediction models and other conventional algorithms (MLP, CNN, LSTM, GRU) to verify the advantages of the deep learning fusion model for high-energy electron flux prediction. The experimental data used mainly comes from NASA's official data platform (https: / / cdaweb.gsfc.nasa.gov). Specifically, the historical high-energy electron flux observation data comes from the secondary dataset of the magneto-electron ion spectrometer aboard the Van Allen Probe A satellite, spanning from March 31, 2018 to December 31, 2022. To comprehensively reflect the state of the space environment, the dataset also integrates multi-source auxiliary information, including solar wind parameters (such as solar wind speed and density) and geomagnetic indices (such as...). index, (Indices, etc.). This fusion of multi-dimensional data not only improves the completeness of the data, but also provides rich information support for building more accurate forecasting models.

[0167] The performance comparison data of other conventional algorithms (MLP, CNN, LSTM, GRU) with the deep learning fusion model of this invention is shown in the table below:

[0168]

[0169] Experimental results show that the CNN-LSTM-AM deep learning fusion model performs well in terms of ACC, Recall, Precision, and The proposed deep learning fusion model significantly outperforms the comparative model, especially in capturing long-term dependencies and handling nonlinear features, demonstrating a clear advantage and fully verifying the effectiveness and superiority of the proposed deep learning fusion model in high-energy electron flux forecasting tasks.

[0170] Performance comparison data of shallow learning prediction model and deep learning fusion model of this invention combined Figure 7 As shown, although the optimized shallow model-SVM prediction model has made great progress in the field of high-energy electron flux prediction, based on the characteristics of high-energy electron flux samples, the feature extraction capability and nonlinear modeling capability of the deep learning fusion model are more suitable for high-energy electron flux prediction. In all performance evaluation indicators, the CNN-LSTM-AM deep learning fusion model proposed in this invention is better than the shallow model.

[0171] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0172] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A high-energy electron flux prediction method based on a deep learning fusion model, characterized in that: Includes the following steps: Step 1: High-energy electron flux data acquisition and processing Data sets are acquired based on historical high-energy electron flux observation data as required, and then data processing is performed, including calibration, noise reduction, redundancy reduction, and imbalance-like processing. Redundancy handling employs principal component analysis to perform feature dimensionality reduction on high-energy electron flux data, including the following steps: (1) Data standardization: The high-energy electron flux samples are centralized; (2) Covariance matrix calculation: Analyze the linear relationship between features; (3) Eigenvalue decomposition: Solving for the eigenvalues ​​and eigenvectors of the covariance matrix; (4) Principal component selection: Sort by eigenvalue size and select the top eigenvalues. The eigenvectors corresponding to the largest eigenvalues ​​are used as principal components; (5) Data projection: Mapping the original data to the selected principal component space; Imbalanced processing employs a partitioned SMOTH optimization method to equalize the high-energy electron flux sample, including the following steps: Define class imbalance: ,in, Indicates the number of low-energy electron samples. Indicates the number of high-energy electron samples. A larger value indicates a more severe imbalance in the dataset; First, the sample space is divided into multiple molecular regions. Within each molecular region, a density-based spatial clustering algorithm is used for local density estimation and noise filtering, while each sample is assigned a class label. Second, based on the clustering results, the molecular regions are further subdivided into multiple atomic regions, and adaptive SMOTE resampling based on the target value is implemented within each atomic region until class balance is achieved. Finally, global sample balance is achieved by iteratively traversing all atomic and molecular regions. Step 2: High-energy electron flux prediction method based on deep learning fusion model A CNN-LSTM-AM deep learning fusion model is constructed, consisting of CNN convolutional layers, LSTM recurrent network layers, and an attention mechanism AM layer, as detailed below: The CNN convolutional layer organizes the processed multidimensional feature vectors into a two-dimensional feature matrix according to the time step. A feature transformation module is designed, including a Flatten layer to flatten the high-dimensional feature vectors into one-dimensional feature vectors and a ReLU activation function to enhance feature representation through nonlinear transformation. The function expression is as follows: For any input Its output Hard zero truncation is implemented in the negative domain, while the identity mapping is maintained in the positive domain. A max-pooling strategy is used to construct the pooling layer. Downsampling reduces the spatial dimension of the feature vector while retaining the most significant feature responses. Its mathematical expression is: In the formula, This indicates the first element in the output feature map after the pooling operation. line, number Column, No. The final output feature of each feature channel position It is the pooling operation function. For input features, subscript The position number corresponding to the height dimension, subscript Position number corresponding to the width dimension, subscript Indicates stride length, subscript Represents the pooling size, subscript The depth of the feature vector; The LSTM recurrent network layer serves as a temporal feature extraction module, where the LSTM unit updates its internal state at each time step.

2. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: In step one, the calibration process uses a linear regression algorithm to calibrate the data samples. In the initial stage of the calibration process, key features are extracted from the dataset and represented as vectors. , To determine the number of key features extracted from the original observation data, the extracted feature values ​​are combined with their corresponding true calibration values ​​to form a training sample set. ,in Indicates the first The feature vector of each sample The calibration function is obtained by fitting the corresponding true value using a linear regression algorithm based on the training samples. , represented as: In the formula, For the weight vector, As a bias term, minimize the predicted value Compared with the true value The mean square error between them is used to determine the optimal value. and , represented as: In the formula, This represents the mean square error.

3. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: The redundancy handling in step one specifically includes: In obtaining the covariance matrix Then, its eigenvalues ​​and eigenvectors are solved through eigenvalue decomposition. and the corresponding feature vector Satisfies the characteristic equation: eigenvectors Mathematical definition of: ,in For the identity matrix, each eigenvalue There is a unique corresponding feature vector , eigenvalue Arrange from largest to smallest, select the first... The largest eigenvalues ​​and their corresponding eigenvectors: ; Given contains A dataset of samples After a linear transformation of the projection matrix, the original high-dimensional feature space is mapped to... The principal component space is represented by the mapping process as follows: In the formula, This represents the output mapped from the original data to the principal component space; Based on the maximum variance theory, the optimal... 3D feature representation, from the covariance matrix Before being selected The eigenvectors corresponding to the largest eigenvalues.

4. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: In step two, a learning rate optimization method is introduced to optimize performance based on the CNN-LSTM-AM deep learning fusion model. The learning rate optimization method adopts the improved Adam optimization algorithm based on the cosine annealing dynamic learning rate scheduler. The specific process is as follows: set up Based on the learning rate, The preset minimum learning rate threshold is based on the maximum number of iterations. and minimum batch size The formula for calculating the total number of system iterations under these two constraints is as follows: After initialization, the algorithm enters the iterative optimization phase. For the ... In this iteration, the corresponding cosine annealing learning rate scheduling coefficient is calculated using the following formula: Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and at the end of the training cycle. The learning rate update mechanism during this iteration is as follows: In each training cycle After completion, the base learning rate Adaptive updates are performed according to the following formula: The improved adaptive learning rate expression is derived as follows: In the formula, and These are the first-order moment estimates and second-order moment estimates after bias correction, respectively.

Images