High-energy electron flux prediction method based on deep learning fusion model

Abstract

The invention discloses a high-energy electron flux prediction method based on a deep learning fusion model, and relates to the technical field of spatial physics. According to the method, a data set is acquired through historical high-energy electron flux observation data according to requirements, then a data processing process including calibration processing, noise reduction processing, redundancy processing and class imbalance processing is performed, and a CNN-LSTM-AM deep learning fusion model sequentially including a CNN convolutional layer, an LSTM cyclic network layer and an attention mechanism AM layer is constructed. Redundancy processing and class imbalance processing are added to the data, the prediction accuracy and robustness are improved, a CNN-LSTM-AM deep learning fusion model with the learning rate optimized is designed in a targeted mode, and the high-energy electron flux prediction requirement is effectively met.

Description

Technical Field

[0001] The present invention relates to the technical field of space physics, and in particular to a high-energy electron flux prediction method based on a deep learning fusion model. Background Art

[0002] As a crucial spacecraft operational region, the geosynchronous orbit hosts thousands of satellites. The safe and stable operation of these satellites is directly linked to the normal functioning of critical infrastructure such as communications and meteorological monitoring in modern society. During the recovery period of a magnetospheric substorm, a surge in high-energy electron flux persists in the geosynchronous orbit, a phenomenon known as an enhanced high-energy electron flux event (HEFEE). Due to the extremely strong penetrating power of high-energy electron flux, HEFEEs can cause deep charging effects on spacecraft, posing a serious threat to the safe operation of near-Earth satellites. Therefore, establishing a reliable high-energy electron flux forecast model can effectively help satellites take protective measures, minimizing the risk of extreme space weather to satellites and, ultimately, human life.

[0003] Current high-energy electron flux prediction techniques can be broadly categorized into traditional prediction methods based on physical mechanisms and data-driven prediction methods. Traditional methods rely heavily on expert knowledge bases, and their subjective parameterization leads to significant uncertainty in prediction results. In the field of artificial intelligence methods, the application of deep learning technology, while showing great potential, still faces challenges, specifically the following key issues:

[0004] (1) Problems in handling data sample complexity: Most existing forecast models fail to fully consider the complexity of high-energy electron flux datasets. Specifically, there is a lack of systematic analysis and treatment of issues such as feature redundancy and class imbalance that are prevalent in samples. This lack of feature engineering will directly affect the generalization performance and prediction stability of the model. Studies have shown that modeling methods that ignore the inherent characteristics of the data may lead to an increased risk of overfitting and reduce the applicability of the model under different spatial environmental conditions;

[0005] (2) The problem of a single prediction model: Existing prediction models generally use a single model to predict high-energy electron flux. However, high-energy electron flux data itself has many uncertainties. Therefore, a single model cannot integrate the complexity of high-energy electron flux data. It is necessary to consider using a multi-model fusion method to make a more accurate prediction of high-energy electron flux based on the characteristics of high-energy electron flux samples.

[0006] (3) Performance optimization of prediction models: Although traditional machine learning methods have obvious advantages in computational efficiency, their prediction accuracy often fails to meet the needs of practical applications. In contrast, prediction models based on deep learning have made significant progress in prediction accuracy through hierarchical feature extraction and representation learning. However, due to limitations in model architecture design and training strategies, the performance of existing deep learning methods still has room for optimization, especially in dealing with the nonlinear dynamic characteristics of high-energy electron flux. Further algorithm improvements are needed to improve prediction accuracy.

[0007] In summary, the ability to predict high-energy electron flux is closely linked to the development of space weather forecasting systems and is a crucial technical support for ensuring the safe operation of space infrastructure. To meet this demand, it is imperative to develop a high-precision high-energy electron flux prediction model to mitigate the impact of the space radiation environment on spacecraft. This is crucial for critical decisions such as mission planning and orbit design. Summary of the Invention

[0008] In order to address the shortcomings of the background technology, the present invention provides a high-energy electron flux prediction method based on a deep learning fusion model, which adds redundancy processing and class imbalance processing to the data, improves the accuracy and robustness of the prediction, and specifically designs a CNN-LSTM-AM deep learning fusion model with optimized learning rate to effectively meet the needs of high-energy electron flux prediction.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a high-energy electron flux prediction method based on a deep learning fusion model, comprising the following steps:

[0010] Step 1: High-energy electron flux data acquisition and processing

[0011] According to the requirements, a data set is obtained through historical high-energy electron flux observation data, and then a data processing process is performed, including calibration processing, noise reduction processing, redundancy processing and class imbalance processing, where:

[0012] The redundancy processing uses principal component analysis to reduce the feature dimension of high-energy electron flux data, including the following steps:

[0013] (1) Data standardization: centralize high-energy electron flux samples;

[0014] (2) Covariance matrix calculation: Analyze the linear relationship between features;

[0015] (3) Eigendecomposition: solving the eigenvalues ​​and eigenvectors of the covariance matrix;

[0016] (4) Principal component selection: Sort by eigenvalue and select the top The eigenvector corresponding to the largest eigenvalue is taken as the principal component;

[0017] (5) Data projection: mapping the original data to the selected principal component space;

[0018] Class imbalance processing uses the partition-SMOTH optimization method to equalize the high-energy electron flux sample, including the following steps:

[0019] Define class imbalance: ,in, represents the number of low-energy electron samples, represents the number of high-energy electron samples, A larger value indicates that the dataset is more imbalanced;

[0020] First, the sample space is divided into multiple molecular regions. A density-based spatial clustering algorithm is used in each molecular region to perform local density estimation and noise filtering, while assigning a class label to each sample. Second, based on the clustering results, the molecular region is further subdivided into multiple atomic regions. Adaptive SMOTE resampling based on the target value is implemented in each atomic region until class balance is achieved. Finally, global sample balancing is achieved by iteratively traversing all atomic and molecular regions.

[0021] Step 2: High-energy electron flux prediction method based on deep learning fusion model

[0022] Construct a CNN-LSTM-AM deep learning fusion model, which includes the CNN convolutional layer, the LSTM recurrent network layer, and the attention mechanism AM layer in sequence. The specific design is as follows:

[0023] The CNN convolution layer constructs a two-dimensional feature moment based on the multi-dimensional feature vector obtained by processing according to the time step, and designs a feature conversion module, which includes a Flatten layer for flattening high-dimensional feature vectors into one-dimensional feature vectors and a ReLU activation function for enhancing feature expression capabilities through nonlinear transformation. The function expression is:

[0024]

[0025] For any input Its output , hard zero truncation is implemented in the negative domain, while maintaining the identity mapping in the positive domain. The maximum pooling strategy is used to construct the pooling layer, which reduces the spatial dimension of the feature vector through downsampling operations while retaining the most significant feature responses. Its mathematical expression is:

[0026]

[0027] Where, is the input feature, is the corresponding output feature, Indicates stride length, represents the pooling size, is the depth of the feature vector;

[0028] The LSTM recurrent network layer is used as a temporal feature extraction module, and at each time step , the LSTM unit updates its internal state through the following formula group:

[0029]

[0030] in, is the sigmoid activation function, represents element-wise multiplication, and denote the weight matrix and bias term respectively, represents the forget gate, represents the input gate, represents the output gate, is the hyperbolic tangent function, and any real number Mapped to the continuous interval [-1,1].

[0031] Furthermore, the calibration process in step 1 is based on the linear regression algorithm to calibrate the data sample. In the initial stage of the calibration process, key features are extracted from the data set and represented as vectors. , the extracted feature values ​​and the corresponding true calibration values ​​form a training sample set ,in Indicates the The feature vector of the sample, The calibration function is obtained by fitting the linear regression algorithm based on the training samples to its corresponding true value. , expressed as: , where is the weight vector, is the bias term, which is obtained by minimizing the predicted value and the true value The mean square error between the two determines the optimal and , expressed as: , where represents the mean square error.

[0032] Furthermore, the noise reduction process in step 1 uses the moving average method to smooth the data, and the given length is The original data sequence , select the window size as , then the smoothed sequence pass Calculate and select window size Strike a balance between noise suppression and data fidelity.

[0033] Furthermore, the redundancy processing in step 1 specifically includes:

[0034] Covariance is defined as follows:

[0035]

[0036] The covariance matrix is ​​defined as follows:

[0037]

[0038] In obtaining the covariance matrix Then, the eigenvalue and eigenvector are solved by eigendecomposition. and the corresponding eigenvector Satisfies the characteristic equation: , the eigenvector Mathematical definition of: ,in is the identity matrix, each eigenvalue There is a unique corresponding eigenvector , the eigenvalue Arrange from largest to smallest, select the The largest eigenvalues ​​and corresponding eigenvectors: ;

[0039] Given contains A dataset of samples , after linear transformation of the projection matrix, the original high-dimensional feature space is mapped to dimensional principal component space, the mapping process is expressed as:

[0040]

[0041] Based on the maximum variance theory, determine the optimal dimensional feature representation, from the covariance matrix Before the selection The eigenvector corresponding to the largest eigenvalue.

[0042] Furthermore, in step 2, a learning rate optimization method is introduced based on the CNN-LSTM-AM deep learning fusion model to optimize performance. The learning rate optimization method adopts an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler. The specific process is as follows:

[0043] set up is the basic learning rate, is the preset minimum learning rate threshold, based on the maximum number of iterations and minimum batch size With these two constraints, the total number of iterations of the system is calculated as follows:

[0044]

[0045] After the initialization is completed, the algorithm enters the iterative optimization stage. The corresponding cosine annealing learning rate scheduling coefficient is calculated by the following formula:

[0046]

[0047] Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and the training cycle. During the tth iteration, the learning rate update mechanism is expressed as follows:

[0048]

[0049] In each training cycle After completion, the base learning rate Adaptive update is performed as follows:

[0050]

[0051] The improved adaptive learning rate expression is derived as follows:

[0052]

[0053] Where, and are the first-order moment estimates and second-order moment estimates after bias correction, respectively.

[0054] Compared with the prior art, the beneficial effect of the present invention is that in the data processing process, in addition to conventional calibration processing and noise reduction processing, the present invention also performs redundancy processing on the data based on the feature dimensionality reduction technology of the principal component analysis method and performs class imbalance processing based on the partition-SMOTH optimization method, which has obvious effects in reducing the feature dimension of the data and improving the prediction performance, effectively reducing the class imbalance characteristics of the data, and improving the accuracy and robustness of the forecast model. In addition, a CNN convolution layer is designed to address the problem of data format differences, an LSTM recurrent network layer is designed according to the difference in data time scale, and an attention mechanism AM layer is designed for dynamic allocation of feature weights to form a CNN-LSTM-AM deep learning fusion model, and the model is optimized by using the dynamic learning rate Adam improved algorithm based on the cosine annealing mechanism to effectively meet the needs of the high-energy electron flux enhancement event prediction task, and provide reliable technical support for space weather forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a technical route diagram of the prediction method of the present invention;

[0056] Figure 2 This is a performance evaluation data graph of different hyperparameter optimization methods for the SVM model in the present invention;

[0057] Figure 3 This is a structural diagram of the CNN-LSTM-AM deep learning fusion model in the present invention;

[0058] Figure 4 It is a structural diagram of the feature conversion module of the CNN convolutional layer in the present invention;

[0059] Figure 5 It is a structural diagram of the LSTM recurrent network layer in the present invention;

[0060] Figure 6 This is the structural diagram of the attention mechanism AM layer in the present invention;

[0061] Figure 7 This is a performance comparison data chart of the deep learning fusion model of the present invention and the shallow model in the embodiment. DETAILED DESCRIPTION

[0062] The technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] The invention is derived from the sub-project of the National Key R&D Program - Spatial Integrated Environmental Effect Modeling Analysis (NO.2022YFF0503601), which aims to improve the reliability and accuracy of the prediction model. The specific technical route is combined with Figure 1 As shown in the figure. First, a systematic data processing process was constructed. After completing the conventional data calibration and noise reduction processing, the feature redundancy and class imbalance of the data were processed. Subsequently, a high-energy electron flux prediction method based on a shallow model was designed. The high-energy electron flux prediction model based on SVM was improved and optimized using a hyperparameter optimization method based on a particle swarm algorithm. Then, based on the analysis of the characteristics of high-energy electron flux samples, a high-energy electron flux prediction method based on a deep learning fusion model was proposed. In order to further improve the performance of the model, a learning rate optimization method was introduced to analyze and test the model performance. Finally, a performance comparison test was conducted between the shallow learning model and the deep learning fusion model to verify the progressiveness and feasibility of the deep learning fusion model.

[0064] S1. High-energy electron flux data acquisition and processing

[0065] First, a data set is acquired, followed by data processing. In addition to calibration and noise reduction, the present invention also addresses redundancy and class imbalance. This systematic data processing approach ensures the performance of subsequent models by using a linear regression algorithm to calibrate data samples, a moving average method to reduce noise, a principal component analysis-based feature dimensionality reduction technique to address redundancy, and a partition-SMOTH optimization method to address class imbalance.

[0066] According to the requirements, the data set is obtained through historical high-energy electron flux observation data, and the data is processed, including calibration processing, noise reduction processing, redundancy processing and class imbalance processing, which are respectively expressed as follows:

[0067] 1.1 Calibration Processing

[0068] The calibration model is constructed based on the linear regression algorithm. In the initial stage of the calibration process, key features are extracted from the data set and represented as vectors. , the extracted feature values ​​and the corresponding true calibration values ​​form a training sample set ,in Indicates the The feature vector of the sample, The calibration function is obtained by fitting the linear regression algorithm based on the training samples to its corresponding true value. , expressed as:

[0069]

[0070] Where, is the weight vector, is the bias term.

[0071] By minimizing the predicted value and the true value The mean square error between the two determines the optimal and , expressed as:

[0072]

[0073] Where, Represents the mean squared error, which quantifies the overall deviation between the model's predicted values ​​and the true values.

[0074] 1.2 Noise Reduction

[0075] The moving average method is used to smooth the data to reduce the noise in the high-energy electron flux data. The given length is The original data sequence , select the window size as ( < ), then the smoothed sequence Calculated by the following formula:

[0076]

[0077] By choosing the window size appropriately , can strike a balance between noise suppression and data fidelity, the window size in this invention is The value of is 50.

[0078] 1.3 Redundancy Processing

[0079] Principal component analysis (PCA) is used to perform feature dimensionality reduction on high-energy electron flux data. The core idea is to map the original high-dimensional feature space to a low-dimensional subspace through orthogonal transformation, thereby reducing the dimensionality while retaining the data information to the maximum extent. The main steps include:

[0080] (1) Data standardization: centralize high-energy electron flux samples;

[0081] (2) Covariance matrix calculation: Analyze the linear relationship between features;

[0082] (3) Eigendecomposition: solving the eigenvalues ​​and eigenvectors of the covariance matrix;

[0083] (4) Principal component selection: Sort by eigenvalue and select the top The eigenvector corresponding to the largest eigenvalue is taken as the principal component;

[0084] (5) Data projection: Map the original data to the selected principal component space.

[0085] Among them, the covariance quantifies the linear correlation between the characteristic dimensions of the high-energy electron flux sample, and the formula is defined as follows:

[0086]

[0087] The covariance matrix formula is defined as follows:

[0088]

[0089] Covariance matrix Elements Reflects the characteristics and The linear correlation between any two features and ,when When and There is a positive correlation, that is, the increase of one feature is often accompanied by the increase of another feature; when When and There is a negative correlation, an increase in one feature is usually accompanied by a decrease in another feature; when When, explain and Statistically independent of each other. The absolute value of the covariance directly reflects the strength of the correlation between features: the larger the absolute value, the stronger the correlation; the smaller the absolute value, the weaker the correlation.

[0090] In obtaining the covariance matrix Then, the eigenvalue and eigenvector are solved by eigendecomposition. and the corresponding eigenvector Satisfies the characteristic equation: , the eigenvector Mathematical definition of: .

[0091] Identity matrix is the core part of the characteristic equation, where each eigenvalue There is a unique corresponding eigenvector , ensuring the certainty of the principal component direction. Arrange from largest to smallest, select the The largest eigenvalues ​​and corresponding eigenvectors: .

[0092] Given contains A dataset of samples , after linear transformation of the projection matrix, the original high-dimensional feature space is mapped to dimensional principal component space, the mapping process is expressed as:

[0093]

[0094] Based on the maximum variance theory, the optimal dimensional feature representation, from the covariance matrix Before the selection The eigenvectors corresponding to the largest eigenvalues ​​constitute The basis of dimensional projective space.

[0095] 1.4. Dealing with Class Imbalance

[0096] Class imbalance is a common problem in machine learning, specifically when the number of samples from different classes in a dataset differs significantly. This phenomenon is particularly evident in the high-energy electron flux prediction task, where the number of high-energy electron (>2MeV) samples is relatively small. To quantify the degree of this imbalance, we define the class imbalance degree: ,in, represents the number of low-energy electron samples, represents the number of high-energy electron samples, Larger values ​​indicate that the dataset is more imbalanced.

[0097] The core mechanism of the synthetic minority oversampling technique (SMOTE) is to perform linear interpolation in the feature space of minority class samples to generate representative synthetic samples, thereby expanding the minority class sample size and achieving a balanced class distribution. However, this algorithm still has limitations in precisely regulating the number of synthetic samples, and cannot achieve fine-grained control over the number of generated samples. To overcome the limitations of the traditional SMOTE method, an improved oversampling algorithm based on density estimation (DS-SMOTE) has been proposed. By incorporating sample distribution density features, this algorithm significantly improves the feature representation capability of imbalanced datasets, demonstrating superior performance in both model training efficiency and classification accuracy. However, both methods tend to generate an excessive number of minority class samples, which not only increases computational complexity but also may lead to model overfitting, thereby compromising generalization performance. In addition, the existing methods' neglect of intra-class imbalance further restricts their application in complex scenarios such as high-energy electron flux forecasting.

[0098] To address these issues, a SMOTH optimization method based on variable-scale sample partitioning (Partition-SMOTH) was proposed. The core of this method is to first divide the sample space into multiple molecular regions. Within each molecular region, a density-based spatial clustering algorithm is used for local density estimation and noise filtering, while simultaneously assigning a class label to each sample. Secondly, based on the clustering results, the molecular region is further subdivided into multiple atomic regions. Within each atomic region, adaptive SMOTE resampling based on the target value is implemented until class balance is achieved. Finally, global sample balancing is achieved by iteratively traversing all atomic and molecular regions. This hierarchical processing strategy not only preserves the physical properties of the data but also achieves unified modeling from the data level to the physical level, significantly improving the robustness and interpretability of the algorithm.

[0099] S2. High-energy electron flux prediction method based on shallow model

[0100] A typical shallow learning prediction model in the field of high-energy electron flux prediction is a prediction model based on support vector machine (SVM) combined with a hyperparameter optimization method. This paper introduces a hyperparameter optimization algorithm based on particle swarm optimization to optimize the SVM model, and tests the optimization performance of the SVM model using different performance evaluation indicators.

[0101] As a classic supervised learning algorithm, SVM has demonstrated unique advantages in predicting high-energy electron flux enhancement events. The choice of kernel function is crucial in algorithm implementation. Commonly used mapping functions include linear kernel functions, polynomial kernel functions, and radial basis kernel functions. The SVM algorithm can be solved using the quadratic programming method, as shown in the following formula:

[0102]

[0103] Where, and represents the normal vector of the hyperplane, represents the bias term.

[0104] For the sample points ( , ) and the distance between the hyperplane is expressed as follows:

[0105]

[0106] The core idea of ​​SVM is to construct an optimal decision boundary that satisfies the optimization criterion of maximizing the class interval. Its mathematical expression is as follows:

[0107]

[0108] In the formula The categories of the sample points are express, represents the number of samples, Representative vector and This optimization problem can be transformed into its dual problem, which is calculated as follows:

[0109]

[0110] Satisfy the constraints:

[0111]

[0112] In the formula represents the Lagrange multiplier, Represents the penalty constant to control the model complexity.

[0113] Finally, the classification hyperplane is expressed as:

[0114]

[0115] In the formula represents the optimal solution to the dual problem, Represents the categories of support vectors on the boundary.

[0116] Based on the complexity of samples, in order to improve the algorithm performance of the SVM model, a hyperparameter optimization algorithm is introduced on the basis of data processing to tune the parameters of the SVM model to improve the prediction performance.

[0117] In the SVM classifier, the key hyperparameter configuration mainly includes the penalty constant and kernel function. Penalty constant It is a regularization parameter, which is an important factor in regulating the complexity of the model. Its value directly affects the generalization ability of the classifier. The choice of kernel function determines the mapping method of the feature space, and realizes nonlinear classification by converting the original feature space into a high-dimensional space. By systematically adjusting The value and kernel function can find the optimal balance between model complexity and classification accuracy. Therefore, hyperparameters need to be carefully selected to ensure the performance and accuracy of machine learning models.

[0118] Conventional hyperparameter optimization methods generally use the grid search method, the core idea of ​​which is to find the hyperparameter configuration that optimizes model performance by traversing all possible combinations in the predefined hyperparameter space.

[0119] The implementation steps of the grid search method are as follows:

[0120] (1) Define the hyperparameter space: define a set of candidate values ​​for each hyperparameter.

[0121] (2) Generate parameter grid: Combine the candidate values ​​of each hyperparameter into a multidimensional grid.

[0122] (3) Traverse the grid points: For each hyperparameter combination in the grid, train the model and evaluate its performance. Cross-validation is usually used to evaluate the generalization performance of the model.

[0123] (4) Select the optimal combination: Select the hyperparameter combination with the best performance on the validation set as the final result.

[0124] The mathematical form of the grid search method is expressed as: , where Represents a hyperparameter combination Corresponding model performance indicators.

[0125] This paper introduces a hyperparameter optimization method based on the particle swarm optimization (PSO), and its specific implementation mechanism is as follows:

[0126] During the particle state update process, The velocity vector of a particle The evolution of is influenced by the inertial component that maintains the original motion trend of the particles, the cognitive component that guides the particles to approach the individual's historical optimal position, and the social component that drives the particles to converge to the group's historical optimal position. This multi-factor coordinated update mechanism can be expressed as a mathematical model as follows:

[0127]

[0128] Where inertia weight Control the influence of historical speed, cognitive coefficient and social coefficient Respectively regulate the relative importance of individual experience and group information in the search process.

[0129] In the hyperparameter optimization framework, the set of hyperparameters to be optimized is modeled as a high-dimensional search space. Each particle is defined as a candidate solution in this space, and its position vector encodes a specific configuration of the hyperparameters. Each dimension corresponds to a value of the hyperparameter. This mapping relationship is formally expressed as:

[0130]

[0131] During the hyperparameter optimization process, each Indicates the The particle in The specific values ​​of the hyperparameter dimensions are determined by the particle swarm algorithm. A hyperparameter optimization strategy based on the particle swarm algorithm is used to find the optimal solution in the parameter space through an iterative search mechanism. Its advantage lies in that through information sharing and collaborative exploration between particles, the algorithm can conduct a global search in the high-dimensional parameter space, effectively avoiding falling into the local optimum. In addition, the parallel nature of the algorithm makes its computational efficiency significantly higher than that of the traditional naive network method.

[0132] Next, we analyze the performance of the two hyperparameter optimization methods. In binary classification tasks, model performance evaluation typically relies on the confusion matrix and its derived metrics. The confusion matrix consists of four key elements: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). TP represents instances where the model correctly predicts the presence of high-energy electron flux enhancement events, while TN reflects instances where the model accurately identifies no enhancement events. In contrast, FP represents instances where the model misclassifies normal events as enhancement events, while FN indicates instances where the model fails to detect actual enhancement events. From a model evaluation perspective, TP and TN reflect the model's ability to correctly predict, with higher values ​​indicating better classification performance. FP and FN, on the other hand, represent instances where the model misclassifies, with lower values ​​indicating better performance. By analyzing the proportional relationship between these metrics, we can comprehensively assess the model's predictive accuracy and reliability across different categories.

[0133] Based on the analysis framework of confusion matrix, this paper adopts accuracy (ACC), recall rate (Recall), precision (Precision) and The coefficient is used as a performance evaluation index to quantify the prediction performance of the SVM model under different hyperparameter optimization methods. The control variable method is used. First, in the data processing stage, all experimental groups use the same data processing method. Then, the effects of the hyperparameter optimization methods are compared. Different optimization methods are set to optimize the SVM model, including: 0 processing, grid search method and PSO method. The specific analysis results are combined with Figure 2 As shown in the figure, through comparative analysis, it can be seen that the SVM model optimized by the PSO method shows significant performance in all evaluation dimensions.

[0134] S3. High-energy electron flux prediction method based on deep learning fusion model

[0135] Based on the characteristics of high-energy electron flux data samples, a deep learning fusion model design scheme is proposed. It mainly consists of three core modules: a CNN convolutional layer designed to address data format variability, an LSTM recurrent network layer designed to address data time scale variability, and an attention mechanism (AM) layer for dynamic feature weight allocation. This forms a deep learning fusion model based on CNN-LSTM-AM. In addition, to further optimize model performance, an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler is introduced to optimize the deep learning fusion model.

[0136] Construct a CNN-LSTM-AM deep learning fusion model, whose basic structure combines Figure 3 As shown in the figure, it includes: CNN convolutional layer, LSTM recurrent network layer and attention mechanism AM layer in sequence. The specific design is as follows:

[0137] The CNN convolutional layer processes the multidimensional feature vector obtained by the time series data set and constructs a two-dimensional feature moment by time step. Figure 4 As shown in Figure 1, to adapt to this structured feature representation, a feature conversion module is designed. The module contains two core components: the Flatten layer is used to flatten the high-dimensional feature vector into a one-dimensional feature vector, and the ReLU activation function enhances the feature expression ability through nonlinear transformation. Its mathematical expression is:

[0138]

[0139] For any input Its output This piecewise linear function has a unique nonlinear property: hard zero truncation is implemented in the negative domain (x<0), while maintaining the identity mapping in the positive domain (x≥0).

[0140] After completing the design of the CNN convolutional layer, the maximum pooling strategy is used to construct the pooling layer. The spatial dimension of the feature vector is reduced by downsampling operation while retaining the most significant feature response. Its mathematical expression is:

[0141]

[0142] Where, is the input feature, is the corresponding output feature, Indicates stride length, represents the pooling size, is the depth (number of channels) of the feature vector.

[0143] LSTM recurrent network layer is used as a temporal feature extraction module, combined with Figure 5 As shown, at each time step , the LSTM unit updates its internal state through the following formula group:

[0144]

[0145] in, is the sigmoid activation function, represents element-wise multiplication, and Represent the weight matrix and bias term respectively. These formulas together constitute the core calculation process of LSTM, achieving fine control of timing information: forget gate Determine how much historical information to retain, input gate Controls the storage of new information, output gate Adjust the output intensity of the current state, is the hyperbolic tangent function, , which converts any real number Mapped to the continuous interval [-1,1].

[0146] By fusing the CNN convolutional layer with the LSTM recurrent network layer, automatic extraction and learning of input data features are achieved, thereby significantly reducing the workload of traditional feature engineering. This fusion model not only improves the prediction accuracy, but also provides a more effective technical approach for high-energy electron flux prediction.

[0147] The attention mechanism AM layer is used to further improve the feature extraction capability of the model. Figure 6 As shown, the attention mechanism, as an efficient deep learning technology, can automatically identify and learn the intrinsic correlation between data by simulating the selective attention mechanism in the human cognitive process. This feature makes it show significant advantages in the task of high-energy electron flux enhancement prediction. The present invention combines the self-attention mechanism with the CNN-LSTM fusion model, so that the model can adaptively extract key features according to the dynamic characteristics of high-energy electron flux data, significantly improving the adaptability of the model to samples; through the self-attention mechanism's ability to model long-range dependencies, it effectively solves the limitations of traditional neural networks in capturing long-range temporal dependencies; the model provides a visual explanation of feature weights, enhancing the interpretability and user-friendliness of the prediction results; thanks to the parallelization characteristics of the attention mechanism, the model training and reasoning efficiency are greatly improved, providing computational efficiency guarantees for processing large-scale high-energy electron flux data sets. This fusion model not only improves the prediction accuracy, but also provides a more reliable and efficient technical solution for the high-energy electron flux prediction task.

[0148] On this basis, the performance of the CNN-LSTM-AM deep learning fusion model is optimized through the learning rate optimization method. The traditional Adam optimization algorithm dynamically adjusts the learning rate of each parameter by calculating the first-order moment and second-order moment estimation of the gradient, thereby accelerating convergence while avoiding falling into the local optimum. However, this algorithm significantly improves the learning efficiency of the model, but slightly reduces the convergence efficiency, and still has several limitations. Therefore, the present invention introduces an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler. The specific process is as follows:

[0149] set up is the basic learning rate, The preset minimum learning rate threshold. Based on the maximum number of iterations and minimum batch size With these two constraints, the total number of iterations of the system is calculated as follows:

[0150]

[0151] After the initialization is completed, the algorithm enters the iterative optimization stage. The corresponding cosine annealing learning rate scheduling coefficient is calculated by the following formula:

[0152]

[0153] Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and the end of the training cycle. Specifically, during the tth iteration, the learning rate update mechanism is expressed as follows:

[0154]

[0155] In each training cycle After completion, the base learning rate Adaptive update is performed as follows:

[0156]

[0157] Substituting the basic learning rate update strategy in the above formula into the model parameter update formula of the Adam optimization algorithm, the improved adaptive learning rate expression can be derived as follows:

[0158]

[0159] Where, and They are the first-order moment estimation and second-order moment estimation after bias correction, respectively. From the above theoretical derivation, it can be concluded that the improved algorithm architecture significantly enhances the flexibility of the adaptive learning rate by introducing a dynamic adjustment mechanism. This improvement strategy enables the improved Adam algorithm based on the dynamic learning rate scheduler to effectively solve several inherent problems of the traditional Adam algorithm.

[0160] In summary, the specific research results and innovations of the present invention can be summarized as follows:

[0161] (1) The high-energy electron flux data were systematically processed. In terms of redundancy processing, the feature dimensionality reduction technology based on principal component analysis was used to process the redundancy of the high-energy electron flux data. The experimental results show that this method has a significant effect in reducing the feature dimension of the data and improving the prediction performance. In terms of class imbalance processing, the SMOTE optimization method based on variable-scale high-energy electron flux sample partitioning was used to process the class imbalance of the high-energy electron flux data. The comparative experiments show that this method effectively reduces the class imbalance characteristics of the data and improves the accuracy and robustness of the prediction model.

[0162] (2) Based on the current research status of high-energy electron flux prediction, the SVM model, a typical shallow prediction model in high-energy electron flux prediction tasks, was improved. The SVM prediction model was optimized using a hyperparameter optimization method based on the particle swarm algorithm. Through control variable experiments and comparative analysis, the significant advantages of this optimization method in improving model performance were confirmed. It effectively solved the time-consuming and labor-intensive problem of traditional manual parameter adjustment, significantly improved the parameter configuration efficiency, and improved the model performance.

[0163] (3) Based on the study of the characteristics of high-energy electron flux data samples, a high-energy electron flux prediction method based on a deep fusion model was proposed. The CNN convolutional layer was designed to address the differences in data formats, and the LSTM recurrent network layer was designed based on the differences in data time scales. On this basis, an attention mechanism AM layer for dynamic feature weight allocation was introduced. To address the learning rate optimization problem, an improved dynamic learning rate Adam algorithm based on the cosine annealing mechanism was adopted. Experimental results show that this optimization method effectively meets the requirements of deep learning models in the prediction of high-energy electron flux enhancement events, providing reliable technical support for space weather forecasting.

[0164] In addition, the performance comparison test of other conventional algorithms (MLP, CNN, LSTM, GRU) and shallow models with the CNN-LSTM-AM deep learning fusion model was carried out through the embodiments to verify the effectiveness of the deep learning fusion model of the present invention in predicting high-energy electron flux.

[0165] Example

[0166] This embodiment compares the performance of the CNN-LSTM-AM deep learning fusion model with the shallow learning prediction model and other conventional algorithms (MLP, CNN, LSTM, GRU) to verify the advantages of the deep learning fusion model for high-energy electron flux prediction. The experimental data used are mainly from the official data platform of the National Aeronautics and Space Administration (NASA) (https: / / cdaweb.gsfc.nasa.gov). Specifically, the historical high-energy electron flux observation data comes from the secondary data set of the magnetic electron ion spectrometer on the Van Allen Probe A satellite, covering the period from March 31, 2018 to December 31, 2022. In order to fully reflect the state of the space environment, the data set also integrates multi-source auxiliary information, including solar wind parameters (such as solar wind speed, density, etc.) and geomagnetic indices (such as index, This fusion of multi-dimensional data not only improves the integrity of the data, but also provides rich information support for building more accurate forecast models.

[0167] The performance comparison data of other conventional algorithms (MLP, CNN, LSTM, GRU) and the deep learning fusion model of the present invention are shown in the following table:

[0168]

[0169] Experimental results show that the CNN-LSTM-AM deep learning fusion model has good performance in ACC, Recall, Precision, The above models are significantly better than the comparison models, especially in capturing long-term dependencies and processing nonlinear characteristics, which fully verifies the effectiveness and superiority of the deep learning fusion model proposed in this invention in the high-energy electron flux prediction task.

[0170] Performance comparison data of shallow learning prediction model and deep learning fusion model of the present invention Figure 7 As shown, it can be seen that although the optimized shallow model - SVM prediction model has made great progress in the field of high-energy electron flux prediction, based on the characteristics of high-energy electron flux samples, the feature extraction ability and nonlinear modeling ability of the deep learning fusion model are more suitable for high-energy electron flux prediction. In various performance evaluation indicators, the CNN-LSTM-AM deep learning fusion model proposed in this invention is better than the shallow model.

[0171] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other configurations without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations coming within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0172] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A high-energy electron flux prediction method based on a deep learning fusion model, characterized by: The following steps are involved: Step 1: High-energy electron flux data acquisition and processing According to the requirements, a data set is obtained through historical high-energy electron flux observation data, and then a data processing process is performed, including calibration processing, noise reduction processing, redundancy processing and class imbalance processing, where: The redundancy processing uses principal component analysis to reduce the feature dimension of high-energy electron flux data, including the following steps: (1) Data standardization: centralize high-energy electron flux samples; (2) Covariance matrix calculation: Analyze the linear relationship between features; (3) Eigendecomposition: solving the eigenvalues ​​and eigenvectors of the covariance matrix; (4) Principal component selection: Sort by eigenvalue and select the top The eigenvector corresponding to the largest eigenvalue is taken as the principal component; (5) Data projection: mapping the original data to the selected principal component space; Class imbalance processing uses the partition-SMOTH optimization method to equalize the high-energy electron flux sample, including the following steps: Define class imbalance: ,in, represents the number of low-energy electron samples, represents the number of high-energy electron samples, A larger value indicates that the dataset is more imbalanced; First, the sample space is divided into multiple molecular regions. A density-based spatial clustering algorithm is used in each molecular region to perform local density estimation and noise filtering, while assigning a class label to each sample. Second, based on the clustering results, the molecular region is further subdivided into multiple atomic regions. Adaptive SMOTE resampling based on the target value is implemented in each atomic region until class balance is achieved. Finally, global sample balancing is achieved by iteratively traversing all atomic and molecular regions. Step 2: High-energy electron flux prediction method based on deep learning fusion model Construct a CNN-LSTM-AM deep learning fusion model, which includes the CNN convolutional layer, the LSTM recurrent network layer, and the attention mechanism AM layer in sequence. The specific design is as follows: The CNN convolution layer constructs a two-dimensional feature moment based on the multi-dimensional feature vector obtained by processing according to the time step, and designs a feature conversion module, which includes a Flatten layer for flattening high-dimensional feature vectors into one-dimensional feature vectors and a ReLU activation function for enhancing feature expression capabilities through nonlinear transformation. The function expression is: For any input Its output , hard zero truncation is implemented in the negative domain, while maintaining the identity mapping in the positive domain. The maximum pooling strategy is used to construct the pooling layer, which reduces the spatial dimension of the feature vector through downsampling operations while retaining the most significant feature responses. Its mathematical expression is: Where, is the input feature, is the corresponding output feature, Indicates stride length, represents the pooling size, is the depth of the feature vector; The LSTM recurrent network layer is used as a temporal feature extraction module, and at each time step , the LSTM unit updates its internal state through the following formula group: in, is the sigmoid activation function, represents element-wise multiplication, and denote the weight matrix and bias term respectively, represents the forget gate, represents the input gate, represents the output gate, is the hyperbolic tangent function, and any real number Mapped to the continuous interval [-1,1].

2. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1 is characterized in that: The calibration process in step 1 is to calibrate the data sample based on the linear regression algorithm. In the initial stage of the calibration process, key features are extracted from the data set and represented as vectors. , the extracted feature values ​​and the corresponding true calibration values ​​form a training sample set ,in Indicates the The feature vector of the sample, The calibration function is obtained by fitting the linear regression algorithm based on the training samples to its corresponding true value. , expressed as: , where is the weight vector, is the bias term, which is obtained by minimizing the predicted value and the true value The mean square error between the two determines the optimal and , expressed as: , where represents the mean square error.

3. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: In the step 1, the noise reduction process uses the moving average method to smooth the data. The given length is The original data sequence , select the window size as , then the smoothed sequence pass Calculate and select window size Strike a balance between noise suppression and data fidelity.

4. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: The redundancy processing in step 1 specifically includes: Covariance is defined as follows: The covariance matrix is ​​defined as follows: In obtaining the covariance matrix Then, the eigenvalue and eigenvector are solved by eigendecomposition. and the corresponding eigenvector Satisfies the characteristic equation: , the eigenvector Mathematical definition of: ,in is the identity matrix, each eigenvalue There is a unique corresponding eigenvector , the eigenvalue Arrange from largest to smallest, select the The largest eigenvalues ​​and corresponding eigenvectors: ; Given contains A dataset of samples , after linear transformation of the projection matrix, the original high-dimensional feature space is mapped to dimensional principal component space, the mapping process is expressed as: Based on the maximum variance theory, determine the optimal dimensional feature representation, from the covariance matrix Before the selection The eigenvector corresponding to the largest eigenvalue.

5. The high-energy electron flux prediction method based on a deep learning fusion model according to claim 1, characterized in that: In step 2, a learning rate optimization method is introduced based on the CNN-LSTM-AM deep learning fusion model to optimize performance. The learning rate optimization method adopts an improved Adam optimization algorithm based on a cosine annealing dynamic learning rate scheduler. The specific process is as follows: set up is the basic learning rate, is the preset minimum learning rate threshold, based on the maximum number of iterations and minimum batch size With these two constraints, the total number of iterations of the system is calculated as follows: After the initialization is completed, the algorithm enters the iterative optimization stage. The corresponding cosine annealing learning rate scheduling coefficient is calculated by the following formula: Introducing a learning rate adjustment factor As a dynamically adjusted parameter, this parameter is updated synchronously at the end of each iteration and the end of the training cycle. During the tth iteration, the learning rate update mechanism is expressed as follows: In each training cycle After completion, the base learning rate Adaptive update is performed as follows: The improved adaptive learning rate expression is derived as follows: Where, and are the first-order moment estimates and second-order moment estimates after bias correction, respectively.

Images