Global ionospheric TEC prediction method based on encoding and decoding spatiotemporal Transformer model

Abstract

The present invention discloses a global ionospheric TEC prediction method based on a coding and decoding spatiotemporal Transformer model, belonging to the technical field of ionospheric total electron content prediction. The method comprises the following steps: S1: acquiring and preprocessing TEC, Dst, Kp, SSN, and F10.7 data, creating samples, and dividing the data into a training set, a validation set, and a test set; S2: constructing a coding and decoding spatiotemporal Transformer model, wherein the encoder consists of a multi-branch convolutional layer and a spatiotemporal Transformer layer, and the decoder consists of a convolutional decoding layer; S3: training the model using the training set, optimizing parameters using a loss function, tuning using a validation set, and evaluating performance and generalization ability using a test set. The method significantly improves the accuracy of global TEC prediction, maintains high prediction performance and robustness under severe space weather conditions, and achieves efficient extraction of local and global spatiotemporal features.

Description

Technical Field

[0001] The present invention belongs to the technical field of ionospheric total electron content prediction, and specifically relates to a global ionospheric TEC prediction method based on a coding and decoding space-time Transformer model. Background Art

[0002] The total electron content (TEC) of the ionosphere is a key parameter for measuring the state of the ionosphere. It is defined as the total number of electrons integrated over a unit cross-sectional area along the radio wave propagation path. TEC fluctuations are significantly affected by solar and geomagnetic activity, such as solar flares and geomagnetic storms. These events can cause large fluctuations in TEC values, which in turn affect the propagation of radio signals on the ground and in the air. Therefore, accurate TEC prediction is crucial for improving the stability of satellite communication and navigation systems and ensuring the safety of space missions.

[0003] With the continuous increase in data volumes and advancements in computing power, deep learning methods have gradually become a hot topic in ionospheric TEC prediction research. Compared to traditional methods, deep learning can better capture the nonlinear characteristics and complex spatiotemporal dependencies in ionospheric TEC variations, thus demonstrating stronger predictive capabilities. Currently, there are two main approaches to constructing TEC spatiotemporal series prediction models: one is to combine convolutional neural networks (CNNs) with feature extraction units such as LSTMs, BiLSTMs, and ConvLSTMs; the other is to sequentially stack spatiotemporal feature extraction units such as ConvLSTMs and BiConvLSTMs. However, these models still have two limitations. In terms of spatial feature extraction, CNNs can efficiently extract local features and capture local information through convolutional kernels. However, due to the limited receptive field, CNNs have certain limitations when processing long-range spatial dependencies, making it difficult to fully extract global spatial features. In terms of temporal feature extraction, although LSTMs and their variants can effectively model long-term dependencies, their recursive structure still causes them to suffer from the vanishing and exploding gradient problems when processing long sequences, and they also have difficulty capturing global temporal dependencies. Summary of the Invention

[0004] In response to the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a global ionospheric TEC prediction method based on the encoding and decoding spatiotemporal Transformer model, which significantly improves the global TEC prediction accuracy, especially under severe space weather conditions, and can still maintain high prediction performance and robustness; shows strong stability and generalization ability, can accurately predict the global ionospheric TEC map; and realizes the efficient extraction of spatiotemporal local features and global features.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] The global ionospheric TEC prediction method based on the encoding and decoding spatiotemporal Transformer model includes the following steps:

[0007] S1: Acquire and preprocess TEC, Dst, Kp, SSN and F10.7 data, make samples and divide the data into training set, validation set and test set; among them, TEC is the total electron content, Dst is the disturbance storm time index, Kp is the planet K index, SSN is the sunspot number, F10.7 is the solar F10.7 radiation index,

[0008] S2: Build a spatiotemporal Transformer model based on encoding and decoding. The encoder consists of multi-branch convolutional layers and spatiotemporal Transformer layers, which are responsible for extracting local spatiotemporal features and modeling global dependencies. The decoder consists of convolutional decoding layers, which are responsible for decoding the features extracted by the encoder and generating TEC prediction results.

[0009] S3: Use the training set to train the model, optimize the parameters through the loss function, tune it using the validation set, and finally use the test set to evaluate the performance and generalization ability.

[0010] As a further improvement of the above technical solution:

[0011] Step S1 also includes the following steps:

[0012] S101: Acquire TEC, Dst, Kp, SSN and F10.7 data for 21 consecutive years;

[0013] S102: Use cubic spline interpolation to fill in the missing data in F10.7 data;

[0014] S103: Unify the temporal resolution of TEC, Dst, Kp, SSN, and F10.7, and the spatial resolution of TEC, Dst, Kp, SSN, and F10.7;

[0015] S104: Merge the TEC, Dst, Kp, F10.7, and SSN data at the same time in the channel dimension to generate multi-channel data with the same dimension at the corresponding time;

[0016] S105: scaling the value range of each channel to the same range;

[0017] S106: Generate samples using a sliding window method;

[0018] S107: Divide the data into training set, validation set and test set.

[0019] The step S2 comprises the following steps:

[0020] S201: Construct multi-channel grid data with the past 24 hours as input;

[0021] S202: extracting spatiotemporal local features by inputting input data into a multi-branch convolutional layer;

[0022] S203: The data output by the multi-branch convolutional layer is passed to the spatiotemporal Transformer layer. After dimension reshaping, convolution embedding, and spatial position encoding, the data is passed to the spatial Transformer layer. This layer extracts global spatial features through the self-attention mechanism. The output of the spatial Transformer layer is dimensionally reshaped and transposed, and temporally encoded before being input to the temporal Transformer layer. This layer captures temporal dependencies through the self-attention mechanism and extracts global temporal features. Finally, the output of the temporal Transformer layer is dimensionally reshaped and transposed before being input to the convolutional decoding layer.

[0023] S204: The convolutional decoding layer decodes the features extracted by the encoder to generate the final TEC prediction result.

[0024] In step S202, the multi-branch convolutional layer includes two parallel sub-branches: the first sub-branch is a layer of Conv3D+BN+ReLU units, and the second sub-branch is three layers of stacked Conv3D+BN+ReLU units. Finally, the outputs of the two parallel sub-branches are spliced ​​in the channel dimension and then passed through a layer of Conv3D+BN+ReLU units, where Conv3D represents three-dimensional convolution, BN represents batch normalization, ReLU represents rectified linear unit, and "+" represents the sequential execution of multiple operations.

[0025] In step S203, both the spatial Transformer layer and the temporal Transformer layer are composed of 6 layers of Transformer encoders. Each layer of encoder is composed of a stack of multi-head self-attention modules and multi-layer perceptron modules. Layer normalization is applied before each module, and a residual connection is added after each module.

[0026] In step S203, when convolution embedding is performed, the dimension is ( ) is convolved to obtain a shape of Output; Flatten the spatial dimension into a one-dimensional sequence, and get a shape of ( ), where , , , the convolution kernel size is , the step size is consistent with the window size.

[0027] In step S203, spatial position encoding is as follows: defining a trainable position encoding matrix S and adding it to the convolved embedded data to introduce spatial position information. The formula is as follows:

[0028] ;

[0029] in, is the embedded data, is a trainable positional encoding matrix initialized to a normal distribution, As input to the spatial Transformer.

[0030] In step S203, the temporal position encoding is to define a trainable position encoding matrix T and combine it with the output of the spatial Transformer layer after dimension reshaping and transposition. Add to introduce time position information, the formula is as follows:

[0031] ;

[0032] in is the data after dimension reshaping and transposition, Initialized to a normal distribution, As input to the temporal Transformer.

[0033] In step S204, the convolutional decoding layer is composed of two layers of UpSampling+Conv3D+BN+ReLU units, one layer of UpSampling+Conv3D+ReLU units, and one layer of Conv3D+ReLU units connected in sequence, where UpSampling represents upsampling, Conv3D represents three-dimensional convolution, BN represents batch normalization, ReLU represents rectified linear unit, and "+" represents the sequential execution of multiple operations.

[0034] In step S3, the mean square error (MSE) is used as the loss function and the Adam optimizer is used for optimization. The root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R²) are used as evaluation indicators of the model, and the performance of the trained model is evaluated based on these indicators and the test set data. MSE, RMSE, MAE and R² are the best performance indicators of the model. 2 The formula is as follows:

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] in, is the total number of data points, is the i-th true value, represents the i-th predicted value, is the mean value representing the true value.

[0040] The beneficial effects of the present invention are:

[0041] (1) Compared with the existing TEC spatiotemporal prediction models, the present invention has significantly improved the global TEC prediction accuracy, especially under severe space weather conditions, and can still maintain high prediction performance and robustness; it shows strong stability and generalization ability, and can accurately predict the global ionospheric TEC map; the method helps to ensure the safety of space missions and improve the stability of satellite navigation systems.

[0042] (2) The method combines the advantages of convolutional neural networks (CNNs) in local feature extraction with the ability of Transformer self-attention mechanism in modeling global dependencies, achieving efficient extraction of spatiotemporal local features and global features.

[0043] (3) The encoder can effectively model local and global spatiotemporal features and effectively avoid the problems of gradient vanishing and gradient exploding, thereby ensuring the stability of model training. The decoder part consists of a convolutional decoding layer, which is responsible for decoding the features extracted by the encoder and generating prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the present invention.

[0045] Figure 2 It is the overall structure diagram of the model of the present invention.

[0046] Figure 3 It is a detailed structural diagram of the model of the present invention. DETAILED DESCRIPTION

[0047] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0048] For ease of description, spatially relative terms such as "above," "above," "on the upper surface of," and "upper" may be used herein to describe the spatial positional relationship of a device or feature to other devices or features as shown in the figures. It should be understood that spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in a drawing is inverted, a device described as "above" or "on top of" another device or structure would then be positioned as "below" or "below" the other device or structure. Thus, the exemplary term "above" can include both the "above" and "below" orientations. The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used herein should be interpreted accordingly.

[0049] A global ionospheric TEC prediction method based on the encoding and decoding spatiotemporal Transformer model, such as Figure 1 As shown, the following steps are included:

[0050] S1: Acquire and preprocess TEC, Dst, Kp, SSN, and F10.7 data, create samples, and divide them into training, validation, and test sets. TEC is the total electron content; Dst is the disturbance storm time index, a measure of the intensity of geomagnetic storms; Kp is the planetary K index, an index describing the intensity of geomagnetic activity on a global scale; SSN is the sunspot number, a measure of solar activity; and F10.7 is the solar F10.7 radiation index, which measures the intensity of solar radiation at a wavelength of 10.7 cm.

[0051] Step S1 specifically includes the following steps:

[0052] S101: Acquires TEC, Dst, Kp, SSN, and F10.7 data from 2003 to 2023. The global TEC map is from the International GNSS (Global Navigation Satellite System) service (IGS); the Kp, SSN, and F10.7 indices are from the Goddard Space Flight Center; and the Dst index is from the World Geomagnetic Data Center (WDC) in Kyoto. The TEC map has a temporal resolution of 2 hours and covers the longitude of 180°W–180°E and latitude of 87.5°N–87.5°S, with resolutions of 5° and 2.5°, respectively. Each map has a pixel size of 71×73. Dst, Kp, F10.7, and SSN are global indices with varying temporal resolutions, as shown in Table 1.

[0053] Table 1 Units and time resolutions of the geomagnetic activity index and solar activity index

[0054]

[0055] S102: Fill in the missing values ​​in the F10.7 index using the cubic spline interpolation method to restore the complete data series.

[0056] S103: To align with the 2-hour temporal resolution of the TEC data, the Dst index (originally 1-hour) was downsampled. The Kp index (originally 3-hour), F10.7 index (originally 24-hour), and SSN index (originally 24-hour) were upsampled to a uniform 2-hour temporal resolution. To match the 71×73 spatial resolution of the TEC data, the global indices (Dst, Kp, SSN, and F10.7) were scaled to a 71×73 grid. Dst, Kp, SSN, and F10.7 are global indices, measuring global space weather phenomena and having a single value at a given moment. TEC data, on the other hand, covers the entire globe, with a total of 71×73 data points. Therefore, to construct multi-channel data, the Dst, Kp, SSN, and F10.7 indices needed to be scaled to the same 71×73 grid as the TEC data.

[0057] S104: Merge the TEC, Dst, Kp, F10.7, and SSN data at the same time in the channel dimension to generate multi-channel data of the corresponding time and dimension (71, 73, 5).

[0058] S105: In order to eliminate the influence of the magnitude of data in different channels, this paper uses the Min-Max normalization method to process the data of each channel and scale the value range of each channel to [0,1].

[0059] S106: Generate samples using a sliding window method, using 48 consecutive hours (24 frames) of multi-channel grid data as the sliding window. The first 24 hours (12 frames) of data are used as input, and the last 24 hours (12 frames) of data are de-normalized after removing the geomagnetic activity and solar activity index channels. After each sample is generated, the window slides backward by one frame of multi-channel grid data.

[0060] S107: Divide the data into a training set, a validation set, and a test set. The training set is used for model training and parameter optimization, the validation set is used to evaluate and tune hyperparameters, and the test set is used to evaluate the model's generalization ability on unseen data to ensure model reliability and performance.

[0061] The division method is based on the fact that the accuracy of TEC forecasts is closely related to the intensity of solar activity. Therefore, three years of data were selected as the test set: one year with relatively high solar activity, one year with relatively low solar activity, and one year with activity somewhere in between. The remaining years were randomly divided into a training set (85%) and a validation set (15%).

[0062] In this example, we selected 2015, a year of relatively high solar activity, 2019, a year of relatively low solar activity, and 2017, a transitional year between the two, as the test set. Data from the remaining years served as the training set (85%) and the validation set (15%).

[0063] S2: Construct an ED-ST-Transformer model (encoding-decoding spatiotemporal Transformer model) based on the encoder-decoder architecture. The structure is as follows Figure 2 As shown in the figure. The encoder consists of a multi-branch convolutional layer and a spatiotemporal Transformer layer, which is responsible for extracting local spatiotemporal features and modeling global dependencies. The decoder consists of a convolutional decoding layer, which is responsible for decoding the features extracted by the encoder and generating TEC prediction results. The specific steps include:

[0064] S201: Build a model that uses multi-channel gridded data (with a time resolution of 2 hours) from the past 24 hours as input to predict the global TEC grid map for the next 24 hours. Figure 2 Where X represents the multi-channel grid input data, the subscript t represents time, and Xt represents the multi-channel input data corresponding to time t.

[0065] S202: Extracting spatiotemporal local features by inputting input data into a multi-branch convolutional layer.

[0066] like Figure 3As shown, the multi-branch convolutional layer contains two parallel sub-branches: one extracts simple features using a single Conv3D+BN+ReLU layer, and the other extracts complex features using three stacked Conv3D+BN+ReLU layers. Conv3D represents three-dimensional convolution, BN represents batch normalization, and ReLU represents rectified linear units. The "+" denotes the sequential combination and execution of multiple operations. Conv3D + BN + ReLU indicates that these three operations are applied sequentially: a three-dimensional convolution (Conv3D) is performed first, followed by batch normalization (BN) on the convolution result, and finally by the rectified linear unit. Similar explanations apply below and are not repeated here. Finally, the two outputs are concatenated (concatenation: combining multiple tensors along a specific dimension (here, the channel dimension)) and further processed using Conv3D+BN+ReLU to extract multi-level spatiotemporal features. The first sub-branch is a layer of Conv3D+BN+ReLU unit with 32 channels, a convolution kernel size of 3×3×3, and a step size of 2×2×2. The second sub-branch is a three-layer stacked Conv3D+BN+ReLU unit. The number of channels of the first convolution layer is 16, the size of the convolution kernel is 3×3×3, and the step size is 2×2×2. The number of channels of the second convolution layer is 32, the size of the convolution kernel is 3×3×3, and the step size is 1×1×1. Padding='same' is used. The number of channels of the third convolution layer is 64, the size of the convolution kernel is 3×3×3, and the step size is 1×1×1. Padding='same' is used. Finally, the outputs of the two parallel branches are spliced ​​in the channel dimension and then passed through a layer of Conv3D+BN+ReLU unit with 64 channels, a convolution kernel size of 3×3×3, a step size of 1×1×1, and padding='same'. Conv3D is used to capture local features, batch normalization (BN) accelerates training and improves stability, and ReLU activation enhances nonlinear modeling. Padding='same' is a method used in convolutional neural networks to pad the input data so that the output size matches the input size.

[0067] S203: spatiotemporal Transformer layer, the structure is as follows Figure 3 The input of the spatiotemporal Transformer layer is the output of the multi-branch convolutional layer, and the spatiotemporal Transformer layer converts the output of the multi-branch convolutional layer into The data is passed into the spatiotemporal Transformer layer, and the data is reshaped by dimension ( ), convolutional embedding and spatial position encoding, and then passed to the Spatial Transformer layer, which extracts global spatial features through the self-attention mechanism. The output of the Spatial Transformer layer is reshaped and transposed ( ) and temporal position encoding, and then input to the Temporal Transformer layer, which captures temporal dependencies through the self-attention mechanism and extracts global temporal features; finally, the output of the Temporal Transformer layer is reshaped and transposed ( ) and then fed into the convolutional decoding layer. Both the spatial and temporal Transformer layers consist of six Transformer encoder layers, each of which consists of alternating stacks of Multi-Head Self-Attention (MSA) and Multi-Layer Perceptron (MLP) modules. Layer normalization (Layer Norm) is applied before each module, and residual connections are added after each module. The embedding dimension of each layer is 128, and the hidden layer dimension of the MLP is 256. Figure 3 In , Multi-Head Attention is multi-head attention.

[0068] The following explains the relevant concepts and processes:

[0069] The self-attention mechanism uses scaled dot-product attention (Scaled Dot-ProductAttention). By calculating the correlation weights between elements in the sequence, a weighted representation is generated for each element, thereby capturing global dependencies.

[0070] First, enter By combining three weight matrices 、 、 Multiply them together to generate the query (Q), key (K), and value (V), respectively.

[0071] ;

[0072] Next, calculate the dot product between Q and K and divide it by the square root of the dimension of the key , to avoid the gradient vanishing problem caused by too large input values. The function calculates the normalized attention weight and multiplies it with V to finally get the weighted attention output. The calculation formula is as follows:

[0073] ;

[0074] in, , represents the set of real numbers, is the number of image patches or time steps, is the embedding dimension; is the learnable parameter matrix; , is the number of attention heads. Indicates that X is an N-row Column matrix, each element in the matrix is ​​a real number, that is, N, are the number of rows and columns of the matrix, corresponding to the sequence length of the input data and the feature dimension of each data point, respectively. represents the transpose of Q times K.

[0075] Multi-Head Attention allows the model to focus on subspace information at different locations in parallel. Multiple attention heads compute in parallel, learning different features and dependencies. Finally, the outputs are concatenated and linearly transformed to obtain the result.

[0076] The calculation formula is as follows:

[0077] ;

[0078] Among them, Concat means splicing.

[0079] ;

[0080] in, , , is the learnable parameter matrix.

[0081] For residual connection, a residual connection mechanism based on pre-normalization (Pre-Layer Normalization) is adopted, which is normalized first and then residual connection. The calculation formula is as follows:

[0082] ;

[0083] in, represents the input of the sub-layer, represents a sub-layer module (such as a multi-head attention mechanism or a feed-forward network), is the layer normalization operation.

[0084] For the multi-layer perceptron (MLP), the multi-layer perceptron (MLP) consists of two fully connected layers and the activation function ReLU, which is used to enhance the expressive power of the model. Its calculation formula is as follows:

[0085] ;

[0086] in, Represents the input of the sub-layer, and the hidden layer dimension of the first fully connected layer is , the hidden layer dimension of the second fully connected layer is . is the output of the Multi-Layer Perceptron (MLP). is the application of the ReLU (Rectified Linear Unit) activation function: for each input value, if the value is greater than 0, then output that value; if the value is less than or equal to 0, then output 0. W1 is a weight matrix and b1 is the first bias term. : Indicates input data A linear transformation (i.e. matrix multiplication) with the weight matrix W1, and the addition of the bias term b1. W2 is another weight matrix, and b2 is the bias term.

[0087] For convolutional embedding, convolutional embedding is done by two-dimensional convolution (the convolution kernel size is , Represents the height and width of the convolution kernel respectively. The step size is consistent with the window size), the dimension is ( ) is convolved to obtain a shape of Output; Flatten the spatial dimension into a one-dimensional sequence, and get a shape of ( ) is an embedded representation of .in , , The window typically refers to the size of the convolution kernel used in the convolution operation. In this step, the kernel size is patch_size_h × patch_size_w, which means that the kernel slides across the input image, "scanning" an area of ​​size patch_size_h × patch_size_w each time. The "window" refers to this area, that is, the area of ​​the image covered by the convolution kernel at each operation. The stride is the distance the convolution kernel moves across the image, which can be in the vertical direction (up and down) and the horizontal direction (left and right).

[0088] For spatial position coding, the spatial position coding is to define a trainable position coding matrix S and combine it with the data after convolution embedding. Add together to introduce spatial location information. The formula is as follows:

[0089] ;

[0090] in, is the input patch embedding representation, is a trainable positional encoding matrix initialized to a normal distribution. As input to the spatial Transformer.

[0091] For temporal position encoding, the temporal position encoding is to define a trainable position encoding matrix T and combine it with the output of the Spatial Transformer layer after dimension reshaping and transposition. Add together to introduce time position information. The formula is as follows:

[0092] ;

[0093] in is the data after dimension reshaping and transposition, Initialized to a normal distribution. As input to the temporal Transformer.

[0094] S204, convolutional decoding layer, the structure is as follows Figure 3 As shown in the figure. This layer decodes the features extracted by the encoder and generates the final TEC prediction result. It is composed of two layers of UpSampling+Conv3D+BN+ReLU units, one layer of UpSampling+Conv3D+ReLU units, and one layer of Conv3D+ReLU units, connected in sequence. UpSampling represents upsampling. In the two layers of UpSampling+Conv3D+BN+ReLU units, the size of the first UpSampling layer is 3×3×3, the number of convolution channels is 128, the size of the convolution kernel is 3×3×5, and the stride is 2×1×1; the size of the second UpSampling layer is 3×3×3, the number of convolution channels is 64, the size of the convolution kernel is 3×3×5, and the stride is 2×1×1. In the one layer of UpSampling+Conv3D+ReLU units, the size of UpSampling is 3×2×2, the number of convolution channels is 32, the size of the convolution kernel is 3×3×3, and the stride is 2×1×1. In a Conv3D+ReLU layer, the convolution channel is 1, the kernel size is 3×2×2, and the stride is 1×1×1. Upsampling and expansion are performed through two layers of UpSampling+Conv3D+BN+ReLU. Then, UpSampling+Conv3D+ReLU is used to enhance spatial information. Finally, Conv3D+ReLU is used for final optimization to generate the prediction result.

[0095] S3: Use the training set to train the model, optimize the parameters through the loss function, use the validation set to tune, and finally use the test set to evaluate the performance and generalization ability. Specifically:

[0096] S301: During model training, select Adam (Adaptive Moment Estimation) as the optimizer and mean squared error (MSE) as the loss function. The Adam optimizer has the advantages of adaptive learning rate and efficient computation, accelerating convergence and improving generalization. The mean squared error (MSE) loss function is computationally simple, easy to optimize, and performs well for most regression problems. The formula for MSE is as follows:

[0097] ;

[0098] in, is the total number of data points, is the i-th true value, represents the i-th predicted value.

[0099] S302: In each training step, the training set data is fed into the model for forward propagation. The loss between the predicted values ​​and the actual labels is calculated, and the model parameters are updated using the backpropagation algorithm. Subsequently, the current model is evaluated using the validation set. The validation set data is used only to evaluate model performance and adjust hyperparameters and does not participate in the model training process. The evaluation results on the validation set are used to tune hyperparameters to optimize model performance and ensure generalization to unseen data.

[0100] S302: After the model training is completed and adjusted by the validation set, the final model is evaluated using the test set. The root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R²) are used as the evaluation indicators of the model. 2 The formula is as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] in, is the total number of data points, is the i-th true value, represents the i-th predicted value, is the mean value representing the true value.

[0105] S303: After the model performs well on the test set and meets the expected performance metrics, save the trained model weights and structure. Archiving the model allows it to be directly loaded into future applications, avoiding the computational overhead of retraining. The saved model can be used for actual deployment and real-time prediction.

[0106] Finally, it is necessary to explain here that the above embodiments are only used to further illustrate the technical solution of the present invention in detail and cannot be understood as limiting the scope of protection of the present invention. Some non-essential improvements and adjustments made by technicians in this field based on the above content of the present invention all fall within the scope of protection of the present invention.

Claims

1. A global ionospheric TEC prediction method based on the encoding and decoding spatiotemporal Transformer model is characterized by: The following steps are involved: S1: Acquire and preprocess TEC, Dst, Kp, SSN and F10.7 data, make samples and divide the data into training set, validation set and test set; among them, TEC is the total electron content, Dst is the disturbance storm time index, Kp is the planet K index, SSN is the sunspot number, F10.7 is the solar F10.7 radiation index, S2: Construct a spatiotemporal Transformer model based on encoding and decoding, in which the encoder consists of a multi-branch convolutional layer and a spatiotemporal Transformer layer. The spatiotemporal Transformer layer consists of a spatial Transformer layer and a temporal Transformer layer, which is responsible for extracting local spatiotemporal features and modeling global dependencies. The input data is input into the multi-branch convolutional layer to extract spatiotemporal local features, and the data output by the multi-branch convolutional layer is passed into the spatiotemporal Transformer layer. After dimension reshaping, convolution embedding and spatial position encoding, the data is passed to the spatial Transformer layer, which extracts global spatial features through the self-attention mechanism. The output of the spatial Transformer layer is dimensionally reshaped and transposed, and temporally encoded, and then input into the temporal Transformer layer, which captures temporal dependencies through the self-attention mechanism and extracts global temporal features. The decoder consists of a convolutional decoding layer, which is responsible for decoding the features extracted by the encoder and generating TEC prediction results. The multi-branch convolutional layer consists of two parallel sub-branches: the first sub-branch is a layer of Conv3D+BN+ReLU units with 32 channels, and the second sub-branch is three stacked layers of Conv3D+BN+ReLU units with 16, 32, and 64 channels, respectively. Finally, the outputs of the two parallel sub-branches are concatenated in the channel dimension and then passed through a layer of Conv3D+BN+ReLU units. Conv3D represents three-dimensional convolution, BN represents batch normalization, and ReLU represents rectified linear unit. "+" indicates the sequential execution of multiple operations. Both the spatial and temporal Transformer layers consist of a 6-layer Transformer encoder. Each encoder layer is composed of a stack of multi-head self-attention modules and multi-layer perceptron modules. Layer normalization is applied before each module, and residual connections are added after each module. When convolution is embedded, through two-dimensional convolution, the dimension is ( ) is convolved to obtain a shape of Output; Flatten the spatial dimension into a one-dimensional sequence, and get a shape of ( ), where , , , the convolution kernel size is , the step size is consistent with the window size; The convolutional decoding layer consists of two layers of UpSampling+Conv3D+BN+ReLU units, one layer of UpSampling+Conv3D+ReLU units, and one layer of Conv3D+ReLU units connected in sequence, where UpSampling means upsampling; S3: Use the training set to train the model, optimize the parameters through the loss function, tune it using the validation set, and finally use the test set to evaluate the performance and generalization ability.

2. The prediction method according to claim 1, wherein: Step S1 also includes the following steps: S101: Acquire TEC, Dst, Kp, SSN and F10.7 data for 21 consecutive years; S102: Use cubic spline interpolation to fill in the missing data in F10.7 data; S103: Unify the temporal resolution of TEC, Dst, Kp, SSN, and F10.7, and the spatial resolution of TEC, Dst, Kp, SSN, and F10.7; S104: Merge the TEC, Dst, Kp, F10.7, and SSN data at the same time in the channel dimension to generate multi-channel data with the same dimension at the corresponding time; S105: scaling the value range of each channel to the same range; S106: Generate samples using a sliding window method; S107: Divide the data into training set, validation set and test set.

3. The prediction method according to claim 1, wherein: The step S2 comprises the following steps: S201: Construct multi-channel grid data with the past 24 hours as input; S202: extracting spatiotemporal local features by inputting input data into a multi-branch convolutional layer; S203: The data output by the multi-branch convolutional layer is passed to the spatiotemporal Transformer layer. After dimension reshaping, convolution embedding, and spatial position encoding, the data is passed to the spatial Transformer layer. This layer extracts global spatial features through the self-attention mechanism. The output of the spatial Transformer layer is dimensionally reshaped and transposed, and temporally encoded before being input to the temporal Transformer layer. This layer captures temporal dependencies through the self-attention mechanism and extracts global temporal features. Finally, the output of the temporal Transformer layer is reshaped and transposed before being input into the convolutional decoding layer; S204: The convolutional decoding layer decodes the features extracted by the encoder to generate the final TEC prediction result.

4. The prediction method according to claim 1, wherein: In step S203, spatial position encoding is as follows: defining a trainable position encoding matrix S and adding it to the convolved embedded data to introduce spatial position information. The formula is as follows: ； in, is the embedded data, is a trainable positional encoding matrix initialized to a normal distribution, As input to the spatial Transformer.

5. The prediction method according to claim 1 or 4, characterized in that: In step S203, the temporal position encoding is to define a trainable position encoding matrix T and combine it with the output of the spatial Transformer layer after dimension reshaping and transposition. Add to introduce time position information, the formula is as follows: ； in is the data after dimension reshaping and transposition, Initialized to a normal distribution, As input to the temporal Transformer.

6. The prediction method according to claim 1, wherein: In step S3, the mean square error (MSE) is used as the loss function and the Adam optimizer is used for optimization. The root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R²) are used as evaluation indicators of the model, and the performance of the trained model is evaluated based on these indicators and the test set data. MSE, RMSE, MAE and R² are the best performance indicators of the model. 2 The formula is as follows: ； ； ； ； in, is the total number of data points, is the i-th true value, represents the i-th predicted value, is the mean value representing the true value.

Images