Ionospheric foF2 Spatial Reconstruction Method Based on Self-Attention Mechanism

Abstract

The present invention proposes an ionospheric foF2 spatial reconstruction method based on the self-attention mechanism, and the implementation steps are as follows: obtaining a training sample set and a test sample set; constructing a prediction network model based on the self-attention mechanism; performing iterative training on the prediction network model based on the self-attention mechanism; obtaining the spatial reconstruction result of the ionospheric foF2. The present invention analyzes the contribution degrees of different features, the similarity and dependence between features through the self-attention module, enables the model to extract richer feature information, overcomes the shortcomings of information loss and insufficient mining of the spatio-temporal correlation of foF2 values in the prior art, improves the spatial reconstruction accuracy of the ionospheric foF2, and thus improves the working performance of radio systems such as communication, positioning, radar, and navigation.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data and machine learning applications in ionospheric research, and relates to a method for reconstructing ionospheric parameters, in particular to a method for spatially reconstructing the ionospheric critical frequency foF2 based on a self-attention mechanism, which can be used to determine the operating parameters of radio systems such as communication, positioning, radar, and navigation. Background Art

[0002] The ionosphere is an ionized region of the Earth's atmosphere, which is approximately the entire space between 60 km and 1000 km above the ground up to the magnetopause. It is an important part of the near-Earth space environment. According to the height from the ground, the ionosphere is usually divided into D, E, F1, and F2 layers. Among them, the F2 layer (220 km - 500 km) has the highest electron density and mainly determines the characteristics of the ionosphere. The maximum electron density in the F2 layer is mainly related to the ordinary wave critical frequency foF2 of the F2 layer. Therefore, foF2 is one of the key parameters for quantifying plasma density changes and describing ionospheric characteristics. It can reflect the ionospheric environmental conditions and disturbance changes, and at the same time greatly affects the propagation of radio waves. To improve the operating performance of radio systems such as communication, positioning, radar, and navigation, and to ensure the safety of space flight, it is necessary to accurately quantify the real-time report / forecast of the ionospheric parameter foF2. Since the ionosphere is affected by many factors such as solar activity and high-altitude electric fields, and its variation law is closely related to time variation and spatial geographical longitude and latitude changes, the accuracy of the real-time report / forecast of foF2 is related to the analysis degree of its related factors and the mining degree of time and space distribution.

[0003] The ionospheric reconstruction problem refers to how to obtain the foF2 values at other unknown locations in the region through interpolation or extrapolation only when the measured or forecast values of foF2 of a limited number of ionospheric observation stations in a certain region are known. For the reconstruction problem, the internationally referenced ionospheric parameter model, the International Reference Ionosphere IRI model, is widely used. It can provide empirical estimates based on monthly averages for a given time and location, but it cannot perform real-time reporting on the current state of the ionosphere and has low accuracy. Therefore, for ionospheric real-time reporting, detection methods are mostly used, including ground active radio detection methods and passive detection methods. Active detection methods such as vertical sounding, oblique sounding, and incoherent scatter radar have defects such as complex detection system equipment, high cost, and limited detection range. Passive detection, such as using high-frequency non-cooperative radiation sources for ionospheric forecasting, has the problem of low forecasting accuracy. To reduce costs and expand the scope of real-time reporting, many studies have carried out ionospheric reconstruction by using data from a limited number of ionospheric detection stations and adopting statistical methods such as data assimilation method and autocorrelation analysis method.

[0004] At present, the commonly used statistical method for ionospheric foF2 reconstruction is the data assimilation method. For example, in the patent application publication No. CN110288117A, titled "A Regional Reconstruction Method for Ionospheric Parameter Critical Frequency", a regional reconstruction method for ionospheric parameter critical frequency is disclosed. This method first determines the grid point longitude and latitude of the reconstruction area and the measured value of foF2 at the sounding station, calculates the foF2 value of the grid point using the Chinese reference ionosphere based on the input time, and finally reconstructs the ionospheric foF2 using the Kalman filter assimilation method. Based on the real-time detected foF2, the reconstruction accuracy of this invention has been improved. However, its defect is that the background field correlation matrix calculated by the Gaussian correlation function only uses time and longitude and latitude information, and the Kalman filter has a shallow mining degree of the spatio-temporal correlation of foF2 in the non-linear system, which affects the further improvement of the reconstruction accuracy. Summary of the Invention

[0005] The object of the present invention is to address the deficiencies of the above-mentioned prior art and propose a spatial reconstruction method for ionospheric foF2 based on the self-attention mechanism, aiming to solve the technical problem of low reconstruction accuracy caused by insufficient mining of the spatio-temporal correlation of foF2 due to information loss and poor recognition of the contribution degree of features in the prior art.

[0006] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Obtain the training sample set R and the test sample set E:

[0008] (1a) Normalize the influencing factors of the ionospheric critical frequency foF2 value of the prediction target point at M consecutive moments, as well as the measured values and influencing factors of the ionospheric critical frequency foF2 of each of the N auxiliary points respectively. Then, form the prediction target point vector set B = {b m |1 ≤ m ≤ M} with the M influencing factors corresponding to the prediction target point after normalization; at the same time, form the M auxiliary point vectors A = {A n |1 ≤ n ≤ N} with the measured values and influencing factors corresponding to the N auxiliary points after normalization. A n = {a nm |1 ≤ m ≤ M}, where M ≥ 8000, N ≥ 3. The influencing factors of the ionospheric foF2 value of the prediction target point include the day of the year and the solar altitude angle of the prediction target point, and the influencing factors of the ionospheric foF2 of each auxiliary point include the day of the year and the solar altitude angle of each auxiliary point. b m is the prediction target point vector at the m-th moment, A n is the prediction target point vector of the n-th auxiliary point at the m-th moment, and a nm is the auxiliary point vector of the n-th auxiliary point at the m-th moment;

[0009] (1b) For each auxiliary point vector a at the m-th moment nm Concatenate it with the predicted target point vector b m to obtain M concatenated vectors C = {c m |1 ≤ m ≤ M}, where c m = [a 1m , …, a nm , … a Nm , b m T , and use K concatenated vectors in C as the training sample set R, and the remaining M - K concatenated vectors as the test sample set E. Here, c m is the concatenated vector at the m-th moment, T is the transpose symbol;

[0010] (2) Construct a prediction network model H based on the self-attention mechanism:

[0011] Construct a prediction network model H including an Embedding layer, multiple self-attention modules, and multiple fully connected-activation function layers connected in sequence; each self-attention module includes a self-attention layer and a normalization layer stacked in sequence, and there is a cross-layer identity path connection between the input end and the output end of each self-attention layer;

[0012] (3) Iteratively train the prediction network model H based on the self-attention mechanism:

[0013] (3a) Initialize the iteration number as t, the maximum iteration number as T, T ≥ 500, the weight parameter of the prediction network model H at the t-th iteration t is w t , and the bias parameter is b t , and let t = 1;

[0014] (3b) Use the training sample set R as the input of the prediction network model H based on the self-attention mechanism. The Embedding layer performs a linear transformation on each training sample; multiple self-attention modules extract features from each predicted target point and auxiliary point vector after the linear transformation; multiple fully connected-activation function layers perform non-linear integration on each extracted feature information to obtain K foF2 prediction values

[0015] (3c) Add to the predicted target point vector b k of the k-th sample in the training sample set R to obtain a new predicted target point vector b' k ; at the same time, delete the measured value of the ionospheric foF2 in the γ-th auxiliary point vector a γk of the k-th sample to obtain a new γ-th auxiliary point vector a' ​​γk ; and splice a′ γk , {a nk |1≤n≤N, 1≤k≤K} and b′ k to obtain K new spliced vectors C′ = {c′ k |1≤k≤K}, where c′ k = [a 1k ,..., b′ k , …, a nk , a′ γk T , where γ is a constant, 1≤γ≤N, and n≠γ, {a nk |1≤n≤N, 1≤k≤K} are the other auxiliary point vectors in the k - th sample excluding the γ - th auxiliary point vector, and c′ k is the k - th new spliced vector;

[0016] (3d) Use the measured value of the ionospheric foF2 in the γ - th auxiliary point vector a γk of the k - th sample as the label of c′ , form a new training sample set R′, and use the new training sample set R′ as the input of the model H for forward propagation to obtain K foF2 predicted values k where, is the label of the k - th new training sample, is the foF2 predicted value of the k - th new training sample;

[0017] (3e) Adopt the mean - square error loss function, calculate the loss value L of H through the label of the new spliced vector t and its corresponding foF2 predicted value t , and calculate the network parameter gradient of H t through L t . Then, adopt the Adam optimization method to update the weight parameter w t and the bias b t through the network parameter gradient to obtain the predicted network model H t for this iteration;

[0018] (3f) Judge whether t≥T holds. If so, obtain the trained predicted network model H * , otherwise, let t = t + 1, H = H t , and execute step (3b);

[0019] (4) Obtain the spatial reconstruction result of the ionospheric foF2:

[0020] Use the test sample set E as the trained predicted network model H​​* Perform forward propagation on the input to obtain M-K foF2 prediction values, and perform denormalization operations on each foF2 prediction value to achieve spatial reconstruction of the ionospheric foF2.

[0021] The present invention has the following advantages compared with the existing technologies:

[0022] First, the prediction network model constructed by the present invention includes multiple self-attention modules. During the training of the model and the process of obtaining the spatial reconstruction result of the ionospheric foF2, the self-attention module calculates the weights of different feature information in the input and the correlation coefficients between features, focuses on important features and suppresses unimportant features, thereby deeply mining the correlation between foF2 and its influencing factors and the correlation between the target point and the foF2 values of multiple auxiliary points, avoiding the defects in the prior art that it is difficult to capture the correlation between foF2 values and influencing factors and the spatio-temporal correlation of foF2 values, and effectively improving the reconstruction accuracy of the foF2 value of the target point.

[0023] Second, the training sample set constructed by the present invention not only includes the time, longitude and latitude information of the prediction target point and auxiliary points, but also includes the day of the year and solar altitude angle information. The information for feature extraction is more abundant, and at the same time, the input of the prediction network model is characterized and enhanced, which helps the self-attention module to more effectively extract features and mine feature correlations, avoiding the defects of information loss and weak characterization ability in the prior art, and further improving the reconstruction accuracy of the foF2 value of the target point. Description of the Drawings

[0024] Figure 1 is the implementation flowchart of the present invention.

[0025] Figure 2 is the structural schematic diagram of the ionospheric foF2 spatial reconstruction model constructed in the embodiment of the present invention. Detailed Embodiments

[0026] The present invention will be further described below in conjunction with the drawings and specific embodiments.

[0027] Refer to Figure 1 , the present invention includes the following steps:

[0028] (1) Obtain a training sample set R and a test sample set E:

[0029] (1a) Since the ionosphere is affected by many factors such as solar activities and high-altitude electric fields, and its variation law is closely related to time variation and spatial geographical longitude and latitude changes, it is necessary to analyze the correlation degree between the foF2 value and its influencing factors. The influencing factors of the ionospheric critical frequency foF2 value of the prediction target point at M consecutive moments, as well as the measured values and influencing factors of the ionospheric critical frequency foF2 of each auxiliary point among N auxiliary points, are normalized respectively; among them, the influencing factors of the ionospheric foF2 value of the prediction target point include the longitude and latitude of the prediction target point, Beijing time, the IRI model value of foF2, the day of the year and the solar altitude angle, and the influencing factors of the ionospheric foF2 of each auxiliary point include the longitude and latitude of each auxiliary point, Beijing time, the IRI model value of foF2, the day of the year and the solar altitude angle. The richness and effectiveness of the sample set information have a great impact on the prediction performance of the prediction network model. Usually, the performance of the model after feature enhancement is better and the generalization ability is stronger.

[0030] The M influencing factors corresponding to the prediction target point after normalization are composed into a prediction target point vector set B = {b m |1 ≤ m ≤ M}, At the same time, the measured values and influencing factors corresponding to the N auxiliary points after normalization are composed into M auxiliary point vectors A = {A n |1 ≤ n ≤ N}, A n = {a nm |1 ≤ m ≤ M}, where M ≥ 8000, N ≥ 3, b m is the prediction target point vector at the m-th moment, A n is the prediction target point vector of the n-th auxiliary point at the m-th moment, a nm is the auxiliary point vector of the n-th auxiliary point at the m-th moment, NS tar , EW tar , LT m , DOY m , and are the normalized longitude, latitude, Beijing time, day of the year, solar altitude angle, and IRI model value of foF2 of the prediction target point at the m-th moment respectively, NS aux_n , EW aux_n , LT m , DOY m , and are the normalized longitude, latitude, Beijing time, day of the year, solar altitude angle, IRI model value of foF2, and measured value of the n-th auxiliary point at the m-th moment respectively.

[0031] (1b) For each auxiliary point vector a nm at the m-th moment and the prediction target point vector bm Perform splicing to obtain M splicing vectors C = {c m |1 ≤ m ≤ M}, where c m = [a 1m ,..., a nm , … a Nm , b m T , and use K splicing vectors in C as the training sample set R, and the remaining M - K splicing vectors as the test sample set E. Here, c m is the splicing vector at the m-th moment, T is the transpose symbol.

[0032] In this example, M = 8000, N = 4, K = 7300.

[0033] (2) Construct a prediction network model H based on the self-attention mechanism:

[0034] Refer to Figure 2 , and construct a prediction network model H including a sequentially connected Embedding layer, multiple self-attention modules, and multiple fully connected-activation function layers; where each self-attention module includes a self-attention layer and a normalization layer stacked in sequence, and there is a cross-layer identity path connection between the input end and the output end of each self-attention layer.

[0035] The number of self-attention modules is 2; the number of fully connected-activation function layers is 2. Among them, the number of fully connected neurons in both fully connected layers is 32, and the first and second activation functions use sigmoid and tanh respectively.

[0036] (3) Iteratively train the prediction network model H based on the self-attention mechanism:

[0037] (3a) Initialize the iteration number as t, the maximum iteration number as T, T ≥ 500, and the weight parameter of the prediction network model H t at the t-th iteration is w t , and the bias parameter is b t , and let t = 1.

[0038] (3b) Use the training sample set R as the input of the prediction network model H based on the self-attention mechanism. The Embedding layer performs a linear transformation on each training sample, maps the features to a high-dimensional space, and explores which factors in the input are similar in the low-dimensional space, facilitating the extraction of data features by the self-attention module;

[0039] ​Multiple self-attention modules extract features from each predicted target point and auxiliary point vector after linear transformation, where the self-attention layer is used to extract temporal and spatial features in the input and analyze the correlation between features, thereby inferring the foF2 value based on the feature information. First, input I = {i k |1≤k≤K} generates query matrix Q=IW Q , key matrix K = IW K Sum matrix V = IW V , then the output of the self-attention layer O={o k |1≤k≤K} is:

[0040] O=softmax(QK T )·V

[0041] Among them, W Q , W K 、 d is the number of elements in each predicted target point vector, d=7, softmax is the normalization operation, K T is the transpose of K.

[0042] Cross-layer identity path connections superimpose corresponding elements of the self-attention layer's input and output, preserving information integrity. This significantly increases model training speed, eases training difficulty, and improves training effectiveness. The normalization layer normalizes the superimposed elements of the self-attention layer's input and output, ensuring consistent data distribution and preventing vanishing and exploding gradients while accelerating network convergence.

[0043] Multiple fully connected activation function layers perform nonlinear integration on each extracted feature information to obtain K foF2 prediction values

[0044] (3c) Use The predicted target point vector b of the kth sample in the permutation training sample set R k The 0 at the foF2 value position in the middle obtains the new vector of the predicted target point At the same time, the γth auxiliary point vector a of the kth sample γk The values at the positions of the foF2 values are determined by the measured values of the ionospheric foF2. Replace it with 0 and get the new vector of the γth auxiliary point And a γ ' k 、{a nk |1≤n≤N,1≤k≤K} and b k ′ to obtain K new splicing vectors C′={c′ k |1≤k≤K}, c′ k =[a 1k,...,b′ k ,…,a nk ,a′ γk T , where γ is a constant, 1 ≤ γ ≤ N, and n ≠ γ, is the other auxiliary point vectors in the k-th sample excluding the γ-th auxiliary point vector, c′ k is the k-th new spliced vector.

[0045] (3d) Use the measured value of the ionospheric foF2 in the γ-th auxiliary point vector a γk of the k-th sample as the label of c′ k to form a new training sample set R′, and use the new training sample set R′ as the input of the model H for forward propagation to obtain K foF2 predicted values where, is the label of the k-th new training sample, is the foF2 predicted value of the k-th new training sample.

[0046] (3e) Adopt the mean square error loss function, and calculate the loss value L of H through the label of the new spliced vector and its corresponding foF2 predicted value t , and calculate the network parameter gradient of H t through L t , then adopt the Adam optimization method to update the weight parameter w t and the bias b t using the network parameter gradient to obtain the prediction network model H t for this iteration; the calculation and update formulas are respectively: t ; the calculation and update formulas are as follows:

[0047]

[0048]

[0049]

[0050]

[0051] v t = β1·v t-1 +(1 - β1)·g t

[0052] s t = β2·s t-1 +(1 - β2)·g t 2 ​

[0053]

[0054] where P t ∈ {ω t , b t}, g t is the gradient at iteration t, β1 and β2 are the exponential decay rates of the first moment and the second moment respectively, v t is the first moment estimate of the gradient of the H t network parameters, s t is the second moment estimate of the gradient of the H t network parameters, is the correction for v t , is the correction for s t , are the t-th powers of β1 and β2 respectively, α is the learning rate, and ε is a constant.

[0055] (3f) Judge whether t ≥ T holds. If so, obtain the trained prediction network model H * , otherwise, set t = t + 1, H = H t , and execute step (3b).

[0056] In the above one iteration process, two forward propagations and one backward update are completed. Since there is no measured value as a label for the predicted target point, the prediction result of the model for the predicted target point for the first time is filled into its foF2 position, which can be as close as possible to the measured value in iterative training; at the same time, a random auxiliary point is selected as a new predicted target of the prediction network model, and its measured value can be used as a training label, enabling the model to have a relatively accurate understanding of the foF2 measured value. By minimizing the difference between the predicted value of the auxiliary point and its corresponding measured value, the prediction performance of the model can be improved. Therefore, it not only solves the defect of low prediction accuracy caused by unsupervised learning in the deep learning method under the reconstruction problem, but also enhances the generalization ability of the model through iterative training of randomly selecting any auxiliary point.

[0057] (4) Obtain the spatial reconstruction result of the ionospheric foF2:

[0058] Use the test sample set E as the input of the trained prediction network model H * to perform forward propagation, obtain M - K foF2 predicted values, and perform denormalization operations on each foF2 predicted value to achieve the spatial reconstruction of the ionospheric foF2.

Claims

1. An ionospheric foF2 spatial reconstruction method based on the self-attention mechanism, characterized in that It includes the following steps: (1) Obtain a training sample set R and a test sample set E: (1a) The influence factors of the ionospheric critical frequency foF2 value of the predicted target points at M consecutive moments, and the measured values and influence factors of the ionospheric critical frequency foF2 of each of the N auxiliary points are normalized respectively. The M influence factors corresponding to the normalized predicted target points are composed into a predicted target point vector set B = {b m | 1 ≤ m ≤ M}; at the same time, the measured values and influence factors corresponding to the N normalized auxiliary points are composed into M auxiliary point vectors A = {A n | 1 ≤ n ≤ N}, A n = {a nm | 1 ≤ m ≤ M}, where M ≥ 8000, N ≥ 3. The influence factors of the ionospheric foF2 value of the predicted target points include the day of the year and the solar elevation angle of the predicted target points. The influence factors of the ionospheric foF2 of each auxiliary point include the day of the year and the solar elevation angle of each auxiliary point. b m is the predicted target point vector at the m-th moment, A n is the predicted target point vector of the n-th auxiliary point at the m-th moment, and a nm is the auxiliary point vector of the n-th auxiliary point at the m-th moment; (1b) For each auxiliary point vector a at the m-th moment nm and the predicted target point vector b m are concatenated to obtain M concatenated vectors C = {c m | 1 ≤ m ≤ M}, where c m = [a 1m ,..., a nm ,... a Nm , b m T . And K of the concatenated vectors in C are used as the training sample set R, and the remaining M - K concatenated vectors are used as the test sample set E. Here, c m is the concatenated vector at the m-th moment, T is the transpose symbol; (2) Construct a prediction network model H based on the self-attention mechanism: Construct a prediction network model H including an Embedding layer, multiple self-attention modules, and multiple fully connected-activation function layers connected in sequence; each self-attention module includes a self-attention layer and a normalization layer stacked in sequence, and the input end and the output end of each self-attention layer are connected by a cross-layer identity path; (3) Iteratively train the prediction network model H based on the self-attention mechanism: (3a) Initialize the number of iterations as t, the maximum number of iterations as T, where T ≥ 500. The weight parameters of the prediction network model H t at the t-th iteration are w t , and the bias parameter is b t , and set t = 1; (3b) Use the training sample set R as the input of the prediction network model H based on the self-attention mechanism. The Embedding layer performs a linear transformation on each training sample. Multiple self-attention modules extract features from each predicted target point and auxiliary point vector after the linear transformation. Multiple fully connected-activation function layers perform non-linear integration on each extracted feature information to obtain K foF2 prediction values (3c) Add to the predicted target point vector b of the k-th sample in the training sample set R k to obtain a new predicted target point vector b′ k ; at the same time, delete the γ-th auxiliary point vector a of the k-th sample γk from the measured value of the ionospheric foF2 to obtain a new γ-th auxiliary point vector a′ γk ; and splice a′ γk , {a nk |1 ≤ n ≤ N, 1 ≤ k ≤ K} and b k ′ to obtain K new spliced vectors C′ = {c′ k |1 ≤ k ≤ K}, where c′ k = [a 1k ,..., b′ k ,..., a nk , a′ γk T , where γ is a constant, 1 ≤ γ ≤ N, and n ≠ γ, {a nk |1 ≤ n ≤ N, 1 ≤ k ≤ K} is the other auxiliary point vectors of the k-th sample excluding the γ-th auxiliary point vector, and c′ k is the k-th new spliced vector;​ (3d) Take the measured value of the ionospheric foF2 in the γ-th auxiliary point vector a of the k-th sample γk as c' and use it as the label to form a new training sample set R'. Then, use the new training sample set R' as the input of the model H for forward propagation to obtain K foF2 predicted values k where, is the label of the k-th new training sample, and is the foF2 predicted value of the k-th new training sample; (3e) Adopt the mean square error loss function, and calculate the loss value L of H through the label of the newly spliced vector and its corresponding foF2 predicted value Calculate H t ; and calculate the network parameter gradient of H through L t , and then adopt the Adam optimization method to update the weight parameter w t and the bias b t through the network parameter gradient, and obtain the predicted network model H t of this iteration; t t ; (3f) Determine whether t≥T holds. If so, obtain the trained prediction network model H * Otherwise, set t = t + 1, H = H t and execute step (3b); (4) Obtain the spatial reconstruction result of the ionospheric foF2: Use the test sample set E as the input of the trained prediction network model H * to perform forward propagation, obtain M-K foF2 prediction values, and perform an inverse normalization operation on each foF2 prediction value to achieve spatial reconstruction of the ionospheric foF2.

2. The ionospheric foF2 spatial reconstruction method based on the self-attention mechanism according to claim 1, wherein: The influencing factors of the ionospheric foF2 value at the predicted target point in step (1) also include the longitude, latitude, Beijing time, and IRI model value of the foF2 at the predicted target point; the influencing factors of the ionospheric foF2 at each auxiliary point also include the longitude, latitude, Beijing time, and IRI model value of the foF2 at each auxiliary point.

3. The ionospheric foF2 spatial reconstruction method based on the self-attention mechanism according to claim 1, wherein For the prediction network model H described in step (2), where: The number of self-attention modules is 2; the number of fully connected-activation function layers is 2, where the number of fully connected neurons in both fully connected layers is 32, and the first and second activation functions are sigmoid and tanh respectively.

4. The ionospheric foF2 spatial reconstruction method based on the self-attention mechanism according to claim 1, wherein The calculation of H described in step (3e) t The loss value L t For the weight parameter w t And the bias b t Perform updates. The calculation and update formulas are respectively:[[]] v t = β1·v t-1 + (1 - β1)·g t Among them, P t ∈ {ω t , b t}, g t is the gradient at iteration t, β1 and β2 are the exponential decay rates of the first moment and the second moment respectively, v t is the first moment estimate of the gradient of the H t network parameters, s t is the second moment estimate of the gradient of the H t network parameters, is the correction to v t , is the correction to s t , are the t-th powers of β1 and β2 respectively, α is the learning rate, and ε is a constant.

Images