A real-time processing method for missing and outlier radiation dose data of MEO satellites

Abstract

The present invention provides a real-time processing method for missing and outlier data of MEO satellite radiation dose, which learns the oscillation period T of radiation dose data from the data recorded by each radiation dose detector carried by the satellite; learns the radiation dose prediction model based on trend and Fourier characteristics; performs outlier detection on the newly arrived radiation dose data based on the prediction model; and replaces the missing values ​​and outliers in the newly arrived radiation dose data with the predicted values. Based on the prediction model, the present invention processes the missing values ​​and outliers in the latest downlinked data in real time, achieves a more accurate effect, and significantly reduces the fitting error, providing a complete and clean data set for subsequent data release, modeling, visualization and other tasks.

Description

Technical Field

[0001] The invention belongs to the field of space-based space environment detection and is suitable for medium-orbit satellite radiation dose detection data processing. Background Art

[0002] Medium Earth Orbit (MEO) refers to a satellite orbit at an altitude of about 2000 to 20000 km from the Earth's surface. It is the main orbit for navigation satellites. Most satellites in the US GPS and China's BeiDou system operate in this orbit. MEO satellites cross the center of the Earth's outer radiation belt about four times a day. The high-energy electron radiation they are exposed to poses a great threat to satellite safety. Studies have shown that more than half of MEO satellite failures are caused by "killer electrons" in the outer radiation belt (Yang Xiaochao, Zhu Guangwu, Fu Yang, Sun Yueqiang, Liang Jinbao, Jing Tao, Chang Zheng, Shi Chunyan. Analysis of the characteristics of high-energy electron radiation environment in medium earth orbit [J]. Journal of Space Science, 2015, 35(5): 581-587). In order to cope with such threats, some of my country's MEO satellites are equipped with space environment detection payloads and have accumulated a certain amount of detection data, which provides important data support for satellite failure analysis, disaster prevention and mitigation, etc.

[0003] However, in actual application, radiation dose detection data has the problem of missing and abnormal data in some time periods. The reasons may be abnormal load operation, crowded signal transmission channels, untimely data download or failure of original detection data to de-telemetry. The missing and abnormal detection data not only affect the monitoring of radiation dose in the satellite cabin, but also have a serious impact on subsequent tasks such as spacecraft design and device screening, and on-orbit satellite health status assessment. Therefore, studying the method of processing missing and abnormal data is a problem that must be solved in the application of Beidou MEO satellite radiation dose data. The data is downloaded and received in batches, usually a batch is downloaded in each circle. The present invention aims to process the missing values ​​and abnormal values ​​in the latest downloaded data in real time.

[0004] Common missing value filling methods in numerical analysis include forward interpolation, backward interpolation, linear interpolation, etc. For the problem of real-time data processing, only forward interpolation can be used, because both backward interpolation and linear interpolation rely on the observed value after the missing value (for real-time data, such an observed value obviously does not exist). Forward interpolation refers to replacing the missing value with the most recent observed value before the missing value. The accuracy of this method is usually very low and it is difficult to meet the needs. There are also some methods that are better than forward interpolation, but these methods are usually used to deal with the problem of random loss of individual data (Qiao Zhufeng, Tian Fengzhan, Huang Houkuan, Chen Jingnian. Comparative study of missing data processing methods [J]. Computer Research and Development, 2006.43:171-175). However, the missing radiation dose data of Beidou MEO satellites has obvious continuity characteristics, that is, if the data starts to be missing at a certain moment, then the data will most likely be missing continuously for a period of time in the next period of time. This feature brings great challenges to the current data interpolation methods, especially when the data is missing for a long time, these methods are almost ineffective. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a real-time processing method for missing values ​​and outliers in MEO satellite radiation dose data. Based on the prediction model, the missing values ​​and outliers in the latest downlinked data are processed in real time, thereby providing a complete and clean data set for subsequent data publishing, modeling, visualization and other tasks.

[0006] The technical solution adopted by the present invention to solve the technical problem comprises the following steps:

[0007] Step 1, select the detection data of a set duration as the training set;

[0008] Step 2, learning the oscillation period T of the radiation dose data from the data recorded by each radiation dose detector carried by the satellite;

[0009] Step 3, learning a radiation dose prediction model based on trend and Fourier features;

[0010] Step 4, performing outlier detection on the newly arrived radiation dose data based on the prediction model;

[0011] Step 5, replace missing values ​​and outliers in the newly arrived radiation dose data with predicted values;

[0012] Step 6, repeat steps 3 to 5 to process the data recorded by each radiation dose detector.

[0013] The step 1 selects 2 to 3 days of detection data as a training set.

[0014] The step 2 described above fits the linear trend of the recorded data and removes it; the detrended radiation dose data is used to fit a sine function with parameters. , where a and are the amplitude and phase respectively, so as to extract the oscillation period T.

[0015] The step 2 uses a sine function to fit the detrended data of each detector, extracts the residual noise and performs a Gaussianity test, selects the detector with the strongest Gaussianity of the residual noise, and learns the oscillation period of the radiation dose data from the data recorded by the detector.

[0016] The step 2 selects Dose3 to learn the oscillation period of the radiation dose data for the data Dose1, Dose2 and Dose3 generated by the three radiation dose detectors carried by the Beidou MEO satellite.

[0017] The radiation dose prediction model Among them, y t is the measured value of radiation dose, f(t) represents the trend of the data, S(t) represents the periodicity of the data, n is the number of data points, represents random noise.

[0018] The step 4 uses the radiation dose prediction model to predict the future radiation dose value; learns the standard deviation of noise from the training set; takes the predicted value as the expectation, and constructs the expected normal value range based on the noise standard deviation using the 3σ principle. If the newly arrived data exceeds this range, it is judged as an abnormal value and removed, otherwise, the original value is retained without processing.

[0019] The beneficial effects of the present invention are as follows: because step S2 adopts a data-driven method to dynamically learn the oscillation period of the radiation dose, the defect of subjective error caused by artificially specified periods is overcome, and a more accurate effect is achieved; because step S3 adopts multi-order Fourier features to model the volatility of data, the defect that the cosine function is difficult to characterize complex periodic changes is overcome, and the fitting error is greatly reduced.

[0020] The present invention selects the radiation dose detection data of a certain Beidou star in March 2019 as the object for experimentation, and the process is as follows: 1) Select the time point to be predicted, such as 00:00 on March 7, 2019; 2) Use the data of the past two days (N=2) as a training set to learn the prediction model, and make a prediction for the next day (assuming that the data for the next day is missing); 3) Compare the true value and the predicted value of the next day to evaluate the model effect. The model effect is evaluated by the root mean square error (RMSE), which is defined as follows:

[0021]

[0022] Where n is the number of data points, y t and are the true value and the predicted value respectively.

[0023] Figure 3 The prediction results (black), true values ​​(dark gray) and training data (light gray) used by the prediction model for the next day on March 7, 2019 are shown, where (a) and (b) represent the results of Dose2 and Dose3, respectively. It can be seen from the experimental results that the prediction results given by the prediction model based on trend and Fourier features proposed in the present invention can well reflect the changing law of the true value; although the fluctuation law of Dose2 is relatively complex, appropriately increasing the order (K) of the Fourier feature enables the model to capture the changing law.

[0024] In order to more accurately measure the prediction method of the present invention, the experiment selected March 6 to 10, 2019 as the prediction date, predicted the radiation dose values ​​in the three directions for the next day, and compared them with the prediction results of the forward interpolation method, and the results are shown in Table 1. It can be seen that no matter which direction of the radiation dose data is for, no matter which day the prediction result is, the RMSE of the prediction value of the method of the present invention is significantly better than that of the forward interpolation method.

[0025] Table 1 Comparison results between the proposed method and the forward interpolation method

[0026]

[0027] BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of radiation dose detection data of a Beidou satellite;

[0029] Figure 2 This is a trend and periodicity analysis chart of Dose3 radiation dose data of a Beidou satellite. The upper figure shows the raw data and linear trend, and the lower figure shows the detrended data and the seasonality fitted by the sine function.

[0030] Figure 3 The figure is a comparison chart of the predicted values, true values ​​and training data used by the prediction model, where (a) and (b) show the results of Dose2 and Dose3 respectively;

[0031] Figure 4 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and embodiments. The present invention includes but is not limited to the following embodiments.

[0033] In practical applications, users usually have historical data for a period of time (processed) and the latest batch of downloaded data (unprocessed). The present invention proposes a real-time processing method for missing values ​​and outliers in MEO radiation dose detection data based on a prediction model. The purpose is to learn rules from historical data and use the rules to perform real-time processing of missing values ​​and outliers in unprocessed data.

[0034] The present invention mainly comprises the following steps:

[0035] S1: Select the detection data of the past N days (usually N is 2 or 3) as the training set;

[0036] S2: Learn the oscillation period T of radiation dose data from Dose3 data (the satellite carries three radiation dose detectors, and the data generated are recorded as Dose1, Dose2, and Dose3 respectively);

[0037] S3: Learning radiation dose prediction models based on trend and Fourier features;

[0038] S4: Outlier detection based on prediction model;

[0039] S5: Use predicted values ​​to handle missing values ​​and outliers in the newly arrived radiation dose data;

[0040] S6: Repeat steps S3 to S5 for Dose1, Dose2, and Dose3 data respectively, and end after the processing is completed.

[0041] The specific contents of steps S2 to S4 are as follows:

[0042] 1. Learning the radiation dose data oscillation period (step S2)

[0043] Figure 1 The data show the detection data of three radiation dosimeters carried by a Beidou satellite from March 2 to March 4, 2019. It can be seen that the basic characteristics of the data are: 1) It shows an upward trend overall, and this trend can be regarded as linear growth in a short period of time (this method generally selects 2 to 3 days of data to train the model, and the trend in this time period is basically linear); 2) It has a periodic fluctuation law, among which the oscillation law of Dose3 is relatively simple and close to a sine wave, that is, the residual noise after fitting the periodicity of Dose3 with a sine function is close to the characteristics of Gaussian white noise. The above characteristics are not only applicable to Figure 1The data shown in the figure have been tested and all the data obtained so far have the above two characteristics. Therefore, it is a scientific and reasonable way to select Dose3 data and use the sine function fitting method to learn the oscillation period. Specifically, it is divided into three steps:

[0044] 1) Fit the linear trend of the data and remove it;

[0045] 2) Fitting a sine function with parameters to the detrended radiation dose data, that is,

[0046]

[0047] Among them, a and are amplitude and phase respectively, and T is the key parameter to be learned - the oscillation period;

[0048] 3) Extract the parameter T from the function fitted in step 2).

[0049] At this point, the oscillation cycle learning of the radiation dose detection data is completed.

[0050] It should be noted that for the current satellite radiation dose detection data, the user can directly specify the Dose3 data to learn the oscillation period, because the sinusoidal characteristics of Dose3 have been experimentally verified. In order to further increase the generalization of the method of the present invention and expand its application value, a judgment basis for selecting a subset of learning oscillation period data is given here, that is, when there are more satellites and more detectors in the future, which instrument's detection data should be selected to learn the oscillation period. The basis is: select the instrument data whose residual noise is closest to Gaussian white noise after the periodicity is removed by sinusoidal fitting. Specifically, 1) fit the data of a detector after trending with a sinusoidal function; 2) extract the residual noise and perform a Gaussianity test, such as calculating the autocorrelation coefficient of the noise; 3) traverse all detectors and select the detector with the strongest Gaussianity of the residual noise.

[0051] 2. Radiation dose prediction model based on trend and Fourier characteristics (step S3)

[0052] according to Figure 1 The fluctuation characteristics of the radiation dose data are shown, and the model shown in formula (1) is used to describe the variation law of the data.

[0053] y=f(t)+S(t)+∈, (1)

[0054] Among them, f(t)=c+βt represents the trend of the data (c and β represent the intercept and slope of the trend respectively), S(t) represents the periodicity of the data, represents random noise. Figure 2It can be seen that the radiation dose detection data targeted by the present invention presents a linear growth trend in the short term, so f(t) is selected here as a linear function. The periodic oscillation of the data is determined by the temperature sensitivity of the radiation dose detector. Its periodicity reflects the periodic change of the satellite's spatial position, which is an important characteristic of the data. According to the relevant theory of Fourier transform, any periodic function can be represented by an infinite series composed of sine and cosine functions; in practical applications, the first K orders of the infinite series are usually used to approximate the original function. Based on the above theory, S(t) is selected as

[0055]

[0056] Where T is the period of the original function, It is called Fourier feature, there are 2K in total, a k and b k are the coefficients of the cosine and sine parts of the Fourier characteristic respectively.

[0057] In this model, K and T are hyperparameters, which need to be specified before model learning or obtained using hyperparameter learning techniques. Please refer to Section 1 for the selection of T. The selection of K is related to the complexity of the periodic fluctuation law. According to the data characteristics targeted by the present invention, after experimental verification, K=5 is set here (for simple cases, K=2 or 5 has almost no effect on the results). After the hyperparameters are selected, the prediction model described in this section can be obtained by fitting the curve shown in formula (1) with the radiation dose data.

[0058] 3. Detection of abnormal values ​​of radiation dose data based on predicted values ​​(step S4)

[0059] The basic principle of this detection method is: use the model learned in Section 2 to predict the radiation dose value in the future (the time span depends on the time span of the newly arrived data, and the two are consistent). The predicted value can be regarded as the expected value of the new data; if the newly arrived radiation dose value deviates seriously from the expected value, it is regarded as an abnormal value. Here, the 3σ principle is used to determine whether the newly arrived data is abnormal. Specifically, it is divided into three steps:

[0060] 1) Use the model shown in formula (1) to predict future radiation dose values;

[0061] 2) Learn the standard deviation of the residual (or noise) from the training set;

[0062] 3) Take the predicted value as the expectation, and construct the expected normal value range based on the noise standard deviation using the 3σ principle. If the newly arrived data exceeds this range, it will be judged as an outlier and removed. Otherwise, the original value will be retained without processing.

[0063] Combine the following Figure 4The basic process shown is used to explain the embodiments of the present invention in detail.

[0064] Step 1: Select model training data

[0065] The data from the past N days (N is usually selected as 2 or 3) are selected as the training set to learn the hidden patterns in the data.

[0066] Step 2: Learn the radiation dose data oscillation period from Dose3

[0067] Figure 1 The data show the detection data of three radiation dosimeters carried by a Beidou satellite from March 2 to March 4, 2019. It can be seen that the basic characteristics of the data are: 1) the overall trend is upward, and this trend can be regarded as linear growth in a short period of time; 2) it has a periodic fluctuation law, among which the oscillation law of Dose3 is close to a sine wave.

[0068] Therefore, we choose Dose3 data to learn the oscillation period. Specifically, it is divided into two steps: 1) Fitting the linear trend and removing it, such as Figure 2 As shown above; 2) Fitting a sine function with parameters to the detrended time series, that is,

[0069]

[0070] Among them, a and are amplitude and phase respectively, T is the key parameter to be learned, and each parameter can be learned from the detrended data by the Ordinary Least Squares (OLS), that is,

[0071]

[0072] Among them, y t -f(t) is the detrended term obtained in step 1).

[0073] At this point, the oscillation period of the radiation dose detection data can be obtained, that is,

[0074] Step 3: Learn a prediction model based on trend and Fourier features

[0075] For one set of data (there are three sets of radiation dose data, namely Dose1, Dose2, and Dose3, and the method described in this step is only used for one set of data at a time), a prediction model based on trend and Fourier features is learned on the training set, as follows:

[0076] according to Figure 1 The fluctuation characteristics of the radiation dose data are shown, and the model shown in the following formula is used to describe the change law of the data.

[0077] y=f(t)+S(t)+∈,

[0078] Among them, f(t)=c+βt represents the linear trend of the data (c and β represent the intercept and slope of the trend respectively); S(t) represents the periodic law of the data, Represents random noise. According to Fourier transform theory, any periodic function can be represented by an infinite series consisting of sine and cosine functions; in practical applications, the first K orders of the infinite series are usually used to approximate the original function. Based on the above theory, S(t) is selected as

[0079]

[0080] Where T is the period of the original function, It is called Fourier feature, and there are 2K of them in total.

[0081] In this model, K and T are hyperparameters, which need to be specified before model learning or obtained using hyperparameter learning techniques. T is learned in step 2. The choice of K is related to the complexity of the periodic fluctuation law. The principle is: the higher the complexity, the larger K should be selected. For Dose3, its law is approximately consistent with the sine wave, so K = 2 can meet the conditions; for Dose1 and Dose2, it is set to K = 5. Experiments show that this setting is sufficient to make S (t) describe the periodic law of Dose1 and Dose2, and a larger K has little effect on increasing the accuracy of the model.

[0082] θ={c,β,a1,...,a K ,b1,...,b K} is the parameter of the model, which needs to be learned from the training data and can be obtained by OLS, that is,

[0083]

[0084] Where n is the number of sample data points, y t is the measured value.

[0085] Step 4: Outlier detection based on predicted values

[0086] The present invention uses the 3σ principle to perform outlier detection on newly arrived data. Specifically, it is divided into two steps:

[0087] 1) Learn the standard deviation of the residual (or noise) from the training set, that is,

[0088]

[0089] Where n is the number of data points, y t and are the true value and predicted value on the training set respectively;

[0090] 2) Use the model learned in step 3 to make predictions for the future, and use the 3σ principle to construct the expected normal value range. If the newly arrived data exceeds this range, it will be judged as an outlier and removed. Otherwise, the original value will be retained without processing, that is,

[0091]

[0092] Among them, y h is the newly arrived data at time h, is the model's predicted value for time h, and NA represents a null value.

[0093] Step 5: Use the predicted values ​​to handle missing values ​​and outliers in the newly arrived data

[0094] After using step 4, the outliers in the newly arrived data are marked as NA (missing values ​​are also automatically marked as NA), so filling NA in the data is equivalent to processing missing values ​​and outliers in the data. The specific method is: if NA appears in the newly arrived data, replace it with the corresponding predicted value; otherwise, keep the original value. That is,

[0095]

[0096] Step 6, repeat steps 3 to 5 for Dose1, Dose2, and Dose3 data respectively;

[0097] Steps 3, 4, and 5 are applied to all groups of radiation dose data (3 groups in total: Dose1, Dose2, and Dose3) to complete the processing of all data. Finally, the processed data is stored.

Claims

1. A real-time processing method for missing and outlier data of MEO satellite radiation dose, characterized in that: The following steps are involved: Step 1, select the detection data of a set duration as the training set; Step 2, learning the oscillation period T of the radiation dose data from the data recorded by each radiation dose detector carried by the satellite; Step 2 described above is to fit the linear trend of the recorded data and remove it; Fitting a sine function with parameters to detrended radiation dose data Among them, a and are the amplitude and phase respectively, so as to extract the oscillation period T; Step 3, learning a radiation dose prediction model based on trend and Fourier features; The radiation dose prediction model Among them, y t is the measured value of radiation dose, f(t) represents the trend of the data, S(t) represents the periodicity of the data, n is the number of data points, represents random noise; Step 4, performing outlier detection on the newly arrived radiation dose data based on the prediction model; Step 5, replace missing values ​​and outliers in the newly arrived radiation dose data with predicted values; Step 6, repeat steps 3 to 5 to process the data recorded by each radiation dose detector.

2. The method for real-time processing of missing and abnormal values ​​of MEO satellite radiation dose data according to claim 1 is characterized in that: The step 1 selects 2 to 3 days of detection data as a training set.

3. The real-time processing method for missing and abnormal values ​​of MEO satellite radiation dose data according to claim 1 is characterized in that: The step 2 uses a sine function to fit the detrended data of each detector, extracts the residual noise and performs a Gaussianity test, selects the detector with the strongest Gaussianity of the residual noise, and learns the oscillation period of the radiation dose data from the data recorded by the detector.

4. The method for real-time processing of missing and abnormal values ​​of MEO satellite radiation dose data according to claim 1, characterized in that: The step 2 selects Dose3 to learn the oscillation period of the radiation dose data for the data Dose1, Dose2 and Dose3 generated by the three radiation dose detectors carried by the Beidou MEO satellite.

5. The method for real-time processing of missing and abnormal values ​​of MEO satellite radiation dose data according to claim 1, characterized in that: The step 4 uses the radiation dose prediction model to predict the future radiation dose value; learns the standard deviation of the noise from the training set; The predicted value is taken as the expectation, and based on the noise standard deviation, the expected normal value range is constructed using the 3σ principle. If the newly arrived data exceeds this range, it is judged as an outlier and removed. Otherwise, the original value is retained without processing.

Images