Telemetry parameter prediction method based on improved attention mechanism ensemble machine learning

Abstract

This invention discloses a telemetry parameter prediction method based on an improved attention mechanism combined machine learning approach. The method includes: preprocessing historical telemetry data and historical covariate data to form combined data; inputting the combined data and pre-obtained future-time covariate data into a pre-established and trained telemetry parameter prediction model to obtain output data; passing the output data through a quantile function to obtain point and interval prediction results for the telemetry parameter at future times; the telemetry parameter prediction model includes a global model and a local model, obtaining global and local components respectively, which are coupled through a Gaussian likelihood function to obtain the output data. This invention utilizes a global-local structure to capture the global trend characteristics and local volatility characteristics of the target sequence, providing both point and interval prediction results. By using an attention mechanism in the global model to obtain the influence weight factors and long-term dependency information of row vectors on the predicted sequence, the prediction accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the field of satellite telemetry parameter prediction technology, and particularly relates to a telemetry parameter prediction method based on an improved attention mechanism combined with machine learning. Background Technology

[0002] During their long-term operation in orbit, spacecraft are affected by various factors such as sunlight, radiation, solar particles, and the complex systems within the spacecraft itself. This can lead to deviations from expected operational status, or even anomalies and malfunctions. By predicting the changing trends of satellite status parameters, operators can anticipate anomalies and increase response time for handling them. Telemetry data contains rich information about the satellite's on-orbit operational status and is a crucial input for predicting its changing trends. Since satellite on-orbit status is influenced by various factors, telemetry data exhibits both slow trend changes and irregular fluctuations, lacking a clear periodicity. Predictable or forecastable influencing factors, such as the satellite's orbit, the period of its passage through the South Atlantic Anomaly (SAA), and other predictable factors that aid in predicting parameter trends, can be defined as covariates.

[0003] Currently, telemetry data prediction is divided into point prediction (predicting the exact value of a time series) and interval prediction (predicting the possible fluctuation range of a time series). Point prediction can provide intuitive prediction results, but cannot reflect the uncertainty of the prediction results. Interval prediction can assess the uncertainty of the prediction, provide the possible fluctuation range of the prediction results, and improve the reliability of the prediction. Based on mathematical statistics, autoregressive models (AR), autoregressive moving average models (ARMA), autoregressive composite moving average models (ARIMA), and exponential smoothing have achieved good results in single-sequence point prediction of telemetry data. In terms of deep learning, Roman D uses deep learning models such as deep neural networks (DNN) and Bayesian ridge regression (BRR) to use the capacity of satellite lithium batteries as a health indicator to predict the remaining life of lithium batteries; O'MEARA uses autoencoders and long short-term memory networks (LSTM) for automatic feature extraction, anomaly detection, and telemetry data prediction, and trains iteratively to provide advance warnings of satellite anomaly patterns or behaviors. Li Zhiqiang [7] divides telemetry data into trend and fluctuation features using HP filtering, and captures them using global and local structure independent modeling, which improves the prediction accuracy. Datong, based on least squares support vector machines, utilized the Gaussian distribution of model estimation errors and confidence intervals to obtain the fluctuation range of anomalous segments. Fang used an ARMA time series prediction model to predict satellite voltage parameters, assuming the data followed a Gaussian distribution to obtain a 95% confidence interval range. Shi Mengxin, based on an attention mechanism combined with a quality-driven interval prediction method, performed multi-step interval prediction of lithium battery discharge current.

[0004] In recent years, data-driven ensemble machine learning methods have become a research hotspot both domestically and internationally in telemetry data prediction research. Different combinations of methods have different applicable scenarios. Mathematical statistics-based methods such as ARIMA, Support Vector Regression (SVR), Gaussian Regression (GPR), and Gray Model (GM) are not suitable for predictions involving covariates. Furthermore, Attention, which focuses on the time step information of a single sequence, cannot capture the correlation information between multiple parameters after introducing covariates. LSTM, due to its unique structure, has advantages in predicting time series data. Its separate modeling of the capture of the trend and volatility characteristics of the sequence improves prediction accuracy. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a telemetry parameter prediction method based on an improved attention mechanism combined with machine learning. To achieve high-accuracy prediction of telemetry parameters and assess the uncertainty of model predictions, thereby providing reliable support for ground decision-making and ensuring the safe operation of satellites, this invention, based on a deep learning model for time-series data, and taking into account the characteristics of slow trend changes and irregular fluctuations in telemetry parameter sequences, proposes a prediction model based on an improved attention mechanism combined with machine learning, providing point prediction and interval prediction results for telemetry sequences.

[0006] To achieve the above objectives, this invention proposes a telemetry parameter prediction method based on an improved attention mechanism combined machine learning approach, the method comprising:

[0007] Step S1) The historical telemetry data and historical covariate data are preprocessed to form combined data;

[0008] Step S2) Input the combined data and the pre-obtained future time covariate data into the pre-established and trained telemetry parameter prediction model to obtain the output data;

[0009] Step S3) The output data is processed through a quantile function to obtain the point prediction and interval prediction results of the telemetry parameter at future times;

[0010] The telemetry parameter prediction model includes a global model and a local model, which respectively obtain global components and local components, and then couple them through a Gaussian likelihood function to obtain output data.

[0011] As an improvement to the above method, step S1) specifically includes:

[0012] The satellite telemetry parameters and covariate data at historical moments are processed by outlier removal, missing value filling and normalization to obtain combined data; wherein the covariates include relevant influence parameter data obtained from satellite orbit data and planning instructions from operators.

[0013] As an improvement to the above method, the input to the global model is a combination of historical time-time data and pre-obtained future time-time covariate data, and the output is a global component. The global model includes an LSTM network and an attention mechanism; wherein...

[0014] The LSTM network is a linear combination of several RNN networks used to obtain preliminary global components;

[0015] The attention mechanism obtains hidden state information by introducing a CNN network and then performs a weighted summation of the row vectors of the hidden state information to obtain the global component.

[0016] As an improvement to the above method, the processing procedure of the LSTM network specifically includes:

[0017] X the historical combination data from the previous time step t-1 and the output g of LSTM t-1 and the covariate B at the next time step t As input to the next time step, the LSTM output g for the next time step is obtained. t :

[0018] g t =G(g) t-1 ,X t-1 B t )

[0019] Where G(·) represents the LSTM network.

[0020] As an improvement to the above method, the attention mechanism specifically includes the following processing steps:

[0021] Use CNN to obtain g t Variable patterns:

[0022]

[0023] in, The convolution value in the i-th row and j-th column represents the value of the LSTM; w represents the time window length of the LSTM; l represents the l-th column of the hidden state; C j,t-w+l This represents the j-th row and t-w+l-th column of the convolution weight matrix;

[0024] Then, attention is weighted and normalized using sigmoid for multivariate selection:

[0025]

[0026]

[0027] in, It is the hidden state H C The i-th row, W a W represents network parameters. a ∈R k×m ;

[0028] Based on the attention weights, a weighted sum is performed on each row to obtain the global variable h. t :

[0029]

[0030] As an improvement to the above method, the local model takes the combined historical time data as input and the pre-obtained future time covariate data as input, and outputs local components that satisfy a normal distribution.

[0031] As an improvement to the above method, the global and local components are used as the mean and standard deviation (μ,σ) of the Gaussian likelihood for parameterization, and the global and local components are coupled through a Gaussian function to obtain the output result.

[0032] As an improvement to the above method, step S3) specifically includes:

[0033] For the output results, a quantile function is used to obtain the corresponding telemetry parameter point prediction and interval prediction results for a given quantile ρ∈(0,1).

[0034] As an improvement to the above method, the telemetry parameter prediction model is trained by using the Loss function to estimate the target value with maximum likelihood during the training phase, so as to obtain the optimal (μ,σ) and model parameters.

[0035] On the other hand, this invention proposes a telemetry parameter prediction system based on an improved attention mechanism combined machine learning, the system comprising:

[0036] The preprocessing and combination module is used to preprocess historical telemetry data and historical covariate data to form combined data;

[0037] The prediction module is used to combine data and pre-obtained future time-time covariate data as input to a pre-built and trained telemetry parameter prediction model to obtain output data; and

[0038] The result output module is used to process the output data through a quantile function to obtain the point prediction and interval prediction results of the telemetry parameter at future times;

[0039] The telemetry parameter prediction model includes a global model and a local model, which respectively obtain global components and local components, and then couple them through a Gaussian likelihood function to obtain output data.

[0040] Compared with the prior art, the advantages of the present invention are:

[0041] 1. This invention draws on the powerful learning capabilities of deep learning models for time series data and proposes a satellite telemetry parameter prediction method based on improved combinatorial machine learning. It can use covariates whose sequence values ​​are known at any time and which have an auxiliary effect on the prediction target sequence (calculated in advance through satellite orbit and mission schedule, etc.) as model inputs, and the prediction target sequence as model inputs. Through a global-local fusion model structure and network learning with improved attention mechanism, it provides point prediction and interval prediction results of telemetry parameter sequences. This provides highly reliable decision support for ground personnel to monitor satellite status and handle anomalies, ensuring long-term safe and reliable operation.

[0042] 2. This invention utilizes a global-local structure to capture the global trend characteristics and local fluctuation characteristics of the target sequence, and can provide point prediction and interval prediction results;

[0043] 3. A new attention mechanism is used instead of the traditional attention mechanism that focuses on time step information. This mechanism obtains the weight factors of the row vectors on the predicted sequence and long-term dependency information, thereby improving the accuracy of the prediction.

[0044] 4. The algorithm proposed in this invention can be used for point prediction and interval prediction of satellite telemetry parameter sequences. By acquiring more temporal information of the sequence and long-term dependent global and local models, it captures the global trend and local fluctuation characteristics of the telemetry parameter sequence, respectively. This enables point prediction of the exact value at a certain time point and interval prediction of the range of value fluctuations, providing high-precision prediction results. Furthermore, by including covariates that have an auxiliary effect on the predicted target sequence as model input, the accuracy of the prediction is improved, providing decision support for satellite ground support systems. Attached Figure Description

[0045] Figure 1 This is a flowchart of the telemetry parameter prediction method based on improved attention mechanism combined machine learning according to the present invention;

[0046] Figure 2 It uses univariate telemetry data to predict the next 30 moments.

[0047] Figure 3 It uses univariate telemetry data to predict the next 50 moments.

[0048] Figure 4 It uses multidimensional data to predict the next 30 moments;

[0049] Figure 5 It uses multidimensional data to predict the next 50 moments. Detailed Implementation

[0050] This invention proposes a prediction model based on improved Attention ensemble machine learning, which acquires different features separately by referring to the global-local fusion structure, providing high-accuracy point prediction and interval prediction results.

[0051] The technical process of this method is as follows: After preprocessing the telemetry parameter data and covariate data by outlier removal, missing value filling and normalization, the data is divided into training data and prediction data. The training data is used to train the model. After being input into the model, the global component and local component are obtained by passing through the global model and local model respectively. The two components are coupled by the Gaussian likelihood function to obtain the prediction model. The test data is used to train the prediction model. The output data of the prediction model is passed through the quantile function to obtain the final point prediction and interval prediction results.

[0052] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0053] Example 1

[0054] like Figure 1 As shown, Embodiment 1 of the present invention proposes a telemetry parameter prediction method based on an improved attention mechanism combined with machine learning. The method includes:

[0055] Step S1) The historical telemetry data and historical covariate data are preprocessed to form combined data;

[0056] Step S2) Combine the data and the pre-obtained future time covariate data and input them into the pre-established and trained telemetry parameter prediction model to obtain the output data;

[0057] Step S3) The output data is processed through a quantile function to obtain the point prediction and interval prediction results of the telemetry parameter at future times;

[0058] The telemetry parameter prediction model includes a global model and a local model, which respectively obtain global components and local components, and then couple them through a Gaussian likelihood function to obtain output data.

[0059] The following is a detailed analysis:

[0060] The satellite's on-orbit status is influenced by various factors, resulting in telemetry data exhibiting both slow trend changes and irregular fluctuations, lacking a clear periodicity. Predictable or foreseeable influencing factors, such as the satellite's position within the Earth's shadow or sunlight zones, the time it passes through the South Atlantic Anomaly (SAA), and other predictable factors that aid in predicting the trend of the parameter data, can be defined as covariates. Definition 1: At time t, the covariate is:

[0061] B t =(x 1,t ,x 2,t ,...,x n,t )

[0062] In the formula x i,t It is the value of the i-th covariate at time t, and n represents the dimension of the covariate.

[0063] Definition 2: Based on the satellite telemetry data at time t and the known covariate B t Obtain the combined satellite data at time t:

[0064] X t =(t,B t ,y t )

[0065] Where yt This represents the value of the telemetry data at time t.

[0066] Define the telemetry data prediction result of satellite 3 at time t as Z. t :

[0067]

[0068] Where d t ,s t Let represent the upper and lower bounds of the telemetry data at time t, respectively. This represents the prediction result of the telemetry data points at time t.

[0069] Definition 4: The problem of predicting satellite telemetry data can be abstracted as follows:

[0070]

[0071] in This represents historical combination data from time t1 to t2. The covariate representing the future time intervals from t2+1 ​​to t3. It is the predicted result from t2+1 ​​to t3 in the future.

[0072] 1. Global Model

[0073] (1) Preliminary global components

[0074] The initial global components are provided by a Long Short-Term Memory (LSTM) network consisting of k linearly combined recurrent neural networks (RNNs), which takes the telemetry parameter sequence value Y from the previous time step as an input. t-1 and the covariate B at the next time step t =(x 1,t ,x 2,t ,...,x n,t ) and the output g of LSTM t-1 As input for the next time step, i.e.:

[0075] g t =G(g) t-1 ,X t-1 B t )

[0076] Where x n,t is the value of a covariate in the covariate matrix at time step t. The initial state of g is the output of the LSTM network G(X,B). The same network structure is used in both the training and prediction phases, and weights are shared to reduce computation.

[0077] (2) Attention mechanism

[0078] Traditional time series attention mechanisms primarily focus on improving task performance by modifying the network structure of the attention model, selecting information relevant to the current time step. The context vector is a column-weighted sum of the current RNN hidden state matrix. This design is suitable for tasks where each time step contains a single sequence, focusing only on a few time steps and struggling to capture periodic information across multiple time steps. While LSTNet incorporates a skip mechanism and traditional attention to alleviate this problem, it is designed for strongly periodic sequence data. Due to satellite orbiting the Earth, telemetry data exhibits a certain degree of pseudo-periodicity, which traditional methods cannot satisfy. The attention mechanism of this invention pays more attention to row vectors rather than time steps, considering the weighting factors of the influence of covariates on the predicted target sequence, making it suitable for multivariate time series. Because convolutional neural networks (CNNs) can capture various important patterns, CNNs are introduced to extract important information from the hidden state of each individual variable, using g... t As input to the attention mechanism function, the Attention function is defined as follows:

[0079] Use CNN to obtain g t Variable patterns:

[0080]

[0081] in, The convolution value in the i-th row and j-th column represents the value of the LSTM; w represents the time window length of the LSTM; l represents the l-th column of the hidden state; C j,t-w+l This represents the j-th row and t-w+l-th column of the convolution weight matrix;

[0082] Then, attention is weighted and normalized using sigmoid for multivariate selection:

[0083]

[0084]

[0085] in, It is the hidden state H C The i-th row, W a W represents network parameters. a ∈R k×m ;

[0086] Based on the attention weights, a weighted sum is performed on each row to obtain the global variable h. t :

[0087]

[0088] At this time, the global vector h tIt contains a weighted sum of row vectors with multiple time steps, so it can capture long time step information and time information of covariates in the target sequence.

[0089] 2. Local Model

[0090] The local model R can be any classical probability time series model. To efficiently compute the Gaussian likelihood function P(Z)... i |R i Based on the analysis of the data, the local component r is obtained. t Satisfies the normal distribution assumption. in (i.e. l) t ) is composed of the same input features X as the global model. t-1, B t The result obtained through RNN: It should be noted that this embodiment uses RNN, but is not limited to it.

[0091] l t =R(X) t-1 B t )

[0092] r t ~N(0,l t )

[0093] 3. Optimize objectives

[0094] This paper assumes that the data follows a Gaussian distribution, and uses the global and local components as the mean and standard deviation (μ, σ) of the Gaussian likelihood to parameterize it, that is:

[0095]

[0096] During the overall training phase, the optimal (μ,σ) and model parameters are obtained by using the Loss function to estimate the target value with maximum likelihood, i.e.:

[0097]

[0098] When outputting prediction results, the quantile function is used, which can provide point prediction and interval prediction results for a given quantile ρ∈(0,1).

[0099] Simulation example:

[0100] Taking a certain on-orbit scientific satellite as an example, since the power system is the basis for the normal operation of the satellite payload, and the bus voltage can directly reflect the state of the power system, the bus voltage telemetry parameters are selected for model training. The model parameters are set using indicators such as whether the satellite is in the shadow zone at a certain time, the payload working mode, and whether the satellite is in the SAA zone as covariates. At the same time, publicly available datasets Solar Energy and Electricity in the field of time series prediction are selected for experiments to verify the robustness of the algorithm proposed in this invention.

[0101] (1) Univariate telemetry data

[0102] The model input is 1-dimensional, and the results of predicting at 30 and 50 time steps are shown below when only the target sequence is present:

[0103] The single-input prediction results for the target sequence are shown in the figure above, displaying the values ​​at the 50th, 10th, and 90th quantiles. The dashed lines p10 quantile and p90 quantile represent the 10th and 90th quantiles, respectively, and are considered as the range of values ​​for the prediction results. The dashed line p50forecast represents the 50th quantile, which is considered as the accurate value of the prediction, i.e., the result of the point prediction. The solid line true represents the actual data.

[0104] Figure 2 To predict the result map for 30 future moments using data from 120 historical moments, the evaluation formula MAPE was used. The results of point prediction (i.e., 50th percentile) and the actual values ​​were used for evaluation. On the test set, the MAPE value of the prediction results was 0.23241 and the RMSE value was 0.45372. Figure 3 The graph shows the results of using data from 120 historical time points to predict 50 future time points. The MAPE value is 0.29636 and the RMSE value is 0.53844. The experiments demonstrate that the model proposed in this invention is quite accurate in time series prediction.

[0105] (2) Model validation on public datasets

[0106] Experiments were conducted on two public datasets using single-dimensional target sequences as input, and the benchmark algorithms in Table 1 were used to evaluate the prediction results of each model using MAPE and RMSE regression evaluation metrics to verify the robustness of the model proposed in this invention.

[0107] The results of the evaluation index MAPE are shown in the table below:

[0108] Table 1. MAPE for each dataset and corresponding algorithm

[0109]

[0110] Table 1 shows the average MAPE results obtained by each comparative model after 20 rounds of validation with a training step size of 100 and a prediction step size of 30. On the Solar Energy dataset, the proposed model shows a maximum reduction in MAPE of 0.1144 compared to SVR and a minimum reduction of 0.0108 compared to LSTM. On the Electricity dataset, compared to other algorithms, the MAPE shows a maximum reduction of 0.3231 compared to GP and a minimum reduction of 0.1073 compared to LSTM.

[0111] The results show that the model proposed in this invention achieves better results than other models on both satellite telemetry datasets and Electricity datasets. In particular, on the Electricity dataset, the MAPE value of the model proposed in this invention decreases significantly compared to other models. This is because the electricity dataset is more affected by human randomness, and the data periodicity is less obvious compared to the Solar dataset. The model proposed in this invention is more advantageous for such data.

[0112] (3) Model robustness experiment

[0113] To verify the robustness of the model, a telemetry dataset was used, and covariates were added along with the telemetry parameter sequence as model inputs to validate the model.

[0114] The prediction results are shown below when the sampling rate is 30 seconds, the model input is 4-dimensional, including covariates and the target sequence:

[0115] Figure 4 The two methods respectively use data from 120 historical time points to predict 30 time points. Figure 3 ) and predict 50 time points ( Figure 5 The results were evaluated using the point prediction results and the actual values. On the test set, the MAPE value for predicting the next 30 time points was 0.17361 and the RMSE was 0.31695; when the sampling interval was 30 seconds, the MAPE value for predicting the next 50 time points was 0.24161 and the RMSE was 0.34058.

[0116] Experiments show that the model proposed in this invention is relatively accurate in predicting time series. Furthermore, by adding covariates that affect the target series, the model's learning of time series patterns and local volatility becomes more accurate.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A telemetry parameter prediction method based on an improved attention mechanism combined with machine learning, for predicting satellite telemetry parameters, the method comprising: Step S1) The historical telemetry data and historical covariate data are preprocessed to form combined data; The covariates include relevant influencing parameter data obtained from satellite orbit data and planning instructions from operators; Step S2) Input the combined data and the pre-obtained future time covariate data into the pre-established and trained telemetry parameter prediction model to obtain the output data; Step S3) The output data is processed through a quantile function to obtain the point prediction and interval prediction results of the telemetry parameter at future times; The telemetry parameter prediction model includes a global model and a local model, which respectively obtain global and local components. The global and local components are used as the mean and standard deviation of the Gaussian likelihood. The parameters are parameterized, and the global and local components are coupled by a Gaussian likelihood function to obtain the output data. The global model includes an LSTM network and an attention mechanism. The attention mechanism obtains hidden state information by introducing a CNN network and then performs a weighted summation of the row vectors of the hidden state information to obtain the global components. The processing procedure of the LSTM network specifically includes: Combine historical data from the previous time step and the output of LSTM and covariates at the next time step As the input to the next time step, the LSTM output for the next time step is obtained. : in, Indicates an LSTM network. t- 1 represents the previous time step. t Indicates the next time step; The attention mechanism's processing steps specifically include: Use CNN to obtain Variable patterns: in, Representing the Line number The convolution value of the column, w Indicates the length of the LSTM time window; l The hidden state is represented by the first... l List; Represents the convolution weight matrix j OK tw +l column; Then, attention is weighted and through sigmoid Function normalization is used for multivariate selection. in, It is in a hidden state. The OK, Indicates network parameters; Based on the attention weights, a weighted sum is performed on each row to obtain the global variable. : ； The input to the local model is a combination of historical time data and pre-obtained future time covariate data, and the output is the local components, which follow a normal distribution.

2. The telemetry parameter prediction method based on improved attention mechanism combined machine learning according to claim 1, characterized in that, Step S1) specifically includes: The combined data is obtained by removing outliers, filling missing values, and normalizing the satellite telemetry parameters and covariate data at historical moments.

3. The telemetry parameter prediction method based on improved attention mechanism combined machine learning according to claim 2, characterized in that, The input to the global model is a combination of historical time-time data and pre-obtained future time-time covariate data; the output is the global components. The LSTM network is a linear combination of several RNN networks used to obtain preliminary global components.

4. The telemetry parameter prediction method based on improved attention mechanism combined machine learning according to claim 1, characterized in that, Step S3) specifically includes: For the output results, a quantile function is used, for a given quantile... The results of point prediction and interval prediction for the corresponding telemetry parameters are obtained.

5. The telemetry parameter prediction method based on improved attention mechanism combined machine learning according to claim 4, characterized in that, The telemetry parameter prediction model is trained using the Loss function and maximum likelihood estimation of the target value during the training phase to obtain the optimal result. and model parameters.

6. A system based on the telemetry parameter prediction method based on improved attention mechanism combined machine learning as described in claim 1, characterized in that, The system includes: The preprocessing and combination module is used to preprocess historical telemetry data and historical covariate data to form combined data; The prediction module is used to combine data and pre-obtained future time-time covariate data as input to a pre-built and trained telemetry parameter prediction model to obtain output data; and The result output module is used to process the output data through a quantile function to obtain the point prediction and interval prediction results of the telemetry parameter at future times; The telemetry parameter prediction model includes a global model and a local model, which respectively obtain global components and local components, and then couple them through a Gaussian likelihood function to obtain output data.

Images