An anomaly detection method applicable to spacecraft telemetry data

Abstract

The present invention belongs to the field of satellite telemetry technology, and particularly relates to an anomaly detection method applicable to spacecraft telemetry data. The detection method includes the following steps: S1: Data acquisition, select a specific spacecraft, obtain its telemetry data, and select parameters related to electrical characteristics from the telemetry data, and divide the parameter dataset into training data and test data according to a ratio of 4:6; S2: Preprocess the data, which includes the following steps: S21: Perform data filling. According to the trend of the change of telemetry parameters, fill the missing data points by taking the average value of the previous second and the next second. If a certain parameter is always missing, fill it with 0; The present invention can combine the ability of the LSTM model to capture long-term dependencies and the ability of the Transformer model to capture global dependencies, and designs a Transformer-LSTM model, which can more accurately detect anomalies in telemetry data.

Description

Technical Field

[0001] The present invention belongs to the technical field of satellite telemetry, and particularly relates to an anomaly detection method applicable to spacecraft telemetry data. Background Art

[0002] With the rapid development of modern aerospace, the functions of spacecraft are becoming increasingly perfect. As a type of spacecraft, communication satellites have an important impact on the development of the national aerospace industry. With the increasing complexity of the structure of communication satellites, the on-orbit failure rate has increased significantly. Satellite telemetry parameters are one of the key indicators for evaluating the satellite state, and their high dynamicity and diversity make them a key reference for satellite performance and health status. Therefore, it has become an urgent task in the aerospace field to monitor, analyze, and interpret satellite telemetry data to achieve fault detection and alarm of communication satellites. In this context, it is crucial to develop an efficient anomaly detection method for satellite telemetry data, which is also one of the current main concerns in this field.

[0003] Currently, there are mainly four types of methods for detecting anomalies in spacecraft telemetry data in the field: methods based on manual monitoring combined with thresholds, expert systems, models constructed based on expert experience, and machine learning. Among them, the method based on manual monitoring combined with thresholds requires a large amount of time and human resources, is difficult to apply to large-scale data, and does not have good scalability. The method based on expert systems cannot effectively detect some unknown anomalies. Constructing a model based on expert experience requires a large amount of time to learn expert experience, and it is impossible to establish corresponding models for each dimension of parameters for ultra-large-scale data, and all anomalies cannot be recorded. Machine learning can establish a clustering model in the case of unlabeled data, reducing the need for labeled data. At the same time, deep learning technology shows extremely strong adaptability in processing time series data, can better capture the trend information of time series data, and also has the ability to process large-scale data. Therefore, the method based on machine learning has better adaptability and performance in the field of satellite fault detection compared with traditional methods.

[0004] However, the current fault detection still mainly relies on simple threshold detection, and can only detect some specific faults. Moreover, satellite telemetry data has the characteristics of large data volume, high dimension, and complex data relationships, and it is difficult to accurately detect faults using traditional detection methods. Summary of the Invention

[0005] The purpose of the present invention is to provide an anomaly detection method applicable to spacecraft telemetry data, which can combine the ability of the LSTM model to capture long-term dependencies and the ability of the Transformer model to capture global dependencies, design a Transformer-LSTM model, and can more accurately detect anomalies in telemetry data.

[0006] The technical solution adopted by the present invention is as follows:

[0007] An anomaly detection method applicable to spacecraft telemetry data, the detection method comprising the following steps:

[0008] S1: Data acquisition, select a spacecraft, this spacecraft is specific, acquire its telemetry data, and select parameters related to electrical characteristics from the telemetry data, and divide the parameter data set into training data and test data according to a ratio of 4:6;

[0009] S2: Preprocess the data, which includes the following steps:

[0010] S21: Perform data filling. According to the trend of the change of telemetry parameters, fill the missing data points by taking the average of the previous second and the next second. If a certain parameter is always missing, fill it with 0;

[0011] S22: Directly delete the data points with discontinuous jump characteristics;

[0012] S23: Parameter normalization, ensure that different features have similar numerical ranges, and prevent certain features from dominating the calculation process;

[0013] S3: Define the training data as y train y train and the test data as y test y test The predicted result data obtained by inputting into the model is defined as y pre y pre Set 3σ as the normal variance standard of the sample. Input the training data y train y train into the Transformer-LSTM model;

[0014] S4: Calculate the distance between y test and y pre and use the 3σ criterion to determine the normal interval, that is, the data samples with variances exceeding the 3σ interval are considered abnormal;

[0015] S5: After the model training is completed, deploy it for anomaly detection of spacecraft telemetry data.

[0016] In the above S3, the training process of the model includes the following steps:

[0017] S31: According to the input data y train , design a Transformer encoder with an appropriate number of layers;

[0018] S32: The Transformer encoder continuously learns the changing trends and patterns of the spacecraft time series through the Multi-HeadAttention mechanism, and obtains an output that can effectively represent the changing patterns of a number of input spacecraft parameter sequences;

[0019] S33: The LSTM decoder receives the output of the Transformer encoder as input in chronological order and uses this output as the initial hidden state;

[0020] S34: The LSTM decoder receives a special start symbol to generate the target sequence, and at each time step, it receives the output of the previous time step and the hidden state of the current time step to generate the output of the current time step and update the current hidden state until the output y is obtained pre 。

[0021] In S3, the modeling method is as follows:

[0022] a. Design N layers of Transformer encoders, each layer containing a Multi-HeadAttention sublayer and a feed-forward neural network sublayer. Assume the input is X ∈ R L×F , representing L vectors of dimension F;

[0023] b. The Transformer encoder represents a function T: R L×F → R L×F ;

[0024] Among them, T is composed of N layers of T1(·), T2(·), …, T N (·). Each layer can be expressed as T m (X) = f m (Attention m (x) + x), f m (·) is a simple feed-forward neural network layer, and Attention m (x) is the self-attention mechanism function;

[0025] c. The LSTM decoder receives the output X output ∈ R L×F of the Transformer encoder as input, and sets it as the initial hidden state H0, initializing the weight parameters and memory cell C0 of the LSTM decoder;

[0026] d. At each time step t, the LSTM decoder receives the input X t and the hidden state H t-1 of the previous time step, and then calculates the input gate I t , the forget gate F t , and the output gate Ot and candidate memory cells The values are calculated as follows:

[0027] I t = σ(X t W xi + H t-1 W hi + b i )

[0028] F t = σ(X t W xf + H t-1 W hf + b f )

[0029] O t = σ(X t W xo + H t-1 W ho + b o )

[0030]

[0031] Among them, W xi , W xf , W xo , W hi , W hf , W ho , b i , b f , b o , W xc , W hc are the weight parameters of the LSTM model.

[0032] Update the memory cell C t and the hidden state H t , where the input gate I t and the forget gate F t are used to update the memory cell C t , and the output gate O t and the updated memory cell C t are used to calculate the hidden state H t at the current time step.

[0033] In step d, the update process is modeled as follows:

[0034]

[0035] H t = O t ⊙ tanh(C t )

[0036] Among them, ⊙ represents element-wise multiplication; until the end of the time step, the output y is obtained. pre .

[0037] In the above-mentioned S4, the method for determining the credibility of the normal interval using the 3σ criterion is as follows:

[0038] S41: Introduce the Gaussian distribution parameters μ and σ for the prediction error, take the standard deviation multiplier N as the adjustment parameter, and follow the principle of maximizing the correlation coefficient.

[0039] S42: To measure the credibility of the standard, r is used to represent the correlation coefficient between the actual fault value y and the predicted fault value The range is from -1 to 1. If the value is 1, it indicates a perfect linear match between these two variables.

[0040] In the above-mentioned S41, define the binary variables y and y represents the actual fault value, represents the predicted fault value. A value of 0 indicates no fault, and a value of 1 indicates the occurrence of a fault. Define the fault value predicted by the i-th time series with the following calculation formula:

[0041]

[0042] The calculation formula in the above-mentioned S42 is as follows:

[0043]

[0044] The technical effects achieved by the present invention are as follows:

[0045] An anomaly detection method applicable to spacecraft telemetry data according to the present invention can capture the relationships between different time steps in the sequence through the Multi-Head Attention mechanism. Moreover, due to the parallel computing ability of the Transformer, it does not assign higher weights to the data at the nearest time points and can also extract features of more distant time series. Therefore, the output of the Transformer encoder can effectively represent the variation law of the input sequence of several spacecraft parameters. In this process, the Transformer encoder helps to model the normal operation mode of the spacecraft and provides a basis for subsequent anomaly detection.

[0046] An anomaly detection method applicable to spacecraft telemetry data according to the present invention is applicable to the anomaly detection of spacecraft telemetry data. By combining the Transformer encoder and the LSTM decoder, they complement each other, can not only learn the global laws of time series data but also grasp the local time-dependent relationships, thus achieving precise detection of abnormal data. Description of the Drawings

[0047] Figure 1 is the flowchart of an embodiment of the present invention;

[0048] Figure 2 is the structural diagram of the LSTM model of an embodiment of the present invention;

[0049] Figure 3 is the structural diagram of the Transformer model of an embodiment of the present invention;

[0050] Figure 4 is the performance comparison diagram of the detection using the LSTM model, the Transformer model and the Transformer-LSTM model in an embodiment of the present invention. Detailed implementation manners

[0051] In order to make the objectives and advantages of the present invention clearer, the present invention will be specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific implementation manners of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0052] As Figures 1-4 shown, LSTM is a special recurrent neural network, which consists of an input gate, a forget gate, an output gate, and memory cells for recording additional information. Based on learning the rules between complex data, LSTM takes into account time and non-linear relationships, and has excellent modeling capabilities for the temporal information of sequence data. The LSTM decoder takes the output of the Transformer encoder as input and predicts the future values of several parameter time series of the spacecraft in chronological order. The output of each time step is affected by the input of the current time step and the memory of the previous time step, which enables LSTM to capture temporal information and trends well. This dependency between time steps is transmitted through the memory of the cell state, giving full play to the unique advantages of LSTM in local order modeling.

[0053] The Transformer model is a deep learning model with powerful sequence modeling and feature extraction capabilities. In the field of anomaly detection for spacecraft telemetry data, the role of the Transformer encoder is to process the input telemetry data sequence to learn the changing trends and patterns of the spacecraft time series. Through the Multi-Head Attention mechanism, it can capture the relationships between different time steps in the sequence. Moreover, due to the parallel computing ability of the Transformer, it does not assign higher weights to the data at the most recent time points and can also extract features from more distant time series. Therefore, the output of the Transformer encoder can effectively represent the changing patterns of the input sequence of several spacecraft parameters. In this process, the Transformer encoder helps to model the normal operating mode of the spacecraft, providing a basis for subsequent anomaly detection.

[0054] The present invention is applicable to anomaly detection of spacecraft telemetry data. By combining the Transformer encoder with the LSTM decoder, the two complement each other, being able to learn both the global patterns of time series data and grasp local time-dependent relationships, thus achieving precise detection of abnormal data. The specific implementation steps are as follows:

[0055] An anomaly detection method applicable to spacecraft telemetry data, the detection method comprising the following steps:

[0056] S1: Data acquisition. Select a specific spacecraft, obtain its telemetry data, and select the parameters related to electrical characteristics from the telemetry data. Since it is difficult to obtain fault data, this experiment uses spacecraft periodic events as abnormal data. The total number of datasets involved in the finally selected periodic events is 125,000, and the number of abnormal points is 25,000. Divide the parameter dataset into training data and test data according to a ratio of 4:6;

[0057] S2: Preprocess the data, which includes the following steps:

[0058] S21: Perform data filling. According to the trend of telemetry parameter changes, fill in the missing data points by taking the average of the previous second and the next second. If a certain parameter is always missing, fill it with 0;

[0059] S22: Directly delete the data points with discontinuous jump characteristics;

[0060] S23: Parameter normalization to ensure that different features have similar numerical ranges and prevent some features from dominating the calculation process;

[0061] S3: Define the training data as y train y train and the test data as y test y test, the prediction result data obtained by the input model is defined as y pre y pre , set 3σ as the normal variance standard of the sample. The training data y train y train is input into the Transformer-LSTM model;

[0062] S4: Calculate the distance between y test and y pre , and use the 3σ criterion to determine the normal interval, that is, the data samples with variances exceeding the 3σ interval are considered abnormal;

[0063] S5: After the model training is completed, deploy it for anomaly detection of spacecraft telemetry data.

[0064] In S3, the training process of the model includes the following steps:

[0065] S31: According to the input data y train , design a Transformer encoder with an appropriate number of layers;

[0066] S32: The Transformer encoder continuously learns the change trends and rules of the spacecraft time series through the Multi-HeadAttention mechanism, and obtains an output that can effectively represent the change rules of a number of input spacecraft parameter sequences;

[0067] S33: The LSTM decoder receives the output of the Transformer encoder in chronological order as input and uses this output as the initial hidden state;

[0068] S34: The LSTM decoder receives a special start symbol, generates the target sequence, and receives the output of the previous time step and the hidden state of the current time step at each time step, so as to generate the output of the current time step and update the current hidden state until the output y pre .

[0069] In S3, the modeling method is as follows:

[0070] a. Design N layers of Transformer encoders, each layer containing a Multi-HeadAttention sublayer and a feed-forward neural network sublayer. Assume the input is X ∈ R L×F , representing L vectors of dimension F;

[0071] b. The Transformer encoder represents a function T: R L×F → R L×F ;

[0072] Among them, T consists of T1(·), T2(·), …, TN These N layers are composed, and each layer can be expressed as T m (X) = f m (Attention m (x) + x), where f m (·) is a simple feedforward neural network layer, and Attention m (x) is the self-attention mechanism function;

[0073] c. The LSTM decoder receives the output X output ∈R L×F from the Transformer encoder as the input, and sets it as the initial hidden state H0. Initialize the weight parameters of the LSTM decoder (such as W xi , W xf , W xo , W hi , W hf , W ho , b i , b f , b o , W xc , W hc , b c , etc.) and the memory cell C0; these parameters will be gradually updated during the training process.

[0074] d. At each time step t, the LSTM decoder receives the input X t and the hidden state H t-1 from the previous time step, and then calculates the input gate I t , the forget gate F t , the output gate O t and the value of the candidate memory cell . Their calculation formulas are shown as follows respectively:

[0075] I t = σ(X t W xi + H t-1 W hi + b i )

[0076] F t = σ(X t W xf + H t-1 W hf + b f )

[0077] O t = σ(X t W xo + H t-1 W ho + bo )

[0078]

[0079] Among them, W xi 、W xf 、W xo 、W hi 、W hf 、W ho 、b i 、b f 、b o 、W xc 、W hc are the weight parameters of the LSTM model.

[0080] Update the memory cell C t and the hidden state H t , where the input gate I t and the forget gate F t are used to update the memory cell C t , and the output gate O t and the updated memory cell C t are used to calculate the hidden state H t at the current time step.

[0081] In step d, the update process is modeled as follows:

[0082]

[0083] H t = O t ⊙ tanh(C t )

[0084] where ⊙ represents element-wise multiplication; until the end of the time step, the output y pre is obtained.

[0085] In S4, the method for determining the credibility of the normal interval using the 3σ criterion is as follows:

[0086] S41: Introduce the Gaussian distribution parameters μ and σ for the prediction error, take the standard deviation multiplier N as the adjustment parameter, and follow the principle of maximizing the correlation coefficient;

[0087] S42: To measure the credibility of the standard, use r to represent the correlation coefficient between the actual fault value y and the predicted fault value ranging from -1 to 1. If the value is 1, it indicates a perfect linear match between these two variables.

[0088] In S41, define the binary variables y and y represents the actual fault value, Indicates the predicted fault value. A value of 0 indicates no fault, and a value of 1 indicates a fault occurrence. Define the fault value predicted by the i-th time series The calculation formula is as follows:

[0089]

[0090] The calculation formula in S42 is as follows:

[0091]

[0092] Finally, by adjusting different parameters N, the change in the correlation between y and is shown in the following table:

[0093] Nσ r 2σ 0.9027 3σ 0.9428 4σ 0.8976 5σ 0.8791 6σ 0.7863

[0094] It can be seen from the data in the table that the correlation coefficient between the actual fault value and the predicted fault value corresponding to 3σ is closer to 1, which indicates the credibility of using the 3σ criterion to determine the normal interval.

[0095] As Figure 4 shown, the precision, recall rate, and F1 value obtained after the three models perform outlier data detection directly reflect the accuracy, coverage rate, and comprehensive performance of the models. In terms of precision, the Transformer-LSTM model shows higher accuracy compared to the other two models, indicating that this model can misjudge normal data as abnormal data less frequently. In terms of recall rate, the Transformer-LSTM model also shows better performance, which means that our model can better capture real outlier data. The Transformer-LSTM model is also significantly superior to the other two models in terms of the F1 value, indicating that it achieves a better balance in the outlier data detection task and has better performance.

[0096] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. The structures, devices, and operation methods not specifically described and explained in the present invention, unless otherwise specifically stated and limited, are implemented according to the conventional means in the art.

Claims

1. A method for detecting anomalies in spacecraft telemetry data, characterized in that: The detection method comprises the following steps: S1: Data acquisition: select a spacecraft, obtain its telemetry data, and select parameters related to electrical characteristics from the telemetry data. Divide the parameter data set into training data and test data in a ratio of 4:6; S2: Preprocess the data, which includes the following steps: S21: Perform data filling. According to the trend of telemetry parameter changes, the missing data points are filled by taking the average value of the previous second and the next second. If a parameter is always missing, it is filled with 0. S22: directly delete the data points with discontinuous jump characteristics; S23: Parameter normalization, ensuring that different features have similar value ranges; S3: Define training data as y train ,y train , the test data is y test y test , the prediction result data obtained by inputting the model is defined as y pre y pre , set 3σ as the sample normal variance standard, and transform the training data y train y train Input Transformer-LSTM model; In S3, the modeling method is as follows: a. Design an N-layer Transformer encoder, each layer contains a Multi-Head Attention sublayer and a feedforward neural network sublayer. Assume that the input is X∈R L×F , represents L F-dimensional vectors; b. Transformer encoder represents a function T: R L×F →R L×F ; Where T is composed of T1(·), T2(·), …, T N (·) These N layers, each of which can be represented by T m (X) = f m (Attention m (x)+x), f m (·) is a simple feed-forward neural network layer, Attention m (x) is the self-attention mechanism function; c. The LSTM decoder receives the output X of the Transformer encoder output ∈R L×F As input, and set as the initial hidden state H0, initialize the weight parameters and memory cell C0 of the LSTM decoder; d. At each time step t, the LSTM decoder receives input X t and the hidden state H of the previous time step t-1 , then calculate the input gate I t 、Forget Gate F t , output gate O t and candidate memory cells The calculation formulas are as follows: I t =σ(X t W xi +H t-1 W hi +b i ) F t =σ(X t W xf +H t-1 W hf +b f ) O t =σ(X t W xo +H t-1 W ho +b o ) Among them, W xi , W xf , W xo , W hi , W hf , W ho 、b i 、b f 、b o , W xc , W hc is the weight parameter of the LSTM model; Renew memory cells C t and the hidden state H t , where the input gate I t and forget gate F t To update memory cells C t , using output gate O t and the updated memory cell C t To calculate the hidden state H of the current time step t ; S4: Calculate y test With y pre The distance is 3σ, and the normal interval is determined using the 3σ criterion, that is, data samples with variance exceeding the 3σ interval are considered abnormal; S5: Once the model training is complete, it is deployed for anomaly detection in spacecraft telemetry data.

2. The method for detecting anomalies of spacecraft telemetry data according to claim 1, characterized in that: In S3, the model training process includes the following steps: S31: According to the input data y train , design a Transformer encoder with a suitable number of layers; S32: The Transformer encoder continuously learns the changing trends and laws of the spacecraft time series through the Multi-Head Attention mechanism, and obtains an output that can effectively represent the changing laws of several parameter sequences of the input spacecraft; S33: The LSTM decoder receives the output of the Transformer encoder as input in chronological order and uses the output as the initial hidden state; S34: The LSTM decoder receives a special start symbol to generate the target sequence, and receives the output of the previous time step and the hidden state of the current time step at each time step to generate the output of the current time step and update the current hidden state until the output y is obtained. pre .

3. The method for detecting anomalies of spacecraft telemetry data according to claim 1, characterized in that: In step d, the updating process is modeled as follows: H t =O t ⊙tanh(C t ) Among them, ⊙ represents element-wise multiplication; until the time step ends, the output y is obtained pre .

4. The method for detecting anomalies of spacecraft telemetry data according to claim 1, characterized in that: In S4, the credibility method of determining the normal interval using the 3σ criterion is as follows: S41: Introduce Gaussian distribution parameters μ and σ for prediction error, use standard deviation multiplier N as adjustment parameter, and follow the principle of maximizing correlation coefficient; S42: In order to measure the credibility of the standard, r is used to represent the actual fault value y and the predicted fault value The correlation coefficient between the two variables ranges from -1 to 1, with a value of 1 indicating a perfect linear match between the two variables.

5. The method for detecting anomalies of spacecraft telemetry data according to claim 4, characterized in that: In S41, binary variables y and y represents the actual fault value, Indicates the predicted fault value. A value of 0 indicates no fault, and a value of 1 indicates a fault occurs. Defines the fault value predicted by the i time series The calculation formula is as follows:

6. The method for detecting anomalies of spacecraft telemetry data according to claim 4, characterized in that: The calculation formula in S42 is modeled as follows:

7. The method for detecting anomalies of spacecraft telemetry data according to claim 1, characterized in that: In S2, parameter normalization is used to reduce the local features dominating the calculation process.

Images