A time series prediction method for intelligent anomaly perception of satellite monitoring data

Abstract

The present invention relates to a time series prediction method for intelligent anomaly perception of satellite monitoring data. The distribution shift problem that is prevalent in satellite time series data is handled by reversible instance normalization, thereby improving the performance of time series prediction. The DTSMixer model that combines the decomposition scheme used in Autoformer and FEDformer with the linear layer of the TSMixer full MLP architecture is used to regress historical time series and directly predict future time series. Since the RevIN‑DTSMixer method is a simple full MLP architecture linear model, its operating efficiency and number of parameters are far superior to the Transformer-based prediction method. Experimental results on real data sets show that the RevIN‑DTSMixer method can significantly improve the accuracy of time series data prediction, and has the advantages of simplicity, efficiency and precision.

Description

Technical Field

[0001] The present invention relates to the technical field of software engineering, and in particular to a time series prediction method for intelligent anomaly perception of satellite monitoring data. Background Art

[0002] With the development of the global information society, people's demand for real-time communication, broadcasting, Internet and data transmission is increasing. Satellites provide a means of communication with wide coverage and long transmission distance, which is essential for connecting remote areas and providing global services. In addition, the precise positioning, long-distance communication and intelligence collection provided by satellites play a key role in environmental monitoring, military and defense fields. However, affected by the extreme working environment and the complexity of satellites and the existing technical level, although various measures are taken in the design and manufacturing process of satellites to improve their reliability, due to the existence of the following factors, such as satellites in space are subject to extreme temperature, vacuum, radiation and other environmental effects, which may cause material aging, equipment failure, electronic component failure, etc., satellite anomalies and failures are always inevitable. The launch and maintenance of satellites require huge funds and are important assets of the country, especially satellites for data transmission, communication, navigation and military purposes. Once a failure occurs, it may lead to data loss, interruption of communication links, loss of navigation systems or catastrophic consequences, seriously affecting the normal operation of related services. Therefore, through early warning of satellite failures, potential problems can be discovered in advance, measures can be taken to avoid or reduce the impact of anomalies and failures, and ensure the stability and reliability of satellite operation, which is of great significance to satellite safety.

[0003] The core of fault warning is fault prediction. Satellite fault prediction refers to the use of technical means and data analysis to identify and predict possible satellite faults or abnormal conditions in advance, so as to take preventive measures in time, reduce risks, and ensure the normal operation of the satellite to the greatest extent. This concept covers aspects such as satellite health monitoring, fault prediction, fault diagnosis, and preventive maintenance. At present, prediction methods based on satellite time series data can be divided into three categories: knowledge-based, model-based, and data-driven.

[0004] (1) Knowledge-based anomaly detection methods rely on prior domain expertise, which is usually provided by expert personnel. These methods use known system behavior patterns and rules to identify anomalies.

[0005] (2) Model-based approaches typically involve modeling the system and then using the model to detect anomalies.

[0006] (3) Data-driven methods rely on a large amount of real data and use this data to capture the underlying complex patterns and associations in the data. Since they do not rely on prior rules or the knowledge of domain experts and do not require too much domain expertise, they have become the focus of academic and industrial circles.

[0007] In the field of satellites, due to the unknown abnormal patterns that may appear, traditional model-based or knowledge-based methods cannot cover all possible abnormal situations. Data-driven methods can detect unknown abnormalities by learning patterns in historical data and have better generalization capabilities. In addition, satellite systems generate a large amount of real-time and historical data. Data-driven methods can effectively process large-scale data and discover abnormal patterns hidden in a large amount of information, so that they can adapt to the ever-changing and complex environment, and can also cope with large-scale and diverse data. Therefore, the data-driven method system is more suitable for being applied to satellite fault prediction problems.

[0008] The paper (Satellite fault prediction model based on health analysis) proposes a multi-index fusion fault prediction method based on fuzzy analytic hierarchy process, which combines adaptive complete integrated empirical mode decomposition and particle swarm extreme learning machine. This method can accurately reflect the changes in battery health and detect faults early. The paper (Research on data-driven satellite attitude control system micro-fault detection and prediction method) proposes to apply the vector autoregression model (VAR) to the prediction of satellite attitude control system, and improves the accuracy of the model by improving the model and assigning different weights to historical data. The paper (A FP-CNN method for aircraft fault prognostics) proposes a new convolutional neural network for aircraft system fault prediction, named FP-CNN. Starting from the raw sensor data, this method learns sensor-related features and time features through convolutional layers to predict the remaining effective life (RUL). This method does not require predetermined feature selection to eliminate useless sensor values ​​and is more accurate than other methods when dealing with the same problem.

[0009] The LSTM-based prediction method is used to detect anomalies in the SMAP and MSL satellite time series data. The anomaly score is calculated by the error between the predicted value and the true value, and specific predicted data is excluded according to a certain strategy, which achieves better results than the traditional method. The LSTM-based prediction method combines the matrix norm to realize multi-mode mining of remote control commands, and effectively marks anomalies in telemetry data through the construction and effective integration of multiple LSTM prediction models. Summary of the invention

[0010] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: how to accurately predict satellite data.

[0011] In order to solve the above technical problems, the present invention adopts the following technical solution: a time series prediction method for intelligent anomaly perception of satellite monitoring data, comprising the following steps:

[0012] S1: Get historical data Where C represents the number of variables, T represents the number of time steps, is the value of the cth variable at the tth time step;

[0013] S2: Construct and train the time series prediction model RevIN-DTSMixer, and train RevIN-DTSMixer. RevIN-DTSMixer includes the following three parts: time series data distribution offset correction module RevIN, time series data decomposition module based on Autoformer and FEDformer, and satellite time series prediction module TSMixer.

[0014] S2-1: Given a set of input and its corresponding goals Where N represents the number of sequences. C, T x and T y Represents the number of variables, the length of the input sequence, and the prediction length of the model. There is a corresponding predicted value

[0015] From historical data Select L time steps to get the input sequence x (i) , where L<T.

[0016] RevIN computes the input sequence x using a specific mean and standard deviation for each variable. (i) Perform positive normalization to obtain

[0017] S2-2: Get S2-1 The input time series data decomposition module decomposes it into periodic trend components and seasonal components

[0018] S2-3: The columns of TSMixer input represent different variables, and the length of the column represents the time step, the period trend component and seasonal components They are input into an N-layer mixing layer Mixer Layer respectively, and then sent to the temporal projection layer Temporal Projection after mixing, and finally form the corresponding periodic trend component prediction results and seasonal component prediction results.

[0019] Then add the prediction results of the periodic trend component and the seasonal component to get x (i) The prediction results

[0020] S2-4: Input RevIN to perform inverse normalization and obtain x (i) The final prediction result

[0021] S2-5: Calculation With x (i) The corresponding true target y (i) According to the loss, the parameters of RevIN-DTSMixer are updated in reverse order until the loss no longer decreases to obtain the optimal RevIN-DTSMixer.

[0022] S3: Get the current set of time series X′, input X′ into the optimal RevIN-DTSMixer to obtain the corresponding optimal prediction result, then input the optimal prediction result into the optimal One-Class SVM, and output the anomaly detection of the immediate optimal prediction result.

[0023] Specifically, the S2-1 input sequence x (i) The steps for normalization are as follows: the mean and standard deviation are calculated for each instance of the input data. Perform the calculation:

[0024]

[0025]

[0026] Next, the input data is normalized using the following equation:

[0027]

[0028] in Respectively represent x (i) The mean and standard deviation of is the learnable affine parameter vector, represents a C-dimensional vector space over the real number field; ε is a small positive number, which is often called a “smoothness term” or “disturbance term”. Its purpose is to ensure numerical stability during the standardization process. Represents the input sequence x (i) The j-th number in the c-th dimension of .

[0029] Specifically, the S2-2 will Decomposition into periodic trend variables and seasonal variables The steps are as follows: For a length Tx The input sequence

[0030]

[0031]

[0032] AvgPool(·) is used for sliding average operation, and padding operation is used to keep the sequence length unchanged. We write the above formula as The sequence decomposition unit is embedded as an internal unit in the encoder and decoder. Assume that the input of the encoder is the data of the past I time steps Each initialization consists of two parts: input from the encoder The second half of the decomposition is a component of length I half to provide the most recent information, and a placeholder of length O filled by a scalar, as follows:

[0033]

[0034]

[0035]

[0036] in, Respectively represent The periodic trend term and seasonal term of Contains respectively from The second half consists of the cyclical trend item and seasonal item components as well as the placeholder content with a length of O. Respectively represent placeholders filled with zeros and The mean of .

[0037] In the encoding process, there are N encoder layers, and the overall equation of the l-th layer encoder is summarized as follows: The detailed formula is as follows:

[0038]

[0039]

[0040] Where “-” indicates the periodic trend item that has been removed. represents the seasonal term after the i-th series decomposition block of the l-th layer. Auto-Correlation is the autocorrelation mechanism proposed by Autoformer. They represent the seasonal terms after the first and second series decomposition blocks of the lth layer, respectively. Represent the outputs of the l-1th and lth layer encoders respectively.

[0041] In the decoding process, there are M decoder layers, and the overall equation of the l-th layer decoder is summarized as follows: in is the hidden variable from the encoder layer, and the formula is as follows:

[0042]

[0043]

[0044]

[0045]

[0046] in They represent the seasonal term and the cyclic trend term after the i-th series decomposition block of the l-th layer, Indicates the trend of the i-th extraction The projector, They represent the outputs of the lth and l-1th layer decoders respectively.

[0047] FEDformer calculates periodic trend variables The final result or seasonal variable The final result is as follows:

[0048] X trend =Softmax(L(x))*(F(x)) (18);

[0049] Among them, F(x) represents a set of average pooling filters used to extract multiple trend components from the input signal, L(x) represents a function that generates weights for mixing the extracted trends, and X trend It has no actual meaning and is the final result of the periodic trend variable or the final result of the seasonal variable.

[0050] Specifically, the TSMixer in S2-3 includes the following components:

[0051] Temporal Mixed MLP: The temporal mixed MLP is located in the mixing layer and adopts a single-layer MLP consisting of a fully connected layer, an activation function and a random dropout layer. It is responsible for modeling the temporal pattern of the time series. By transposing the input, the temporal mixed MLP is shared on all features.

[0052] Feature Mixture MLP: The feature mixture MLP is located in the mixture layer and is followed by the time mixture MLP. A two-layer MLP is used to analyze the correlation between different features. Since it is applied to the feature domain, it is shared across all time steps.

[0053] Temporal Projection: The temporal projection is a fully connected layer applied in the time domain. The matrix output from the mixing layer is transposed again here and predicted by a fully connected layer. After the prediction, it is restored by transposition. They not only learn the temporal pattern, but also map the time series from the original input length L to the target prediction length T.

[0054] Residual Connection: The residual connection is applied in each MLP component of the mixing layer, namely the time mixing MLP and the feature mixing MLP.

[0055] Normalization: TSMixer applies 2D normalization in both time and feature dimensions, which is a fusion of batch normalization and layer normalization, and normalizes the data in both time and feature dimensions.

[0056] As an improvement, the final prediction result is obtained by S2-4 The method is as follows: through the normalized reciprocal of the following formula Perform an anti-normalization operation:

[0057]

[0058] Compared with the prior art, the present invention has at least the following advantages:

[0059] The present invention proposes a new satellite time series data prediction method (Reversible InstanceNormalization Decomposition Time-Series Mixer) RevIN-DTSMixer. The reversible instance normalization (RevIN) is used to deal with the distribution offset problem that is prevalent in satellite time series data, thereby improving the performance of time series prediction. The DTSMixer model that combines the decomposition scheme used in Autoformer and FEDformer with the linear layer of the TSMixer full MLP architecture is used to regress historical time series and directly predict future time series. Since the RevIN-DTSMixer method is a simple full MLP architecture linear model, its operating efficiency and number of parameters are far superior to the current Transformer-based prediction method. Experimental results on three real data sets show that the RevIN-DTSMixer method can significantly improve the accuracy of time series data prediction, and has the advantages of simplicity, efficiency and precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is the Revin-DTSMixer algorithm architecture diagram.

[0061] Figure 2This is the RevIN algorithm architecture diagram.

[0062] Figure 3 This is the Dlinear architecture diagram.

[0063] Figure 4 This is the TSMixer architecture diagram.

[0064] Figure 5 This is a hybrid layer architecture diagram.

[0065] Figure 6 This is the architecture diagram of the time projection layer, where Fully Connected represents the fully connected layer.

[0066] Figure 7 This is the bus voltage data diagram.

[0067] Figure 8 This is a graph of spacecraft bus current data.

[0068] Fig. 9 This is a battery temperature data graph.

[0069] Fig.10 This is the temperature data diagram of the reaction wheel.

[0070] Fig.11 This is the reaction wheel speed data diagram. DETAILED DESCRIPTION

[0071] The present invention is described in further detail below.

[0072] Problem description:

[0073] The time series data prediction task is defined as follows: For a time series containing C variables, given historical data Where L is the size of the sliding window, and is the value of the i-th variable at the t-th time step. The task of time series prediction is to predict the value of the next T time steps. When T>1, iterative multi-step forecasting (IMS) learns a single-step predictor and applies iteratively to obtain multi-step forecasts, while direct multi-step forecasting (DMS) directly optimizes the multi-step forecast target at one time. Usually, the input will be a multivariate time series data with multiple variables whose length is much larger than the window size. For example, if a time series data training set X of size w is input into the model with a sliding window size of L and a step size of 1, then wL groups of sample results will be obtained:

[0074]

[0075]

[0076] in Represents the c-dimensional vector at time step i. According to the input data, the corresponding prediction sequence will be generated after prediction in It represents the predicted value of the c-dimensional vector at time step i, and then the difference between the true value and the predicted value is calculated by the regression model's indicators MAE and MSE to measure the quality of the model's predictive ability.

[0077] A time series prediction method for intelligent anomaly perception of satellite monitoring data includes the following steps:

[0078] S1: Get historical data Where C represents the number of variables, T represents the number of time steps, is the value of the cth variable at the tth time step.

[0079] S2: Build and train the time series prediction model RevIN-DTSMixer [see Figure 1 】, and trained RevIN-DTSMixer, which consists of the following three parts: time series data distribution offset correction module RevIN, time series data decomposition module based on Autoformer and FEDformer, and satellite time series prediction module TSMixer; [RevIN is used to correct the distribution offset in satellite time series data, solve the problem of inconsistent distribution of training data and test data, and then Dlinear is used to split the time series into different components to better understand the patterns and trends in the sequence, and finally DTSMixer is used to predict historical time series data. 】

[0080] S2-1: Given a set of input and its corresponding goals Where N represents the number of sequences. C, T x and T y Represents the number of variables, the length of the input sequence, and the prediction length of the model. There is a corresponding predicted value

[0081] In RevIN, since the observations have been normalized and denormalized in the time dimension, the input sequence length T x and the model prediction length T y It doesn’t have to be consistent.

[0082] RevIN has two steps: normalization and denormalization. This is because after normalization, the model receives transformed data. as inputs and predict their future values. However, the input data has different statistics from the original distribution, and only by observing the normalized input It is difficult to capture input x (i) Therefore, in order to make it easier for the model to handle this problem, RevIN performs an inverse normalization step at the symmetric position, that is, the output layer, to explicitly return the non-stationary information removed from the input data to the model output. Through the inverse normalization step, the model output can also be restored to the original time series value.

[0083] From historical data Select L time steps to get the input sequence x (i) , where L<T.

[0084] RevIN computes the input sequence x using a specific mean and standard deviation for each variable. (i) Perform positive normalization to obtain

[0085] S2-2: DLinear is a model that combines the decomposition scheme used in Autoformer and FEDformer with linear layers. Since DLinear is a linear model with at most two linear layers, its memory cost and number of parameters are much lower than existing Transformers, and its inference speed is faster; the decomposition scheme uses a seasonal trend decomposition method to divide satellite time series data into trend and seasonal components, which helps to deal with long-term trends and seasonal fluctuations in satellite time series data, and improves the robustness of the model to abnormal data in satellite time series data; the linear network directly performs multi-step predictions, avoiding the error accumulation effect of traditional autoregressive models; the single-layer linear network has the shortest signal traversal path, and the shorter the path, the better the dependency captured, enabling DLinear to capture short-term and long-term temporal relationships; the structure of the DLinear network is as follows Figure 3 As shown in Figure 2, the two components are predicted separately by the linear network, and then the results are added together to obtain the final model prediction output.

[0086] S2-1 is obtained The input time series data decomposition module decomposes it into periodic trend components and seasonal components

[0087] S2-3: TSMixer refers to the MLP Mixer architecture proposed in computer vision. It naturally enhances the model by stacking linear models with nonlinearity to form multi-layer perceptrons (MLPs). It also applies common deep learning techniques such as normalization and residual connections to promote effective learning. At the same time, it alternately uses multi-layer perceptrons in the time domain and feature domain, which not only retains the ability of linear models to capture temporal patterns, but also utilizes cross-variable information. The TSMixer architecture diagram is shown below. Figure 4 As shown, the mixed layer Figure 5 As shown, the time projection layer is Figure 6 As shown:

[0088] The columns of TSMixer input represent different variables, and the length of the column represents the time step, the period trend component and seasonal components They are input into an N-layer mixing layer Mixer Layer respectively, and then sent to the temporal projection layer Temporal Projection after mixing, and finally form the corresponding periodic trend component prediction results and seasonal component prediction results.

[0089] Then add the prediction results of the periodic trend component and the seasonal component to get x (i) The prediction results

[0090] S2-4: Input RevIN to perform inverse normalization and obtain x (i) The final prediction result

[0091] S2-5: Calculation With x (i) The corresponding true target y (i) The loss between [using the existing loss calculation method, please confirm or modify], reversely update the parameters of RevIN-DTSMixer according to the loss until the loss no longer decreases to obtain the optimal RevIN-DTSMixer.

[0092] S3: Get the current set of time series X′, input X′ into the optimal RevIN-DTSMixer to obtain the corresponding optimal prediction result, then input the optimal prediction result into the optimal One-Class SVM, and output the anomaly detection of the immediate optimal prediction result.

[0093] Specifically, the S2-1 input sequence x (i) The steps for normalization are as follows: the mean and standard deviation are calculated for each instance of the input data. Perform the calculation:

[0094]

[0095]

[0096] Next, the input data is normalized using the following equation:

[0097]

[0098] in Respectively represent x (i) The mean and standard deviation of is the learnable affine parameter vector, represents a C-dimensional vector space over the real number field; ε is a small positive number, usually used to avoid division by zero. This small positive number is often called a "smoothness term" or "disturbance term", and its purpose is to ensure numerical stability during the normalization process. Represents the input sequence x (i) The j-th number in the c-th dimension of .

[0099] Since the non-stationary information is removed, the normalized series has a more consistent mean and variance. On this basis, the normalized model can more accurately predict the local dynamics within the series.

[0100] Specifically, the S2-2 will Decomposition into periodic trend variables and seasonal variables The steps are as follows: For a length T x The input sequence

[0101]

[0102]

[0103] AvgPool(·) is used for sliding average operation, and padding operation is used to keep the sequence length unchanged. We write the above formula as The sequence decomposition unit is embedded as an internal unit in the encoder and decoder. Assume that the input of the encoder is the data of the past I time steps Each initialization consists of two parts: input from the encoder The second half of the decomposition is a component of length I half to provide the most recent information, and a placeholder of length O filled by a scalar, as follows:

[0104]

[0105]

[0106]

[0107] in, Respectively represent The periodic trend term and seasonal term of Contains respectively from The second half consists of the cyclical trend item and seasonal item components as well as the placeholder content with a length of O. Respectively represent placeholders filled with zeros and The mean of .

[0108] In the encoding part, Autoformer focuses on modeling seasonal terms by eliminating periodic trend terms. The output of the encoder contains past seasonal information and will be used as cross-information to help the decoder improve the prediction results.

[0109] In the encoding process, there are N encoder layers, and the overall equation of the l-th layer encoder is summarized as follows: The detailed formula is as follows:

[0110]

[0111]

[0112] Where “_” indicates the periodic trend item that has been removed. represents the seasonal term after the i-th series decomposition block of the l-th layer. Auto-Correlation is the autocorrelation mechanism proposed by Autoformer. They represent the seasonal terms after the first and second series decomposition blocks of the lth layer, respectively. Represent the outputs of the l-1th and lth layer encoders respectively.

[0113] In the decoding process, Autoformer models the periodic trend term and the seasonal term separately. For the seasonal term, the autocorrelation mechanism uses the seasonal nature of the sequence to aggregate self-sequences with similar processes of different periods; for the periodic trend term, the periodic trend information is gradually extracted from the predicted latent variables in an accumulated manner, so that Autoformer can gradually improve the trend prediction and eliminate the interference information in the discovery of periodic dependencies. There are M decoder layers, and the overall equation of the l-th layer decoder is summarized as follows: in is the hidden variable from the encoder layer, and the formula is as follows:

[0114]

[0115]

[0116]

[0117]

[0118] in They represent the seasonal term and the cyclic trend term after the i-th series decomposition block of the l-th layer, Indicates the trend of the i-th extraction The projector, They represent the outputs of the lth and l-1th layer decoders respectively.

[0119] FEDformer uses a Mixture of Experts Decomposition block (MOEDecomp) based on seasonal trend decomposition in the decomposition module. Since the fixed window average pooling used in Autoformer may make it difficult to extract trends, FEDformer designs a set of average filters with different sizes to extract multiple trend components and data-related weights from the input signal, and finally combines them into the final result. FEDformer combines them into the final result.

[0120] FEDformer calculates periodic trend variables The final result or seasonal variable The final result is as follows:

[0121] X trend =Softmax(L(x))*(F(x)) (18);

[0122] Among them, F(x) represents a set of average pooling filters used to extract multiple trend components from the input signal, L(x) represents a function that generates weights for mixing the extracted trends, and X trend It has no actual meaning and is the final result of the periodic trend variable or the final result of the seasonal variable;

[0123] Each filter may focus on trends at different scales or frequencies. The L(x) part represents a function that generates weights for mixing the extracted trends. The calculation of the weights may depend on the characteristics of the input signal to enable the model to dynamically adjust the contribution of the trend components. The Softmax function normalizes the weights to ensure that their sum is 1. In this way, they can be interpreted as a probability distribution of the mixture of individual trend components. The final trend component X trend It is obtained by applying normalized weights to the linear combination of the trend components extracted by the average pooling filter, which ensures that the contribution to each trend component is weighted by the corresponding weight.

[0124] In summary, Autoformer uses a sliding average method to extract the periodic trend component of the input sequence, and then obtains the seasonal component by subtracting the original input sequence from the periodic trend component, thus decomposing the original input sequence into a periodic trend component and a seasonal component, which is conducive to the subsequent more accurate formulation of the prediction model; FEDformer proposes a hybrid expert strategy, the core idea of ​​which is to mix the sliding average methods of different kernel sizes in a specific weighted manner to calculate the trend component, and Dlinear is based on the linear model and adds the above data preprocessing method. In addition, the present invention also improves the decomposition model according to the characteristics of satellite time series data. In calculating the periodic trend component, a longer moving average kernel is used to smooth the sequence, remove short-term noise, and obtain the periodic trend component, and then subtract the periodic trend component from the original sequence to obtain a de-periodic trended sequence, and then use a shorter moving average kernel to smooth the de-trended sequence to capture the seasonal component. When calculating the two components, a hybrid expert strategy is used, and the corresponding moving average kernel method is used according to the characteristics of the corresponding components and the final result is calculated according to the weight. The residual component finally obtained is the part of the sequence that cannot be explained by the trend and seasonality.

[0125] Specifically, the TSMixer in S2-3 includes the following components:

[0126] Temporal Mixing MLP

[0127] The temporal mixing MLP is located in the mixing layer and adopts a single-layer MLP, which consists of a fully connected layer, an activation function and a random dropout layer. It is responsible for modeling the temporal pattern of the time series. By transposing the input, the temporal mixing MLP is shared on all features.

[0128] Feature Mixture MLP

[0129] The feature mixing MLP is located in the mixing layer and is followed by the time mixing MLP. A two-layer MLP is used to analyze the correlation between different features. Since it is applied to the feature domain, it is shared across all time steps.

[0130] Time projection

[0131] The temporal projection is a fully connected layer applied in the time domain. The matrix output from the mixing layer is transposed again here and predicted by a fully connected layer. After the prediction, it is restored by transposition. They not only learn the temporal pattern, but also map the time series from the original input length L to the target prediction length T.

[0132] Residual Connection

[0133] The residual connections are applied in each MLP component of the mixing layer, namely the time mixing MLP and the feature mixing MLP. These connections enable the model to learn deeper architectures more effectively while keeping the computational cost reasonable and enable the model to effectively ignore unnecessary time mixing and feature mixing operations.

[0134] Normalization

[0135] Due to the existence of time mixing and feature mixing operations, TSMixer applies 2D normalization in both time and feature dimensions, which is the fusion of batch normalization and layer normalization, and standardizes the data in both time and feature dimensions.

[0136] Specifically, the S2-4 obtains the final prediction result The method is as follows: through the normalized reciprocal of the following formula Perform an anti-normalization operation:

[0137]

[0138] RevIN is very versatile. As a flexible, end-to-end, simple symmetrical structure, RevIN can be applied to neural network structures of any depth to alleviate the distribution differences in time series. The location where RevIN is added is almost symmetrical, and it can be easily applied to the input and output layers of our existing model structure.

[0139] Experimental setup

[0140] 1. Data Description

[0141] The present invention uses MSL, SMAP and LASP to evaluate the effect of the model. The original data sets of Mars Science Laboratory (MSL) and Soil Moisture Active Passive (SMAP) are from the Incident Surprise, Anomaly (ISA) report. They are telemetry data collected by NASA from the Mars Curiosity Rover telemetry satellite and the soil moisture active passive telemetry satellite. These data contain abnormal annotations marked by experts for use by mission personnel to handle unexpected events that affect spacecraft and may cause potential risks to them in post-launch operations. All data have been anonymized, do not contain time information, and all telemetry values ​​are pre-scaled between (-1, 1) according to the minimum / maximum values ​​in the test set. The channel ID has also been anonymized, where the SMAP satellite contains 55 channels, each channel has 25 dimensions, and the MSL satellite contains 27 channels, each channel has 55 dimensions. Although the time information is anonymous, the sampling interval is consistent, with a granularity of 1 minute, that is, the training and test data of the channel are all continuous and equally spaced sequences. As shown in Table 1.

[0142] Table 1 Time series prediction dataset information

[0143]

[0144] The LASP data comes from the telemetry data set maintained by the Laboratory for Atmospheric and Space Physics (LASP), which contains five time subseries data sets:

[0145] ①Bus voltage

[0146] The bus voltage is a univariate time series with 51,704 bus voltage readings recorded over a period of approximately 18 years. “Univariate” means that it tracks the value of only one variable over time. The sampling interval (regular frequency at which measurements are taken) of the data is every 3 hours. This cadence is a result of average resampling, as the original data was sampled at a 5-minute interval. Missing values ​​are filled using linear interpolation. Predictably, the voltage drops once a year, and the trend of the data remains constant until a change point occurs approximately in early 2008. The trend peaks at the change point and then slopes downward, as shown in Figure 2. Figure 7 shown.

[0147] ② Spacecraft bus current

[0148] The spacecraft bus current is a univariate time series consisting of 5,345 bus current readings recorded over approximately 15 years. The sampling interval is one measurement per day, which is the result of an average resampling from its original 5-minute sampling interval. Missing values ​​are filled using linear interpolation. The current does not follow a predictable pattern, but its values ​​are consistently between approximately 10 and 15 amps until late 2012, and between 5 and 15 amps thereafter, as shown in Figure 2. Figure 8 shown.

[0149] ③Battery temperature

[0150] Battery Temperature is a univariate time series with 128,288 battery temperature readings recorded over approximately 10 years. The cadence is hourly measurements, which are the result of average resampling from their original 16-second sampling interval. Missing values ​​are filled using linear interpolation. The temperature shows regular annual variations with increasing magnitude, as Fig. 9 shown.

[0151] ④Reaction wheel temperature

[0152] The reaction wheel temperature is a univariate time series with 769,745 wheel temperature readings recorded over a period of approximately 15 years. The sampling interval is 10-minute measurements, which is the result of resampling the original 5-minute sampling interval by averaging. Missing values ​​are filled using linear interpolation. The temperature follows a predictable interannual variability pattern until a turning point after 2011, as shown in Figure 2. Fig.10 shown.

[0153] ⑤Reaction wheel speed

[0154] The reaction wheel speed is a univariate time series with 984,911 speed readings recorded over a period of about 10 years. The sampling interval is 5-minute measurements, which are the result of resampling the original 1-second sampling interval by averaging. Missing values ​​are filled with zeros. Since the reaction wheel is not always in motion, the data is sparse, as shown in Figure 2. Fig.11 shown.

[0155] 2. Experimental Environment

[0156] In order to eliminate the influence of experimental environment factors on the results, the experiments of the present invention were carried out under the same hardware and software conditions, as shown in Table 2. The programming language used in the experiment was Python, the deep learning framework was Pytorch, and the main development tools were Visual Studio Code and Anaconda.

[0157] Table 2 Experimental environment

[0158]

[0159] 3. Evaluation Metrics

[0160] The present invention uses two commonly used indicators to evaluate the performance of the prediction model, namely MAE (Mean Absolute Error) and MSE (Mean Squared Error). MAE is the average value of the absolute error between the predicted value and the true value, and the absolute error refers to the absolute value of the difference between each predicted value and the corresponding true value. The calculation formula of MAE is as follows:

[0161]

[0162] Where n is the number of samples, y i is the true value of the i-th sample, is the value predicted by the model. The smaller the value of MAE, the better. It is very sensitive to the magnitude of the prediction error because it only considers the absolute value of the error.

[0163] MSE is the average of the squares of the errors between the predicted values ​​and the true values. The difference between each predicted value and the true value is squared and averaged. The formula for calculating MSE is as follows:

[0164]

[0165] Where n is the number of samples, yi is the true value of the i-th sample, is the value predicted by the model. The smaller the MSE value, the better. Compared with MAE, MSE gives higher weight to larger errors because the errors are squared.

[0166] 4. Parameter settings

[0167] The parameter details of the time series prediction method for intelligent anomaly perception of satellite monitoring data proposed in the present invention are shown in Table 3.

[0168] Table 3. Details of RevIN-DTSMixer prediction model parameters

[0169] Parameter name Parameter Value Input sequence length 720 Forecasting task type Multivariate prediction Multivariate Temporal feature encoding frequency minute Predicting sequence length 1 Sliding average window size 25 Number of hidden layer units 64 Dropout 0.05 Activation Function ReLU Batch size 64 Optimizer Adam Loss Function Mean square error Number of training iterations 30 Learning Rate 0.0001

[0170] 5. Model verification and results analysis

[0171] In order to verify the effectiveness of the RevIN-DTSMixer time series prediction method, this paper selects five baseline methods: Transformer, Autoformer, Informer, Flashformer, and Reformer, and conducts comparative experiments on the same training set and test set.

[0172] Transformer: It uses a self-attention mechanism to capture dependencies between different positions in the input sequence. This enables the model to better understand the associations between different time steps in the time series and use these associations for prediction. At the same time, its multi-head attention mechanism allows the model to focus on different aspects of the input sequence in different representation subspaces. This helps capture the complex relationships between different features in the sequence, thereby improving the accuracy of time series prediction.

[0173] Autoformer: It proposes a deep decomposition architecture, embedding the sequence decomposition unit into the encoding and decoding process to achieve progressive prediction, and realize the alternation and mutual promotion of decomposition and prediction result optimization. Based on the theory of random processes, it proposes an autocorrelation mechanism to replace the original attention mechanism, including cycle-based dependency discovery and time-delay information aggregation.

[0174] Informer: ProbSpare self-attention is proposed to solve the problem of quadratic computational complexity in the original self-attention mechanism. The attention distillation mechanism is used in the encoder dimension, so that the features of a single layer are reduced to half of the original in the time dimension, so that longer sequence inputs can be processed in the encoder. Finally, the generative decoder is used to avoid cumulative error propagation.

[0175] Flashformer: Flashformer refers to the Transformer model that uses the hardware-accelerated FlashAttention module. FlashAttention optimizes the standard attention in engineering, achieving training acceleration and space complexity optimization. By performing operations such as calculations in SRAM instead of GPU HBM and not storing N*N attention matrices, it achieves faster training speed, supports modeling of longer sequences (because memory is optimized, it can be longer), and has no loss compared to standard attention (this is better than methods that sacrifice effect for efficiency, such as linformer).

[0176] Reformer: Replaces the fully connected attention mechanism in the traditional Transformer model with Locality-Sensitive Hashing (LSH). By using LSH, Reformer can significantly reduce the consumption of computing resources while maintaining model performance, while dividing the input sequence into fixed-size chunks and then processing each chunk. This block processing method enables the model to save a lot of memory space when processing long sequences, so that it can process longer sequences.

[0177] Table 4 Performance comparison of all time series prediction models on MSL

[0178]

[0179] Table 5 Performance comparison of all time series prediction models on SMAP

[0180]

[0181] Table 6 Performance comparison of all timing prediction models on LASP-BusVoltage

[0182]

[0183] Table 7 Performance comparison of all time series prediction models on LASP-TotalBusCurrent

[0184]

[0185] Table 8 Performance comparison of all time series prediction models on LASP-BatteryTemperature

[0186]

[0187] Table 9 Performance comparison of all time series prediction models on LASP-WheelTemperature

[0188]

[0189] Table 10 Performance comparison of all time series prediction models on LASP-WheelRPM

[0190]

[0191] Tables 4 to 10 show the performance comparison of the time series prediction model RevIN-DTSMixer of the present invention and five baseline methods on the three datasets of MSL, SMAP, and LASP, and the best results are indicated in bold. Although both SMAP and SMAP are satellite telemetry data collected by NASA, they represent very different tasks, so the difficulty of prediction is completely different. Compared with MSL, SMAP's operation is more routine, and only requires less training and data to easily and accurately predict telemetry results, while MSL has many kinds of behaviors with different regularities, some of which are generated during the activities of the rover, and these behaviors are not available in the limited training data. This also explains why the results of each prediction model on the MSL dataset are relatively poor. Even so, RevIN-DTSMixer still achieved the best results in MAE and MSE indicators of different prediction step sizes (1, 3, 12). In addition, on the SMAP dataset and each sub-dataset of LASP, the performance indicators of RevIN-DTSMixer are also basically better than the current mainstream time series data prediction methods. On the SMAP dataset, the MSE and MAE of RevIN-DTSMixer at different step sizes reached 0.631, 0.922, 1.969, 0.161, 0.197 and 0.294, respectively, which are 71.3%, 60.1%, 21.5%, 77.3%, 73.1% and 58.9% higher than the suboptimal Autoformer.

[0192] The average performance comparison of all prediction algorithms on the LASP dataset shows that the average MSE and MAE values ​​of the prediction model RevIN-DTSMixer of the present invention are 0.373 and 0.237, respectively. Compared with other benchmark models, the MSE improvement rates are 94.7%, 60.2%, 72.9%, 61.9% and 77.6%, respectively, and the MAE improvement rates are 81.2%, 46.4%, 62.2%, 44.6% and 71.6%, respectively, which are higher than all benchmark models.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A time series prediction method for intelligent anomaly perception of satellite monitoring data, characterized by: The steps include: S1: Get historical data Where C represents the number of variables, T represents the number of time steps, is the value of the cth variable at the tth time step; S2: Construct and train the time series prediction model RevIN-DTSMixer, and train RevIN-DTSMixer. RevIN-DTSMixer includes the following three parts: time series data distribution offset correction module RevIN, time series data decomposition module based on Autoformer and FEDformer, and satellite time series prediction module TSMixer; S2-1: Given a set of input and its corresponding goals Where N represents the number of sequences; C, T x and T y Represent the number of variables, the length of the input sequence, and the prediction length of the model; for a given input sequence There is a corresponding predicted value From historical data Select L time steps to get the input sequence x (i) , where L <T; RevIN computes the input sequence x using a specific mean and standard deviation for each variable. (i) Perform positive normalization to obtain S2-2: Get S2-1 The input time series data decomposition module decomposes it into periodic trend components and seasonal components The S2-2 will Decomposition into periodic trend variables and seasonal variables The steps are as follows: For a length T x The input sequence AvgPool(·) is used for sliding average operation, and padding operation is used to keep the sequence length unchanged. The above formula is recorded as The sequence decomposition unit is embedded as an internal unit in the encoder and decoder; assuming that the input of the encoder is the data of the past I time steps Each initialization consists of two parts: input from the encoder The second half of the decomposition is a component of length I half to provide the most recent information, and a placeholder of length O filled by a scalar, as follows: in, Respectively represent The periodic trend term and seasonal term of Contains respectively from The second half contains the periodic trend item and seasonal item components and the placeholder content with a length of O; Respectively represent placeholders filled with zeros and The mean of In the encoding process, there are N encoder layers, and the overall equation of the l-th layer encoder is summarized as follows: The detailed formula is as follows: Where "_" indicates the periodic trend item that has been removed. represents the seasonal term after the i-th series decomposition block of the l-th layer. Auto-Correlation is the autocorrelation mechanism proposed by Autoformer. They represent the seasonal terms after the first and second series decomposition blocks of the lth layer, respectively. Represent the outputs of the l-1th and lth layer encoders respectively; In the decoding process, there are M decoder layers, and the overall equation of the l-th layer decoder is summarized as follows: in is the hidden variable from the encoder layer, and the formula is as follows: in They represent the seasonal term and the cyclic trend term after the i-th series decomposition block of the l-th layer, Indicates the trend of the i-th extraction The projector, Represent the outputs of the lth and l-1th layer decoders respectively; FEDformer calculates periodic trend variables The final result or seasonal variable The final result is as follows: X trend =Softmax(L(x))*(F(x)) (18); Among them, F(x) represents a set of average pooling filters used to extract multiple trend components from the input signal, L(x) represents a function that generates weights for mixing the extracted trends, and X trend It has no actual meaning and is the final result of the periodic trend variable or the final result of the seasonal variable; S2-3: The columns of TSMixer input represent different variables, and the length of the column represents the time step, the period trend component and seasonal components They are input into an N-layer mixer layer respectively, and then sent to the temporal projection layer after mixing, and finally form the corresponding periodic trend component prediction results and seasonal component prediction results; Then add the prediction results of the periodic trend component and the seasonal component to get x (i) The prediction results S2-4: Input RevIN to perform inverse normalization and obtain x (i) The final prediction result S2-5: Calculation With x (i) The corresponding true target y (i) According to the loss, the parameters of RevIN-DTSMixer are updated in reverse order until the loss stops decreasing to obtain the optimal RevIN-DTSMixer. S3: Get the current set of time series X′, input X′ into the optimal RevIN-DTSMixer to obtain the corresponding optimal prediction result.

2. A time series prediction method for intelligent anomaly perception of satellite monitoring data according to claim 1, characterized in that: The S2-1 is for the input sequence x (i) The steps for normalization are as follows: The mean and standard deviation are calculated for each instance of the input data. Perform the calculation: Next, the input data is normalized using the following equation: in Respectively represent x (i) The mean and standard deviation of is the learnable affine parameter vector, represents a C-dimensional vector space over the real number field; ε is a small positive number, which is usually called a "smoothness term" or "disturbance term" and its purpose is to ensure numerical stability during the standardization process; Represents the input sequence x (i) The j-th number in the c-th dimension of .

3. A time series prediction method for intelligent anomaly perception of satellite monitoring data according to claim 2, characterized in that: The TSMixer in S2-3 contains the following components: Time Mixed MLP: The time mixed MLP is located in the mixed layer and adopts a single-layer MLP, which consists of a fully connected layer, an activation function and a random dropout layer. It is responsible for modeling the temporal pattern of the time series. By transposing the input, the time mixed MLP is shared on all features. Feature Mixing MLP: The feature mixing MLP is located in the mixing layer and is followed by the time mixing MLP. A two-layer MLP is used to analyze the correlation between different features. Since it is applied to the feature domain, it is shared across all time steps. Temporal projection: The temporal projection is a fully connected layer applied in the time domain, where the matrix output from the mixing layer is transposed again and predicted through a fully connected layer, and then restored by transposition after the prediction. They not only learn the temporal pattern, but also map the time series from the original input length L to the target prediction length T; Residual connection: The residual connection is applied in each MLP component of the mixing layer, i.e., time mixing MLP and feature mixing MLP; Normalization: TSMixer applies 2D normalization in both time and feature dimensions, which is a fusion of batch normalization and layer normalization, and normalizes the data in both time and feature dimensions.

4. A time series prediction method for intelligent anomaly perception of satellite monitoring data according to claim 3, characterized in that: S2-4 obtains the final prediction result The method is as follows: By the normalized reciprocal pair Perform an anti-normalization operation:

Images