Flight trajectory anomaly detection method based on LSTM-GBSVDD model

Abstract

The invention relates to a flight trajectory anomaly detection method based on an LSTM-GBSVDD model, in order to deal with the problem of length change of flight trajectory data, key features in a time sequence are extracted by using an LSTM, and variable-length trajectory data are converted into fixed-length representation, so that anomaly detection can be performed in a fixed-length feature space, and the detection accuracy is improved. And the universality of the model is improved. On the basis of feature extraction, an SVDD algorithm is introduced to construct a multi-dimensional hypersphere classifier, and a normal flight trajectory is modeled. Through the model, potential abnormal tracks can be identified under an unsupervised framework, dependence on data labels is avoided, and the bottleneck that abnormity cannot be identified under an unsupervised environment is solved. According to the invention, joint optimization of LSTM network parameters and an SVDD scoring function is realized for the first time, and a gradient-based training method is provided. According to the method, the anomaly detection precision and the calculation efficiency are remarkably improved, and the detection performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of flight trajectory anomaly detection methods, and specifically to a flight trajectory anomaly detection method based on an LSTM-GBSVDD model. Background Art

[0002] With the rapid development of aviation technology, flight safety issues have become increasingly prominent, and anomaly detection of flight trajectories has become a key research area. Traditional trajectory anomaly detection methods mostly rely on optoelectronic devices, but these methods have many shortcomings in terms of detection accuracy, response speed, and anti-interference ability. In recent years, with the widespread application of machine learning and deep learning technologies, new means have been provided for flight trajectory anomaly detection. We explore anomaly detection in an unsupervised framework and introduce a flight trajectory anomaly detection method based on the LSTM-GBSVDD model to address the complexity, dynamics, and length variation of flight trajectory data.

[0003] Flight trajectory data has complex time dependencies and variable length characteristics, so traditional anomaly detection methods are not accurate enough when processing these data. Using LSTM networks, we can better process time series data, and the support vector data description (SVDD) method can provide an effective anomaly detection method.

[0004] Flight trajectory anomaly detection has become a hot topic of research at home and abroad. Coelho e Silva et al. proposed a flight data analysis framework for flight anomaly detection in order to provide a comprehensive model for model building, application and interpretation. The method is designed to be applicable to both online and offline and multiple scales. In order to jointly solve the tasks of abnormal trajectory detection and pattern recognition in flight trajectory data analysis, Zhang et al. proposed an end-to-end framework based on a deep autoencoder network. Considering the coupling relationship between the two tasks, a structured sparsity-induced norm is introduced in the reconstruction-based loss function to separate abnormal trajectories from the whole. Some models cannot reflect the changes in data patterns. To address this problem, Zhao et al. proposed an incremental anomaly detection method based on Gaussian Mixture Model (GMM) to identify common patterns from flight data and detect outliers in flight. Real-time Trajectory Optimization (RTOP) is a flight path optimization framework based on machine learning that can continuously and automatically perform in-flight planning based on the latest weather information. The above methods are still insufficient in processing the time dependency of data, and it is difficult to effectively capture the long-term dependency in the trajectory, which may affect the detection accuracy.

[0005] In recent years, with the development of deep learning, more and more researchers have begun to pay attention to the application of neural networks in flight trajectory anomaly detection. The interception problem of aircraft has always been an important research issue. In order to provide accurate target trajectory information, Sun et al. proposed a target interception model based on an improved LSTM trajectory prediction pipeline network, designed an improved LSTM network, estimated parameters in a dynamically updated manner, and used the estimated results for the calculation of the final trajectory prediction pipeline. The algorithm has high accuracy and is conducive to improving the interception success rate. Chevrot et al. proposed a new multivariate anomaly detection model called contextual auto-encoder (CAE). The model is based on a conventional LSTM-based autoencoder, but uses multiple decoders, each of which obtains data for a specific flight phase (such as climb, cruise, or descent) during training. It has been verified that CAE has achieved better results in terms of detection accuracy and speed. Shafienya et al. proposed a new hybrid deep learning model to extract spatial and temporal features of Hartsfield-Jackson Atlanta International Airport taking into account uncertainty. A large amount of spatial and temporal flight attribute data is used as input to the model. Prediction is performed by hybrid convolutional neural network and gated recurrent unit (CNN-GRU) and three dimensional (3D-CNN) model. Traditional drone anomaly detection models have some defects. They fail to integrate data from multiple sensors across time and frequency domains, which hinders the ability of anomaly detection models to accurately assess the status of drones. They use the same prediction error loss for all categories, resulting in too many false positives in some key categories. Finally, most of them use single-modal classification models to process data from multiple heterogeneous sensors, which makes it difficult for the model to extract target features. Deng et al. proposed a drone anomaly detection model based on sensor information fusion and hybrid multimodal neural network (IF-HMNN). Although these methods have improved the effect of flight trajectory anomaly detection to a certain extent, further research and improvement are still needed in terms of multi-dimensional data integration and real-time performance.

[0006] The support vector data description (SVDD) method is an anomaly detection method based on statistical learning theory. Li et al. proposed an anomaly recognition model based on long short-term memory (LSTM) and support vector data description (SVDD). SVDD is used to generate a multidimensional hypersphere classifier to train healthy samples. The classifier is used to diagnose the data obtained by the LSTM prediction model and realize anomaly recognition. In order to accurately detect (Automatic Dependent Surveillance-Broadcast, ADS-B) data attacks, Luo et al.

[15] proposed an anomaly detection model that fully considers the time correlation and distribution characteristics of ADS-B data. In order to solve the problem of adaptive anomaly detection threshold, the difference between the reconstructed value and the actual value is put into SVDD (support vector data description) for training. Support vector data description (SVDD) is a classic anomaly detection algorithm. Zhang et al. proposed an improved SVDD model that combines deep feature extraction with data structure preservation, namely deep structure preservation (DSPSVDD). In order to extract deep data features more effectively, an enhanced comprehensive optimization objective is designed for the deep SVDD model while considering the minimization of the hypersphere volume and the minimization of the network reconstruction error. In summary, although some progress has been made in the application of the SVDD method, there is still room for improvement in deep feature extraction and data structure preservation, high-dimensional feature processing, and optimization of target effects. Summary of the invention

[0007] Aiming at the problems of inaccurate feature extraction and low detection efficiency when processing complex and dynamic flight trajectory data with real-time changing data length, the present invention provides a flight trajectory anomaly detection method based on the LSTM-GBSVDD model, and achieves performance improvement in multiple aspects.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] The flight trajectory anomaly detection method based on the LSTM-GBSVDD model includes the following steps:

[0010] Step 1: Establish a flight trajectory training set, and input the flight trajectory sequence samples in the flight trajectory training set into the LSTM model;

[0011] Step 2: The LSTM model integrates the features of each time step of the sample, performs feature mean pooling, generates a fixed-length feature vector, and inputs the feature vector into the SVDD model;

[0012] Step 3: The SVDD classifier constructs a hypersphere and performs joint training based on the gradient to update the LSTM model parameters and the radius and center of the hypersphere constructed by the SVDD classifier. The trained LSTM model and SVDD model are finally obtained by minimizing the loss function.

[0013] Step 4: Input the flight trajectory sequence to be detected into the trained LSTM model to obtain a feature vector of fixed length, and input the feature vector into the trained SVDD model to determine whether there is data outside the hypersphere. If so, the flight trajectory is considered abnormal and the abnormal result is output.

[0014] Furthermore, each flight trajectory sequence sample includes four types of data sequences during a flight, namely, speed, altitude, traffic, and sideslip angle data at each moment during the flight.

[0015] Furthermore, in step 2, the specific content of the gradient-based training algorithm is as follows:

[0016] Step 3.1. For the LSTM model, calculate the gradient of the objective function with respect to the LSTM parameter θ:

[0017]

[0018] For the SVDD classifier, the objective function is calculated relative to the center of the hypersphere And the gradient of radius R:

[0019]

[0020]

[0021] In the formula, the objective function S τ (·) is a smooth function;

[0022] Step 3.2: For the LSTM weights W and bias B parameters, the center of the SVDD classifier And radius R are updated using the following formula:

[0023] For LSTM weights W and bias B:

[0024]

[0025]

[0026]

[0027] M k =M

[0028] Where M is a matrix with n rows and n columns, and the elements in the matrix are W (·) is a matrix with n rows and n columns. The elements in the matrix is the smooth function S τ (·) is the weight between the i-th unit and the j-th unit of the LSTM model, i,j∈{1,2,…,n}, n is the number of units in the LSTM model;

[0029] For the center of the SVDD classifier and radius R:

[0030]

[0031]

[0032] In the formula, the meaning of each parameter is as follows:

[0033] k: number of iterations;

[0034] The output sequence of the i-th LSTM model unit is an indicator of the neural network's response to the input;

[0035] λ: R 2 The trade-off parameter between the total classification error controls the relative importance between the classification error and the square of the radius;

[0036] is the radius of the hypersphere corresponding to the output sequence of the i-th unit of the LSTM model;

[0037] μ: an estimate of the scaling factor, A hyperparameter used to adjust the influence range of the support vector, affecting the aggregation degree of samples in SVDD;

[0038] ξ: standard deviation of sample data, which measures the degree of dispersion of data and is used to determine the size of the scaling factor;

[0039] τ: smoothing parameter, which controls the smoothness of the model;

[0040] ψ: trade-off parameter, used to adjust the trade-off between different loss terms;

[0041] Step 3.3: Calculate the LSTM-SVDD joint loss function. The LSTM-SVDD joint loss function is a loss function used to measure the degree of matching between the features extracted by LSTM and the SVDD target, expressed as:

[0042] Loss=λ1·MSE+λ2·SVDDSphereLoss

[0043] Among them, λ1 and λ2 are weight coefficients used to balance the importance of the two loss terms; SVDD Sphere Loss is the hypersphere loss, which is used to measure the distance between the extracted feature points and the SVDD hypersphere, and MSE is the mean square error of the LSTM model;

[0044] Determine whether the LSTM-SVDD joint loss function has reached the global optimal value or the local optimal value. If it has, then the training is terminated and the LSTM weight W and bias B obtained at the last iteration and the center of the SVDD classifier are output. and radius R, otherwise the number of iterations increases by 1, and go to step 3.1 to continue training with the updated parameters.

[0045] After adopting the above technical solution, the present invention has the following advantages compared with the prior art:

[0046] (1) Processing variable-length data sequences: In order to deal with the problem of length variation of flight trajectory data, the present invention uses LSTM to extract key features in the time series and converts variable-length trajectory data into fixed-length representation. This enables anomaly detection to be performed in a fixed-length feature space, thereby improving the universality of the model.

[0047] (2) Anomaly detection modeling: Based on feature extraction, we introduce the SVDD algorithm to construct a multidimensional hypersphere classifier for modeling normal flight trajectories. This model can identify potential abnormal trajectories in an unsupervised framework, avoiding dependence on data labels and solving the bottleneck of being unable to identify anomalies in an unsupervised environment.

[0048] (3) Joint optimization of LSTM and SVDD: We have realized the joint optimization of LSTM network parameters and SVDD scoring function for the first time and proposed a gradient-based training method. This method significantly improves the accuracy and computational efficiency of anomaly detection and solves the suboptimal solution problem caused by training LSTM or SVDD alone, thereby improving detection performance.

[0049] (4) Simulation test verification: Through a large number of simulation tests, we verified the effectiveness of the LSTM-GBSVDD algorithm in anomaly detection tasks across multiple time scales. The experimental results show that the algorithm can efficiently detect various types of anomalies and achieve significant improvements in detection performance, further verifying its application potential in complex flight trajectory anomaly detection.

[0050] In summary, the LSTM-GBSVDD algorithm proposed in the present invention can effectively address the key challenges in flight trajectory anomaly detection, achieve breakthroughs in both accuracy and computational efficiency, and provide an efficient and reliable solution for unsupervised anomaly detection of complex time series data.

[0051] The present invention is described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the flight trajectory anomaly detection method of the present invention;

[0053] Figure 2 This is the speed change diagram of the aircraft in Fault 1;

[0054] Figure 3 This is the height change diagram of fault 1;

[0055] Figure 4 This is the angle of attack change diagram of the flight process after the 10th second of Fault 1;

[0056] Figure 5 This is the side slip angle change diagram for fault 1;

[0057] Figure 6 This is the height change diagram of fault 2;

[0058] Figure 7 This is the change diagram of the angle of attack during the flight process after the 10th second of Fault 2;

[0059] Figure 8 This is the side slip angle change diagram for fault 2;

[0060] Fig. 9 This is the speed change diagram of the aircraft in Fault 2;

[0061] Fig.10 It is the speed anomaly detection graph of LSTM-GBSVDD method (fault 1);

[0062] Fig.11 It is the highly outlier detection graph of LSTM-GBSVDD method (fault 1);

[0063] Fig.12 This is the attack angle anomaly detection diagram of the LSTM-GBSVDD method (fault 1);

[0064] Fig.13 It is the sideslip angle abnormal point detection diagram of LSTM-GBSVDD method (fault 1);

[0065] Fig.14 It is the speed anomaly detection graph of LSTM-GBSVDD method (fault 2);

[0066] Fig.15 It is the highly outlier detection graph of LSTM-GBSVDD method (fault 2);

[0067] Fig.16 This is the attack angle anomaly detection diagram of the LSTM-GBSVDD method (fault 2);

[0068] Fig.17 It is the sideslip angle abnormal point detection map (fault 2) of LSTM-GBSVDD method;

[0069] Fig.18 This is a comparison chart of ROC curves of the flight trajectory dataset based on the LSTM-GBSVDD algorithm (fault 1 and fault 2);

[0070] Fig.19 It is a comparison chart of the accuracy, F1 and AUC score methods for fault one anomaly detection;

[0071] Fig. 20 It is a comparison chart of the accuracy, F1 and AUC score methods for detecting anomalies in fault 2;

[0072] Fig.21 This is a comparison chart of the training time of the LSTM-GBSVDD anomaly detection method (fault 1 and fault 2). DETAILED DESCRIPTION

[0073] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0074] 1 Model and Problem Description

[0075] The process of detecting time series anomalies by the LSTM-GBSVDD model we proposed is as follows: Figure 1 shown.

[0076] In the present invention, all vectors are column vectors. For vector a, a T is its ordinary transpose, is l2 normal. The time index is indicated by a subscript, for example, a i is the i-th vector. Here, 1(and 0) is the vector of all 1(and 0), and I represents the identity matrix.

[0077] We observe the data series {X i} n =1, the definition is as follows:

[0078] X i =[x i,1 x i,2 … x i,di ] (1)

[0079] in and Yes X iThe number of columns in can vary with i. Here, we assume that most of the observed sequences are normal and the rest are abnormal. Our goal is to find a decision function that determines X based on the observed data. i Is it abnormal? +1 and -1 represent the output of the required scoring function for normal data and abnormal data, respectively.

[0080] In order to find a scoring function l(·) such that

[0081]

[0082] Support Vector Domain Description (SVDD) is a classification algorithm based on Support Vector Machine (SVM). The SVDD algorithm can be used for data classification, anomaly detection and other problems, especially for small samples, high-dimensional data and other problems. The basic idea of ​​SVDD is to establish a minimum hypersphere and use this hypersphere to contain all data as much as possible. When testing new data, we only need to determine whether the data is inside or outside the hypersphere to achieve classification. However, these algorithms can only process sequences of fixed length. Therefore, we use the LSTM architecture for each X i To obtain a fixed-length vector, we use an LSTM architecture without peephole connections. We first transform X i Enter the LSTM architecture, the internal equation of LSTM is as follows:

[0083] z i,j =g(W (z) x i,j )+R (z) h i,j-1 +b (z) (3)

[0084] s i,j =σ(W (s) x i,j )+R (s) h i,j-1 +b (s) (4)

[0085] f i,j =σ(W (f) x i,j )+R (f) h i,j-1 +b (f) (5)

[0086] c i,j =s i,j ⊙z i,j +fi,j ⊙c i,j-1 (6)

[0087] o i,j =σ(W (o) x i,j )+R (o) h i,j-1 +b (o) (7)

[0088] h i,j =o i,j ⊙g(c i,j ) (8)

[0089] in, is the state vector, is the input vector, is the output vector of the jth LSTM unit. In addition, s i,j ,f i,j and i,j are the input gate, forget gate, and output gate, respectively. Here, g() is set to the hyperbolic tangent function, i.e., tanh, and is applied point-by-point to the input vector. Similarly, σ(·) is set to a hybrid optimizer (RMSProp+Nesterov Momentum, Nadam), and ⊙ is the element-wise multiplication of two vectors of the same size. In addition, W(), R(·), and b(·) are parameters of the LSTM architecture, and the size of each parameter is selected according to the dimensions of the input and output vectors. Basically, in our LSTM architecture, c i,j-1 Represents the network cell state of the previous LSTM module. This cell state provides the information flow between consecutive LSTM blocks. For LSTM architecture, it is very important to determine how much information should be retained in the cell state. Therefore, in order to determine the amount of information that needs to be retained, we use f i,j , which outputs a number between 0 and 1 and scales the cell state in Equation (6). The next step is to determine how much new information we should learn from the data.

[0090] Then, we multiply the old and new information with the forget gate and the input gate respectively to generate new state information. Finally, we need to determine the content of the output. To obtain the output, we use c i,j However, we still need to decide which parts of the cell state should be preserved on output. Therefore, we first compute o i,j to filter some parts of the cell state. Then, we push the cell state into the tanh layer and multiply it with the output gate to get the final output of the LSTM module, as shown in formula (8).

[0091] After applying the LSTM architecture to each column of the data sequence, we take the mean pooling method of the LSTM output of each data sequence. In this way, we get a set of new fixed-length sequences, which are represented as We also use the same method to obtain each X i Status information

[0092] 1.1 Anomaly Detection Using SVDD Algorithm

[0093] In this section, we introduce an anomaly detection algorithm based on SVDD and provide a joint update to learn the LSTM and SVDD parameters. In the SVDD algorithm, our goal is to find a hypersphere that contains the normal data while leaving the abnormal data outside the hypersphere. For the sequence We have the following SVDD optimization problem:

[0094]

[0095]

[0096]

[0097] W (·)T W (·) =I,R (·)T R (·) =I and b (·)T b (·) =1 (11)

[0098] Where λ>0 is R 2 The trade-off parameter between and the total classification error. R is the radius of the hypersphere, is the center of the hypersphere. In addition, θ and ξ represent the LSTM parameters and slack variables, respectively. After solving the optimization problem of formulas (9)-(11), we use the following scoring function to detect anomalies:

[0099]

[0100] (1) Gradient-based training algorithm:

[0101] In this section we introduce a first-order gradient-based training algorithm. We use the G(·) function to eliminate the constraints in formula (10) as follows:

[0102]

[0103]

[0104] st:W (·)T W(·) =I,R (·)T R (·) =I

[0105] and b (·)T b (·) =1 (14)

[0106] in

[0107]

[0108] Since there is no difference function G(·), the gradient-based method cannot optimize formula (13). We use S τ (·) replaces G(·), and (13) is modified as follows:

[0109]

[0110]

[0111] st:W (·)T W (·) =I,R (·)T R (·) =I

[0112] and b (·)T b (·) =1 (16)

[0113] where F τ (,,) is the objective function of (15). In order to obtain the optimal values ​​of formula (15) and formula (16), we will update R and θ until a local or global optimum is reached. To update R, we use the gradient descent algorithm. The gradient is calculated as follows. First, calculate The gradient of is:

[0114]

[0115] Using (17), we can get the following update:

[0116]

[0117] Where K represents the number of iterations. Similarly, we calculate the objective function relative to R 2 The derivative of is:

[0118]

[0119] According to (19), we set R 2 Updated to:

[0120]

[0121] For θ, the gradient is calculated as follows:

[0122]

[0123] Using formula (21), we can get the following update:

[0124]

[0125] in

[0126]

[0127] Remark 1: For R(·) and b(·), we first calculate the gradient of the objective function with respect to the selected parameters, as shown in (23). Then we get B according to the selected parameters. k . Using B k , we update the selected parameters as shown in (22).

[0128] Note 2: In the semi-supervised framework, our SVDD-based algorithm has the following optimization problem:

[0129]

[0130]

[0131]

[0132] W (·)T W (·) =I,R (·)T R (·) =I and b (·)T b (·) =1 (27)

[0133] in, is a slack variable, such as is the margin of labeled data, C1, C2 and C3 are trade-off parameters, k and l are the number of labeled and unlabeled data instances, respectively, and y j ∈{-1,+1} represents the jth th The label of each data instance.

[0134] In LSTM-GBSVDD, the model can optimize the parameters of LSTM and SVDD at the same time through the gradient optimization method. This joint optimization method ensures that the LSTM network is consistent with the anomaly detection boundary of the SVDD model when extracting time series features. LSTM not only extracts the time features of the flight trajectory, but also directly guides these features to the anomaly detection task of SVDD. This joint optimization can keep the goals of LSTM and SVDD consistent and avoid the problem of local optimality. Because optimizing the parameters of LSTM or SVDD alone may cause the model to perform better in a certain part, but the overall performance is poor. Joint optimization ensures that the model reaches the global optimal in extracting features and detecting anomalies, effectively improving the detection performance.

[0135] Traditional support vector machines (SVM) divide data into two categories through "planes", but this approach may not be flexible enough for anomaly detection tasks. SVM can only distinguish whether the data is "online" or "outside the line", and has poor recognition effect on abnormal points with high-dimensional complex distribution. The SVDD model introduces a "sphere" encirclement strategy, which constructs a sphere boundary to wrap normal data inside and exclude abnormal data from the sphere. In contrast, the sphere encirclement method of SVDD is more suitable for anomaly detection, especially in complex distribution data such as flight trajectories. It can adapt to the complexity of the data more flexibly and capture more subtle differences between normal and abnormal data. Mathematically, this makes SVDD far better than SVM in encircling abnormal data, reducing the possibility of false positives and false negatives.

[0136] 2 Case Analysis

[0137] We will verify the performance of the algorithm on the dataset. First, we evaluate the performance of the dataset containing variable-length data sequences, including speed, altitude, and yaw angle. In addition, we also measure the training time of the algorithm. For all experiments in this section, we normalize each dimension of the dataset to [-1,1]. In this section, we will train the LSTM-GBSVDD anomaly detection model based on the gradient algorithm. The method is verified by simulation and experiments.

[0138] 2.1 Anomaly Detection on Simulated Datasets

[0139] In this section, we first simulate the flight trajectory through MATLAB's Simulink, collect historical flight data from the public ADSB database, further test the algorithm, simulate different flight states (such as climb, cruise, descent, etc.) for a long time, and set up multiple simulation scenarios such as normal and abnormal. In order to evaluate the performance, we first select "0" as the normal number and "1" as the abnormal number. 60% of the samples are used for the training part and 40% for the test part. In both the training and test parts, 10% of the samples we selected have abnormal samples. At the same time, we select the key parameters μ, the output dimension m of the LSTM architecture, and the regularization parameter λ. The AUC score is a common indicator for evaluating the performance of a binary classification model. It represents the area under the ROC curve. The ROC curve is a curve drawn with the false positive rate (FPR) as the horizontal axis and the true positive rate (TPR) as the vertical axis. The AUC score ranges from 0 to 1. The larger the value, the better the model performance. It is an effectiveness performance indicator for anomaly detection tasks.

[0140] 2.2 Benchmark Test Dataset

[0141] In this section, we compare the AUC scores of the LSTM-GBSVDD algorithm in actual experiments. We collected four features in this dataset, namely speed, altitude, angle of attack, and sideslip angle. Our goal is to determine whether the aircraft trajectory is abnormal based on these features. The first flight state is 25s, and the second flight state is 12s. Every second, 1024 data are collected, 25600 data are collected in the first flight state, and 11264 data are collected in the second flight state. We divide these digital samples into two parts: training and testing. For all simulation experiments in this section, the performance of the algorithm is maximized.

[0142] The flight feature dimensions of the aircraft are set, such as speed, altitude, angle of attack, sideslip angle, etc. The dimension of each input feature vector is 3,256 hidden units, and the number of layers of the LSTM network is 2 to ensure that the complex relationship in the time series is captured. The activation function uses a hybrid optimizer (RMSProp+Nesterov Momentum, Nadam), μ is appropriately lowered to 0.0005-0.001, and the momentum is set to 0.9. It is used as the activation function of the input gate, forget gate, and output gate to optimize the network parameters.

[0143] Parameter settings for the SVDD classifier: The initial value of the hypersphere radius is 1, which is then updated through optimization. The initial value of the sphere center is set to the mean of all normal data, which is gradually optimized during training. The initial value of the slack variable is set to 0.1, allowing a small number of outliers to be outside the hypersphere. The hyperparameter is set to 0.05 to control the proportion of outliers.

[0144] Using the global average pooling technique, the output of each LSTM unit is averaged to generate a fixed-length vector representation. The dimension of the output fixed-length vector is 256, which is used as the input of the subsequent SVDD classifier.

[0145] During the construction and training of the SVDD classifier, the specific parameters are set as follows:

[0146] The regularization parameter λ is used to balance the size of the hypersphere and the training error. It can be optimized by grid search, λ = 10. The radius R is initialized to 1, and the center C is initialized to the mean of the normal data points. The initial value of the slack variable ξ is set to 0.1, allowing some abnormal points to be outside the hypersphere.

[0147] In the gradient-based training process, for the LSTM network, the initial learning rate is set to 0.001, and the Nadam optimizer is used for parameter update. For the sphere center and radius optimization of SVDD, the learning rate is 0.01, which is used to update the sphere center C and radius R. The Nadam optimization algorithm is used to update the parameters of LSTM and SVDD. Nadam has the advantage of adaptive learning rate and can converge faster. Momentum parameters β1=0.9, β2=0.999 prevent gradient oscillation and ensure model convergence. The number of iterations is set to 1000, and the weights of LSTM and the radius and sphere center of SVDD are updated each time. The optimal number of iterations is determined by cross-validation. The change of the loss function is checked after each iteration to ensure that the model converges within a reasonable range. In order to balance the losses of LSTM and SVDD, the regularization parameter λ is set to 0.1 to balance the reconstruction error of LSTM and the hypersphere optimization objective of SVDD.

[0148] 2.3 Simulation Results

[0149] Here, we use the area under the receiver operating characteristic curve (ROC) as a performance metric. In the ROC curve, we plot the true positive rate (TPR) as a function of the false positive rate (FPR). The area under this curve (also known as AUC) is a performance metric for anomaly detection tasks. In it, we mark the numbers "0" and "9" as normal and abnormal, respectively. Fig.18 In Figure 2, we observe that the AUC scores of the LSTM-based algorithms are much higher than those of the SVDD algorithms. Among the LSTM-based methods, LSTM-GBSVDD slightly outperforms LSTM-GBSVM. In summary, the gradient-based training method has a higher performance, which is due to its learning ability.

[0150] Table 1. Precision (Prec), F1, AUC and Train Time of 200 average simulation tests of anomaly detection by different methods under two fault modes

[0151]

[0152]

[0153] 3. Analysis of anomaly detection results

[0154] This section comprehensively analyzes the performance of the proposed LSTM-GBSVDD anomaly detection algorithm and compares it with two commonly used anomaly detection methods, including SVDD and LSTM-GBSVM. Figure 2-9 The anomaly detection results of the LSTM-GBSVDD method on the aircraft speed, altitude, angle of attack, and sideslip angle datasets under two fault modes are shown and compared. Figure 10-17 The anomaly detection results of the test set samples of the aircraft's speed, altitude, angle of attack, and sideslip angle under two fault modes based on the LSTM-GBSVDD method are shown.

[0155] As shown in Table 1, the accuracy of SVDD is low and the false positive rate is high, indicating that its detection results contain quite a lot of false positives. In contrast, LSTM-GBSVM performs relatively well in terms of accuracy, F1 score and AUC. The LSTM-GBSVDD algorithm proposed in the present invention further improves the accuracy, significantly reduces the number of false positives, and performs outstandingly in terms of accuracy, F1 score and AUC score.

[0156] In fault mode 1, the speed, altitude, angle of attack, and sideslip angle all showed abnormalities. The speed and altitude showed significant abnormalities from the 10th second to the 25th second in the data set, the angle of attack showed slight fluctuations at the 10th second and between the 18th and 20th seconds, and the sideslip angle fluctuated at the 10th second and the 24th second. However, fault mode 1 had little effect on the flight trajectory of the aircraft, and the aircraft was still able to land smoothly near the target point.

[0157] In fault mode 2, abnormal changes in speed, altitude, angle of attack, and sideslip angle were also observed. The speed and altitude showed significant abnormalities at the 10th second, and dropped to zero at the 11th second; the angle of attack and sideslip angle both showed violent fluctuations at the 10th second. Fault mode 2 had a significant impact on the flight trajectory, causing the aircraft to fall prematurely.

[0158] According to Table 1 and Fig.19 and Fig. 20The analysis results show the accuracy, F1 score and AUC score of the speed, altitude, angle of attack and sideslip angle under two failure modes of the aircraft. The simulated failure modes of the two aircraft verify the effectiveness of our proposed LSTM-GBSVDD algorithm. We compare the algorithm with two other advanced anomaly detection algorithms: SVDD and LSTM-GBSVM. It can be seen that compared with SVDD and LSTM-GBSVM algorithms, our proposed LSTM-GBSVM algorithm achieves the highest AUC score among all algorithms, which is due to the gradient separation update process, which avoids falling into a local minimum.

[0159] In addition, from Table 1 and Fig.21 It can be seen that although the accuracy, F1, and AUC scores of the LSTM-GBSVM algorithm and LSTM-GBSVM are not much different, there is a significant difference in the training time between the two. The training time of SVDD and LSTM-GBSVDD is shorter, while the training time of LSTM-GBSVM is relatively long, so LSTM-GBSVDD has an advantage in comprehensive performance.

[0160] In summary, the LSTM-GBSVDD algorithm can accurately, timely and effectively detect anomalies in both fault modes of the aircraft, showing superior anomaly detection performance, and providing strong support for aircraft health monitoring and fault diagnosis.

[0161] The above is an example of the best implementation of the present invention, and the parts not described in detail are common knowledge of ordinary technicians in the field. The protection scope of the present invention shall be based on the content of the claims, and any equivalent transformation based on the technical enlightenment of the present invention is also within the protection scope of the present invention.

Claims

1. The flight trajectory anomaly detection method based on the LSTM-GBSVDD model is characterized by: The following steps are involved: Step 1: Establish a flight trajectory training set, and input the flight trajectory sequence samples in the flight trajectory training set into the LSTM model; Step 2: The LSTM model integrates the features of each time step of the sample, performs feature mean pooling, generates a fixed-length feature vector, and inputs the feature vector into the SVDD model; Step 3: The SVDD classifier constructs a hypersphere and performs joint training based on the gradient to update the LSTM model parameters and the radius and center of the hypersphere constructed by the SVDD classifier. The trained LSTM model and SVDD model are finally obtained by minimizing the loss function. Step 4: Input the flight trajectory sequence to be detected into the trained LSTM model to obtain a feature vector of fixed length, and input the feature vector into the trained SVDD model to determine whether there is data outside the hypersphere. If so, the flight trajectory is considered abnormal and the abnormal result is output.

2. The flight trajectory anomaly detection method based on the LSTM-GBSVDD model according to claim 1 is characterized in that: Each flight trajectory sequence sample includes four types of data sequences during a flight, namely, speed, altitude, traffic, and sideslip angle data at each moment during the flight.

3. The flight trajectory anomaly detection method based on the LSTM-GBSVDD model according to claim 1, characterized in that: In step 2, the specific content of the gradient-based training algorithm is as follows: Step 3.

1. For the LSTM model, calculate the gradient of the objective function with respect to the LSTM parameter θ: For the SVDD classifier, the objective function is calculated relative to the center of the hypersphere And the gradient of radius R: In the formula, the objective function S τ (·) is a smooth function; Step 3.2: For the LSTM weights W and bias B parameters, the center of the SVDD classifier And radius R are updated using the following formula: For LSTM weights W and bias B: M k =M Where M is a matrix with n rows and n columns, and the elements in the matrix are W (·) is a matrix with n rows and n columns. The elements in the matrix is the smooth function S τ (·) is the weight between the i-th unit and the j-th unit of the LSTM model, i,j∈{1,2,…,n}, n is the number of units in the LSTM model; For the center of the SVDD classifier and radius R: In the formula, the meaning of each parameter is as follows: k: number of iterations; The output sequence of the i-th LSTM model unit is an indicator of the neural network's response to the input; λ: R 2 The trade-off parameter between the total classification error controls the relative importance between the classification error and the square of the radius; is the radius of the hypersphere corresponding to the output sequence of the i-th unit of the LSTM model; μ: an estimate of the scaling factor, A hyperparameter used to adjust the influence range of the support vector, affecting the aggregation degree of samples in SVDD; ξ: standard deviation of sample data, which measures the degree of dispersion of data and is used to determine the size of the scaling factor; τ: smoothing parameter, which controls the smoothness of the model; ψ: trade-off parameter, used to adjust the trade-off between different loss terms; Step 3.3: Calculate the LSTM-SVDD joint loss function. The LSTM-SVDD joint loss function is a loss function used to measure the degree of matching between the features extracted by LSTM and the SVDD target, expressed as: Loss=λ1·MSE+λ2·SVDDSphereLoss Among them, λ1 and λ2 are weight coefficients used to balance the importance of the two loss terms; SVDD Sphere Loss is the hypersphere loss, which is used to measure the distance between the extracted feature points and the SVDD hypersphere, and MSE is the mean square error of the LSTM model; Determine whether the LSTM-SVDD joint loss function has reached the global optimal value or the local optimal value. If it has, then the training is terminated and the LSTM weight W and bias B obtained at the last iteration and the center of the SVDD classifier are output. and radius R, otherwise the number of iterations increases by 1, and go to step 3.1 to continue training with the updated parameters.

Images