Flight data anomaly detection method and system based on dual-view enhanced Transform

Abstract

The invention relates to a flight data anomaly detection method and system based on dual-view enhanced Transform, and belongs to the technical field of aviation safety. According to the invention, the task of flight safety anomaly detection is expanded beyond a traditional overrun event, aiming at the subtle anomaly which is often not detected in the flight process; according to the fact that QAR data is a typical form of time series data, a flight safety problem is re-expressed as an anomaly detection task in time series analysis; in addition, an unsupervised anomaly detection model specially designed for QAR data is designed in order to solve the challenge of excessively depending on expert marking data in flight safety research. And finally, designing a time sequence anomaly detection model architecture based on reconstruction according to the feature sparse characteristic of the QAR data. According to the method, the condition of deviating from the normal flight mode can be identified in time, early warning information is provided for airlines, experts are effectively assisted in analyzing abnormal flight behaviors, and the method has a wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the field of aviation safety technology and relates to a flight data anomaly detection method and system based on a dual-view enhanced Transformer. Background Art

[0002] Aviation safety has always been a core issue in the civil aviation industry. According to Airbus's statistical analysis of commercial aviation accidents, the number of flight accidents in 2022 was significantly lower than more than a decade ago. However, considering the economic losses and serious harm caused by safety accidents, aviation safety is still highly concerned by airlines and has become an important direction for the continuous improvement and optimization of the civil aviation industry. As a flight data recording device, the Quick Access Recorder (QAR) is now widely installed in various types of commercial passenger aircraft in China. It records multiple parameters such as aircraft status, pilot operations, and environmental factors throughout the flight phase in real time. Analyzing the deviation from the reference to determine whether there is a fault is the key to ensuring flight safety. At present, the analysis of abnormal data for QAR is still very limited. Most airlines use the exceedance detection (ED) method to analyze QAR data, but this method is limited to pre-defined anomalies. It cannot analyze a large number of unknown anomalies in QAR data, so there are still certain safety risks. In addition, the scale of today's aviation operations and the complexity of QAR data make threshold-based methods no longer available because they are not scalable. Therefore, how to use advanced big data and artificial intelligence technologies to improve the analysis and processing capabilities of flight data, thereby improving the level of civil aviation safety management, is a key focus of the civil aviation industry.

[0003] In recent years, scholars have conducted extensive research on flight safety. It can be roughly divided into two categories: flight safety research based on expert knowledge analysis and data-driven flight safety event research. Flight safety research based on expert knowledge analysis mainly relies on the flight experts' profound insights into civil aviation flight operations, explores the risk factors of unsafe events from the aspects of risk causes, crew psychology, and flight management, and further proposes preventive measures. Chen Zhihua et al. analyzed the hard landing event from the perspective of physical principles, and further found that the psychological reasons of the crew (fear of hard landing and pursuit of standard landing operation mode) are closely related to hard landing, tail scratching, and runway ... Flight-level anomalies refer to sequences or subsequences that form unusual shapes in flight data. For example, Li Xu et al. used curve clustering to identify the types of hard landings. First, they summarized the causes of hard landing events through group and individual comparison analysis, and then established a secondary classification directory tree. On this basis, they extracted curve features for the identification of hard landing patterns. Transient anomalies refer to abnormal values ​​in flight data that deviate from expectations. Usually, over-limit detection is used to determine whether a flight safety event has occurred. For example, Wang Lei et al. expanded the hard landing risk assessment model to a landing risk assessment model, while considering three types of over-limit events: tail strike, hard landing, and runway runoff. Chen et al. explored the influencing parameters of the maximum vertical acceleration during the landing phase through quantile regression. In addition, some machine learning-based methods such as support vector machines and random forests have also been used for prediction and warning of hard landing events. Kang Zongwei et al. used multi-layer spatiotemporal encoders and TG-Attention to predict the landing speed of the aircraft to directly warn of the risk of runway runoff. Tong Chao and others used the LSTM model to predict the vertical load and landing speed of the aircraft during landing, in order to warn of hard landing and runway overrun events.

[0004] In addition, since QAR data is a typical high-dimensional time series data, the problem of abnormal event detection in QAR data is also a typical time series anomaly detection problem. As a crucial practical problem, it has been widely discussed by scholars in the academic community. According to whether the training data contains abnormal labels, existing research is mainly divided into supervised time series anomaly detection and unsupervised time series anomaly detection. Supervised methods can perform better when abnormal labels are available or accessible. Such methods can be traced back to AutoEncoder, LSTM-VAE, etc. On the other hand, under the premise that abnormal labels are difficult to obtain, unsupervised methods can often obtain more generalized models, including OmniAnomaly, GDN, DAGMM, etc. Due to the characteristics of massive high-dimensional, irregular, and difficult to obtain abnormal labels of time series data in the real world, unsupervised time series anomaly detection research is more extensive. According to different abnormality determination criteria, these studies roughly include density estimation-based, clustering-based, reconstruction-based, and autoregression-based methods. The principle of density estimation-based methods is to find outliers in low-density areas. It is assumed that non-outliers appear in dense neighborhoods, and the density of sample points is compared with the density of the sample point's neighbors. Classic methods such as LOF and COF calculate the local density and local connectivity of outliers for outlier determination. Clustering-based methods usually formalize the anomaly score as the distance from the data point to the cluster center. SVDD and Deep SVDD cluster the representation of normal data points into a compact cluster. THOC fuses the multi-scale time series features of the intermediate layer through a hierarchical clustering mechanism and detects anomalies through multi-layer distances. Reconstruction-based methods attempt to detect anomalies through the reconstruction error of data points. Park et al. proposed an LSTM-VAE model that uses LSTM for time series modeling and variational autoencoder (VAE) for reconstruction. OmniAnomaly adds a normalization flow to further extend the LSTM-VAE model and uses reconstruction probability for detection. Autoregressive-based models detect anomalies through the prediction error of the next moment. Similarly, LSTM can also be used to replace the autoregressive model.

[0005] In addition, in recent years, many studies have proposed many novel methods to improve the accuracy of anomaly detection. For example, MTAD-GAT and GDN combine graph neural networks to model feature and time series correlation; MAD-GAN and USAD use adversarial training ideas for modeling; MSCRED and CAE-M use convolutional neural networks and LSTM as encoder-decoder structures; TranAD and Anomaly Transformer use Transformer to further improve the time series representation ability; DCdetector and CARLA use contrastive learning to more clearly distinguish between normal and abnormal data point representations.

[0006] Currently, most existing aviation safety incident studies have the following common problems when applied to QAR data:

[0007] Limited to certain specific safety events: Most of the existing QAR data anomaly detection is targeted at a certain over-limit event. For example, hard landing, runway overrun, tail strike, etc. Although these analyses of flight over-limit events can avoid the occurrence of aircraft safety accidents, the anomaly detection efficiency of these studies is low and will cause pilots to become QAR pilots, that is, they only ensure that there are no over-limit events during the flight process and ignore other potential flight safety issues.

[0008] Reliance on manual annotation: Anomaly detection of QAR data relies on rich expert experience and a large amount of manually annotated data. Most current research is based on supervised learning to analyze flight safety events, but this requires a large amount of high-quality annotated data to train the model to improve its generalization and accuracy. Collecting large-scale annotated data requires sufficient professional domain knowledge and a large amount of manpower costs.

[0009] Feature sparsity: Current research mainly focuses on features related to specific events, which may lead to feature sparsity. In a multidimensional dataset, feature sparsity occurs when only a small number of features contribute significantly to the task, while most features remain irrelevant or inactive, increasing the difficulty of effective feature selection and extraction. Since QAR data contains thousands of flight parameters, the selection of QAR parameters varies depending on the specific research task, resulting in different feature densities, which greatly affects the model performance. Summary of the invention

[0010] In view of this, the purpose of the present invention is to provide a flight data anomaly detection method and system based on dual-view enhanced Transformer. The present invention expands the task of flight safety anomaly detection beyond traditional over-limit events, targeting subtle and often undetected anomalies during flight. Given that QAR data is a typical form of time series data, the flight safety problem is restated as an anomaly detection task in time series analysis. In addition, in order to address the challenge of over-reliance on expert labeled data in flight safety research, an unsupervised anomaly detection model specifically designed for QAR data is designed. Finally, based on the sparse feature characteristics of QAR data, a reconstruction-based time series anomaly detection model architecture is designed.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A flight data anomaly detection method based on dual-view enhanced Transformer, the method specifically comprises the following steps:

[0013] S1: Collect flight data in the Quick Access Recorder (QAR);

[0014] S2: Input the collected time series data into the constructed reconstruction-based time series anomaly detection model for detection;

[0015] S3: Determine whether the flight data is abnormal based on the model detection output results.

[0016] Further, in step S2, the constructed reconstruction-based time series anomaly detection model uses a dual-view enhanced transformer (DUVET) to achieve high-precision time series anomaly detection; specifically, it includes:

[0017] 1) DUVET is based on the Transformer, which has stronger temporal modeling capabilities, and uses the Transformer encoder as the base encoder of the model;

[0018] 2) DUVET treats the data in a time window as a time block patch as the basic unit of model input, extracts local semantic information in the time series, and improves the model's time series modeling ability;

[0019] 3) In order to further solve the problem of poor time series reconstruction ability, DUVET, on the one hand, models the global correlation between data points and other data points through the self-attention mechanism, and on the other hand, adopts the designed bell-shaped attention mechanism (Bell Attention) based on the bell-shaped function to model the local correlation between data points and neighboring data points, and combines the representations of the two views to enhance the time series reconstruction ability of Transformer for anomaly detection.

[0020] Furthermore, the DUVET includes four components from bottom to top: (1) Instance Normalization: used to stabilize the distribution of input time series data and improve training stability; (2) Channel Independence & Patching: channel independence is used to reduce the number of model parameters and alleviate the overfitting problem, and patching is used to enhance local semantics while reducing the computational complexity of the attention mechanism; (3) Dual-View Transformer Encoder: used to capture the global and local dependencies between patches and learn the periodicity and trend of time series data; (4) Reconstruction Linear Layer: used to map the representation that combines global and local associations to the same dimension as the input time series, thereby reconstructing the input.

[0021] Furthermore, the instance normalization is used to adjust the distribution of the network layer output, specifically including:

[0022]

[0023] Where m is the number of variables in the multivariate time series, i = 1,...,m, T is the length of the series; u i and σ i is the mean and standard deviation of the time series of the ith variable, γ and β are learnable parameters for scaling and offset, respectively, and ε is a small constant used to avoid zero errors caused by dividing the denominator; it is worth noting that after obtaining the reconstructed sequence, the output needs to be restored to the original data range through the anti-normalization operation, that is, the mean and standard deviation are added back to the output prediction. The specific process is as follows:

[0024]

[0025] Denormalization is necessary to reconstruct the input time series so that the reconstructed data is easier to interpret, visualize, or use for post-processing.

[0026] Furthermore, the channel independently separates the multivariate time series into multiple univariate sequences, inputs one univariate sequence into the Transformer backbone network each time, and then reconstructs each sequence, and concatenates the reconstructed sequence results as the final output of the model, specifically including: for the output X∈R after the instance normalization operation m×T , X is divided into m univariate sequences X i ∈R 1×T ,i=1,...,m, where each univariate sequence X i They are all independently transmitted to the same Transformer backbone network according to the rules, and finally the network outputs the reconstructed univariate time series

[0027] The specific process of Patching includes: i ∈R 1×T It is divided into N time blocks according to the size of the patch P and the stride length S (the non-overlapping area between two consecutive patches). The specific calculation is as follows:

[0028]

[0029] Where N is the number of patches, It is a rounding down operation; the patch sequence is generated by patching the original univariate sequence It is worth noting that before patching, the last value X at the end of the original sequence needs to bei,end ∈R is repeated S times and then padded to the end of the original sequence to ensure the integrity of Patching.

[0030] Furthermore, the dual-view Transformer encoder integrates a dual-view transformer encoder, which combines the self-attention mechanism of global-local bell-shaped attention to capture global and local dependencies in patches; this dual-view integration ensures that the model can distinguish between normal and abnormal data, thereby preventing over-reconstruction. Even in the dataset where the model has feature sparsity, it has a significant error range, enhancing robustness and stability. The task of the reconstructed linear layer is to map the representation, which encapsulates the global-local association of both, back to the same dimension of the input time series, thereby facilitating the reconstruction of the input data, and mitigating the information loss caused by feature sparsity by effectively reconstructing the input model; thereby improving its reconstruction ability and accuracy of anomaly detection.

[0031] Furthermore, the dual-view Transformer Encoder mainly includes two parts: (1) the Transformer encoder based on the self-attention mechanism is used to extract the global temporal dependency between different patches, and learn the periodicity of the temporal data under the global view according to the global correlation between the time blocks, thereby capturing the temporal global representation; (2) the Transformer encoder based on the bell-shaped attention mechanism is used to extract the local dependency between a patch and its adjacent patches, and learn the trend of the temporal data under the local view according to the local correlation of the time blocks, thereby capturing the temporal local representation;

[0032] The self-attention mechanism adopts a multi-head attention mechanism, which enables the encoder to simultaneously focus on information in different subspaces and encode the patch from multiple perspectives, including: firstly initializing the query vector query, key vector key and value vector value required for the l-th layer of self-attention:

[0033]

[0034] in Represents the query, key and value in self-attention respectively; Represents the first layer Q i ,K i ,V i The linear projection matrix of , H represents the number of attention heads; then, the global attention weights between patches are calculated and the semantic information in the patch sequence is weighted and combined according to the attention weights to obtain a richer and more accurate representation. The specific implementation is as follows:

[0035]

[0036] in represents the attention weight learned by the i-th attention head in the l-th layer. The Softmax(·) function is used to normalize the attention weight. Since the global correlation between patches is considered, The patch semantic information represented is more comprehensive. Finally, concatenate the outputs of each attention head and get the final representation:

[0037]

[0038] in is a learnable parameter matrix;

[0039] Through self-attention, DUVET can capture the global dependencies between patches and further characterize patches based on this global association. For time series data, this global association often represents the periodicity in the time series, which helps DUVET reconstruct the input time series based on the periodicity of the time series.

[0040] The bell-shaped attention mechanism first initializes a, b, and value required for the l-th layer BellAttention:

[0041]

[0042] in They represent the mapping matrices of a and b obtained by the i-th attention head of the l-th layer respectively; then, the local attention weights between patches are calculated and the patch representation of the fused temporal trend is obtained according to the weights. The specific implementation is as follows:

[0043] a i ,b i =σ(a i )+ε,σ(b i )+ε

[0044] a i ,b i =repeat(a i ),repeat(b i )

[0045]

[0046] Where σ(·) is the sigmoid function used to scale a and b to the interval (0,1), ε is a small constant to prevent the attention weight value from approaching 0; repeat(·) represents repeated sampling, and after repeated sampling, a i ,bi ∈R N×N , represents the local attention weight learned by the i-th attention head in the l-th layer; finally, the output of each attention head is concatenated to obtain the final representation:

[0047]

[0048] in is a learnable weight matrix.

[0049] Furthermore, in this method, the mean square error loss function (MSE loss) is used. MSE loss is designed to measure the difference between the input sequence X and the reconstructed sequence Specifically, it collects the loss of each channel over m time series and averages them to obtain the overall target loss:

[0050]

[0051] where || ||2 represents the l2 norm, X i,1:T Represents the value of the i-th univariate sequence from the beginning to the end, and minimizes the mean square error loss function to make the input sequence and the output reconstructed sequence as similar as possible;

[0052] For normal data points, the reconstruction model is trained on large-scale time series data, and the model learns the normal time series pattern, so reconstruction is easier. However, due to the rarity of abnormal points, it is difficult for abnormal data points to establish a strong association with the global and local time series patterns, making them difficult to reconstruct. Based on the above idea, this method uses the reconstruction error as the abnormality definition standard, X∈R m×T The anomaly score of can be defined as follows:

[0053]

[0054] Where AnomalyScore(X)∈R T is a moment-by-moment anomaly score. Usually, the anomaly score of an abnormal data point is higher than that of a normal data point. Based on the moment-by-moment anomaly score, a hyperparameter threshold δ is used to determine whether a data point is abnormal. If the score exceeds the threshold, the output is abnormal, that is:

[0055] Y t =I(AnomalyScore(X t )≥δ) t=1,...,T

[0056] Where I(·) is an indicator function, and the output is 1 when the condition is met, otherwise it is 0.

[0057] The present invention also provides a flight data anomaly detection system based on dual-view enhanced Transformer.

[0058] The beneficial effects of the present invention are:

[0059] 1) The method provided by the present invention can timely identify deviations from normal flight patterns by analyzing a large amount of multivariate time series data generated during actual flight operations, provide early warning information to airlines, and effectively assist experts in analyzing abnormal flight behaviors.

[0060] 2) The present invention excels in processing multivariate and univariate time series, especially for complex aviation flight data, and can maintain stable performance in high-dimensional sparse data scenarios.

[0061] 3) This paper transforms the anomaly detection of flight data into unsupervised time series anomaly detection, providing new application scenarios and research challenges for the field of time series data anomaly detection.

[0062] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0064] Figure 1 is the graph of the sine function;

[0065] Figure 2 The overall architecture diagram of the Dual View Enhanced Transformer (DUVET);

[0066] Figure 3 This is a schematic diagram of channel independence;

[0067] Figure 4 This is a Patching diagram;

[0068] Figure 5 The architecture diagram of the dual-view Transformer encoder;

[0069] Figure 6 It is a bell-shaped curve graph;

[0070] Figure 7 Schematic diagram of parameter sensitivity of main hyperparameters in DEVT;

[0071] Figure 8Schematic diagram of the impact of Patch size (P) on training time;

[0072] Fig. 9 This is a visualization of DUVET anomaly detection;

[0073] Fig.10 The following is a graph of anomaly detection results for an example flight. DETAILED DESCRIPTION

[0074] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0075] The main problem of existing reconstruction-based anomaly detection methods is poor time series reconstruction capability. This is mainly reflected in two aspects: (1) The base encoder used has weak time series capture capability. For example, CNN, RNN and its variants have poor performance on long sequence data due to their weak ability to capture long-term time series dependencies; (2) Existing methods use data at one moment as the basic unit, obtain information from a single time step and ignore the information of adjacent time steps, which makes the encoder extract less semantic information from time series data. These two factors will reduce the model's ability to reconstruct time series data, further affecting the accuracy of anomaly detection. At the same time, inspired by real-world observations, when reconstructing a data point, information that is beneficial to the reconstruction of the data point can be obtained from two perspectives. For example Figure 1 As shown in the figure, for the data points to be reconstructed (red data points) of the sine function, they can be reconstructed by the global green asterisk data points, or by the local data points in the purple dotted box. Therefore, when reconstructing time series data, you can reconstruct by capturing the periodicity of the time series data globally, or you can reconstruct by capturing the trend of the time series data locally.

[0076] The invention proposes a reconstruction-based time series anomaly detection model - Dual-View Enhanced Transformer (DUVET) to achieve high-precision time series anomaly detection. Specifically, DUVET is based on the Transformer with stronger time series modeling capabilities, and uses the Transformer encoder as the base encoder of the model. At the same time, DUVET regards the data in a time window as a time block (patch) as the basic unit of model input, extracts local semantic information in the time series, and thus improves the model's time series modeling ability. In order to further solve the problem of poor time series reconstruction ability, DUVET, on the one hand, models the global association between data points and other data points through the self-attention mechanism, and on the other hand, adopts a novel bell-shaped attention mechanism (BellAttention) based on a bell-shaped function to model the local association between data points and neighboring data points, and combines the representations of the two views to enhance the Transformer's time series reconstruction capability for anomaly detection.

[0077] The specific architecture of the DUVET model of the present invention is as follows Figure 2 As shown. DUVET consists of four components from bottom to top: (1) Instance Normalization: used to stabilize the distribution of input time series data and improve training stability; (2) Channel Independence & Patching: Channel independence is used to reduce the number of model parameters and alleviate overfitting problems, and Patching is used to enhance local semantics while reducing the computational complexity of the attention mechanism; (3) Dual-View Transformer Encoder: used to capture the global and local dependencies between patches and learn the periodicity and trend of time series data; (4) Reconstruction Linear Layer: used to map the representation that combines global and local associations to the same dimension as the input time series, thereby reconstructing the input. Specifically including:

[0078] 1) Instance Normalization:

[0079] Instance Normalization is used to adjust the distribution of network layer outputs. Its main functions include reducing internal covariate shift and improving the generalization ability of the model. Most importantly, in the training of large-scale data, it can stabilize the gradient flow of the network, ensure the stable flow of gradients during the back propagation process, help avoid the problem of gradient disappearance or explosion, and thus improve the training stability of the model. Including:

[0080]

[0081] Where m is the number of variables in the multivariate time series, i = 1,...,m, and T is the length of the sequence. i and σ i is the mean and standard deviation of the time series of the ith variable, γ and β are learnable parameters for scaling and offset, respectively, and ε is a small constant used to avoid zero errors caused by dividing the denominator. It is worth noting that after obtaining the reconstructed sequence, the output needs to be restored to the original data range through the denormalization operation, that is, the mean and standard deviation are added back to the output forecast. The specific process is as follows:

[0082]

[0083] Denormalization is necessary to reconstruct the input time series so that the reconstructed data is easier to interpret, visualize, or use for post-processing.

[0084] 2) Channel independence & Patching:

[0085] Channel independence: Channel independence is a generalization strategy that separates multivariate time series into multiple univariate sequences, inputs one univariate sequence into the Transformer backbone network each time, then reconstructs each sequence and concatenates the reconstructed sequence results as the final output of the model. Figure 3 As shown, Figure 3 The specific idea of ​​channel independence is shown. Specifically, for the output X∈R after instance normalization, m×T ,X is divided into m univariate sequences X i ∈R 1×T ,i=1,...,m, where each univariate sequence X i All are independently fed into the same Transformer backbone network according to the rules. Finally, the network outputs the reconstructed univariate time series Most importantly, it is through this parameter sharing mechanism that the dependencies between different variable sequences are captured. Therefore, channel independence allows the model to generalize the input time series to a certain extent, that is, it allows the model to input time series data with a different number of variables during training during inference, which greatly improves the generalization of the model.

[0086] Patching: Patching was first proposed in the field of computer vision. It is mainly used to divide images into small blocks (patches), and then use these image blocks instead of pixels as the basic units to solve the resource limitation problem without losing pixel information, thereby improving the performance of the model. Time series tasks usually face the challenges of huge computing resource costs and limited model performance brought by long sequences. In recent years, Patching has gradually become popular in the field of time series, mainly because it simplifies the computational complexity of the self-attention mechanism in Transformer. Figure 4 The following is the specific process of Patching in time series data. For each univariate time series X i ∈R 1×T It is divided into N time blocks according to the size of the patch P and the stride length S (the non-overlapping area between two consecutive patches). The specific calculation is as follows:

[0087]

[0088] Where N is the number of patches, This is a rounding operation. Generate a patch sequence by patching the original univariate sequence It is worth noting that before patching, the last value X at the end of the original sequence needs to be i,end∈R is repeated S times and then padded to the end of the original sequence to ensure the integrity of Patching.

[0089] Through Patching, the original time series is no longer a single time step, but a time block combined with context as the input token to the Transformer. This means that when the model understands the semantics of a time unit, it no longer obtains information from a single time step, but takes into account the data information of the moments around the time step. This is conducive to enhancing the Transformer's extraction of local semantic information in the time series, thereby improving model performance. More importantly, Patching greatly reduces the number of tokens input to the Transformer, from the original T to about T / S. Therefore, the time complexity and space complexity of the self-attention mechanism in the Transformer are both reduced from O(T 2 ) is reduced to O((T / S) 2 ), which has a more significant effect on reducing memory consumption and computational complexity in long sequence scenarios. Therefore, Patching allows Transformer to process long sequences faster and more easily while improving Transformer's ability to model time series.

[0090] 3) Dual-view Transformer encoder:

[0091] The component integrates a dual-view transformer encoder, which combines the self-attention mechanism with global-local bell-shaped attention to capture global and local dependencies in patches. This dual-view integration ensures that the model can distinguish between normal and abnormal data, thus preventing over-reconstruction. Even in datasets with feature sparsity, the model has a significant error margin, enhancing robustness and stability.

[0092] 4) Reconstruct the linear layer:

[0093] The task of this layer is to map the representation, which encapsulates the global and local associations of the two, back to the same dimensional input time series, thus facilitating the reconstruction of the input data. By effectively reconstructing the input model, the information loss caused by feature sparsity is mitigated; thereby improving its reconstruction capability and accuracy of anomaly detection.

[0094] The architecture of the Dual-View Transformer Encoder is as follows Figure 5As shown in the figure, it mainly consists of two parts: (1) The Transformer encoder based on the self-attention mechanism is used to extract the global temporal dependencies between different patches, and learns the periodicity of the temporal data in the global view based on the global correlation between time blocks, thereby capturing the global representation of the temporal data; (2) The Transformer encoder based on the bell-shaped attention mechanism is used to extract the local dependencies between a patch and its adjacent patches, and learns the trend of the temporal data in the local view based on the local correlation of the time blocks, thereby capturing the local representation of the temporal data.

[0095] 1) Embedding layer:

[0096] The Embedding layer is mainly used for data vectorization and dimension reduction. Its essence is linear mapping transformation. i is divided into a sequence of time blocks A patch is then considered as the basic unit of model input, and then an embedding operation is performed on the patch size P dimension to map it to a high-dimensional vector to represent the semantic information of the patch. The embedding operation is as follows:

[0097] X i e =W emb T X i p

[0098] in is the weight parameter of the Embedding layer, d model is the embedding dimension, Represents the dense representation of each patch.

[0099] 2) Overall architecture of encoder:

[0100] The dual-view Transformer encoder features alternating stacked attention modules and feed-forward neural network layers. This stacked structure helps learn the potential associations between patches from deep multi-level features. Assume that the dual-view Transformer encoder contains L layers, each of which inputs a patch sequence of length N. Then the overall equation of the lth layer can be expressed as:

[0101] Z l =LayerNorm(Attention(χ l-1 )+χ l-1 )

[0102] χ l=LayerNorm(Feed-Forward(Z l )+Z l )

[0103] in represents the output of the encoder at layer l, and the input χ at layer 0 0 That is X i e . is the latent vector representation of the lth layer. Attention(·) is used to extract the global or local association between patches. Feed-Forward(·) is used to l Perform nonlinear transformation to further extract deeper features. The specific process is as follows:

[0104] Feed-Forward(Z l )=max(0,ZlW1+b1)W2+b2

[0105] in are the weight and bias parameters of the feedforward neural network layer, respectively.

[0106] 3) Self-attention mechanism:

[0107] The present invention uses the self-attention mechanism of the original Transformer encoder to extract the global dependencies between patches, such as Figure 5 The architecture on the left is shown. Unlike traditional recurrent neural networks or convolutional neural networks that are limited by high computational complexity and local receptive field problems, the self-attention mechanism can simultaneously capture the relationship between each position and other positions, so it has the ability to capture global semantics. In order to enhance the encoder's ability to represent patches, the present invention introduces a multi-head attention mechanism, which enables the encoder to simultaneously focus on information in different subspaces and encode patches from multiple angles. Specifically, first initialize the query vector (query), key vector (key), and value vector (value) required for the l-th layer of self-attention:

[0108]

[0109] in They represent the query, key and value in self-attention respectively. Represents the first layer Q i ,K i ,V iThe linear projection matrix of , H represents the number of attention heads. Then, the global attention weights between patches are calculated and the semantic information in the patch sequence is weighted and combined according to the attention weights to obtain a richer and more accurate representation. The specific implementation is as follows:

[0110]

[0111] in represents the attention weight learned by the i-th attention head in the l-th layer, and the Softmax(·) function is used to normalize the attention weight. Since the global correlation between patches is considered, The patch semantic information represented is more comprehensive. Finally, concatenate the outputs of each attention head and get the final representation:

[0112]

[0113] in is a learnable parameter matrix.

[0114] Through self-attention, DUVET can capture the global dependencies between patches and further characterize patches based on this global association. For time series data, this global association often represents the periodicity in the time series, which helps DUVET reconstruct the input time series based on the periodicity of the time series.

[0115] 4) Bell-Shaped Attention Mechanism:

[0116] Although self-attention performs well in modeling global relationships in time series and helps capture features such as the periodicity of time series, it is still lacking in reconstructing data points at a certain moment or a certain period of time. Therefore, in order to capture the trend of time series, the present invention hopes that the model will focus on the influence of surrounding patches when learning the dependencies between patches, that is, the current patch establishes a strong association with the surrounding patches, and a weak association with distant patches. This is reflected in the function graph with the characteristics of high in the middle and low on both sides, and the bell curve fits exactly, such as Figure 6 Its mathematical expression is:

[0117]

[0118] Where a controls the height of the peak and b controls the width of the peak.

[0119] This paper proposes a novel attention mechanism, BellAttention, to extract local dependencies between patches. Similarly, the multi-head attention mechanism is used to enhance the patch representation capability. First, a, b, and value required by the l-th layer BellAttention are initialized:

[0120]

[0121] where a i ,b i ∈R N , They represent the mapping matrices of a and b obtained by the i-th attention head in the l-th layer. Then, the local attention weights between patches are calculated and the patch representation of the fused temporal trend is obtained according to the weights. The specific implementation is as follows:

[0122] a i ,b i =σ(a i )+ε,σ(b i )+ε

[0123] a i ,b i =repeat(a i ),repeat(b i )

[0124]

[0125] Where σ(·) is the sigmoid function used to scale a and b to the interval (0,1), and ε is a small constant to prevent the attention weight value from approaching 0. repeat(·) represents repeated sampling. After repeated sampling, a i ,b i ∈R N×N , represents the local attention weight learned by the i-th attention head in the l-th layer. Finally, the output of each attention head is concatenated to get the final representation:

[0126]

[0127] in is a learnable weight matrix.

[0128] In BellAttention, a is mainly used to control the maximum attention weight value, and b is mainly used to control the range width of the association between the current patch and the adjacent patch. Through the multi-head attention mechanism, each head learns different a and b to represent the encoder's attention to different breadths and strengths of temporal trends when representing the patch. This local association helps the model represent the semantics of the patch from the perspective of temporal trends.

[0129] 5) Reconstruct the linear layer:

[0130] After the above dual-view Transformer encoder, for the i-th variable time series, the output of the L-th layer encoder using self-attention as the base attention mechanism is recorded as The output of the last encoder layer using BellAttention as the base attention mechanism is recorded as The average fusion is performed to obtain the final representation vector Z input to the reconstruction linear layer i , the specific calculation is as follows:

[0131]

[0132] in is a two-dimensional vector, which is converted to a one-dimensional vector by flattening Finally, it is mapped into the same dimension as the input sequence through a linear layer. The specific process is as follows:

[0133]

[0134] in b linear ∈R T are the weights and biases of the reconstructed linear layer respectively. Output The i-th variable time series reconstructed by the model.

[0135] In this invention, the loss function commonly used in reconstruction-based anomaly detection methods, namely, mean square error loss function (MSE loss), is used. MSE loss is intended to measure the difference between the input sequence X and the reconstructed sequence Specifically, it collects the loss of each channel over m time series and averages them to obtain the overall target loss:

[0136]

[0137] where || ||2 represents the l2 norm, X i,1:T Represents the value of the i-th univariate sequence from the beginning to the end. By minimizing the MSE loss, the input sequence and the output reconstructed sequence are made as similar as possible.

[0138] For normal data points, the reconstruction model is trained on large-scale time series data. The model learns the normal time series pattern, so reconstruction is easier. However, due to the rarity of abnormal points, it is difficult for abnormal data points to establish a strong association with the global and local time series patterns and difficult to reconstruct. Based on the above idea, the present invention uses the reconstruction error as the abnormality definition standard, X∈R m×T The anomaly score of can be defined as follows:

[0139]

[0140] Where AnomalyScore(X)∈R T is a moment-by-moment anomaly score. Usually, the anomaly score of an abnormal data point is higher than that of a normal data point. Based on the moment-by-moment anomaly score, a hyperparameter threshold δ is used to determine whether a data point is anomaly. If the score exceeds the threshold, the output is anomaly, that is:

[0141] Y t =I(AnomalyScore(X t )≥δ) t=1,...,T

[0142] Where I(·) is an indicator function, and the output is 1 when the condition is met, otherwise it is 0.

[0143] In this embodiment, the proposed DUVET model is evaluated mainly through four experiments. Among them, the first type of experiment is a comparative experiment, which is used to show the performance improvement effect of DUVET relative to the existing mainstream advanced anomaly detection methods, and the experimental results are displayed through visualization methods. The second type of experiment is an ablation experiment, which is used to verify the effectiveness of Patching, channel independence and BellAttention in DUVET. The third type is a hyperparameter sensitivity analysis experiment, which is used to verify the impact of hyperparameters on experimental results. The fourth type of experiment is the application of DUVET on the QAR dataset to detect and analyze abnormal events in flight parameters.

[0144] Table 1 Dataset statistics

[0145]

[0146] Six public datasets (of which NAB is a univariate dataset and the rest are multivariate datasets) and the QAR professional dataset were used in the experiment. Table 1 summarizes the statistical characteristics of each dataset, where the values ​​in brackets are the number of sequences in the dataset repository. Experiments were performed on all sequences in the dataset and the average scores were recorded. The details of each dataset are as follows:

[0147] (1) NAB is a dataset of multiple real-world data traces, including temperature sensor readings, cloud machine CPU utilization, service request latency, and taxi demand in New York City. However, the sequences in this dataset are known to have incorrect anomaly labels, which are excluded in the experiments.

[0148] (2) MBA is a collection of electrocardiogram records from four patients, containing multiple instances of two different types of abnormalities (supraventricular contractions or premature beats). This is a large-scale dataset popular in the data management community.

[0149] (3) SMAP is a dataset of soil samples and telemetry information from NASA's Mars rovers.

[0150] (4)MSL is a dataset similar to SMAP, but corresponds to sensor and actuator data from the Mars rover itself.

[0151] (5) SMD is a five-week dataset that contains stack traces of resource utilization on 28 machines in a computing cluster.

[0152] (6) SWaT dataset: This dataset was collected from a real-world water treatment plant with 7 days of normal operation and 4 days of abnormal operation. The dataset includes sensor values ​​(water level, flow, etc.) and actuator operations.

[0153] (7) The QAR dataset records multiple parameters such as aircraft status, pilot operation, and environmental information during the entire flight process at different sampling frequencies. The entire experiment is implemented based on the Pytorch deep learning framework. For all datasets, the time window size input to the DUVET model is set to 100 (T = 100). DUVET contains three encoder layers (L = 3), and the hidden state vector dimension d model =256, attention head H=8, and the patch size P will be evaluated in the hyperparameter sensitivity analysis experiment. As for the model training process, all methods are trained using the Adam optimizer, with the initial learning rate set to 0.001 and the batch size set to 128. The present invention divides the training time series in all data sets into 70% training data, 10% validation data, and 20% test data, and uses an early stopping strategy based on the performance of the model in the validation set to prevent overfitting. If the anomaly score at a certain time point is greater than a certain threshold δ, it is marked as an anomaly. The determination of δ is the best threshold searched based on the performance of the model on the validation set. Finally, the test set is evaluated using multiple indicators based on δ and the trained model, including precision, recall, and F1 score.

[0154] Benchmark model and comparison results:

[0155] The comparative experiment compares DUVET with existing unsupervised multivariate time series anomaly detection models. These include reconstruction-based methods OmniAnomaly, CAE-M, SimAD, RestAD, LSTM-VAE-based long short-term memory neural network using graph neural network modeling methods MTAD-GAT, GDN, combined with adversarial generation methods MAD-GAN, contrastive learning-based methods DCdetector, Transformer-based methods TranAD, Anomaly Transformer, a total of 11 baseline models for comparison. In order to make an open and fair comparison, this experiment is set according to the hyperparameters described in the original work of each baseline model.

[0156] Table 2 shows the experimental results of DUVET and other methods on three evaluation metrics on all datasets. It is observed that DUVET consistently outperforms the baseline methods in most cases. Specifically, on three datasets (MSL, MBA, and SWaT), DUVET achieves the best performance on all evaluation metrics. In terms of the overall evaluation metric F1-score, DUVET achieves the best results on 6 of the 7 datasets and the second best result on the SMAP dataset. The results show that DUVET can effectively capture the complex temporal dependencies and subtle patterns in time series data, and exhibits significant performance on different datasets, especially on datasets with high feature sparsity and complex temporal dependencies, such as QAR, MSL, MBA, and SWaT. The dual-view encoder of the model cleverly balances local and global dependencies, while the Bell attention mechanism achieves effective capture of short-term temporal features by optimizing multi-head attention. Notably, on the QAR dataset with significant feature sparsity and subtle patterns, DUVET achieves an impressive F1 score of 94.26, which largely outperforms all baseline models. Moreover, on the SMAP, NAB, and SMD datasets, DUVET shows highly competitive performance, on par with or slightly exceeding the leading methods. Transformer-based models, including AnomalyTrans, TranAD, RestAD, and the proposed DUVET, consistently outperform other methods on datasets with strong periodicity, such as SMD, SMAP, and NAB. Built on a dual-view enhanced Transformer architecture, DUVET shows superior performance on both univariate and multivariate time series tasks, highlighting the superior modeling capability of Transformer-based methods for time series data. In contrast, graph neural network-based methods, such as MTAD-GAT and GDN, show relatively weak performance on these datasets due to their reliance on graph-based representations. While these methods aim to capture complex spatiotemporal dependencies, they often have difficulty in effectively modeling long-term temporal relationships and subtle patterns inherent in periodic datasets. CAE-M struggles on NAB, SWaT, and QAR due to its limited ability to model long-term temporal dependencies, sensitivity to feature sparsity, and insufficient handling of multivariate interactions, which are critical for these datasets. LSTM-VAE performs poorly on the high-periodic SMAP dataset, as RNN-based temporal encoding models often have difficulty effectively capturing long-term periodic dependencies in extended sequences, resulting in suboptimal performance. DCdetector shows strong performance on most datasets, using contrastive learning to distinguish between normal and abnormal patterns, enhancing its anomaly detection capabilities. However, despite its dual-branch structure that captures both global and local information, DCdetector still falls short compared to the performance of DUVET.This difference may stem from the fact that DCdetector relies too much on the contrastive loss function, which minimizes the impact of abnormal data but lacks the adaptability to dynamically adjust attention according to different data characteristics. Therefore, this limitation restricts the ability of DCdetector to capture subtle and complex patterns in more complex datasets, while DUVET's dual-view attention mechanism and adaptability are proven to be more effective.

[0157] The experimental results not only verify the effectiveness and superiority of DUVET in the task of time series anomaly detection, but also highlight its potential as an effective tool in the field of flight safety. Its ability to generalize on datasets with different feature densities, from industrial control systems such as SWaT to complex flight safety data in the QAR dataset, distinguishes it from existing time series anomaly detection models.

[0158] Table 2 Experimental results of DUVET and baseline on all datasets, where P, R and F1 represent precision, recall and F1 score (%) respectively. The best results are in bold and the suboptimal results are in horizontal lines.

[0159]

[0160] Ablation experiment:

[0161] In order to explore the importance of each component in the DUVET model, ablation experiments are performed to exclude one main component at a time and observe how it affects the F1 score of each dataset. Specifically, first, this experiment removes the instance normalization in DUVET, and the input sequence is directly fed into the Transformer encoder without normalization. Secondly, this experiment does not adopt the idea of ​​channel independence, but is based on the conventional idea of ​​channel mixing, that is, all variable sequences are input into the Transformer backbone network after unified Patching, and finally the reconstructed sequence of all variables is output. Third, this experiment excludes Patching and takes the observation value at each moment as a basic unit (that is, the token input into the Transformer). Finally, this experiment removes the bell-shaped attention mechanism that learns temporal trends under local views, and only retains the self-attention mechanism that learns temporal global associations.

[0162] Table 3 Ablation experiment results of DUVET, where F1 represents F1 score (%) and the best result is in bold

[0163]

[0164] The final experimental results are shown in Table 3, from which we can observe that:

[0165] (1) After removing instance normalization, the performance of DUVET has slightly decreased. The main reason is that during the training of deep neural networks, the input distribution of each layer is changing dramatically, which makes the model training unstable. Instance normalization helps to reduce the severity of the change, improve the stability of training, and further improve the performance of the model.

[0166] (2) Channel independence helps improve model performance, mainly because models based on the idea of ​​channel independence are more adaptable. Specifically, each time series is processed separately by the Transformer, which means that different variable time series can learn different attention patterns and then reconstruct them according to their respective attention maps. In contrast, in the channel mixing method, all time series share the same attention pattern, which will greatly reduce the performance of the model if different variable time series have different timing patterns.

[0167] (3) When the input sequence is directly input into the Transformer without Patching, the F1 score of DUVET in each dataset decreases slightly. This once again verifies that Patching can increase the receptive field of the Transformer, thereby better capturing the local semantics of time series data.

[0168] (4) After removing Bell Attention, the performance of DUVET on the four datasets has dropped significantly. This is mainly because the input sequence is only reconstructed through the time series periodicity under the global view. When the periodicity of the time series is not obvious, the effect is not satisfactory. Through Bell Attention, the reconstruction of the sequence can also be combined with the trend reconstruction of the time series under the local view, thereby enhancing the reconstruction ability of the time series.

[0169] In short, each major component in DUVET plays a vital role and has a positive impact on the performance of DUVET.

[0170] Experiments on the influence of hyperparameters:

[0171] Figure 7 and Figure 8 This paper shows how different hyperparameters affect the F1 score and training time of the DUVET model on six datasets. Figure 8 It can be seen that the F1 score fluctuates with the change of parameter P, and each dataset has an optimal P value. This shows that the patch size directly affects the model performance. In addition, Fig. 9As shown in the figure, as P increases, the training time per epoch is significantly reduced, mainly because Patching simplifies the computational complexity of the attention mechanism in Transformer. In general, the choice of P affects the overall performance of DUVET, but the model still maintains superior performance over other baseline methods at different P values. Moreover, Patching significantly reduces the training time and memory consumption of the DUVET model, thereby improving training efficiency. Figure 8 The F1 scores of the DUVET model on various datasets are shown as the size of the hidden vector dimension (d model ) and the number of attention heads. As can be seen from the figure, there is no significant fluctuation in the F1 score, indicating that the DUVET model has strong robustness. model May vary by dataset, but performance is generally good in the range {128,512}.

[0172] Visual analysis:

[0173] In order to more intuitively demonstrate the ability of DUVET to detect anomalies on time series data, a series of visualization experiments are conducted to demonstrate the performance of DUVET on the univariate time series dataset NAB and the multivariate time series datasets SMD and MB. Fig. 9 As shown. For multivariate time series datasets, a summary series is generated by averaging all variable dimensions to provide a more concise representation. In addition, if an anomaly occurs in any dimension at a time point, the time point will also be marked as an anomaly in the overall series. In the figure, the blue rectangle represents the actual anomaly data point, and the red area represents the predicted anomaly. The DUVET model determines whether a data point is abnormal based on the anomaly score (reconstruction error) and the corresponding threshold. The visualization results clearly show that the DUVET model can effectively identify abnormal areas in time series data. In addition, the subsequences in the original sequence that deviate from the expected time pattern have higher anomaly scores at these time points, indicating that the reconstruction-based DUVET model is more sensitive to data points that deviate from the expected normal pattern of the overall sample, making it easier to detect anomalies. In summary, the visualization of the detection results intuitively verifies the advanced capabilities of the DUVET model.

[0174] Application of DUVET in flight safety analysis:

[0175] As a typical representative of high-dimensional time series datasets, QAR data has the characteristics of complexity, specialization, and a wide range of parameters. The diversity of QAR parameters leads to equally diverse anomaly detection applications, such as engine fault detection, over-limit event detection, and pilot operation analysis. Therefore, from a technical perspective, time series anomaly detection based on QAR data often has unique challenges, such as different levels of feature density. DUVET effectively addresses these complexity issues and exhibits stable and versatile performance on multiple publicly available datasets and QAR datasets, making it particularly suitable for the diverse technical needs of QAR anomaly detection. From an application perspective, QAR-based anomaly detection research meets different research needs, and each study emphasizes different combinations of QAR parameters. In order to rigorously evaluate the performance of DUVET, the present invention uses QAR to conduct experiments with various parameter combinations, thereby achieving a comprehensive evaluation of different configurations.

[0176] Table 4 Description of QAR parameters used for anomaly detection

[0177]

[0178] In order to detect abnormal data in the flight parameter sequence and further analyze it, this experiment applies the DUVET model to the QAR dataset. Specifically, the dataset used in this experiment comes from the Airbus A321 flight at Beijing Airport in China, with a total of 25,914 samples. Among them, the training set, validation set, and test set are divided in a ratio of 7:1:2. The experiment focuses on the critical approach and leveling stage during the aircraft landing process, that is, 20 seconds before the aircraft touches the ground. The main reason is that these 20 seconds are the stage from 200 feet to landing. At this stage, the pilot needs to ensure that the aircraft gradually approaches the runway at an appropriate descent rate and speed, and maintains a stable attitude for landing. At this time, the pilot needs to accurately evaluate and adjust the wind speed, altitude, speed, and landing point to ensure a safe and smooth landing. The flight parameters selected in the experiment are mainly parameters that reflect the crew operation and the aircraft's own status, as shown in Table 4.

[0179] The experimental results of univariate and multivariate time series anomaly detection are shown in Tables 5 and 6, respectively. In the univariate scenario, DUVET achieves excellent F1 scores in all QAR parameters, with the highest PITCH reaching 98.25%, demonstrating its effectiveness in capturing single time series anomalies. For multivariate experiments, different combinations of QAR parameters are considered by selecting multiple parameter subsets from the 10 parameters in Table 5. As shown in Table 6, the model always provides a stable F1 score and achieves the best performance (F1 = 96.08%) when all parameters are used. The experimental results show that DUVET has strong generalization ability, can effectively handle different feature combinations, and maintain high accuracy in scenarios with different feature complexity.

[0180] Table 5. Experimental results of anomaly detection on QAR data

[0181]

[0182] Table 6 Experimental results of multivariate anomaly detection on QAR data

[0183]

[0184] To further illustrate the ability of DUEVT to detect flight safety risks, this embodiment visualizes the abnormal detection results of 10 flight parameters for an example flight, such as Fig.10 As shown, the parameter curves of this flight are compared with the group pattern, and the detected anomalies are contained in the red dashed box. Note that due to the different sampling frequencies, the curve length (number of data points) of each QAR parameter is also different. The lower half of each sub-figure is the outlier curve and the corresponding threshold (red horizontal line), and the abnormal data points are marked in red. It can be observed that for some parameters, including GS, IVV_CA and N11, the results show no significant anomalies. On the contrary, for pilot operation parameters, including pitch control (PITCH_CPT) and roll control (ROLL_CPT), they show significant fluctuations throughout the final approach phase, significantly deviating from their normal range. The rudder control parameter (RUD_PEDA_POS) deviates from the normal range a few seconds before landing. Fig.10 (e), (f), (g), and (h) show four key parameters of the flight status, including roll angle (ROLL), rudder position (RUDD), pitch angle (PITCH), and vertical load (VRTG). From here we can observe that in the initial stage of the 20-second descent, the aircraft exhibited an abnormally low roll angle (ROLL), followed by an unstable rudder position (RUDD). Fig.10As shown in (g), the pitch angle (angle of pitch) of the aircraft remained within the normal range during the intermediate stage, but experienced significant fluctuations that deviated from its overall safe trend, followed by an abnormal pattern in the final moments before landing. Fig.10 As shown in (h), the vertical load (VRTG) exhibits significant fluctuations, deviating from the normal range before landing, indicating an increased risk of a hard landing.

[0185] The visualization results show that DUVET is able to identify instantaneous and sudden risks from a local perspective, as well as safety hazards throughout the landing phase. From the results, it can be concluded that correct pilot operation and stable flight status are essential for a smooth landing, thereby reducing the possibility of flight safety events such as hard landings and tail strikes. These findings help improve flight safety. On the one hand, DUEVT enables crew members to take corrective actions in a timely manner by identifying abnormal flight patterns in real time, thereby reducing potential risks and ensuring the safety of passengers and crew members; on the other hand, airlines can use DUEVT to develop standardized safety protocols, guide pilot operations, and proactively respond to abnormal behaviors, ultimately improving pilot performance and operational safety.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A flight data anomaly detection method based on dual-view enhanced Transformer, characterized by: The method specifically comprises the following steps: S1: Collect flight data in the quick access recorder QAR; S2: Input the collected time series data into the constructed reconstruction-based time series anomaly detection model for detection; S3: Determine whether the flight data is abnormal based on the model detection output results.

2. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 1, characterized in that: In step S2, the constructed reconstruction-based time series anomaly detection model uses a dual-view enhanced transformer (DUVET) to achieve high-precision time series anomaly detection; specifically, it includes: 1) DUVET is based on the Transformer, which has stronger temporal modeling capabilities, and uses the Transformer encoder as the base encoder of the model; 2) DUVET treats the data in a time window as a time block patch as the basic unit of model input, extracts local semantic information in the time series, and improves the model's time series modeling ability; 3) DUVET, on the one hand, models the global association between data points and other data points through the self-attention mechanism, and on the other hand, adopts the designed bell-shaped attention mechanism based on the bell-shaped function to model the local association between data points and neighboring data points, and combines the representations of the two views to enhance the temporal reconstruction capability of Transformer for anomaly detection.

3. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 2 is characterized in that: The DUVET consists of four components from bottom to top: (1) instance normalization: used to stabilize the distribution of input time series data and improve training stability; (2) Channel independence & Patching: Channel independence is used to reduce the number of model parameters and alleviate the overfitting problem, and Patching is used to enhance local semantics while reducing the computational complexity of the attention mechanism; (3) Dual-view Transformer encoder: used to capture the global and local dependencies between patches and learn the periodicity and trend of time series data; (4) Reconstruction linear layer: used to map the representation that combines global and local associations to the same dimension as the input time series, thereby reconstructing the input.

4. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 3 is characterized by: The instance normalization is used to adjust the distribution of the network layer output, specifically including: Where m is the number of variables in the multivariate time series, i = 1,...,m, T is the length of the series; u i and σ i is the mean and standard deviation of the time series of the i-th variable, γ and β are learnable parameters for scaling and offset, respectively, and ε is a small constant used to avoid zero errors caused by dividing the denominator; after obtaining the reconstructed sequence, the output needs to be restored to the original data range through the anti-normalization operation, that is, the mean and standard deviation are added back to the output prediction. The specific process is as follows: Denormalization is necessary to reconstruct the input time series so that the reconstructed data is easier to interpret, visualize, or use for post-processing.

5. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 4 is characterized in that: The channel independently separates the multivariate time series into multiple univariate sequences, inputs one univariate sequence into the Transformer backbone network each time, then reconstructs each sequence, and concatenates the reconstructed sequence results as the final output of the model, specifically including: for the output X∈R after the instance normalization operation m×T , X is divided into m univariate sequences X i ∈R 1×T ,i=1,...,m, where each univariate sequence X i They are all independently transmitted to the same Transformer backbone network according to the rules, and finally the network outputs the reconstructed univariate time series The specific process of Patching includes: i ∈R 1×T It will be divided into N time blocks according to the size of the patch P and the stride length S. The specific calculation is as follows: Where N is the number of patches, It is a rounding down operation; the patch sequence is generated by patching the original univariate sequence Before patching, you need to replace the last value X at the end of the original sequence. i,end ∈R is repeated S times and then padded to the end of the original sequence to ensure the integrity of Patching.

6. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 5, characterized in that: The dual-view Transformer encoder integrates a dual-view transformer encoder, combined with a self-attention mechanism of global-local bell-shaped attention, to capture global and local dependencies in patches; the task of the reconstructed linear layer is to map the representation, which encapsulates the global-local associations of the two, back to the same dimensional input time series, thereby facilitating the reconstruction of the input data, and reducing the information loss caused by feature sparsity by effectively reconstructing the input model.

7. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 6 is characterized by: The dual-view Transformer encoder mainly consists of two parts: (1) The Transformer encoder based on the self-attention mechanism is used to extract the global temporal dependency between different patches, and learn the periodicity of the temporal data under the global view based on the global correlation between time blocks, thereby capturing the global representation of the temporal data; (2) The Transformer encoder based on the bell-shaped attention mechanism is used to extract the local dependency between a patch and its adjacent patches, and learn the trend of the temporal data under the local view based on the local correlation of the time blocks, thereby capturing the local representation of the temporal data; The self-attention mechanism adopts a multi-head attention mechanism, which enables the encoder to simultaneously focus on information in different subspaces and encode the patch from multiple perspectives, including: firstly initializing the query vector query, key vector key and value vector value required for the l-th layer of self-attention: in Represents query, key and value in self-attention respectively; Represents the first layer Q i ,K i ,V i The linear projection matrix of , H represents the number of attention heads; then, the global attention weights between patches are calculated and the semantic information in the patch sequence is weighted and combined according to the attention weights to obtain a richer and more accurate representation. The specific implementation is as follows: in represents the attention weight learned by the i-th attention head in the l-th layer. The Softmax(·) function is used to normalize the attention weight. Finally, the output of each attention head is concatenated to obtain the final representation: in is a learnable parameter matrix; The bell-shaped attention mechanism first initializes a, b, and value required for the l-th layer BellAttention: where a i ,b i ∈R N , They represent the mapping matrices of a and b obtained by the i-th attention head of the l-th layer respectively; then, the local attention weights between patches are calculated and the patch representation of the fused temporal trend is obtained according to the weights. The specific implementation is as follows: a i ,b i =σ(a i )+ε,σ(b i )+e a i ,b i =repeat(a i ),repeat(b i ) Where σ(·) is the sigmoid function used to scale a and b to the interval (0,1), ε is a small constant to prevent the attention weight value from approaching 0; repeat(·) represents repeated sampling, and after repeated sampling, a i ,b i ∈R N×N , represents the local attention weight learned by the i-th attention head in the l-th layer; finally, the output of each attention head is concatenated to obtain the final representation: in is a learnable weight matrix.

8. The flight data anomaly detection method based on dual-view enhanced Transformer according to claim 7, characterized in that: In this method, the mean square error loss function is used to measure the input sequence X and the reconstructed sequence Specifically, it collects the loss of each channel over m time series and averages them to obtain the overall target loss: where || ||2 represents the l2 norm, X i,1:T Represents the value of the i-th univariate sequence from the beginning to the end, and minimizes the mean square error loss function to make the input sequence and the output reconstructed sequence as similar as possible; This method uses the reconstruction error as the anomaly definition criterion, X∈R m×T The anomaly score of can be defined as follows: Where AnomalyScore(X)∈R T is a moment-by-moment anomaly score. Usually, the anomaly score of an abnormal data point is higher than that of a normal data point. Based on the moment-by-moment anomaly score, a hyperparameter threshold δ is used to determine whether a data point is abnormal. If the score exceeds the threshold, the output is abnormal, that is: Y t =I(AnomalyScore(X t )≥δ) t=1,...,T Where I(·) is an indicator function, and the output is 1 when the condition is met, otherwise it is 0.

9. A flight data anomaly detection system based on dual-view enhanced Transformer, characterized by: The system adopts the method as claimed in any one of claims 1 to 8.

Images