Civil aviation aircraft test flight data anomaly detection method based on multi-stage denoising preprocessing
By employing multi-level denoising preprocessing and dynamic threshold determination, the problems of high false alarm rate and low detection accuracy under noise interference during civil aircraft test flights have been solved, achieving high-precision anomaly detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CIVIL AVIATION FLIGHT UNIV OF CHINA
- Filing Date
- 2026-05-13
- Publication Date
- 2026-06-12
AI Technical Summary
Existing anomaly detection methods have a high false alarm rate in civil aircraft test flights, are not sensitive to minor anomalies, and rely on fault label training, which is not effective in unlabeled environments and makes it difficult to effectively distinguish between environmental noise and real faults.
A multi-level denoising preprocessing method is adopted, including discrete wavelet transform and Kalman filtering to remove high-frequency electromagnetic noise and low-frequency sensor drift. It is combined with LSTM-AE model for unsupervised learning and anomalies are identified by dynamic threshold judgment.
It significantly improves the signal-to-noise ratio of the data, reduces the false alarm rate, improves the detection accuracy, has wide applicability, and can complete the model deployment without manually annotating fault data.
Smart Images
Figure CN122196842A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aerospace technology, and in particular to a method for detecting anomalies in flight data during civil aircraft test flights. Test flights are a crucial step in obtaining airworthiness certificates for civil aircraft, and the quality of the data directly affects the safety and effectiveness assessment of test flights. Background Technology
[0002] Flight testing of civil aircraft is a crucial step in airworthiness certification, and the massive amounts of multi-dimensional time-series data generated are the core basis for assessing aircraft safety. However, unlike routine operations, test aircraft typically undergo complex non-standard modifications and must perform high-intensity maneuvers under extreme envelope and large temperature difference conditions. These extreme conditions result in extremely complex data collected by airborne sensors: on the one hand, the dense testing and modification equipment easily introduces high-frequency electromagnetic noise interference; on the other hand, long-duration cross-airspace flights can easily cause nonlinear low-frequency null drift in sensors. This mixed noise severely degrades data quality, posing significant challenges to subsequent analysis.
[0003] Existing anomaly detection methods struggle to effectively address the aforementioned challenges. Traditional fixed threshold or simple statistical methods are ill-suited to the intense dynamic maneuvers of flight tests, easily generating high false alarm rates even during normal high-G maneuvers. Meanwhile, mainstream supervised learning models rely on a large number of fault labels for training, which is impractical in flight test scenarios where exploration of the unknown is paramount and fault samples are extremely scarce. Furthermore, conventional unsupervised algorithms, when directly applied to unprocessed noisy data, often fail to distinguish between environmental noise and genuine faults, resulting in low detection accuracy. Therefore, a method capable of accurately removing noise and identifying flight anomalies in unlabeled, highly interfering environments is urgently needed. Summary of the Invention
[0004] The purpose of this invention is to provide a method for detecting anomalies in civil aircraft flight test data based on multi-level denoising preprocessing, aiming to solve the problems of high false alarm rate, insensitivity to minor anomalies, and reliance on fault labels in existing technologies under strong noise environments.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method for detecting anomalies in civil aircraft test flight data based on multi-level denoising preprocessing, which mainly includes four stages: data acquisition and alignment, multi-level serial denoising preprocessing, unsupervised model training, and dynamic threshold determination.
[0006] First, to address the various types of noise present in flight data, this invention designs a complementary multi-stage cascaded denoising mechanism. The first stage employs Discrete Wavelet Transform (DWT), utilizing the asymmetry and tight support characteristics of wavelet basis functions to adaptively remove high-frequency electromagnetic noise from the signal. The second stage then employs Kalman Filtering, constructing a state-space model to correct for low-frequency sensor drift remaining in the wavelet-processed signal. This "high-frequency first, low-frequency later" strategy can output high signal-to-noise ratio standard flight data while preserving the true flight maneuver characteristics (such as sharp climbs) to the greatest extent possible.
[0007] Secondly, in constructing the detection model, this invention utilizes LSTM-AE (Long Short-Term Memory-Autoencoder) for unsupervised learning on the aforementioned standard data. The encoder compresses long-term temporal data into latent feature vectors, which are then reconstructed by the decoder. Since the model is trained only on normal data, it will be unable to effectively reconstruct the data when abnormal data is input, resulting in significant reconstruction errors.
[0008] Finally, in the anomaly detection stage, this invention abandons the traditional fixed threshold and instead adaptively generates a dynamic detection threshold based on the statistical distribution of the reconstruction error of the training set (such as mean and variance). Simultaneously, a temporal continuity rule is introduced, triggering an alarm only when multiple consecutive time windows exceed the threshold, further filtering out occasional transient disturbances.
[0009] The beneficial effects of this invention are as follows:
[0010] 1. Strong noise resistance: Experiments show that after adopting the multi-level denoising strategy of this invention, the signal-to-noise ratio (SNR) and root mean square error (RMSE) of the data are better than those of a single filtering method, effectively restoring the real flight waveform.
[0011] 2. High detection accuracy: By combining dynamic thresholds with temporal continuity rules, this method can achieve an F1 score of over 90% when detecting sudden velocity and height anomalies, significantly reducing the false alarm rate.
[0012] 3. Wide applicability: No manual annotation of fault data is required; model deployment can be completed using only normal flight data, greatly reducing the application threshold. Attached Figure Description
[0013] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:
[0014] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;
[0015] Figure 2 This is a schematic diagram showing the principle and effect comparison of multi-stage serial noise reduction preprocessing;
[0016] Figure 3 This is a schematic diagram of the network topology of the LSTM-AE model;
[0017] Figure 4 This is a schematic diagram illustrating the principle of the dynamic threshold determination mechanism;
[0018] Figure 5 This is an example diagram showing the anomaly detection results of the method of the present invention in actual flight data. Detailed Implementation
[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0020] Example 1: Overall Testing Process
[0021] like Figure 1 As shown, this embodiment proposes a method for detecting anomalies in civil aircraft flight test data based on multi-level denoising preprocessing. To ensure the accuracy and real-time performance of the detection, the overall workflow of this method unfolds in a logical time sequence.
[0022] The system first collects multi-dimensional sensor time-series data from the airborne flight data recording system or ground monitoring system. This data covers key flight parameters such as airspeed, barometric altitude, latitude and longitude, and attitude angles. After collection, the original multi-dimensional observation sequence is constructed through timestamp alignment. To address potential missing values in the data, this embodiment employs linear interpolation or direct removal of invalid samples to clean the data, ensuring the integrity and temporal continuity of subsequent input data.
[0023] After acquiring the cleaned raw observation sequence, the system performs a core multi-stage cascaded denoising process. This step is crucial to the invention, aiming to sequentially filter out high-frequency electromagnetic noise and correct low-frequency sensor drift, thereby outputting standard flight data with a high signal-to-noise ratio. The specific denoising logic will be described in detail in Example 2.
[0024] Subsequently, to eliminate the dimensional differences between data from different sensors, the system uses the Z-Score normalization method to normalize the data. After processing, a sliding window mechanism (with a window length of 30 in this embodiment) is used to reconstruct the continuous time series into a sample set suitable for model input, and the sample set is divided into training and testing sets proportionally to prepare for subsequent model training.
[0025] Next, the unsupervised training phase begins, where the system constructs an autoencoder model based on a bidirectional long short-term memory network (LSTM-AE). During training, the model is iteratively trained using only training set data containing normal flight conditions. The training objective is to optimize the model parameters by minimizing the error between the input sequence and the reconstructed sequence until the reconstruction loss converges, thereby enabling the model to learn the temporal patterns of normal flight data.
[0026] Finally, in the real-time anomaly detection phase, the flight data to be tested is input into the trained model, and its real-time reconstruction error is calculated. The system combines adaptively generated dynamic thresholds and connected component analysis rules to determine whether there are any anomalies in the current flight state and outputs the final alarm result.
[0027] Example 2: Multi-stage cascaded noise reduction mechanism
[0028] like Figure 2 As shown, this invention performs a refined "operation" to address the noise mixed in the original data:
[0029] First stage: High-frequency noise reduction. db4 is selected as the wavelet basis function, and the decomposition level is set to 5 levels. Since the waveform of the db4 wavelet matches the characteristics of sudden changes in flight attitude more closely, it can effectively identify abrupt changes in the signal. The system calculates the absolute median deviation (MAD) of the detail coefficients at each level to estimate the noise intensity and generate an adaptive soft threshold to filter out high-frequency glitches.
[0030] Second stage: Low-frequency correction. The wavelet-reconstructed signal is input into a Kalman filter. A one-dimensional state-space model is constructed, with the state transition matrix F and observation matrix H set as identity matrices. The key lies in parameter configuration: setting the process noise covariance. Observation noise covariance This parameter combination means that the model assumes the actual flight state is relatively smooth (small Q), while the current observations contain a large error (large R). Through this parameter game, the filter successfully smooths the baseline drift caused by the sensor's long-term operation.
[0031] Comparative experimental data show that after this two-stage processing, the signal-to-noise ratio of the data is significantly improved without causing phase delay of the real signal.
[0032] Example 3: LSTM-AE Model Construction
[0033] like Figure 3 As shown, the detection model of this invention consists of an encoder, a decoder, and an output layer, and learns the normal flight mode through a "compression-decompression" process. The specific network parameter configuration is as follows:
[0034] Input layer: Receives a time window sequence with shape (Window_Size, Features). In this embodiment, the window length is set to 30.
[0035] The encoder uses an LSTM layer to extract temporal features, employing ReLU as the activation function to avoid gradient vanishing and accelerate convergence. The LSTM unit count is set to 32 (Latent Dim) to extract long-term dependencies in the sequence. The hidden state at the last time step is preserved as a latent vector.
[0036] Decoder: First, a repeat vector layer is used to copy the latent feature vector 30 times in the time dimension. Then, an LSTM layer (also using the ReLU activation function) is connected to gradually restore the features to the time sequence matrix.
[0037] Output Layer: To map the LSTM output back to the dimensions of the original data, a TimeDistributed Dense layer is connected at the end of the model. This layer performs a fully connected operation independently at each time step, ultimately outputting a reconstructed sequence with dimensions completely identical to the input.
[0038] The model was trained using the Adam optimizer, with the loss function being the mean squared error (MSE), the batch size being set to 64, and the number of epochs being set to 50.
[0039] Combination Figure 3 The network topology diagram in the image further explains the formulas and core letter variables in the model: In the coding layer, the network reception length is L (corresponding to the time window length Window Size in this embodiment, i.e.) The input sequence has a feature dimension of M. (The figure shows...) to This represents the input data vectors at different time steps, each containing M collected flight feature variables (i.e., in the figure). ). This represents the hidden state vector extracted by the LSTM encoding layer at time t. As the sequence propagates forward, the hidden state at the final time step... The extracted and retained features serve as latent feature vectors carrying the global temporal patterns of the input sequence. In the decoding layer, the network structure is mirror-symmetric to the encoding layer. The core formulas are marked in the diagram. This indicates that the decoder directly uses the terminal hidden state output by the coding layer as its initial hidden state input to start the reconstruction process. This represents the hidden state on the decoding layer side. (See diagram) to This represents the reconstructed output vector generated at the corresponding time step of the decoding layer. Each output vector also contains M reconstructed features (i.e., in the figure). By minimizing the error between the X sequence and the R sequence, the model can fully learn the temporal evolution patterns of civil aircraft under normal flight conditions.
[0040] Example 4: Dynamic Thresholds and Anomaly Detection
[0041] like Figure 4 As shown, in order to solve the problem that fixed thresholds cannot adapt to complex working conditions, this invention establishes an adaptive judgment logic, which covers three aspects: dynamic threshold generation, window marking, and connected component analysis.
[0042] The first step is the dynamic threshold generation. After model training is complete, the system calculates the reconstruction error (MSE) of all samples in the training set and sets the threshold using the multiple mean method. The threshold calculation formula is:
[0043]
[0044] in To reconstruct the mean error for the training set, This is the sensitivity coefficient. Experiments have verified that when... When the value is 2, the threshold can accurately cover about 99% of the training set error distribution, and the F1 score of the detection reaches its optimum.
[0045] The second step is the window anomaly marking. During the detection phase, the system calculates the reconstruction error of each sliding window in the test set. If the error of a window exceeds the aforementioned dynamic threshold, the system marks all data points within the time period covered by that window (i.e., from the current time to the end time of the window) as potential anomalies.
[0046] Finally, there is the connectivity analysis and alarm process. To prevent false alarms triggered by occasional noise, the system performs connectivity analysis on the marked potential anomalies, merging temporally consecutive anomalies into a single independent anomaly event. Only when the duration of the merged anomaly event exceeds a preset value (e.g., 5 sampling points) does the system consider the anomaly valid, issue a formal alarm, and mark that time period in the result graph (e.g., ...). Figure 5 (The red highlighted area is shown). This strategy effectively avoids false alarms caused by single-point noise and significantly improves the robustness of the system.
Claims
1. A method for detecting anomalies in civil aircraft flight test data based on multi-level denoising preprocessing, characterized in that, Includes the following steps: Step 1: Collect aircraft flight time-series data and construct the original multidimensional observation sequence by aligning the timestamps; The original multidimensional observation sequence is subjected to multi-level cascaded denoising processing to output standard flight data with high signal-to-noise ratio; the multi-level cascaded denoising processing specifically includes: firstly, using discrete wavelet transform to separate high-frequency random noise in the original signal, and then using Kalman filtering to correct low-frequency sensor drift in the signal; Step 2: Divide the standard flight data into training and test sets based on the sliding window mechanism, and standardize the training set data; Step 3: Construct an LSTM-AE model. Use training set data containing only normal flight conditions to train the model in an unsupervised manner. Optimize the model parameters by minimizing the error between the input sequence and the reconstructed sequence until the model converges. Calculate the statistical distribution characteristics of the reconstruction error on the converged model using the training set, and construct a dynamic anomaly judgment threshold by combining the preset anomaly coefficient. Step 4: Input the flight data to be tested into the trained LSTM-AE model, calculate its real-time reconstruction error, and determine whether the current flight state is abnormal based on the dynamic anomaly judgment threshold and time sequence continuity rules.
2. The method according to claim 1, characterized in that, The method of separating high-frequency random noise in the original signal using discrete wavelet transform specifically includes: selecting db4 wavelet as the basis function, setting the decomposition level to 5 levels, decomposing the original signal into approximation coefficients and detail coefficients of different frequency bands; using an adaptive threshold algorithm based on the absolute deviation of the median to calculate the noise level, and performing soft thresholding on the detail coefficients of each level to filter out high-frequency noise caused by electromagnetic interference.
3. The method according to claim 1, characterized in that, The method of using Kalman filtering to correct low-frequency sensor drift specifically includes: constructing a one-dimensional state-space model, setting both the state transition matrix and the observation matrix to be identity matrices to match the slow time-varying characteristics of flight parameters; and suppressing low-frequency drift caused by temperature or time accumulation of the sensor by configuring the ratio of process noise covariance Q to observation noise covariance R.
4. The method according to claim 3, characterized in that, Q=0.01 and R=2 are set to preserve realistic flight maneuver characteristics while smoothing baseline drift.
5. The method according to claim 1, characterized in that, The LSTM-AE model adopts a bidirectional LSTM structure, with the following topology: the encoder part contains multiple LSTM units, which are used to extract deep temporal dependency features of the input time window sequence and compress them into latent feature vectors; the decoder part uses a mirrored LSTM unit structure to gradually decompress the latent feature vectors and reconstruct them into an output sequence with the same dimension as the input.
6. The method according to claim 1, characterized in that, The formula for calculating the dynamic anomaly detection threshold Threshold is as follows: ,in, The outlier coefficient, The mean of the reconstruction error of the training set samples; the anomaly coefficient The selection is based on the distribution characteristics of the reconstruction error in the training set.
7. The method according to claim 6, characterized in that, set up This covers 99.2% of the normal data distribution range.
8. The method according to claim 1, characterized in that, The temporal continuity rule is specifically defined as follows: an anomaly alarm is triggered only when the reconstruction error at N consecutive time points exceeds the dynamic anomaly determination threshold.
9. The method according to claim 8, characterized in that, set up This is to filter out false alarms caused by transient disturbances.