Aviation flight data abnormity real-time restoration method
By performing standard normalization of flight timing data and identifying abnormal types, combining K nearest neighbor repair method and TimeGAN model for real-time repair, the problem of abnormal data deviation in flight tests is solved, and the reliability and accuracy of data are improved.
Patent Information
- Application Number
- CN202411951616.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-25
AI Technical Summary
During flight tests, flight data is prone to severely deviating from the true value of the parameter due to factors such as sensor abnormalities and telemetry signal interference, resulting in false alarms, affecting monitoring effects, and even causing decision-making errors. The existing technology mainly relies on post-event data processing and lacks real-time repair methods.
A real-time repair method for aviation flight data abnormalities is proposed. By standard normalizing the flight timing data, abnormal types are identified, K nearest neighbor repair method and TimeGAN model are used to repair data, and integrity verification is carried out.
It realizes data abnormality repair in real-time monitoring scenarios of flight tests, improves the reliability and accuracy of monitoring data, reduces false alarm rates, and ensures the reliability of data for subsequent task analysis and safety warning.
Smart Images
Figure CN119884621A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of flight test data processing and monitoring, and in particular relates to a real-time repair method for aviation flight data anomalies. Background Art
[0002] Real-time monitoring of the safety status of the test aircraft and evaluation of mission execution through flight data are important technical means to improve test flight efficiency and ensure test flight safety. In the field of flight testing, flight test mission analysis and safety warning are based on the availability and high quality of telemetry data. However, due to factors such as sensor anomalies and telemetry signal interference, telemetry data will seriously deviate from the true value of the parameters, which can easily cause false alarms, affect monitoring effects, and even cause decision-making errors. Therefore, the importance of flight data anomaly repair in the field of flight testing cannot be ignored, and is directly related to flight safety, flight efficiency, and data accuracy.
[0003] At present, the domestic repair of flight data anomalies mainly relies on post-processing, usually using automatic or manual methods, combined with specific rules of various professions to detect data, and clean the detected abnormal data by eliminating or correcting the data; for real-time monitoring, most of them only reach the anomaly detection stage, and use status lights or other visual methods on the front-end monitoring interface to remind monitoring personnel that the current data is unreliable. Some monitoring terminals use a simple threshold method to eliminate outliers, and use the previous moment value to fill in the instantaneous anomaly of single variable parameters for repair. For more complex anomaly types, real-time repair methods are less involved.
[0004] Based on this, it is necessary to provide a new real-time repair method for flight data anomalies to solve anomaly problems such as instantaneous deviation of single variable / multiple correlated variables from the trend and short-term persistent loss, and to ensure the repair effect of abnormal data while considering the real-time requirements of the scene. Summary of the invention
[0005] The present invention aims to provide a method for repairing aviation flight data anomalies in real time, which can repair abnormal data with different characteristics in a real-time monitoring scenario of a flight test to ensure the reliability of the monitoring data for subsequent mission analysis and safety warning.
[0006] The technical solution of the present invention is: According to the first aspect of the present invention, a method for real-time repair of aviation flight data anomalies is proposed, which is characterized by comprising the following steps:
[0007] Step 1: Taking the flight time series data sampled at a fixed time as input, first perform standard normalization on the data to eliminate the dimension and scale differences of the data and perform data preprocessing;
[0008] Step 2: Identify the abnormal type of the current flight data according to the number of sampling points of the flight time series data; count the actual number of sampling points of the data within 1s in real time. If the actual number of sampling points is less than 90% of the periodic sampling points, the flight time series data is considered to be abnormal data with persistent missing; if the actual number of sampling points is greater than or equal to 90% of the periodic sampling points and is less than the periodic sampling points, the flight time series data is considered to be abnormal data with instantaneous deviation from the trend;
[0009] Step 3: For abnormal flight data that deviates from the trend instantaneously, the K nearest neighbor repair method is used to complete the abnormal data repair;
[0010] Step 4: For abnormal flight data that is continuously missing, the TimeGAN model is used to complete abnormal data repair;
[0011] Step 5: Verify the integrity of the repaired flight data obtained in steps 3 and 4.
[0012] In a possible embodiment, in step 1, the specific process of data preprocessing includes: assuming that the original data is X={x 1 ,x 2 ,......,x n}, the following formula (1) is used for data preprocessing:
[0013]
[0014] Among them, μ X and σ X represent the mean and standard deviation of the data set X respectively.
[0015] In a possible embodiment, in step 3, the following steps are specifically included:
[0016] Construct a KNN model and determine the K value; for each abnormal point in the abnormal flight data that deviates from the trend instantaneously, calculate the similarity with its preceding neighbors; select the K samples with the highest similarity as the nearest neighbors of the abnormal point; use the known values of the K nearest neighbors to estimate the abnormal flight data that deviates from the trend instantaneously through weighted averaging.
[0017] In a possible embodiment, the K value is set to a multiple of the periodic sampling points.
[0018] In a possible embodiment, the Euclidean distance of formula (2) is specifically used to measure the similarity between the abnormal point and other sample points;
[0019]
[0020] Among them, d u,i is the Euclidean distance, and ||·|| is the 2-norm.
[0021] In a possible embodiment, the K samples with the highest similarity are used as the outlier point x u The nearest neighbor Y u ={y u1 ,y u2 ,…,y uk}.
[0022]
[0023] in, Indicates taking the first K minimum values, d u is the outlier point x u The set of Euclidean distances to previous order neighbors;
[0024]
[0025] Among them, x u ′ is the outlier point x u The repair value, w u,k is the weighted value of the kth neighbor, which can be calculated by formula (5);
[0026]
[0027] In a possible embodiment, the step 4 specifically includes the following steps:
[0028] Build a generator of the TimeGAN model. Inside the generator, use RNN units to build a multi-layer recurrent neural network to map random noise to the potential representation of generated data.
[0029] In order to adapt TimeGAN to time series data, the time information T is taken as the generator input together with the random noise vector Z. After a fully connected layer, its dimension is mapped to hidden_dim. Using the Sigmoid activation function, the generator returns the generated latent space representation E.
[0030] The training loss of the generator is designed to minimize the difference between the generated data and the real data, hoping that the generated data can be close to the distribution of the real data. Specifically, adversarial loss is used as the loss function of the generator, as shown in formula (6);
[0031]
[0032] Among them, L gs represents the adversarial loss of the generator, and z represents the distribution from the prior distribution p z The random noise extracted from the generator, G(z) represents the sequence generated by the generator, and D is the discriminator;
[0033] Build a discriminator for the TimeGAN model, use RNN units to build a multi-layer recurrent neural network to process the potential representation of the input; map the output dimension to 1 through the fully connected layer to perform binary classification to determine whether the input is real data or generated data, and finally the discriminator returns the classification result;
[0034] The generator output E and the preprocessed normal flight data are used as the discriminator input. The training loss of the discriminator includes the loss of real data and the loss of generated data, as shown in formula (7);
[0035]
[0036] Among them, L ds represents the adversarial loss of the discriminator, and z represents the distribution from the prior distribution p z The random noise extracted from , G(z) represents the sequence generated by the generator, D is the discriminator, and x is the normal flight timing data;
[0037] The training process alternately trains the generator and the discriminator, uses back propagation and the optimizer to continuously update the parameters, and minimizes the two loss functions so that the generator and the discriminator are constantly learning and adapting.
[0038] In a possible embodiment, the flight timing data X with abnormal index is L As the discriminator input of the trained TimeGAN model, the KL divergence of the original data X and the generated data E is calculated. If the KL divergence is less than 0.1, it is considered that the distribution similarity of the generated data is basically the same as that of the original data, and the missing values in the original data are filled with the values of the abnormal index positions of the generated data.
[0039]
[0040] Among them, D KL (P||Q) represents the KL divergence, P(x) represents the probability distribution of the original data X, and Q(e) represents the probability distribution of the generated data E.
[0041] The beneficial effects of this application are:
[0042] 1. In the field of real-time monitoring of flight tests, deep learning algorithms are used for the first time to repair data anomalies, solving the problem that traditional methods cannot obtain data features and only perform repairs based on normal data local to the abnormal data, which is ineffective.
[0043] 2. The K-nearest neighbor repair method can handle the instantaneous jump problem of parameters with trend change rules. This method is simple and easy to implement, and takes into account both real-time performance and algorithm effect in the real-time processing stage of flight data.
[0044] 3. The flight data anomaly repair method based on the TimeGAN model combines the flexibility of the unsupervised generative adversarial network (GAN) and the conditional probability principle of the supervised autoregressive model, ensuring the effect of time series data anomaly repair. At the same time, the introduction of the intrinsic network improves the effectiveness of the model and reduces the dimension of the parameter space. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 This is a flight timing data abnormality repair process of a preferred embodiment of the present invention;
[0046] Figure 2 A flight data anomaly repair process based on the K nearest neighbor method according to a preferred embodiment of the present invention;
[0047] Figure 3 The flight data anomaly repair process based on TimeGAN in the preferred embodiment of the present invention;
[0048] Figure 4 This is a comparison before and after restoration using the K nearest neighbor method of a preferred embodiment of the present invention;
[0049] Figure 5 This is a comparison before and after restoration using the TimeGAN method in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0051] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.
[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0053] The present application is further described in detail below in conjunction with the accompanying drawings of the embodiments.
[0054] like Figure 1 As shown, the real-time repair process of aviation flight data anomalies is divided into five steps.
[0055] 1. First, the flight time series data with abnormal indexes sampled at fixed time is standardized and preprocessed to eliminate the dimension and scale differences of the data. Assume that the original data is X = {x 1 ,x 2 ,......,x n}, the standardized representation is:
[0056]
[0057] Among them, μ X and σ X represent the mean and standard deviation of the data set X respectively.
[0058] 2. The number of sampling points N of flight time series data s Based on this, the actual number of sampling points N of data within 1s is counted in real time r , if N r Less than 90% N s , then the flight time series data is considered to be abnormal data with persistent missing X L Otherwise, the flight time series data is considered to be abnormal data that deviates from the trend instantaneously. D .
[0059] 3. Such as Figure 2 As shown, a KNN model is constructed and the K value is determined, where K represents the number of nearest neighbor data points during repair, which can be set to the number of sampling points N. s It should be noted that if the K value is too small, overfitting is likely to occur. If it is too large, the learning error will also increase.
[0060] 4. For abnormal data X D Each outlier point x in u , and calculate the similarity between its preceding neighbors. Specifically, the Euclidean distance of formula (2) is used to measure the similarity between the outlier point and other sample points.
[0061]
[0062] Among them, d u,iis the Euclidean distance, and ||·|| is the 2-norm.
[0063] 5. Select the K samples with the highest similarity as the outlier point x u The nearest neighbor Y u ={y u1 ,y u2 ,…,y uk}.
[0064]
[0065] in, Indicates taking the first K minimum values, d u is the outlier point x u The set of Euclidean distances to previous-order neighbors.
[0066] 6. Use the known values of the K nearest neighbors to estimate the missing value by weighted average, where the Euclidean distance d between two points u,i The smaller it is, the larger the weight coefficient is, and the higher the similarity is. Repeat the above process for all sample sequences containing outliers to estimate outliers for them. The K value, weight calculation, etc. can be adjusted according to actual needs to obtain the best repair effect.
[0067]
[0068] Among them, x u ′ is the outlier point x u The repair value, w u,k is the weighted value of the kth neighbor, which can be calculated by formula (5).
[0069]
[0070] 7. Such as Figure 3 As shown in Figure 1, a generator of a TimeGAN model is constructed. Inside the generator, a multi-layer recurrent neural network is constructed using RNN units to map random noise to the potential representation of generated data.
[0071] 8. In order to adapt TimeGAN to time series data, the time information T is taken as the generator input together with the random noise vector Z. After a fully connected layer, its dimension is mapped to hidden_dim. Using the Sigmoid activation function, the generator returns the generated latent space representation E.
[0072] 9. The training loss of the generator is designed to minimize the difference between the generated data and the real data, hoping that the generated data can approach the distribution of the real data. Specifically, adversarial loss is used as the loss function of the generator, as shown in formula (6).
[0073]
[0074] Among them, L gs represents the adversarial loss of the generator, and z represents the distribution from the prior distribution p z The random noise extracted from , G(z) represents the sequence generated by the generator, and D is the discriminator.
[0075] 10. Build a discriminator for the TimeGAN model, use RNN units to build a multi-layer recurrent neural network, and process the potential representation of the input. The output dimension is mapped to 1 through the fully connected layer to perform binary classification to determine whether the input is real data or generated data. Finally, the discriminator returns the classification result.
[0076] 11. Use the generator output E and the preprocessed normal flight data as the discriminator input. The training loss of the discriminator includes the loss of real data and the loss of generated data, as shown in formula (7).
[0077]
[0078] Among them, L ds represents the adversarial loss of the discriminator, and z represents the distribution from the prior distribution p z The random noise extracted from , G(z) represents the sequence generated by the generator, D is the discriminator, and x is the normal flight timing data.
[0079] 12. The training process alternately trains the generator and the discriminator, uses back propagation and the optimizer to continuously update the parameters, and minimizes the two loss functions so that the generator and the discriminator are constantly learning and adapting.
[0080] 13. The flight time series data X with abnormal index L As the discriminator input of the trained TimeGAN model, the KL divergence of the original data X and the generated data E is calculated. If the KL divergence is less than 0.1, it is considered that the distribution similarity of the generated data is basically the same as that of the original data, and the missing values in the original data are filled with the values of the abnormal index positions of the generated data.
[0081]
[0082] Among them, D KL (P||Q) represents the KL divergence, P(x) represents the probability distribution of the original data X, and Q(e) represents the probability distribution of the generated data E.
[0083] Example 1
[0084] 1. Based on the flight data and its instantaneous deviation trend abnormal characteristics, a normal time series data set X with 65,000 points was generated. The number of sampling points was set to 10, and the abnormal values of the instantaneous deviation trend were added at 40,800, 44,600 and 55,000 to obtain the abnormal data of the instantaneous deviation trend, as shown in the attached figure. Figure 4 As shown in the figure on the left.
[0085] 2. Perform data preprocessing on the abnormal data that deviates from the trend instantaneously to obtain abnormal data X D .
[0086] 3. Since the data is generated offline, the KNN model is directly constructed, and the K value is set to 4 times the number of sampling points, that is, 40.
[0087] 4. Taking outlier point 1 as an example, calculate the Euclidean distance d between it and its previous neighbor 1 ={0.2,0.2,0.2,0.375,0.475,...,1,1,...,0.875,...,0.23,0.22,0.2}.
[0088] 5. Select the 40 data points with the smallest Euclidean distance as the nearest neighbors Y u .
[0089] 6. For simplicity, the weight is set to 1, then the first instantaneous deviation from the trend is estimated to be 0.21, which can be obtained by multiplying the standard deviation μ X , and add the mean σ X After the inverse transformation, the estimated value of the original abnormal data is obtained. The final repair result is as shown in the attached figure. Figure 4 As shown in the figure on the right.
[0090] Example 2
[0091] 1. Based on the flight data and its persistent missing anomaly features, a normal time series data set X with 10,000 points was generated. The number of sampling points was set to 10, and persistent missing anomaly values were added at the 6100-7100th position to obtain the persistent missing abnormal data, as shown in the attached figure. Figure 5 As shown in the figure on the left.
[0092] 2. Preprocess the abnormal data with persistent missing to obtain abnormal data X L .
[0093] 3. The time information T is used together with the random noise vector Z as the input of the trained generator to obtain the latent space representation E returned by the generator.
[0094] The latent space representation E returned by the generator and the abnormal data X LAs the discriminator input of the trained TimeGAN model, the KL divergence between the original data X and the generated data E is calculated to be 0.063, which is less than 0.1, indicating that the generated values of the TimeGAN model are highly similar to the distribution of the original data. The corresponding values at 6100-7100 are taken as the abnormal data X. L The filling value at the corresponding position is similar to Example 1. After the data is inversely transformed, the repair result is finally obtained as shown in the attached figure. Figure 5 As shown in the figure on the right.
Claims
1. A real-time repair method for aviation flight data anomaly, characterized in that: The steps include: Step 1: Taking the flight time series data sampled at a fixed time as input, first perform standard normalization on the data to eliminate the dimension and scale differences of the data and perform data preprocessing; Step 2: Identify the abnormal type of current flight data according to the number of sampling points of flight time series data; Real-time statistics of the actual sampling points of data within 1s. If the actual sampling points are less than 90% of the periodic sampling points, the flight time series data is considered to be abnormal data with persistent missing. If the actual sampling points are greater than or equal to 90% of the periodic sampling points and less than the periodic sampling points, the flight time series data is considered to be abnormal data with instantaneous deviation from the trend. Step 3: For abnormal flight data that deviates from the trend instantaneously, the K nearest neighbor repair method is used to complete the abnormal data repair; Step 4: For abnormal flight data that is continuously missing, the TimeGAN model is used to complete abnormal data repair; Step 5: Verify the integrity of the repaired flight data obtained in steps 3 and 4.
2. The method for real-time repair of aviation flight data anomalies according to claim 1, characterized in that: In step 1, the specific process of data preprocessing includes: assuming that the original data is X = {x1, x2, ..., x n }, the following formula (1) is used for data preprocessing: Among them, μ X and σ X represent the mean and standard deviation of the data set X respectively.
3. The method for real-time repair of aviation flight data anomalies according to claim 1, characterized in that: In the step 3, the following steps are specifically included: Construct a KNN model and determine the K value; for each abnormal point in the abnormal flight data that deviates from the trend instantaneously, calculate the similarity with its preceding neighbors; select the K samples with the highest similarity as the nearest neighbors of the abnormal point; use the known values of the K nearest neighbors to estimate the abnormal flight data that deviates from the trend instantaneously through weighted averaging.
4. The method for real-time repair of aviation flight data anomalies according to claim 3, characterized in that: The K value is set as a multiple of the periodic sampling points.
5. The method for real-time repair of aviation flight data anomalies according to claim 3, characterized in that: Specifically, the Euclidean distance of formula (2) is used to measure the similarity between the outlier point and other sample points; Among them, d u,i is the Euclidean distance, and ||·|| is the 2-norm.
6. The method for real-time repair of aviation flight data anomalies according to claim 5, characterized in that: The K samples with the highest similarity are taken as the outlier point x u The nearest neighbor Y u ={y u1 ,y u2 ,…,y uk }. in, Indicates taking the first K minimum values, d u is the outlier point x u The set of Euclidean distances to previous order neighbors; Among them, x u ′ is the outlier point x u The repair value, w u,k is the weighted value of the kth neighbor, which can be calculated by formula (5); 7. The method for real-time repair of aviation flight data anomalies according to claim 1, characterized in that: In the step 4, the following steps are specifically included: Build a generator of the TimeGAN model. Inside the generator, use RNN units to build a multi-layer recurrent neural network to map random noise to the potential representation of generated data. In order to adapt TimeGAN to time series data, the time information T is taken as the generator input together with the random noise vector Z. After a fully connected layer, its dimension is mapped to hidden_dim. Using the Sigmoid activation function, the generator returns the generated latent space representation E. The training loss of the generator is designed to minimize the difference between the generated data and the real data, hoping that the generated data can be close to the distribution of the real data. Specifically, adversarial loss is used as the loss function of the generator, as shown in formula (6); Among them, L gs represents the adversarial loss of the generator, and z represents the distribution from the prior distribution p z The random noise extracted from the generator, G(z) represents the sequence generated by the generator, and D is the discriminator; Build a discriminator for the TimeGAN model, use RNN units to build a multi-layer recurrent neural network to process the potential representation of the input; map the output dimension to 1 through the fully connected layer to perform binary classification to determine whether the input is real data or generated data, and finally the discriminator returns the classification result; The generator output E and the preprocessed normal flight data are used as the discriminator input. The training loss of the discriminator includes the loss of real data and the loss of generated data, as shown in formula (7); Among them, L ds represents the adversarial loss of the discriminator, and z represents the distribution from the prior distribution p z The random noise extracted from , G(z) represents the sequence generated by the generator, D is the discriminator, and x is the normal flight timing data; The training process alternately trains the generator and the discriminator, uses back propagation and the optimizer to continuously update the parameters, and minimizes the two loss functions so that the generator and the discriminator are constantly learning and adapting.
8. The method for real-time repair of aviation flight data anomalies according to claim 7, characterized in that: The flight time series data X with abnormal index L As the discriminator input of the trained TimeGAN model, the KL divergence of the original data X and the generated data E is calculated. If the KL divergence is less than 0.1, it is considered that the distribution similarity of the generated data is basically the same as that of the original data, and the missing values in the original data are filled with the values of the abnormal index positions of the generated data. Among them, D KL (P||Q) represents the KL divergence, P(x) represents the probability distribution of the original data X, and Q(e) represents the probability distribution of the generated data E.