ADS-B data filtering method and system based on ADF test

Abstract

The present invention relates to the field of civil aviation flight data processing technology, and in particular to an ADS-B data filtering method and system based on ADF inspection, the method comprising collecting a time series data set of flight equipment, constructing an ADF inspection model, training the ADF inspection model using a training set, and determining the stability of the time series data according to the ADF inspection model; performing a stability inspection on the time series data in the test set using the trained ADF inspection model, identifying and filtering out abnormal frames according to the inspection results, constructing an abnormal frame filtering mechanism, and correcting or eliminating the abnormal frames; evaluating the filtered time series data set using standard data or calibration data, and calculating the error rate and accuracy. The present invention is applicable to various time series data, can accurately judge the stability of the data, and then accurately filter abnormal frames, and provides a quantifiable judgment standard to make the filtering result more interpretable.

Description

Technical Field

[0001] The present invention relates to the technical field of civil aviation flight data processing, and in particular to an ADS-B data filtering method and system based on ADF inspection. Background Art

[0002] In the field of civil aviation technology, the processing of ADS-B data (Automatic Dependent Surveillance-Broadcast System data) and the filtering of abnormal frames are crucial links. At present, statistical methods or threshold-based rule filtering methods are mainly used for processing and filtering. Although abnormal data can be excluded to a certain extent, there are obvious problems and defects, which are mainly manifested in several aspects: First, sensitivity to instability. The above methods are more sensitive to the instability of data and are prone to misjudge normal but highly variable data as abnormal, which seriously affects the accuracy and reliability of the data. Second, the threshold-based rule filtering method usually uses fixed rules and thresholds, which is difficult to adapt to the changes in different data characteristics and environmental conditions, resulting in inflexible filtering effects and poor adaptability. Third, the above statistical methods and rule filtering methods are insufficient in processing complex time series data, especially in the presence of multivariate relationships, and may not be able to effectively identify and filter out real abnormal data, which has limitations. Fourth, the above methods do not fully consider the time series characteristics of ADS-B data, lack a comprehensive judgment of data stability, and cannot effectively filter out unstable data. Summary of the invention

[0003] The present invention aims at the technical problem that the accuracy and reliability of ADS-B data processing and filtering in the prior art are both low, and provides an ADS-B data filtering method and system based on ADF inspection to improve the quality and security of ADS-B data.

[0004] In view of the above technical problems, an embodiment of the present invention provides an ADS-B data filtering method based on ADF inspection, comprising:

[0005] Collecting original ADS-B data of flight equipment, wherein the original ADS-B data includes latitude and longitude information, altitude information and timestamp information of the flight equipment, preprocessing the original ADS-B data to obtain a time series data set, and dividing the time series data set into a training set, a validation set and a test set;

[0006] Construct an ADF test model, use the training set to train the ADF test model, determine the parameters and structure of the ADF test model, and determine the stability of time series data based on the ADF test model;

[0007] Use the trained ADF test model to perform stability test on the time series data in the test set, identify and filter out abnormal frames based on the test results, and correct or eliminate the abnormal frames;

[0008] Use standard data or calibration data to evaluate the filtered time series data set and calculate the error rate and accuracy. Adjust the parameters of the ADF test model through cross-validation based on the training set to optimize the abnormal frame filtering mechanism.

[0009] The present invention also provides an ADS-B data filtering system based on ADF inspection, comprising:

[0010] A data processing module is used to collect the original ADS-B data of the flight equipment, wherein the original ADS-B data includes the latitude and longitude information, altitude information and timestamp information of the flight equipment, pre-process the original ADS-B data to obtain a time series data set, and divide the time series data set into a training set, a validation set and a test set;

[0011] The model building module is used to build an ADF test model, train the ADF test model using a training set, determine the parameters and structure of the ADF test model, and determine the stability of time series data based on the ADF test model;

[0012] The test and filtering module is used to use the trained ADF test model to perform stability tests on the time series data in the test set, identify and filter out abnormal frames based on the test results, and correct or eliminate the abnormal frames;

[0013] The evaluation and optimization module is used to evaluate the filtered time series data set using standard data or calibration data, calculate the error rate and accuracy; adjust the parameters of the ADF test model through cross-validation based on the training set, and optimize the abnormal frame filtering mechanism.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects.

[0015] (1) The present invention significantly improves the accuracy of data filtering by introducing the ADF test. The ADF test can accurately and quantitatively evaluate the stationarity of time series data, thereby effectively identifying and filtering out abnormal frames in ADS-B data. By calculating the first-order difference, building a statistical model, applying OLS to estimate parameters, and calculating the ADF statistic, this process not only ensures an accurate description of the characteristics of the time series data, but also provides a quantitative basis for judging whether the time series is stable by comparing the ADF test statistic with the critical value. In addition, by adjusting multi-dimensional parameters such as the difference threshold, time interval threshold, latitude threshold, longitude threshold, and altitude threshold, the present invention can more accurately capture abnormal changes in the data, further enhance the accuracy of abnormal frame identification, and provide a solid and reliable foundation for data analysis and application.

[0016] (2) The present invention shows a wide range of applicability. It is not only applicable to conventional data types such as longitude, latitude, and altitude in ADS-B data, but can also flexibly cope with various complex time series data. By constructing a filtering mechanism based on the ADF test, a comprehensive capture of time series characteristics is achieved. Whether it is facing a sequence with an obvious trend or seasonally changing data, the present invention can provide an effective stability assessment. At the same time, the flexible adjustment of multiple threshold parameters enables the method to be customized according to different data characteristics and anomaly types, further enhancing its applicability. Whether it is real-time data processing or offline analysis, the present invention can provide a stable and reliable data filtering solution.

[0017] (3) The present invention not only improves the accuracy of anomaly identification, but also reduces the occurrence of misjudgment through the combination of multi-dimensional threshold adjustment and abnormal frame filtering mechanism, ensuring the purity of the data set and the reliability of the analysis results. The present invention can be applied to different business scenarios, data characteristics and real-time requirements. For real-time monitoring systems that require rapid response, the time interval threshold can be adjusted to reduce data processing time; and in scenarios with high requirements for data integrity, the difference threshold can be adjusted to improve the detection capability of abnormal mutations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of a flow chart of an ADS-B data filtering method based on ADF inspection in one embodiment of the present invention;

[0019] Figure 2 The figure is a schematic diagram of the structure of an ADS-B data filtering system based on ADF inspection in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In one embodiment, if Figure 1 As shown, a method for filtering ADS-B data based on ADF inspection is provided, comprising the following steps S10-S40:

[0022] S10. Collect original ADS-B data of flight equipment, wherein the original ADS-B data includes latitude and longitude information, altitude information and timestamp information of the flight equipment, pre-process the original ADS-B data to obtain a time series data set, and divide the time series data set into a training set, a validation set and a test set; understandably, collect and process the original ADS-B data to provide a structured and data basis for subsequent model training and testing, and ensure the effectiveness of model training and evaluation.

[0023] S20, construct an ADF test model, use the training set to train the ADF test model, determine the parameters and structure of the ADF test model, and determine the stability of the time series data based on the ADF test model; understandably, by constructing the ADF test model, the non-stationary components in the time series can be identified, providing a basis for subsequent anomaly detection and data processing.

[0024] S30. Use the trained ADF test model to perform stability test on the time series data in the test set, identify and filter out abnormal frames based on the test results, build an abnormal frame filtering mechanism, and correct or eliminate abnormal frames; understandably, this step effectively filters out abnormal data that may affect data analysis and model prediction performance, thereby improving data quality.

[0025] S40, using standard data or calibration data to evaluate the filtered time series data set, calculate the error rate and accuracy; adjust the parameters of the ADF test model based on the training set through cross-validation method, and optimize the abnormal frame filtering mechanism. Understandably, by understanding the performance level of the current model, the cross-validation method is used to adjust and optimize the model parameters, further improve the accuracy and robustness of the model, and ensure the effectiveness of the model in practical applications.

[0026] In the present invention, based on the preliminary processing of the original ADS-B data, the construction of the ADF inspection model to identify and filter out the abnormal data in the processed ADS-B data to adjust and optimize the ADF inspection model and other processes, based on the solid statistical principles of the ADF inspection, a reliable method is provided to accurately judge the stability of the ADS-B data, and then the abnormal frames can be accurately filtered out, which greatly improves the accuracy and reliability of the processing and filtering of the ADS-B data.

[0027] In one embodiment, the step S10 includes the following steps S101-S103:

[0028] S101. Collect raw ADS-B data through aviation monitoring systems or equipment, and perform preliminary cleaning to remove duplicate and invalid data. Duplicate data not only takes up storage space, but also affects the accuracy of subsequent analysis results. Invalid data (such as data with incorrect format or missing key fields) will also interfere with the analysis process.

[0029] S102. The cleaned original ADS-B data is normalized to meet the requirements of the ADF test, so that the values ​​of the original ADS-B data are all at the same level, so that data with different characteristics have the same weight in model training, and the normalized original ADS-B data constitutes a time series data set; normalization processing helps to eliminate model bias caused by excessive differences in numerical ranges and improve the stability and generalization ability of the model.

[0030] S103, divide the time series data set into a training set, a validation set, and a test set. The training set is used to learn the model, the validation set is used to adjust the model parameters to avoid overfitting, and the test set is used to finally evaluate the generalization ability of the model. This division method helps to find and correct problems in a timely manner during the model training process, thereby improving the efficiency and quality of model training.

[0031] In this embodiment, since each frame of ADS-B data contains a timestamp, the pre-processed data ensures timing and continuity, which is helpful for the subsequent real-time analysis and monitoring. High-quality ADS-B and its processing results (time series data set) can provide important basis for airlines' operational decisions, air traffic management of air traffic control departments, and flight safety monitoring.

[0032] In one embodiment, the step S20 includes the following steps S201-S203:

[0033] S201, construct an ADF test model, the mathematical expression of the test formula of the ADF test model is:

[0034] Δy t =α+βt+γy t-1 +δ1 Δy t-1 +…+δ p Δy t-p +∈ t (1)

[0035] Among them, Δy i represents the first-order difference of the time series at time t, y t represents the value of the time series at time, α represents the constant term, β represents the coefficient of the time trend term, t represents the time trend term, γ represents the coefficient of the lag term, and checks whether it is close to 1. If γ is close to 1, there is a unit root, δ i represents the coefficient of the lagged difference term, p represents the lag order, ∈ t represents the error term.

[0036] S202, train the ADF test model using the training set, determine the parameters and structure of the ADF test model, and test whether the time series data has a unit root, that is, determine whether the time series data tends to be stable. Understandably, the ADF test model is a test method based on statistics, which is used to determine whether the time series data has a unit root. The existence of a unit root means that the time series is non-stationary, that is, its statistical characteristics such as mean, variance and autocorrelation coefficient will change over time, which may cause large errors in the prediction results. If the ADF test determines that the time series data does not have a unit root, it can be considered that the series is stationary, that is, its statistical characteristics do not change significantly over time.

[0037] In one embodiment, the step S202 further includes the following steps S2021-S2025:

[0038] S2021. In order to eliminate the trend term and constant term in the time series and make it closer to a stationary series, since a non-stationary time series may lead to false results when modeling, the first-order difference value of the time series in the training set is calculated. The mathematical expression of the first-order difference of the time series is:

[0039] Δy t =y t -y t-1 (2);

[0040] S2022. According to the time series data and its first-order difference value, the ADF test model is used to perform regression analysis, and a regression equation is constructed to fit the ADF test model. The mathematical expression of the regression equation is:

[0041]

[0042] Among them, Δy trepresents the first-order difference of the time series at time t, α represents the constant term, βt represents the time trend term, and γy t-1 represents the lag term, represents the difference term, ∈ t represents the error term.

[0043] Understandably, the purpose of constructing a regression equation is to statistically test whether there is a unit root in the time series. The regression equation of the ADF test contains a series of variables related to the lag term of the time series, as well as a constant term and a time trend term. Through the regression analysis of these variables, we can evaluate the relationship between the first-order difference sequence of the time series and its lag term. In this way, the main purpose of constructing a regression equation to fit the ADF test model is to test whether there is a unit root in the time series, so as to determine the stationarity of the time series and provide a basis for subsequent modeling and analysis.

[0044] S2023, using the least multiplication OLS method to estimate the parameters in the regression equation, the parameters including the constant term α, the coefficient of the time trend term β, the coefficient of the lag term γ, and the coefficient of the lag difference term δ i ; Through OLS estimation, we can obtain the values ​​of these parameters and provide a basis for subsequent statistical tests.

[0045] S2024. Calculate the statistic τ of the ADF test model. The mathematical expression of the statistic τ is:

[0046]

[0047] in, The estimated value of the lagged one-period coefficient γ is obtained by the minimum multiplication OLS estimation. yes The standard error of ; Understandably, the statistic τ is calculated based on the estimated value of the lagged one-period coefficient obtained by OLS estimation and its standard error. The calculation of the statistic τ is crucial to determine whether there is a unit root in the time series.

[0048] S2025, comparing the calculated statistic τ with a critical value, where the critical value is obtained from a distribution table of the ADF statistic according to a significance level;

[0049] If the statistic τ is less than the critical value, the null hypothesis is rejected, that is, the time series does not have a unit root, and the time series is determined to be stable;

[0050] If the statistic τ is greater than or equal to the critical value, the null hypothesis is accepted, that is, the time series has a unit root, and the time series is determined to be unstable. Understandably, the critical value is used to determine whether the test statistic is small enough to reject the null hypothesis at a given significance level. The critical value is usually not calculated, but is found from the statistical table or the distribution provided by the software package. These values ​​are pre-calculated based on the specific percentile of the ADF statistic distribution when the null hypothesis is true. When performing an ADF test, make sure that the selected critical value matches the confidence level, type, and model setting of the test.

[0051] S203, using the validation set to validate the ADF test model, and further optimizing the ADF test model according to the validation result, adjusting parameters or improving the structure.

[0052] In this embodiment, by calculating the first-order difference value, the time series data is regressed using the ADF test model to construct a statistical model that can capture the characteristics of the time series, and then the model parameters are estimated by the minimum multiplication OLS to provide a quantitative description of the time series characteristics for the model. Then, the ADF test statistic is calculated to provide a quantitative indicator to judge the stationarity of the time series. Based on the comparison result of the statistic and the critical value, a judgment is made on whether the time series is stationary, providing a basis for subsequent data processing or analysis. The above-mentioned ADF test provides a quantitative method based on statistical principles to evaluate the stationarity of the time series, thereby ensuring the accuracy of the analysis results.

[0053] In one embodiment, the step S30 includes the following steps S301-S304:

[0054] S301, using the trained ADF test model to perform stability test on the time series data of each frame in the test set, to test whether the time series data of each frame has a unit root, and to identify normal frames and abnormal frames;

[0055] In one embodiment, the step S301 includes the following steps S3011-S3013:

[0056] S3011, extract the latitude and longitude, altitude and timestamp information of each frame of time series data to form a time series {t i ,lat i ,lon i , dlt i}, and use the ADF test model to calculate the first-order difference value of each element in the frame time series:

[0057]

[0058] Among them, t iIndicates timestamp, lat i Indicates latitude, lon i Indicates longitude, alt i Indicates height;

[0059] S3012. Using the first-order difference value of each element in the time series as input, the ADF test model is applied to calculate the statistics of each element in the time series:

[0060]

[0061] Among them, τ lat Represents the statistic of dimension, τ lon Denotes the statistic of longitude, τ alt A statistic representing height;

[0062] S3013, compare the statistics of each element in the time series with the critical value, and lat , τ lon and τ alt When both are less than the critical value, it is determined that the time series data of the frame does not have a unit root, and the time series data of the frame is determined to be stable, which is a normal frame; otherwise, the time series data of the frame is determined to be unstable, which is an abnormal frame.

[0063] It can be understood that the steps S3011-S3013 show how to apply the trained ADF test model to actual time series data to perform automated stability testing.

[0064] S302, filter out abnormal frames, analyze the causes of abnormality in the two dimensions of time and geographical location, build an abnormal frame filtering mechanism, determine whether the abnormal frames are correctable abnormal frames, and divide the abnormal frames into correctable abnormal frames and serious abnormal frames. Understandably, correctable abnormal frames usually mean that there are temporary or accidental errors in the data that can be identified and corrected, while serious abnormal frames may indicate that the data is missing or damaged. Distinguishing between these two situations helps maintain the integrity of the data set.

[0065] In one embodiment, the step S302 includes the following steps S3021-S3023:

[0066] S3021. If the altitude at the first moment in the time series shows an abnormal mutation, and the altitude at the second moment adjacent to the first moment and the altitude at the third moment change smoothly, then the altitude at the first moment is determined to be a correctable abnormal frame with abnormal time continuity; for example, the altitude data of the flight equipment at a certain moment suddenly increases or decreases abnormally, but the altitude data at the previous and next two moments change smoothly, then the altitude data at this moment can be defined as a correctable abnormal frame.

[0067] S3022. If the trajectory at the first moment in the time series obviously deviates from the predetermined trajectory path, and the trajectory at the second moment adjacent to the first moment and the trajectory at the third moment are continuous and smooth, then the trajectory at the first moment is determined to be a correctable abnormal frame with an abnormal geographical location; for example, the position (latitude and longitude) data of a certain frame of the flight equipment during the flight deviates from the normal route, but the position information of the previous and next frames is relatively consistent, then the position of the frame can be defined as a correctable abnormal frame.

[0068] S3023. If there are data points in the time series that significantly deviate from the expected normal value, lack sufficient previous and subsequent data to support routine corrections, or show unrelated anomalies in multiple dimensions at the same time, they are determined to be severely abnormal frames. For example, the altitude data suddenly shows extremely high or extremely low values, far beyond the reasonable flight altitude range; or there is a long period of data loss in the data stream, resulting in a lack of sufficient data basis for the correction of some frames; or there are extreme values ​​of altitude data and jumps in latitude and longitude data at the same time, and these changes are not obviously related.

[0069] In one embodiment, the step S3023 includes the following steps S30231-S30233:

[0070] S30231. By calculating the difference between the current frame data and the previous and next frame data, a difference threshold is set. If the difference exceeds the preset threshold, it is determined to be seriously deviated from the expected normal value. The determination formula is:

[0071] Δx i =|x i -x i-1 |,Δx i >θ 1 (7)

[0072] Where Δx i Indicates the difference, x i Indicates the next frame of data, x i-1 Represents the previous frame data, θ 1 represents a difference threshold; understandably, by calculating the difference between the current frame and the frames before and after it and comparing it with the preset difference threshold, frames that seriously deviate from the expected normal value are identified.

[0073] S30232. By checking the continuity of the data in the time series, a time interval threshold is set. If there is a long period of data missing, resulting in data discontinuity, it is determined to be a serious abnormal frame that cannot be corrected. The determination formula is:

[0074] Gap = t i -t i-1 , Gap>θ 2 (8)

[0075] Among them, Gap represents the timestamp difference, ti-1 Indicates the timestamp of the previous frame, t i represents the timestamp of the next frame, θ 2 Represents the time interval threshold; understandably, the continuity of the data in the time series is checked, and the discontinuous frames caused by long-term data missing are identified through the timestamp difference and the time interval threshold.

[0076] S30233. By comprehensively analyzing the data changes in multiple dimensions, if multiple dimensions of a frame of data are significantly abnormal at the same time, and there is no correlation between the abnormalities, it is determined to be a serious abnormal frame. The determination formula is:

[0077]

[0078] Among them, Δlat i Indicates the latitude difference, Δlon i Indicates the longitude difference, Δalt i represents the height difference value, represents the latitude threshold, represents the longitude threshold, and represents the height threshold. Understandably, the data changes in multiple dimensions are comprehensively analyzed to identify frames that are significantly abnormal in multiple dimensions at the same time and lack correlation between the abnormalities.

[0079] In the above embodiment, by calculating the difference amount and comparing the threshold, the extreme abnormal points in the data are identified, and by checking the timestamp, the discontinuity caused by the missing data is identified to provide protection for the integrity of the data. By comprehensively considering the data changes in multiple dimensions, the serious abnormal frames in complex situations are identified, the serious abnormal frames are removed, and the data set is purified to avoid the negative impact of these data on the analysis results. The data filtering method is applicable to various time series data, including latitude and longitude, altitude, etc. in ADS-B data, and has wide applicability.

[0080] S303: For the correctable abnormal frames, a linear difference algorithm or a spatial difference algorithm is used to correct them, and for the serious abnormal frames, they are removed from the time series data set.

[0081] In one embodiment, the step S303 includes the following steps S3031-S3032:

[0082] S3031. For the correctable abnormal frames with abnormal time continuity, a linear difference algorithm is used to correct them.

[0083] If the altitude data is abnormal, the flight equipment will 5 Height data alt 5 is 5000 m, and at t 4 and time t 6 The altitude data are 4900 meters and 4950 meters respectively, alt 5Obviously abnormal, but the data before and after change smoothly, so it can be corrected by linear interpolation:

[0084]

[0085] S3032: For the abnormal frames with abnormal geographical location, the spatial difference algorithm is used to correct them. 8 Longitude data lon 8 is 120.5°E, and at t 7 and t 9 The longitude data at this time are 120.3°E and 120.4°E, lon 8 It obviously deviates from the normal trajectory. However, the data before and after change smoothly, so it can be corrected by spatial interpolation:

[0086]

[0087] If the multi-dimensional data is abnormal, the latitude, longitude and altitude data of the flight equipment {lat 10 ,lon 10 , alt 10} are abnormal, with longitude of 121.0°E and altitude of 5500 meters, while at t 9 and t 11 The data at the time are {120.4°E, 5000m} and {120.5°E, 5050m}. The data before and after change slowly, and can be corrected by multidimensional interpolation method:

[0088]

[0089] In this embodiment, the linear difference algorithm is used to correct the mutation points in time and restore the temporal continuity of the data. The spatial difference algorithm is used to correct the abnormal points in the geographical location and maintain the spatial consistency of the trajectory. The corrected frame is more consistent with the overall trend and pattern of the data, which improves the overall data quality.

[0090] S304: Mark and record the correctable abnormal frames and serious abnormal frames respectively.

[0091] In one embodiment, the step S40 includes the following steps S401-S404:

[0092] S401. In order to quantify the effect of data filtering, the filtered time series data set is evaluated using standard data or calibration data. The mathematical formulas of the error rate and accuracy are:

[0093]

[0094] S402, divide the training set into k subsets {D 1 ,D 2 , ..., D k};

[0095] For each subset D i , using the remaining k-1 subsets {D 1 , ..., D i-1 , D i+1 , ..., D k} to train the model and set the subset D i Used as a validation set, repeat the training k times to ensure that each subset is used as a validation set once;

[0096] Calculate the error of each verification and get k error values ​​{e 1 , e 2 , ..., e k};

[0097] Calculate the average error:

[0098]

[0099] Adjust the parameters in the ADF test model one by one, adjust one parameter at a time, repeat the above cross-validation process, and find the average error e avg The smallest parameter combination. Cross-validation ensures that the model performs consistently on different data subsets and improves the generalization ability of the model.

[0100] S403, the optimization abnormal frame filtering mechanism comprises the following steps:

[0101] According to the evaluation results, adjust and optimize the difference threshold, time interval threshold, latitude threshold, longitude threshold and altitude threshold used in the abnormal frame identification process;

[0102] Combined with multi-dimensional data changes, a joint filtering mechanism is used to improve the accuracy of abnormal frame recognition.

[0103] The mathematical expression of the joint filtering mechanism is:

[0104] If(Δlat i >θ lat )&(Δlon i >θ lon )&(Δalt i >θ alt );

[0105] Understandably, by adjusting the threshold and adopting the abnormal frame filtering mechanism, the recognition accuracy of abnormal frames is improved to ensure the accuracy and reliability of the data filtering effect. The difference threshold, time interval threshold, latitude threshold, longitude threshold and altitude threshold can be adjusted according to needs. Specifically, they can be set according to business scenarios, data characteristics, abnormal types, real-time requirements and resource constraints. It can more accurately identify abnormal changes between data points and improve the detection ability of abnormal mutations. The abnormal frame filtering mechanism improves the overall accuracy of abnormal frame recognition by comprehensively considering data changes in multiple dimensions. Through multi-dimensional joint analysis, it reduces the misjudgment that may be caused by a single dimension threshold.

[0106] S404, using machine learning algorithms to train the ADF test model to improve the recognition accuracy and robustness of the model. Understandably, machine learning algorithms such as random forests and support vector machines can be used to train the ADF test model, which can automate the anomaly detection process, reduce manual intervention, and improve the efficiency and speed of data processing.

[0107] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0108] In one embodiment, an ADS-B data filtering system based on ADF inspection is provided, and the ADS-B data filtering system based on ADF inspection corresponds one-to-one to the ADS-B data filtering method based on ADF inspection in the above embodiment. Figure 2 As shown, the ADS-B data filtering system based on ADF inspection includes:

[0109] The data processing module 100 is used to collect the original ADS-B data of the flight equipment, wherein the original ADS-B data includes the latitude and longitude information, altitude information and timestamp information of the flight equipment, pre-process the original ADS-B data to obtain a time series data set, and divide the time series data set into a training set, a validation set and a test set;

[0110] The model building module 200 is used to build an ADF test model, train the ADF test model using a training set, determine the parameters and structure of the ADF test model, and determine the stability of the time series data according to the ADF test model;

[0111] The test and filtering module 300 is used to perform stability test on the time series data in the test set using the trained ADF test model, identify and filter out abnormal frames according to the test results, and correct or remove the abnormal frames;

[0112] The evaluation and optimization module 400 is used to evaluate the filtered time series data set using standard data or calibration data, calculate the error rate and accuracy rate; adjust the parameters of the ADF test model through cross-validation based on the training set, and optimize the abnormal frame filtering mechanism.

[0113] In the ADS-B data filtering system based on ADF inspection in the above-mentioned embodiment of the present invention, based on the preliminary processing of the original ADS-B data, the construction of the ADF inspection model to identify and filter out the abnormal data in the processed ADS-B data to adjust and optimize the ADF inspection model and other processes, based on the solid statistical principles of the ADF inspection, a reliable method is provided to accurately judge the stability of the ADS-B data, and then the abnormal frames can be accurately filtered out, which greatly improves the accuracy and reliability of the processing and filtering of the ADS-B data.

[0114] For the specific definition of the ADS-B data filtering system based on ADF inspection, please refer to the definition of the ADS-B data filtering method based on ADF inspection above, which will not be repeated here. Each module in the above-mentioned ADS-B data filtering system based on ADF inspection can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can be used to execute the operations corresponding to the above modules.

[0115] The above are merely embodiments of the ADS-B data filtering method and system based on ADF inspection of the present invention, and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An ADS-B data filtering method based on ADF test, characterized in that: include: S10, collecting original ADS-B data of the flight equipment, wherein the original ADS-B data includes latitude and longitude information, altitude information and timestamp information of the flight equipment, preprocessing the original ADS-B data to obtain a time series data set, and dividing the time series data set into a training set, a validation set and a test set; S20, constructing an ADF test model, training the ADF test model using a training set, determining the parameters and structure of the ADF test model, and determining the stability of the time series data according to the ADF test model; S30, using the trained ADF test model to perform stability test on the time series data in the test set, identifying and filtering out abnormal frames according to the test results, building an abnormal frame filtering mechanism, and correcting or eliminating abnormal frames; S40. Use standard data or calibration data to evaluate the filtered time series data set and calculate the error rate and accuracy; adjust the parameters of the ADF test model through cross-validation based on the training set to optimize the abnormal frame filtering mechanism.

2. The ADS-B data filtering method based on ADF inspection according to claim 1, characterized in that: The step S10 comprises: S101. Collect raw ADS-B data through aviation monitoring systems or equipment, and perform preliminary cleaning to remove duplicate and invalid data; S102, normalizing the cleaned original ADS-B data so that the values ​​of the original ADS-B data are all at the same level to form a time series data set; S103, dividing the time series data set into a training set, a validation set and a test set.

3. The ADS-B data filtering method based on ADF test according to claim 1, characterized in that: The step S20 comprises: S201, construct an ADF test model, the mathematical expression of the test formula of the ADF test model is: Δy t =α+βt+γy t-1 +δ1Δy t-1 +…+d p Δy t-p +∈ t (1) Among them, Δy t represents the first-order difference of the time series at time t, y t represents the value of the time series at time t, α represents the constant term, β represents the coefficient of the time trend term, t represents the time trend term, γ represents the coefficient of the lag term, and checks whether it is close to 1. If γ is close to 1, there is a unit root, δ i represents the coefficient of the lagged difference term, p represents the lag order, ∈ t represents the error term; S202, using the training set to train the ADF test model, determine the parameters and structure of the ADF test model, and test whether the time series data has a unit root, that is, determine whether the time series data tends to be stable; S203, using the validation set to validate the ADF test model, and further optimizing the ADF test model according to the validation result, adjusting parameters or improving the structure.

4. The ADS-B data filtering method based on ADF inspection according to claim 3 is characterized in that: The step S202 includes the following steps: S2021. Calculate the first-order difference value of the time series in the training set. The mathematical expression of the first-order difference of the time series is: Δy t =y t -y t-1 (2); S2022. According to the time series data and its first-order difference value, the ADF test model is used to perform regression analysis, and a regression equation is constructed to fit the ADF test model. The mathematical expression of the regression equation is: Among them, Δy t represents the first-order difference of the time series at time t, α represents the constant term, βt represents the time trend term, and γy t-1 represents the lag term, represents the difference term, ∈ t represents the error term; S2023, using the least multiplication OLS method to estimate the parameters in the regression equation, the parameters including the constant term α, the coefficient of the time trend term β, the coefficient of the lag term γ, and the coefficient of the lag difference term δ i ; S2024. Calculate the statistic τ of the ADF test model. The mathematical expression of the statistic τ is: in, The estimated value of the lagged one-period coefficient γ is obtained by the minimum multiplication OLS estimation. yes The standard error of S2025, comparing the calculated statistic τ with a critical value, where the critical value is obtained from a distribution table of the ADF statistic according to a significance level; If the statistic τ is less than the critical value, the null hypothesis is rejected, that is, the time series does not have a unit root, and the time series is determined to be stable; If the statistic τ is greater than or equal to the critical value, the null hypothesis is accepted, that is, the time series has a unit root, and it is determined that the time series is unstable.

5. The ADS-B data filtering method based on ADF inspection according to claim 1, characterized in that: The step S30 comprises the following steps: S301, using the trained ADF test model to perform stability test on the time series data of each frame in the test set, to test whether the time series data of each frame has a unit root, and to identify normal frames and abnormal frames; S302, filtering out abnormal frames, analyzing the causes of abnormality for the abnormal frames in terms of time and geographical location, building an abnormal frame filtering mechanism, determining whether the abnormal frames are correctable abnormal frames, and dividing the abnormal frames into correctable abnormal frames and serious abnormal frames; S303: For the abnormal frames that can be corrected, a linear difference algorithm or a spatial difference algorithm is used to correct them, and for the serious abnormal frames, they are removed from the time series data set; S304: Mark and record the correctable abnormal frames and serious abnormal frames respectively.

6. The ADS-B data filtering method based on ADF inspection according to claim 5, characterized in that: The step S301 includes the following steps: S3011, extract the latitude and longitude, altitude and timestamp information of each frame of time series data to form a time series {t i ,lat i ,lon i , alt i }, and use the ADF test model to calculate the first-order difference value of each element in the frame time series: Among them, t i Indicates timestamp, lat i Indicates latitude, lon i Indicates longitude, alt i Indicates height; S3012. Using the first-order difference value of each element in the time series as input, the ADF test model is applied to calculate the statistics of each element in the time series: Among them, τ lat Represents the statistic of dimension, τ lon Denotes the statistic of longitude, τ alt A statistic representing height; S3013, compare the statistics of each element in the time series with the critical value, and lat , τ lon and τ alt When both are less than the critical value, it is determined that the time series data of the frame does not have a unit root, and the time series data of the frame is determined to be stable, which is a normal frame; otherwise, the time series data of the frame is determined to be unstable, which is an abnormal frame.

7. The ADS-B data filtering method based on ADF inspection according to claim 6, characterized in that: The step S302 includes the following steps: S3021. If the height at the first moment in the time series shows an abnormal sudden change, and the height at the second moment adjacent to the first moment and the height at the third moment change smoothly, then determine that the height at the first moment is a correctable abnormal frame with abnormal time continuity; S3022: if the trajectory at the first moment in the time series obviously deviates from the predetermined trajectory path, and the trajectory at the second moment adjacent to the first moment and the trajectory at the third moment are continuous and smooth, then the trajectory at the first moment is determined to be a correctable abnormal frame with abnormal geographical location; S3023: If there are data points in the time series that significantly deviate from the expected normal value, lack sufficient previous and subsequent data to support routine correction, or show unrelated anomalies in multiple dimensions at the same time, they are determined to be severely abnormal frames; S30231. By calculating the difference between the current frame data and the previous and next frame data, a difference threshold is set. If the difference exceeds the preset threshold, it is determined to be seriously deviated from the expected normal value. The determination formula is: Δx i =|x i -x i-1 |,Δx i >θ1 (7) Where Δx i Indicates the difference, x i Indicates the next frame of data, x i-1 represents the previous frame data, θ1 represents the difference threshold; S30232. By checking the continuity of the data in the time series, a time interval threshold is set. If there is a long period of data missing, resulting in data discontinuity, it is determined to be a serious abnormal frame that cannot be corrected. The determination formula is: Gap=t i -t i-1 ,Gap>θ2 (8) Among them, Gap represents the timestamp difference, t i-1 Indicates the timestamp of the previous frame, t i represents the timestamp of the next frame, θ2 represents the time interval threshold; S30233. By comprehensively analyzing the data changes in multiple dimensions, if multiple dimensions of a frame of data are significantly abnormal at the same time, and there is no correlation between the abnormalities, it is determined to be a serious abnormal frame. The determination formula is: Among them, Δlat i Indicates the latitude difference, Δlon i Indicates the longitude difference, Δalt i represents the height difference value, represents the latitude threshold, represents the longitude threshold, and represents the height threshold.

8. The ADS-B data filtering method based on ADF inspection according to claim 5, characterized in that: The step S303 includes: S3031, for the correctable abnormal frames with abnormal time continuity, a linear difference algorithm is used to correct them; S3032: For the correctable abnormal frames with abnormal geographical locations, a spatial difference algorithm is used to correct them.

9. The ADS-B data filtering method based on ADF test according to claim 1, characterized in that: The step S40 comprises: S401, using standard data or calibration data to evaluate the filtered time series data set, the mathematical formulas of the error rate and accuracy are: S402, divide the training set into k subsets {D1, D2, ..., D k }; For each subset D i , using the remaining k-1 subsets {D1, ..., D i-1 , D i+1 , ..., D k } to train the model and set the subset D i Used as a validation set, repeat the training k times to ensure that each subset is used as a validation set once; Calculate the error of each verification and get k error values ​​{e1, e2, ..., e k }; Calculate the average error: Adjust the parameters in the ADF test model one by one, adjust one parameter at a time, repeat the above cross-validation process, and find the average error e avg Minimum parameter combination; S403, the optimization abnormal frame filtering mechanism comprises the following steps: According to the evaluation results, adjust and optimize the difference threshold, time interval threshold, latitude threshold, longitude threshold and altitude threshold used in the abnormal frame identification process; Combined with the multi-dimensional data changes, a joint filtering mechanism is used to improve the accuracy of abnormal frame recognition. The mathematical expression of the joint filtering mechanism is: If(Δlat i >θ lat )&(Δlon i >θ lon )&(Dalt i >θ alt ); S404. Use a machine learning algorithm to train the ADF test model to improve the recognition accuracy and robustness of the model.

10. An ADS-B data filtering system based on ADF inspection, characterized in that: include: A data processing module is used to collect the original ADS-B data of the flight equipment, wherein the original ADS-B data includes the latitude and longitude information, altitude information and timestamp information of the flight equipment, pre-process the original ADS-B data to obtain a time series data set, and divide the time series data set into a training set, a validation set and a test set; The model building module is used to build an ADF test model, train the ADF test model using a training set, determine the parameters and structure of the ADF test model, and determine the stability of time series data based on the ADF test model; The test and filtering module is used to use the trained ADF test model to perform stability tests on the time series data in the test set, identify and filter out abnormal frames based on the test results, and correct or eliminate the abnormal frames; The evaluation and optimization module is used to evaluate the filtered time series data set using standard data or calibration data, calculate the error rate and accuracy; adjust the parameters of the ADF test model through cross-validation based on the training set, and optimize the abnormal frame filtering mechanism.

Images