Method for detecting an anomaly in a flight, data processing system, computer program

Abstract

A computer-implemented method for detecting an anomaly in a flight is disclosed. In one arrangement the method comprises: receiving first flight data corresponding to a first time window, the first flight data derived from a plurality of sensor readings taken during the first time window; predicting, using a machine learning model given the first flight data as input, predicted second flight data corresponding to a second time window, the second time window being subsequent to the first time window; receiving actual second flight data corresponding to the second time window, the second flight data derived from a plurality of sensor readings taken during the second time window; calculating a prediction error by comparing the predicted second flight data to the actual second flight data; determining whether the prediction error is greater than a predetermined threshold; and outputting an alert in response to the prediction error being determined to be greater than the threshold.

Description

[0001] METHOD FOR DETECTING AN ANOMALY IN A FLIGHT, DATA PROCESSING

[0002] SYSTEM, COMPUTER PROGRAM

[0003] The disclosure relates to detecting anomalies in aircraft flights.

[0004] Modem aircraft include a number of monitoring systems to ensure the safety of an aircraft and provide updates on the status of the aircraft. Output from such monitoring systems may be analysed after flights to identify components of the aircraft that may need to be checked or repaired. Such monitoring systems are not normally configured to identify anomalies in real time or during a flight.

[0005] It is an object of the present disclosure to enhance flight monitoring.

[0006] According to an aspect of the invention, there is provided a computer-implemented method for detecting an anomaly in a flight, the method comprising: receiving first flight data corresponding to a first time window, the first flight data derived from a plurality of sensor readings taken during the first time window; predicting, using a machine learning model given the first flight data as input, predicted second flight data corresponding to a second time window, the second time window being subsequent to the first time window; receiving actual second flight data corresponding to the second time window, the second flight data derived from a plurality of sensor readings taken during the second time window; calculating a prediction error by comparing the predicted second flight data to the actual second flight data; determining whether the prediction error is greater than a predetermined threshold; and outputting an alert in response to the prediction error being determined to be greater than the threshold.

[0007] Thus, a method is provided in which flight data is compared to a computer generated prediction of the flight data and a warning is generated if a prediction error suggests that the flight data contains an anomaly.

[0008] Unlike typical known monitoring systems on flights, the method allows the performance of the aircraft to be assessed in real time and / or while the flight itself is in progress. The method can therefore provide a warning to the pilot during the flight when an anomaly in the flight data is detected. Providing a warning during the flight rather than afterwards allows corrective measures to be taken during the flight, such as an emergency landing or diverting the flight to a nearby airport in the case of serious anomalies, or otherwise provides useful feedback to the pilot. Unlike typical known monitoring systems, the method also allows the performance of the pilot to be assessed, instead of only monitoring the components of the aircraft. Monitoring the pilot’s performance allows a warning to be issued, for example, when an aircraft is being flown in an unsafe or inefficient way.

[0009] Using a machine learning model to generate the prediction allows for the method to be more generalisable than other techniques, such as clustering, that simply compare the acquired flight data to historical flight data. The method is therefore able, for example, to perform well on data that is dissimilar to training data but nevertheless not anomalous.

[0010] Additionally, using the first flight data (i.e., flight data obtaining during the flight) to generate the predicted second flight data that is used for comparison ensures that the method takes into account the factors that are specific to the flight in question, such as weather conditions on the day. False alerts are therefore reduced.

[0011] The machine learning model may further be provided with the first flight data in raw form, for example in an uncompressed form. The first flight data and the predicted second flight data may thus contain actual flight parameters rather than compressed representations of the flight parameters. The predicted second flight data can then be compared to the actual second flight data to identify the flight parameters providing the highest contribution to the prediction error. Identifying not only the existence of an anomaly but also which particular flight parameter or flight parameters is or are anomalous provides more information to the pilot and facilitates identification of the cause of the anomaly and / or what corrective action might be effective to respond to it.

[0012] Optionally, the method further comprises: a) predicting, using the machine learning model given the actual second flight data as input, further predicted flight data corresponding to a further time window, the further time window being subsequent to the second time window; b) receiving actual further flight data corresponding to the further time window, the actual further flight data derived from a plurality of sensor readings taken during the further time window; c) calculating a prediction error by comparing the further predicted flight data to the actual further flight data; d) determining whether the prediction error is greater than the predetermined threshold; and e) outputing an alert in response to the prediction error being determined to be greater than the threshold.

[0013] Optionally, the method further comprises performing a plurality of iterations of at least steps (a) to (d) with the further time window for each iteration corresponding to one or more respective subsequent time windows, each subsequent time window being subsequent to the further time window of the previous iteration.

[0014] Performing the method iteratively for multiple time windows allows the method to provide continual monitoring with a high time resolution (small time windows), allowing warnings to be issued within a short time period of an anomalous event occurring.

[0015] Optionally, the machine learning model comprises a machine learning model trained using training data that comprises historical flight data.

[0016] Historical flight data (i.e., flight data obtained from real flights that have happened in the past) is available in large quantities and can thus be used to provide effect training of the machine learning model. Furthermore it may not be necessary to identify anomalous flight data in the historical flight data because while the historical flight data may contain some anomalies the majority of the historical flight data will be non-anomalous. A close match between the first flight data and the predicted second flight data predicted by a machine learning model trained on the historical flight data should therefore indicate with a high degree of confidence that the first flight data is non-anomalous. Conversely, a high prediction error should indicate with a high degree of confidence that an anomaly is present.

[0017] Optionally, the method further comprises identifying and selecting non-anomalous training data from candidate training data as selected training data used for training the machine learning model.

[0018] Preventing the machine learning model from being trained on anomalous flight data increases the chances of the model generating predicted second flight data with a high prediction error when the first flight data or actual second flight data is anomalous. The number of false alerts output is therefore reduced.

[0019] Optionally, the non-anomalous training data is identified and selected by calculating local outlier factor values for the candidate training data and selecting the selected training data used for training the machine learning model based on the calculated local outlier factor values. Optionally, the method further comprises selecting the selected training data dependent on calculated local outlier factors being greater than a predetermined value.

[0020] The use of the local outlier factor to determine the anomalous candidate training data allows for the process to be automated, meaning that the size of the candidate training data can be greater than if an expert was used to select the selected training data. Additionally, an expert may introduce bias into the selection process.

[0021] Optionally, the method further comprises assessing training data that has not been selected in the selected training data and including the assessed training data where it is determined that the assessed training data was incorrectly identified as anomalous based on a calculated local outlier factor value.

[0022] Reviewing the candidate training data that was not selected allows for data that was incorrectly labelled as anomalous to be included in the selected training data. Training data that is incorrectly labelled as anomalous may represent uncommon flights. Including the incorrectly labelled data in the selected training data used to train the machine learning model may allow the machine learning model to generate a more accurate prediction, and reduce the number of false alerts generated due to uncommon but non-anomalous flights.

[0023] Optionally, the method further comprises selecting a duration of one or more of the time windows to be different to a duration of one or more other time windows. Optionally, a duration of each of one or more of the time windows is less than 1 second.

[0024] Using a shorter time window allows for more frequent determination of whether an alert should be output by the method, thereby increasing a responsiveness of the method. Using a longer time window reduces the computational load on the system implementing the method and / or improves the accuracy of the prediction. Varying the size of the time window during a flight allows a balance between responsiveness and computational load / accuracy to be varied during the flight, for example to favour responsiveness at the expense of higher computational load / accuracy during portions of the flight where anomalies may have more serious consequences and / or are more likely to occur (e.g., take off and / or landing) and / or to favour lower computational load and / or increased accuracy at the expense of lower responsiveness during other portions of the flight (e.g., taxiing or cruising phases of the flight).

[0025] Optionally, the method further comprises analysing the predicted second flight data, before or during the second time window, to identify anomalies in the predicted second flight data and outputting an alert, before or during the second time window, in response to detection of an anomaly in the predicted second flight data.

[0026] Analysing the predicted second flight data before or during the second time window to identify anomalies allows anomalies to be identified before they have occurred or sooner than might otherwise be possible. The pilot can then be issued a warning in advance of an anomaly (or soon after an anomaly) and thereby seek to avoid the anomaly or reduce the magnitude and / or consequences of the anomaly.

[0027] Embodiments of the disclosure will be further described by way of example only with reference to the accompanying drawings.

[0028] Fig. 1 is a flow chart of steps of an example method for detecting an anomaly in a flight.

[0029] Fig. 2 is a diagram illustrating an example configuration of a machine learning model.

[0030] Fig. 3 is a graph showing first flight data and predicted second flight data over a number of time steps.

[0031] Fig. 4a is a graph showing a second time window, predicted second flight data and actual second flight data.

[0032] Fig. 4b is a graph showing a further time window, further predicted flight data and actual further flight data.

[0033] Embodiments of the disclosure comprise computer-implemented methods. The methods may be implemented using any general purpose computer system. Such computer systems are well known in the art and may comprise any suitable combination of hardware (e.g. processors, motherboards, memory, storage, input / output ports, etc.), firmware, and / or software to carry out the methods described. The computer system may be located in one location or may be distributed between multiple different locations. A computer program may be provided to implement the methods when executed by the computer system. The computer program may be provided to a user as a computer program product. The computer program product may be distributed by download or provided on a non-transitory storage medium such as an optical disk or USB storage device.

[0034] An example of a computer-implemented method for detecting an anomaly in a flight is described below with reference to the figures. As depicted in step SI of Fig. 1, the method comprises receiving first flight data corresponding to a first time window. The first flight data is derived from a plurality of sensor readings taken during the first time window. The first time window represents a specific period of time during the course of a flight. The duration of the first time window may be selected to be the same as or different to a duration of one or more other time windows. A duration of each of one or more of the time windows may for example be less than 1 second. Such time windows may be referred to as short time windows. Alternatively or additionally, a duration of each of one or more of the time windows may be equal to or greater than 10 seconds. Such time windows may be referred to as long time windows. Optionally, the duration of each of one or more of the time windows is about 30 seconds. The duration of the time window may change (e.g., increase or decrease) over the course of a flight. Using a shorter time window allows for more frequent warnings to be provided. Using a longer time window allows for a reduction in computational load.

[0035] The first flight data is derived from a plurality of sensor readings taken during the first time window. The sensor readings may be gathered using one or more sensors in or on the aircraft. The plurality of sensor readings may be directly included within the first flight data. Alternatively or additionally, the first flight data may include the output of a data filtering or other data processing application that receives the plurality of sensor readings as input and performs additional processing. For example, the data processing application may normalise the data or filter out data readings that are caused by errors in the sensors. The data processing application may also upsample the plurality of sensor readings, if the sampling rate desired is greater than the sampling rate of one or more sensors.

[0036] The sensor readings may include one or more of: the direction of travel of the aircraft; the latitude and / or longitude of the aircraft; a position or state of one or more flight controls; the current speed and / or acceleration of the aircraft; and the current altitude of the aircraft.

[0037] In one example implementation, the first flight data includes 130 flight parameters recorded at a sampling rate of 16 Hz. In another example implementation, the first flight data includes 132 flight parameters. These flight parameters may be a subset of the set of flight parameters recorded by the aircraft

[0038] As depicted in step S2 of Fig. 1, the method further comprises predicting, using a machine learning model given the first flight data as input, predicted second flight data corresponding to a second time window. The second time window is subsequent to the first time window. The predicted second flight data aims to predict the flight data that would be generated by the flight during the second time window, based on the first flight data gathered during the first flight window. The second time window can immediately follow the first time window. Alternatively, there can be a further time window between the end of the first time window and the beginning of the second time window.

[0039] One of the parameters of the first flight data may be the altitude of the aircraft. The graph of Fig. 3 shows an example of a first time window 12 showing variation of the altitude of the aircraft in example first flight data 2. The graph of Fig. 3 further shows the predicted altitude of the aircraft for a corresponding predicted second flight data 6 over the course of a second time window 14. The duration of the first time window 12 and second time window 14 are both 200 timesteps in the example shown. In this example, a single timestep is 0.0625 seconds, meaning 200 timesteps is 12.5 seconds. In other examples, a timestep may have a different length of time. If timestep 200 is considered as a present time, the first time window 12 in Fig. 3 corresponds to a preceding 200 timesteps and the second time window 14 corresponds to a future 200 timesteps. The first flight data and predicted second flight data may contain further parameters in addition to the altitude of the aircraft.

[0040] Using the first flight data to generate the prediction allows for local conditions, such as wind or rain, to be considered when generating the prediction. Equally, other operating parameters of the aircraft can be considered, such as the weight of the aircraft (which will depend on how the aircraft is loaded on that day). A method that generated a prediction without taking into account such factors and compared the current flight data to that prediction would not be as reliable. For example, the aircraft may accelerate slower than usual due to adverse weather conditions, which may be viewed as an anomaly by comparing the aircraft’s position to where it is expected to be if the adverse weather conditions were not taken into account (by basing the prediction on the first flight data).

[0041] As depicted in step S3 of Fig. 1, the method further comprises receiving actual second flight data corresponding to the second time window. The actual second flight data is derived from a plurality of sensor readings taken during the second time window.

[0042] As with the first flight data, the sensor readings may be directly included within the actual second flight data. Alternatively or additionally, the actual second flight data may include the output of a data filtering or other data processing application that receives the sensor readings as input. For example, the data processing application may normalise the data, perform upsampling, or filter out data readings that are caused by errors in the sensors.

[0043] The sensor readings may include the same readings included within the first flight data, and may be gathered using the same one or more sensors used to gather the sensor readings used to derive the first flight data.

[0044] As depicted in step S4 of Fig. 1, the method further comprises calculating a prediction error by comparing the predicted second flight data to the actual second flight data. The prediction error represents the error between the predicted second flight data generated by the machine learning model and the actual second flight data. The prediction error may also be called the reconstruction error. Optionally, the calculation of the prediction error comprises calculating one or more of: a mean square error; the average squared distance; and / or a root mean square error between the predicted second flight data and the actual second flight data. Alternative methods may be used to calculate the prediction error. Fig. 4a depicts a first time window 12 and a second time window 14. The second time window 14 contains both the predicted second flight data 6 and the actual second flight data 4. Differences between the predicted second flight data 6 and actual second flight data 4 correspond to prediction error. The predicted second flight data 6 is seen to diverge from the actual second flight data 4 as a function of time in the second time window 14.

[0045] As depicted in step S5 of Fig. 1, the method further comprises determining whether the prediction error is greater than a predetermined threshold. As the machine learning model is trained using normal flight data, the machine learning model will be able to produce a better prediction when the first flight data is not abnormal, meaning that the prediction error will be lower. Equally, if the actual second flight data is abnormal then the actual second flight data will not match the predicted second flight data. A prediction error above a certain threshold therefore indicates an anomaly in the flight data. Fig. 4a visually illustrates the prediction error as the difference between the line of the predicted second flight data 6 and the line of the actual second flight data 4. The threshold may be set to the highest prediction error produced on the training data used to train the model. Alternatively, the threshold may be set to be the highest prediction error produced on a set of data selected to be anomaly free.

[0046] As depicted in step S6 of Fig. 1, the method further comprises outputting an alert in response to the prediction error being determined to be greater than the threshold. The alert indicates to the pilot of the aircraft that the current flight is likely abnormal. The pilot may then act upon the alert. The alert may comprise an audio and / or visual and / or tactile signal. The alert may for example comprise a warning displayed on the flight controls. The method is able to detect an anomaly in the flight data early, allowing the pilot to be informed and take corrective action if required.

[0047] In an arrangement, the actual second flight data may be received while the actual second flight data is produced (i.e., during the second time window 14), rather than being received at or after the end of the second time window 14. The method may be configured to update the prediction error during the second time window 14, allowing an alert to be generated when the threshold is exceeded before the end of the second time window 14 instead of waiting for the end of the second time window 14 to calculate the prediction error and generating the alert at or after the end of the second time window 14.

[0048] The machine learning model may be provided the first flight data without any compression or dimensionality reduction having been applied to the first flight data beforehand. This approach contrasts with other methods where compressed representations are provided that do not correspond to actual flight parameters. The individual contribution to the prediction error for each of the flight parameters in the predicted second flight data can then be considered to identify which of the flight parameters is anomalous. For example, if the plane is travelling at an inappropriate velocity for the current altitude, the aircraft velocity will make a greater contribution to the prediction error than other parameters. The cause of the anomaly can therefore more easily be identified.

[0049] In one implementation, the method further comprises analysing the predicted second flight data 6, before or during the second time window 14, to identify anomalies in the predicted second flight data 6 and outputting an alert, before or during the second time window 14, in response to detection of an anomaly in the predicted second flight data 6. Analysing the predicted second flight data 6 allows for an abnormal flight to potentially be detected before the end of the second time window 14, providing an early alert to the pilot. As an example, Fig. 3 shows the altitude of the aircraft in the first flight data 2 and the predicted second flight data 6. Should the predicted altitude be too close to the ground, or contain some other anomaly, the pilot can be provided with a warning in advance. The method therefore allows for anomalies to be warned about before they occur.

[0050] In an arrangement, the method further comprises performing the following steps: a) predicting, using the machine learning model given the actual second flight data as input, further predicted flight data corresponding to a further time window, the further time window being subsequent to the second time window; b) receiving actual further flight data corresponding to the further time window, the actual further flight data derived from a plurality of sensor readings taken during the further time window; c) calculating a prediction error by comparing the further predicted flight data to the actual further flight data; d) determining whether the prediction error is greater than the predetermined threshold; and e) outputting an alert in response to the prediction error being determined to be greater than the threshold.

[0051] Performing the method using a further time window allows for a greater length of time to be covered without increasing the duration of the time windows.

[0052] The method may further comprise performing a plurality of iterations of at least steps (a) to (d) with the further time window for each iteration corresponding to one or more respective subsequent time windows, each subsequent time window being subsequent to the further time window of the previous iteration. Performing a plurality of iterations allows for the flight to be continuously analysed to determine if an anomaly is present in the flight data. When a short window length is used, the method is able to detect an anomaly relatively quickly (in real time or near real time). Alternatively, if a long window is used, the machine learning model is provided with more flight data and so is less likely to falsely detect an anomaly due to sensor error or turbulence causing a large prediction error. Although actual second flight data and predicted second flight data are used when describing other embodiments, the actual second flight data is interchangeable with the actual further flight data, and the predicted second flight data is interchangeable with the predicted further flight data. Fig. 4a and Fig. 4b depicts the method operating iteratively. In Fig. 4a, the first flight data 2 in the first time window 12 is provided as input to the machine learning model, and predicted second flight data 6 is generated predicting the flight data within the second time window 14. The actual second flight data 4 is then received at the end of the second time window 14 and compared to the predicted second flight data 6. If the prediction error is greater than a threshold the alert is given. The pilot may then react to the alert.

[0053] Fig. 4b depicts a further time window 16, further predicted flight data 10, and actual further flight data 8. The further time window 16 depicted in Fig. 4b is immediately after the second time window 14 shown in Fig. 4a. The predicted second flight data 6 is now ignored, with the further predicted flight data 10 being generated using the actual second flight data 4. The actual further flight data 8 is then received at the end of the further time window 16 and compared to the further predicted flight data 10. If the prediction error exceeds the threshold a warning will be generated, or an existing warning will continue if the previous prediction error was over the threshold such that a warning was already being outputted. If the prediction error is below the threshold, the warning will not be output, or an existing warning will stop.

[0054] The machine learning model is configured to receive as input data the first flight data 2 and generate as output data a prediction of the second flight data 6 that would be generated by the same flight during the second time window 14. The machine learning model may comprise an encoding portion and a decoding portion. The encoding portion may be configured to encode the input data, which is the first flight data, to an encoded representation of the first flight data. The decoding portion may be configured to receive the encoded representation and create a prediction of the second flight data 6 generated during a second time window 14 that occurs after the first time window 12. The machine learning model is trained to deconstruct the flight data provided as input to a lower dimensional form, and to then reconstruct the original data. The machine learning model is able to provide a better reproduction for flight data that is non-anomalous in comparison with data used to train the machine learning model. The error in the reproduction can thus be used to determine when a flight is abnormal.

[0055] The machine learning model may be an autoencoder 18, an example of which is depicted in Fig. 2. The autoencoder 18 depicted in Fig. 2 is implemented using a neural network. The encoding portion may be configured to encode the input data into a lower dimensional representation, for example a compressed vector representation, to form the encoded representation. The encoding portion may comprise multiple encoding layers, with each subsequent encoding layer having fewer nodes than the layer before. The decoding portion may then be configured to decode the lower dimensional representation into the predicted second flight data 6. The decoding portion may comprise multiple layers, with each subsequent decoding layer having a greater number of nodes than the layer before.

[0056] Alternatively, the machine learning model may be a transformer. The transformer may be implemented using a neural network. The transformer may take an input time series, such as a sequence of vectors, and transform the time series into an encoding vector. The encoding is then decoded back into another sequence. The transformer may include an attention mechanism that gives more weight to certain parts of the data. The attention mechanism determines the significance of other tokens in the input for the encoding of a specific token. Unlike traditional recurrent neural networks (e.g. LSTMs) that process input sequences sequentially, the transformer utilises the attention mechanism to process input sequences in parallel, thus, rendering them highly suitable for extensive training and improved accuracy. The use of a transformer based machine learning model reduces the time taken to train the model and provides a prediction that is more precise. Transformer based models are able to process long sequences of time series at a greater speed due to identifying focus points in the series.

[0057] The machine learning model may comprise a machine learning model trained using training data that comprises historical flight data (i.e., flight data from actual flights that have occurred in the past). Historical flight data may comprise the flight data for a plurality of flights over a period of months or years. The historical flight data may include at least 500 flights for example. In one implementation, the historical flight data includes between 700 and 1000 flights. The historical flight data may be sampled over the last three minutes of the flight. Optionally, the plurality of flight data may include only flight data that was captured at the airport that the aircraft that the model is used with operates at. Alternatively or additionally, the plurality of flight data may include only flight data corresponding to an aircraft of the same type as the aircraft that the model is used with. The training data used to train the machine learning model may comprise other training data in addition to, or instead of, historical training data. For example, some of the training data may be generated using a flight simulator. Alternatively, a computer program could be used to generate example flight data.

[0058] The method may further comprise training the machine learning model. As discussed, the machine learning model may be trained using historical flight data and / or other flight data. The method may further comprise identifying and selecting non- anomalous training data from candidate training data as selected training data used for training the machine learning model. The method produces fewer false alerts when the machine learning model is not trained using anomalous data, as the prediction error is then greater for anomalous flight data. Not all of the candidate flight data is therefore suitable for training the machine learning model. Training a machine learning model using anomalous data can also lead to the machine learning model generating worse predicted second flight data and further predicted flight data for non-anomalous flight data. An example of anomalous flight data would be data generated during an unusual flight, such as a plane approaching a runway too fast. Identifying anomalous flight data as anomalous and only selecting the non-anomalous training data allows for the machine learning model to be better trained. The number of false alerts produced by the method is then reduced. Training the machine learning model on non-anomalous training data will improve the machine learning model’s performance at generating predicted flight data. The machine learning model will then produce predictions that more closely align with the flight data generated by a non-anomalous flight.

[0059] Equally, removing the anomalous training data will reduce the risk of the machine learning model predicting anomalous flight data. When the machine learning model is provided with anomalous flight data generated by an anomalous flight, the model will then more reliably generate predicted flight data that has a high prediction error when compared to the actual flight data generated by the anomalous flight. Only training the machine learning model using non-anomalous training data therefore increases the method’s accuracy at detecting anomalies in flights.

[0060] The non-anomalous training data may be identified and selected by calculating local outlier factor (LOF) values for the candidate training data, and selecting the selected training data used for training the machine learning model based on the calculated local outlier factor values. The local outlier factor value for each data point in the candidate training data is calculated based on the local density deviation of the data point with respect to the data point’s neighbours. Therefore, a data point with many neighbours positioned close to the data point will have a higher density than a data point with a few neighbours, or a datapoint with many neighbours at a distance. The selected training data may be selected dependent on calculated local outlier factors being greater than a predetermined value. Alternatively, the value may be determined using the average value of all of the points. Selecting the selected training data based on the local outlier factor ensures that data points that are further apart from other data points are not selected. Performing the selection using LOF allows for the selection to be automated, rather than requiring a human to manually identify what data is and is not anomalous. Alternative techniques may be used to automatically detect and remove anomalies within the training data instead of or in addition to the local outlier factor values. For example, K-means clustering, or other clustering techniques may be used.

[0061] Although reviewing the candidate training data to identify anomalous and non- anomalous training data can be performed by a human expert, the process can take a long time and / or be subject to human bias and mistakes. Selecting the selected training data using the local outlier factor values reduces the amount of computation required to select the selected training data in comparison to more complex techniques. Using an automated system to select the selected training data additionally allows for the candidate training data to include a greater number of values than if the training data was being manually selected.

[0062] The above techniques could also be applied to the detection of anomalous flight data directly. However, the present method provides several benefits for detecting anomalies in real world flight data, in comparison to clustering techniques and local outlier factor. Clustering techniques are heavily reliant on the flight data provided as input, the arrival and departure airport, the type of aircraft and the size of the dataset. Clustering techniques therefore cannot be generalised to the same extent as the present method. For a clustering technique to identify flight data as non-anomalous, there must be several points of training data located in close proximity within the vector space. The present method is able to function on unseen data, and establish a specific threshold for when flight data is considered anomalous. Although the present method must have been trained on similar flight data, there is no requirement that every single parameter must be similar, which is the case for clustering techniques.

[0063] Local outlier factor values are therefore best suited only for selecting the selected training data.

[0064] Clustering techniques are similarly unsuited for detecting anomalous flight data, as they require additional hyperparameters to be tuned. These parameters are highly sensitive and are difficult to correctly set, with research in the area offering little guidance on the topic. Clustering techniques also struggle to adapt to changes in the data, such as a change of airport, the number of flights or the flight time.

[0065] Using local outlier factor values to select the training data provides better performance than using a clustering technique instead, as clustering techniques are heavily dependent on dimensionality reduction in comparison to local outlier factor based techniques. Clustering techniques produce clusters, with any values outside of those clusters being viewed as anomalous, and so struggles to identify local outliers that are still near a cluster. Local outlier factor based techniques do not have the same drawback, and therefore allow for the number of false positives to be reduced. Local outlier factor based techniques also only require tuning the parameter for the nearest neighbour.

[0066] In comparison to existing clustering and LOF techniques, the present method can be generalised across aircraft, airports and approaches. Existing techniques require hyperparameters to be adjusted based on the airport and aircraft type, meaning that any prediction method based on these techniques would not be able to be generalised. In contrast, the present method is more universally applicable, and can be generalised to a variety of airports and aircraft types while still producing accurate results.

[0067] The present method provides the further advantage of generating a prediction that can be used to generate a warning in advance. Clustering techniques function by comparing a data point to a large dataset of existing datapoints, to identify outliers that do not fit within clusters. Clustering must therefore always be performed offline. In contrast, the present method can be used in real time, and is able to generate predictions of future time steps to provide advanced warning.

[0068] The method may further comprise assessing training data that has not been selected in the selected training data and including the assessed training data where it is determined that the assessed training data was incorrectly identified as anomalous based on a calculated local outlier factor value (or other clustering technique). Although it may be difficult or impossible to manually select the selected training data, the number of data points that are considered to be anomalous is far fewer, meaning that the anomalous data can more easily be manually sorted through. Identifying and including any data points that were falsely considered to be anomalous due to the data point’s local outlier factor (for example) improves the accuracy of the model trained on the selected training data. Data points that represent uncommon (but not anomalous) situations are not excluded from the selected training data.

[0069] Alternative methods may be used to ensure that the selected training data does not include any anomalies. In an arrangement, the training of the machine learning model is performed using performance data captured from a flight simulator A pilot may perform repeated flights in a flight simulator to generate the selected performance data. The flights can be performed in ideal conditions to ensure that they are not anomalous, and can be varied to ensure that the machine learning model is not over-trained on a specific flight path or using a specific model of aircraft. The method may further involve selecting the selected training data used for training the machine learning model from the captured performance data, instead of using every single flight captured. Selecting the selected training data from the captured flights allows for any flight where the pilot themselves displayed anomalous behaviour, for example landing too fast, to be removed.

[0070] The first flight data may be added to the training data to obtain updated training data. The machine learning model may then be updated using the updated training data. Updating the training data may be useful if the amount of available training data is low, or the current conditions do not match those of the training data (for example the training data may be for predominantly dry weather and the current conditions may be wet).

[0071] Optionally, updating the machine learning model using the updated training data is performed after receiving actual second flight data corresponding to the second time window. The first flight data may only be added to the flight data if an alert is not given, preventing anomalous flight data from being included with the training data.

[0072] The method may be implemented in multiple configurations. A data processing system comprising a processor may be configured to perform the method of any preceding claim. Alternatively or additionally, a computer program may comprise instructions which, when the program is executed by a computer, causes the computer to carry out the method.

[0073] As discussed above, the various embodiments can be implemented in a wide variety of operating environments, which in some cases can include one or more user computers, computing devices, or processing devices which can be used to operate any of a number of applications. User devices can include any of a number of general purpose personal computers, such as desktop or laptop computers running a standard operating system, as well as cellular, wireless, and handheld devices running mobile software and capable of supporting a number of networking and messaging protocols. Such a system also can include a number of workstations running any of a variety of commercially- available operating systems and other known applications for purposes such as development and database management. These devices also can include other electronic devices, such as thin-clients, gaming systems, and other devices capable of communicating via a network, provided the devices include a file system to protect.

[0074] Various aspects also can be implemented as part of at least one service or Web service, such as may be part of a service-oriented architecture. Services such as Web services can communicate using any appropriate type of messaging, such as by using messages in extensible markup language (XML) format and exchanged using an appropriate protocol such as SOAP (derived from the "Simple Object Access Protocol"). Processes provided or executed by such services can be written in any appropriate language, such as the Web Services Description Language (WSDL). Using a language such as WSDL allows for functionality such as the automated generation of client-side code in various SOAP frameworks.

[0075] In embodiments utilizing a Web server, the Web server can run any of a variety of server or mid-tier applications, including HTTP servers, FTP servers, CGI servers, data servers, Java servers, and business application servers. The server(s) also may be capable of executing programs or scripts in response to requests from user devices, such as by executing one or more Web applications that may be implemented as one or more scripts or programs written in any programming language, such as Java®, C, C# or C++, or any scripting language, such as Perl, Python, or TCL, as well as combinations thereof. The server(s) may also include database servers, including without limitation those commercially available from Oracle®, Microsoft®, Sybase®, and IBM®.

[0076] The environment can include a variety of data stores and other memory and storage media as discussed above. These can reside in a variety of locations, such as on a storage medium local to (and / or resident in) one or more of the computers or remote from any or all of the computers across the network. In a particular set of embodiments, the information may reside in a storage-area network (“SAN”) familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers, servers, or other network devices may be stored locally and / or remotely, as appropriate. Where a system includes computerized devices, each such device can include hardware elements that may be electrically coupled via a bus, the elements including, for example, at least one central processing unit (CPU), at least one input device (e.g., a mouse, keyboard, controller, touch screen, or keypad), and at least one output device (e.g., a display device, printer, or speaker). Such a system may also include one or more storage devices, such as disk drives, optical storage devices, and solid-state storage devices such as random access memory (“RAM”) or read-only memory (“ROM”), as well as removable media devices, memory cards, flash cards, etc.

[0077] Such devices also can include a computer-readable storage media reader, a communications device (e.g., a modem, a network card (wireless or wired), an infrared communication device, etc.), and working memory as described above. The computer- readable storage media reader can be connected with, or configured to receive, a computer- readable storage medium, representing remote, local, fixed, and / or removable storage devices as well as storage media for temporarily and / or more permanently containing, storing, transmitting, and retrieving computer-readable information. The system and various devices also typically will include a number of software applications, modules, services, or other elements located within at least one working memory device, including an operating system and application programs . It should be appreciated that alternate embodiments may have numerous variations from that described above. For example, customized hardware might also be used and / or particular elements might be implemented in hardware, software (including portable software, such as applets), or both. Further, connection to other computing devices such as network input / output devices may be employed. Storage media and other non-transitory computer readable media for containing code, or portions of code, can include any appropriate media known or used in the art, such as but not limited to volatile and non-volatile, removable and non-removable non- transitory media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data, including RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the system device. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and / or methods to implement the various embodiments.

[0078] Having described the invention it will be appreciated that variations may be made on the above described embodiment, which is not intended to be limiting. The invention is defined in the appended claims and their equivalents.

Claims

CLAIMS1. A computer-implemented method for detecting an anomaly in a flight, the method comprising: receiving first flight data corresponding to a first time window, the first flight data derived from a plurality of sensor readings taken during the first time window; predicting, using a machine learning model given the first flight data as input, predicted second flight data corresponding to a second time window, the second time window being subsequent to the first time window; receiving actual second flight data corresponding to the second time window, the second flight data derived from a plurality of sensor readings taken during the second time window; calculating a prediction error by comparing the predicted second flight data to the actual second flight data; determining whether the prediction error is greater than a predetermined threshold; and outputting an alert in response to the prediction error being determined to be greater than the threshold.

2. The method of claim 1, wherein the machine learning model comprises an encoding portion configured to generate a low dimensional representation using the first flight data and a decoding portion configured to generate the predicted flight data using the low dimensional flight data.

3. The method of claim 2, wherein the machine learning model is an autoencoder.

4. The method of claim 2, wherein the machine learning model is a transformer.

5. The method of any of claims 1 to 4, further comprising: a) predicting, using the machine learning model given the actual second flight data as input, further predicted flight data corresponding to a further time window, the further time window being subsequent to the second time window;b) receiving actual further flight data corresponding to the further time window, the actual further flight data derived from a plurality of sensor readings taken during the further time window; c) calculating a prediction error by comparing the further predicted flight data to the actual further flight data; d) determining whether the prediction error is greater than the predetermined threshold; and e) outputting an alert in response to the prediction error being determined to be greater than the threshold.

6. The method of claim 5, comprising performing a plurality of iterations of at least steps (a) to (d) with the further time window for each iteration corresponding to one or more respective subsequent time windows, each subsequent time window being subsequent to the further time window of the previous iteration.

7. The method of any preceding claim, wherein the machine learning model comprises a machine learning model trained using training data that comprises historical flight data.

8. The method of any preceding claim, comprising: adding the first flight data to the training data to obtain updated training data; and updating the machine learning model using the updated training data.

9. The method of claim 8, wherein updating the machine learning model using the updated training data is performed after receiving actual second flight data corresponding to the second time window.

10. The method of any preceding claim, comprising training the machine learning model.

11. The method of claim 10, comprising identifying and selecting non-anomalous training data from candidate training data as selected training data used for training the machine learning model.

12. The method of claim 11, wherein the non-anomalous training data is identified and selected by: calculating local outlier factor values for the candidate training data; and selecting the selected training data used for training the machine learning model based on the calculated local outlier factor values.

13. The method of claim 12, comprising selecting the selected training data dependent on calculated local outlier factors being greater than a predetermined value.

14. The method of claim 12 or 13, comprising assessing training data that has not been selected in the selected training data and including the assessed training data where it is determined that the assessed training data was incorrectly identified as anomalous based on a calculated local outlier factor value.

15. The method of any of claims 10 to 14, wherein the training of the machine learning model is performed using performance data captured from a flight simulator.

16. The method of any preceding claim, comprising selecting a duration of one or more of the time windows to be different to a duration of one or more other time windows.

17. The method of claim 16, wherein a duration of each of one or more of the time windows is less than 1 second.

18. The method of claim 16, wherein the duration of each of one or more of the time windows is equal to or greater than 10 seconds.

19. The method of any preceding claim, comprising analysing the predicted second flight data, before or during the second time window, to identify anomalies in thepredicted second flight data and outputting an alert, before or during the second time window, in response to detection of an anomaly in the predicted second flight data.

20. The method of any preceding claim, wherein the calculation of the prediction error comprises calculating one or more of: a mean square error; the average squared distance; and / or a root mean square error.

21. A data processing system comprising a processor configured to perform the method of any preceding claim.

22. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of claims 1 to 20.