Track anomaly detection method and associated device
A neural network-based method for track anomaly detection addresses computational and real-time challenges by calculating average log-likelihood values and setting consistent thresholds, improving performance and reducing inference time for embedded systems.
Patent Information
- Application Number
- FR2023015426
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-12-28
- Publication Date
- 2025-07-04
- Estimated Expiration
- 2043-12-28
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Method for detecting track anomalies and associated device
[0001] The present invention relates to a method for detecting track anomalies.
[0002] The invention relates to the field of discrimination of anomalies in runways, more particularly maritime, avionic or automobile runways, and more particularly still AIS maritime runways (from the English "Automatic Identification System").
[0003] A track is a time series giving at each time step information such as the position, speed and heading of the tracked object. The track allows for example the reconstruction of the trajectory followed by the tracked object.
[0004] An AIS maritime track corresponds to the time series of AIS messages, emitted by an AIS beacon on board a vessel, as a tracked object.
[0005] Each AIS message contains static and dynamic information relating to the vessel. In particular, an AIS message provides instant information on geographic position, speed, heading, navigation status and turning radius.
[0006] AIS beacons have ranges ranging from 5 nautical miles to 20 nautical miles, depending on the category of the AIS maritime system.
[0007] AIS messages are picked up by a listening station, on the ground or on board a maritime surveillance aircraft. AIS messages can be analyzed either in real time or in delayed time.
[0008] Among the possible analyses, we know the method of detecting anomalies of AIS tracks from GeoTrackNet, with an inference using a so-called "contrario" detection.
[0009] This method is for example presented at the address: “https: / / github.com / CIA-Oceanix / GeoTrackNet”.
[0010] This method relies on the use of an artificial intelligence algorithm called GeoTrackNet. Following a learning phase, a suitably parameterized neural network is used in inference to calculate a log likelihood score for each AIS message of a track of interest.
[0011] Then, the score of each message allows this message to be classified as either normal or abnormal.
[0012] If the track of interest has too many abnormal messages, it is annotated as abnormal.
[0013] However, this method requires significant computational resources for inference, especially when the track of interest contains many messages. In effect, this method requires the calculation of an increasing cumulative binomial law on each sub-segment of a track.
[0014] Furthermore, this method is not suitable for real time because it requires an inference time for detecting anomalies from log-likelihoods of the order of 2 minutes for a batch of 100 AIS tracks.
[0015] Furthermore, with this method, the detection threshold must be adjusted according to the duration of the track and varies enormously (of the order of 10 for a 10-minute track, to 1.10 7 for a 4-hour track).
[0016] The anomaly detection performance of this method on short tracks (10 to 30 min) could also be improved.
[0017] It is then understood that, particularly for embedded systems, whose computing resources are constrained, there is a need for a method for detecting track anomalies which is less costly in terms of digital resources and which allows real-time processing.
[0018] Furthermore, it would be desirable to be able to detect an abnormal track as precisely and quickly as possible, after receiving its first AIS messages, and without having to continually modify the detection threshold depending on the duration of the track.
[0019] The aim of the invention is therefore to propose a method for detecting track anomalies which addresses these problems.
[0020] For this purpose, the invention relates to a method for detecting track anomalies, a track comprising a temporal succession of data, the method comprising, in a configuration phase, the steps of: training a neural network on a plurality of training tracks to predict a log-likelihood value of data from a track;calculating an average log-likelihood value for each track in a set of validation tracks and calculating a median over the calculated average log-likelihood values, the average log-likelihood value of a track being obtained by averaging the log-likelihood value of the data of the track under consideration, the log-likelihood value of a data item being estimated using the trained neural network, calculating a score over the average log-likelihood value of each track in a set of test tracks relative to the median and calculating a statistical coefficient from the calculated scores; defining an abnormality threshold from the median and the statistical coefficient;the method further comprising, in an inference phase, the steps of: acquiring at least one track of interest, processing the track of interest by: calculating an average log-likelihood value of the track of interest by averaging the log-likelihood values of the data which make up the track of interest, the log-likelihood value of a data item being estimated by means of the trained neural network,; calculating a score of the average log-likelihood value of the track of interest relative to the median, and comparing the score of the average log-likelihood value of the track of interest to the abnormality threshold, the track of interest being classified as "abnormal" when the score of the average log-likelihood value of the track of interest is greater than the abnormality threshold, and "normal" otherwise.
[0021] According to other advantageous aspects of the invention, the method for detecting track anomalies comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:
[0022] - the statistical coefficient is the coefficient of the MAD algorithm on the scores of the average log-likelihood values of the test tracks across all test tracks relative to the median;
[0023] - the abnormality threshold S is defined according to the following equation:
[0024] S = MEDvALID + nb_MAD x C_MAD
[0025] with MEDvalid the median, C__MAD the coefficient of the MAD algorithm and nb_MAD a coefficient;
[0026] - a track is a track of AIS messages, radar plots, satellite detections, or GPS georeferenced positions;
[0027] - a data being an AIS message, an abnormal track consists of: an anomaly of deviation; an anomaly of impossibility or cut-off of the AIS system; an anomaly of risk of collision, or an anomaly of entry into an unusual zone;
[0028] - the step of processing the track of interest is carried out in real time, preferably at each update of the track of interest leading to the addition of data;
[0029] - a length of a track of interest is configurable by a sliding window;
[0030] - the training and validation tracks are tracks of which the majority are considered normal; and
[0031] - the neural network is the GeoTrackNet network.
[0032] The invention also relates to a computer program product comprising software instructions which, when executed by a computer, implement all or part of the steps of the preceding method.
[0033] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example and made with reference to the drawings in which:
[0034] [Fig-1] [Fig.l] is a schematic view of a system for anomaly detection of tracks, receiving AIS messages from a tracked vessel;
[0035] [Fig.2] [Fig.2] is an illustration of a monitored geographic area; and
[0036] [Fig.3] [Fig.3] is a flowchart of the steps of a preferred embodiment of the track anomaly detection method according to the invention, implemented by the track anomaly detection device of [Fig.l], for identifying abnormal tracks in the geographic area of [Fig.2].
[0037] Although the present description is made for the particular case of tracks based on AIS messages, the invention applies to any type of track, in particular tracks made up of radar plots, satellite detections, GPS readings, etc.
[0038] From a track, consisting of the succession of N AIS messages coming from a ship, we seek to know if this ship is following a normal or abnormal route in relation to a set of routes considered to be normal.
[0039] For example, detectable anomalies are for example: deviation anomalies; AIS impossibility or cut-off; or entry into an unusual zone.
[0040] The invention also relies on the GeoTrackNet algorithm, but modifies the way of using the results provided by this algorithm to facilitate the calculations.
[0041] We seek to determine whether the route followed by a ship 1 is normal or abnormal.
[0042] Vessel 1 is equipped with an AIS beacon 2 periodically transmitting messages AIS, xt, for example by means of a VHF antenna.
[0043] A monitoring station 3, on land or, preferably, on board a maritime surveillance aircraft, monitors the geographical area crossed by the ship 1.
[0044] Monitoring station 3 tracks vessel 1 from the received AIS messages.
[0045] For this, the station 3 comprises at least one sensor 8, configured to acquire the AIS xt messages over time and transmit them, as digital signals, to a runway anomaly detection device 10.
[0046] The device 10 is a computer comprising calculation means, such as a processor, and storage means, such as a memory. The memory stores in particular the instructions of computer programs, in particular a program whose execution allows the implementation of the method according to the invention.
[0047] The device 10 is thus programmed to develop a track from all of the last N AIS messages xt received successively from the ship 1.
[0048] The device 10 is thus programmed to qualify this track of interest as normal or abnormal by implementing the method according to the invention, preferably in real time, as new AIS messages are received.
[0049] Alternatively, the track of interest is analyzed in delayed time, following an AIS message acquisition campaign.
[0050] More specifically, the track anomaly detection device 10 comprises an input module 22, a processing module 24 and an output module 26.
[0051] The input module 22 makes it possible to receive the digital signals from the sensor 8 and reconstruct a track from a series of N successive AIS messages. A track is in fact a sliding window of fixed length, to retain the last N AIS messages.
[0052] The length of a track is fixed, but is configurable by the operator.
[0053] The processing module 24 makes it possible to discriminate whether the track of interest at the output of the module 22 is a normal or abnormal track.
[0054] The processing module 24 comprises a neural network training unit 32, a mean log-likelihood calculation unit 33, a median calculation unit 34, a MAD score calculation unit 35, a MAD coefficient calculation unit 36 and a unit for determining an abnormality threshold 37 and a discrimination unit 38.
[0055] The device 10 further comprises a database 28, storing in particular learning data consisting of batches of training tracks and batches of validation tracks, and batches of test tracks and a plurality of medians of the average log-likelihoods of the tracks of the batches of validation tracks.
[0056] Each median of the plurality of medians is associated with a specific duration of the validation tracks used to calculate it. For example, there is a median of the average log-likelihoods of a set of validation tracks of 10 min, a median for a set of validation tracks of 20 min, etc. for track durations of 40 min, 1 h, 2 h, 3 h and for a validation set whose track duration ranges from 4 h to 24 h.
[0057] Finally, once processed by the processing module 24, the labeled track of interest is transmitted to the output module 26.
[0058] The module 26 is, for example, an application which displays an alert on the screen of a human-machine interface 12 of the device 10. This display of the label of the track of interest allows the operator to concentrate his actions on the abnormal tracks in order to optimize the monitoring and / or intervention resources on the corresponding vessels.
[0059] Alternatively, the module 26 is an automatic application that generates an action based on the abnormal tracks. The action generated is, for example, alerting the crew of the ship whose route is deemed abnormal of a potential risk and / or proposing one or more correction routes.
[0060] In [Fig.2], a geographical area of interest Z, maritime in the case presented, is monitored by station 3.
[0061] The area of interest Z is crossed by a plurality of tracks.
[0062] The tracks do not necessarily all have the same length in terms of the number of AIS messages constituting them.
[0063] Among these tracks, we distinguish tracks of interest to be processed and historical tracks.
[0064] Among these history tracks, there are training tracks (such as tracks 41, 42, 43), validation tracks (such as tracks 51 and 52) and test tracks (such as tracks 61 and 62).
[0065] History tracks are tracks for learning and configuring processing.
[0066] The training and validation tracks are considered mostly normal for training and calculating the average log-likelihood medians.
[0067] The test tracks include tracks considered normal (such as track 62) and tracks considered abnormal (such as track 61).
[0068] The tracks of interest are tracks that the implementation of the method according to the invention will allow to be classified as normal (case of track 71) or as abnormal (case of track 72). In [Fig.2], the last three messages (relative to time 111 ooo current t), xt, xt _i and xt _2, of track 71 and the last three messages xt, x t4 and xt 2 of track 72 are represented.
[0069] The operation of the track anomaly detection device 10 will now be described with reference to [Fig. 3], which illustrates a preferred embodiment of the track anomaly detection method 100.
[0070] In a configuration phase 101, carried out prior to a mission, the method 100 comprises a training step 110, during which the training unit of a neural network 32 is executed to train a neural network on a set of training tracks.
[0071] The neural network is preferably a Variational Recurrent Neural Network - VRNN (“variational recurrent neural network”).
[0072] More preferably, it is the GeoTrackNet model.
[0073] The training is carried out on a batch of training tracks, which is extracted from the database 28, said training tracks being historical tracks recorded in the past.
[0074] The goal of training is to learn a distribution that maximizes a log-likelihood of a sequence of T successive AIS messages extracted from a track. T is an integer less than N, the total number of AIS messages in the track.
[0075] In the following, a track sequence is defined as a set of T successive AIS messages from the same track. A sequence is denoted: Xj.T = {xt}, jT, OR XT is the last AIS message of the sequence.
[0076] We therefore speak of the log-likelihood of an AIS track sequence, evaluated from the last message considered and the previous T1 messages.
[0077] The log-likelihood of the AIS track sequence is defined as:
[0078] logp(x) = logp(x logp(x I x^ i)
[0079] With: - xt; the t-th AIS message of the sequence; - j / x |xh , ),1a conditional probability of obtaining the message xt knowing the sequence of messages which precedes it; - there is a probability of the first message in the sequence; and, - / j the likelihood of the sequence considered. ^tT / •
[0080] Advantageously, the GeoTrackNet neural network training technique presented in detail in the paper is used: https: / / arxiv.org / pdf / 1912.00682.pdf.
[0081] A trained neural network is obtained at the output of step 110. Finally, for a message xt of a track, the trained neural network predicts the log-likelihood of this message as the last message of a sequence of T messages.
[0082] Then, still in the configuration phase 101, the method 100 comprises a step 120 of calculating the average log-likelihood and a step 125 of calculating medians.
[0083] In step 120, for each track of a batch of validation tracks extracted from the database 28, the previously trained neural network is used to predict a log-likelihood value for each of the messages of the validation track considered.
[0084] Then, unit 33 is executed to determine an average log-likelihood value for each validation track.
[0085] The value of the average log-likelihood of a track is equal to the average of the log-likelihood values of each of the messages that make up said track.
[0086] Then, in step 125, unit 34 is executed to determine the median, MEDvAr.tm of the average log-likelihood values for all validation tracks of the batch under consideration.
[0087] This median is finally stored in the database 28.
[0088] As a reminder, the median, MEDvalid, is the average log-likelihood value of a batch of validation tracks, for which 50% of the average log-likelihood values of the validation tracks in that batch are above said value and 50% of the average log-likelihood values of the validation tracks in that batch are below said value.
[0089] Advantageously, several medians are stored according to characteristics specific to the different validation tracks, in particular the track duration.
[0090] Then, still in the configuration phase 101, the method 100 comprises a step 130 of calculating a MAD score and a step 135 of calculating a MAD coefficient.
[0091] In step 130, a batch of test tracks extracted from the database 28 for adjusting the normal / abnormal track detection thresholds is considered.
[0092] For each test track, unit 34 is first executed to determine a value of the average log-likelihood.
[0093] For each test track, unit 35 calculates a score, SCOfe_MAD, of its average log-likelihood value.
[0094] This score corresponds to the absolute deviation between the average log-likelihood value of the test track Li and the median of the average log-likelihood values of the tracks of the validation set determined in step 125:
[0095] score_MAD = |Lj- MEDy^LiDl
[0096] Preferably, the median value is chosen as a function of the duration of the test track. It is that of the validation tracks having a duration closest to the duration of the test track considered.
[0097] Then, in step 135, unit 36 is then executed to implement the Median Absolute Deviation -MAD algorithm (for mean absolute deviation of a median).
[0098] The MAD algorithm then provides a MAD coefficient, C_MAD, from the scores, SCOre_MAD, of the test tracks:
[0099] C_MAD = med(scores_MAD)
[0100] The MAD coefficient is in fact the median of the absolute deviations of the mean log-likelihood values of the test tracks from the median of the mean log-likelihood values of the validation test tracks.
[0101] Still in the configuration phase 101, the method 100 comprises a step 140 of calculating the abnormality threshold.
[0102] In this step, the unit 37 is for example executed to determine an abnormality threshold S from the statistical parameters determined in the previous steps.
[0103] For example, the abnormality threshold is defined as:
[0104] S = MEDvalid + nb_MAD x C_MAD
[0105] where nbMAD is a fixed coefficient throughout the detection process, equal for example to three or four.
[0106] If for a test track, its score, AD score, is higher than the abnormality threshold, S, this track is considered “abnormal”, on the other hand if its score is lower than or equal to the abnormality threshold, S, this track is considered “normal”.
[0107] Alternatively, the abnormality threshold is a value adjustable by the operator according to the needs and / or characteristics of the tracks.
[0108] In an inference phase 102 of the method 100, which is carried out during the detection mission, the method 100 comprises an acquisition step 150, during which the input module 12 is executed in order to develop a track of interest from the succession of AIS messages coming from the same ship 1.
[0109] Then, in a step 160, the discrimination unit 38 is executed to monitor the track of interest and label it normal (case of track 71) or abnormal (track 72).
[0110] First, unit 38 calls unit 33 to calculate an average log-likelihood value of the track of interest. For this, the average of the log-likelihood values of the last N messages of the track of interest is calculated, the log-likelihood value of a message being estimated by the suitably parameterized neural network.
[0111] Then, unit 38 calls unit 35 to calculate a MAD score of the average log-likelihood value of the track of interest.
[0112] Finally, unit 38 compares the MAD score of the track of interest with the abnormality threshold S adjusted in phase 101.
[0113] The track of interest is then considered “abnormal” if its MAD score is higher than the abnormality threshold, and “normal” otherwise.
[0114] Finally, in a step 170, the display module 26 is executed to display the track of interest on the HMI 12 and an alarm when the label associated with this track of interest is “abnormal”. The alarm must help the operator to identify the tracks which deviate from the normal learned behaviors.
[0115] If in [Fig. 1], the different units implemented during the method according to the invention have been represented for convenience as belonging to the same electronic device 10. However, as a variant and preferably, the steps of the configuration phase 101 of the method 100 (and the associated units) are carried out on a first computer (for example on the ground, having normal or high computing capacities), while the steps of the inference phase 102 of the method 100 (and the associated units) are carried out on a second computer (for example on board, having constrained computing capacities). The first and second computers are independent. The content of the database associated with each computer is adapted to the steps actually implemented by this computer.
[0116] Advantageously, the second computer is produced in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array) or in the form of an integrated circuit, such as an ASIC (Application Specific Integrated Circuit).
[0117] Generally speaking, a neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.
[0118] More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.
[0119] Alternatively, more complex neural network structures can be envisaged with a layer that can be connected to a layer further away than the immediately preceding layer.
[0120] Alternatively, the neural network used is of the “Transformers” type.
[0121] Each neuron is also associated with an operation, that is to say a type processing, to be carried out by said neuron within the corresponding processing layer.
[0122] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.
[0123] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the previous layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering at the output of said neuron, in particular to the neurons of the following layer which are connected to it, the value resulting from the application of the activation function. The activation function makes it possible to introduce non-linearity into the processing carried out by each neuron. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.
[0124] As an optional addition, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.
[0125] Such a neural network is trained on a set of training tracks comprising a large majority of tracks considered to be normal.
[0126] The present invention has a number of advantages:
[0127] First of all, the invention makes it possible to discriminate abnormal tracks in a set of tracks of interest in a more economical manner in terms of computation time for similar performances. Indeed, it involves performing a simple thresholding operation on the MAD score of the average log-likelihood of a track.
[0128] Furthermore, the invention makes it possible to discriminate abnormal tracks more economically in terms of digital resources.
[0129] With the invention, an anomaly detection inference time of the order of 1 second is achieved for 100 tracks, for performances equivalent to that of the state-of-the-art method.
[0130] The invention also makes it possible to detect anomalies on a short runway. More precisely, a detection of the order of 15% more abnormal runs for runs ranging from 10 to 30 minutes.
[0131] Finally, the method allows the same detection threshold to be used continuously regardless of the duration of the test track, unlike the state-of-the-art method.
Claims
1. Claims Method (100) for detecting track anomalies, a track comprising a temporal succession of data, the method comprising, in a configuration phase, the steps of: a. training a neural network (110) on a plurality of training tracks to predict a log-likelihood value of a track's data; b. calculating (120) an average log-likelihood value for each track of a set of validation tracks and calculating (125) a median over the calculated average log-likelihood values, the average log-likelihood value of a track being obtained by averaging the log-likelihood value of the data of the track considered, the log-likelihood value of a data being estimated by means of the trained neural network, c. calculating (130) a score on the average log-likelihood value of each track in a set of test tracks relative to the median and calculating (135) a statistical coefficient from the calculated scores; d. definition (140) of an abnormality threshold from the median and the statistical coefficient; the method further comprising, in an inference phase, the steps of: a. acquisition (150) of at least one track of interest (3), b. processing (160) of the track of interest by: • calculating an average log-likelihood value of the track of interest by averaging the log-likelihood values of the data that make up the track of interest, the log-likelihood value of a data item being estimated using the trained neural network, • calculating a score of the average log-likelihood value of the track of interest relative to the median, and • comparing the score of the average log-likelihood value of the track of interest to the abnormality threshold, the track of interest being classified as “abnormal” when the score of the average log-likelihood value of the track of interest is greater than the abnormality threshold, and “normal” otherwise.
2. The method of claim 1, wherein the statistical coefficient is the coefficient of the MAD algorithm on the scores of the average log-likelihood values of the tracks of the set of test tracks relative to the median.
3. Method according to claim 2, in which the abnormality threshold S is defined according to the following equation: S = MEDval1d + nb_MAD x C_MAD With MEDVAI In the median, C_MAD the coefficient of the MAD algorithm and nb_MAD a coefficient.
4. A method according to any preceding claim, wherein a track is a track of AIS messages, radar plots, satellite detections, or GPS georeferenced positions.
5. Method according to any one of the preceding claims, in which, a data being an AIS message, an abnormal track consists of: a deviation anomaly; an anomaly of impossibility or cut-off of the AIS system; a collision risk anomaly, or an anomaly of entry into an unusual zone.
6. Method according to any one of the preceding claims, in which step f is carried out in real time, preferably at each update of the track of interest leading to the addition of data.
7. A method according to any preceding claim, wherein a length of a track of interest (3) is configurable by a sliding window.
8. A method according to any preceding claim, wherein the training and validation tracks are tracks the majority of which are considered normal.
9. A method according to any preceding claim, wherein the neural network is the GeoTrackNet network.
10. A computer program comprising software instructions which, when executed by a computer, implement a method according to any one of the preceding claims.