Threat estimation systems

Abstract

A threat estimation system 100 arranged to provide a first artificial neural network (ANN) 111 that operates in accordance with a first set of weight values, and an engagement planner 120. The threat

Description

Field of the Invention The present invention concerns threat estimation systems; More particularly, but not exclusively^ the invention concerns threat estimation systems that use artificial neural networks (ANNs) to perform threat estimation, methods of updating such threat estimation systems, and methods of training threat state prediction ANNs. Background of the Invention Threat state estimation systems are a key component of systems such as ground-based air defence systems, air-to-air products and weapon aiming. Such systems use sensor data to infer characteristics of a threat in a defence context, for example using radar data in an air-defence context to infer the class / type of a threat. The performance of such systems can be critical. Existing systems require updates to be deployed, to take account of new types of threat, and for general performance improvements. Existing systems rely on a model-based approach, for example using models based on Kalman filters to identify threats. Each threat model consists of a bespoke definition of equations, parameters and heuristics. These definitions need to be created and tested prior to deployment, which is timeconsuming and requires a high degree of specialist skill and knowledge from the engineer developing the definitions. As a consequence, the process of creating and deploying updates can be onerous and slow to provide. It would be desirable to be able to create and deploy updates more quickly and flexibly, with less input by specialist engineers required. The present invention seeks to mitigate some or all of the above-mentioned problems. Alternatively or additionally, the present invention seeks to provide improved threat estimation systems, improved methods of updating threat estimation systems, and improved methods of training threat state prediction ANNs. Summary of the Invention In accordance with a first embodiment of the invention there is provided a 5 threat estimation system comprising: a processor, and memory, wherein the threat estimation system is arranged, using the processor and memory, to provide: 10 a first artificial neural network, ANN, that operates in accordance with a first set of weight values; and an engagement planner; wherein the threat estimation system is arranged to: receive a source of sensor data; 15 estimate from the sensor data, using the first ANN, a threat result for a threat indicated by the sensor data; and determine from the threat result, using the engagement planner, a threat action to be performed; and wherein the threat estimation system is further arranged to: 20 receive an updated first set of weight values for the first ANN; and update the first ANN to operate in accordance with the updated first set of weight values. By using a first ANN, the functionality of the threat estimation system can be provided by ANN training on existing data, without requiring time-consuming 25 designing of bespoke algorithms by highly-skilled engineers. However, by having the first ANN then provide the threat result to an engagement planner, which then . determines the threat action to be performed, a modular system can be provided in which the disadvantages of having an ANN simply provide the complete operation of the threat estimation system can be avoided. In particular, due to their nature the 30 underlying design assumptions and basis of operation of ANN-based systems is obfuscated compared to bespoke algorithms, and their operation can be less predictable, particularly where they receive input dissimilar to that on which they have been trained. By having the output of the first ANN passed to an engagement planner, which then uses that output to determine the threat action to take, those disadvantages can be mitigated. In addition, output from other ANNs, or other non-ANN-based systems, can be used as well by the engagement planner to determine the threat action to take. 5 Further, additional training of the first ANN can be performed to take account of new types of threat, or on better and / or larger datasets to provide improved performance of the first ANN. This can be done remotely, for example by the providers of the threat estimation system, in order to determine an updated set of ' weight values for the first ANN, without the first ANN of the threat estimation system 10 itself being required. As no designing of bespoke algorithms is required, this can be less time-consuming to perform and not require such a high level of skills. The updated set of weight values can then be provided to the threat estimation system, and used to update the set of weight values of the first ANN, so the first operation of the first ANN is updated. This allows updates to be more quickly and flexibly provided. 15 The sensor data may be radar data. Alternatively or additionally, the sensor data may be sonar data, video data, any other suitable type of sensor data, or a combination thereof. The threat estimation system may be a naval threat estimation system, an aircraft threat estimation system, a land threat estimation system, or may be used to 20 evaluate threats in any other setting or combination of settings. The threat result, may be a single result or a set of results. The threat action may be a single action or a set of actions. The threat action may be to perform a threat response such as launching a missile or set of missiles, and / or may be the display of the threat result or results to an operator. The threat estimation system may perform 25 the threat action. The first ANN may be a recurrent neural network, RNN. Where the threat estimation system comprises a plurality of ANNs, some or all of them may be RNNs. The first ANN may be a threat classification ANN arranged to perform threat classification, such that the threat result estimated by the threat classification ANN is 30 a classification of the type of the threat indicated by the sensor data. The type of the threat may identify the type of missile, or vessel / aircraft / vehicle, or any other appropriate classification of the threat. The threat estimation system may be further arranged to: provide a novelty detection ANN that operates in accordance with a novelty detection set of weight values; determine from the threat result estimated by the threat classification ANN if the threat indicated by the sensor data is of a type known to the first ANN; and 5 wherein the engagement planner is arranged to determine the threat action to be performed using the determination by the novelty detection ANN. This can allow the threat estimation system to take account of the threat classification ANN providing a threat classification for a threat which it was not trained upon. The threat estimation system may be further arranged to: 10 receive an updated novelty detection set of weight values for the novelty ' detection ANN; and update the novelty detection ANN to operate in accordance with the updated novelty detection set of weight values. The novelty detection set of weight values for the novelty detection ANN may 15 be determined using a subset of the first set of weight values for the threat classification ANN. The updated novelty detection set of weight values may similarly be determined using a subset of the updated first set of weight values for the threat classification ANN. The threat classification ANN may comprise a layer with internal memory 20 states that operates in accordance with a subset of the first set of weight values, and wherein the novelty detection ANN comprises a layer with internal memory states that operates in accordance with the subset of the first set of weight values. The threat classification ANN and the novelty detection ANN may share a layer with internal memory states. The layers with internal memory states may be long-short-term- 25 memory (LSTM) layers. LSTM layers are commonly found in RNNs, which provide a learned set of features of the input, and in this way can provides a static representation of the threat. Alternatively, the layers with internal memory states may be gated recurrent units (GRUs), for example. By having the threat classification ANN and the novelty detection ANN use the same or related LSTM layers, the risk 30 that the novelty detection ANN might distinguish “novelties” based on threat characteristics different from those used by the threat classification ANN is mitigated, helping to avoid both false positives and negatives. The threat estimation system may be further arranged to store at least one predetermined set of default settings, and the engagement planner may be arranged, in the case that the novelty detection ANN determines that the threat indicated by the sensor data is not of a type known to the threat classification ANN, to determine the 5. , threat action to be performed using the default settings. This can mitigate problems that can arise with systems that utilise ANNs components, as the underlying design assumptions made during development of ANN-based systems are more obfuscated compared to those for existing systems based on bespoke algorithms, and generally the behaviour of systems under conditions where design assumptions have been 10 violated is less predictable and robust for ANN-based systems than for bespoke-designed algorithms. The threat estimation system may be further arranged to: provide a further ANN that operates in accordance with a further set of weight values; 15 estimate from the sensor data, using the further ANN, a further threat result for the threat indicated by the sensor data; and wherein the engagement planner is arranged to determine the threat action to be performed using the further threat result. The threat estimation system may be further arranged to: 20 receive an updated further set of weight values for the further ANN; and update the further ANN to operate.in accordance with the updated further set of weight values. The further ANN may be arranged to use the threat result estimated by the first ANN to estimate the further threat result from the sensor data. In particular, where the 25 first ANN is a threat classification ANN, the threat classification result may be used by the further ANN. The further ANN may be a threat state prediction ANN arranged to perform threat state prediction, such that the further threat result estimated by the threat state prediction ANN is a future position of the threat indicated by the sensor data; 30 The threat state prediction ANN may be arranged to take as input a flag indicating if a valid input is being provided, and may be arranged: when the flag indicates a valid input is provided, determine the position of the threat at a next time step using sensor data; and when the flag indicates a valid input is not provided, determine the position of ■ । • the threat at a next time step using the position of the threat determined by the threat state prediction ANN at the preceding time step. This can allow threat state prediction, which can be of particular importance, 5 to be improved, particularly for “long” prediction teams (for example more than five prediction steps into the future). The flag can be used during training, allowing the multiple training samples from a single trajectory, with different ratios of “observations” (for which the flag is set to indicate a valid input is being provided to the threat state prediction ANN), and “prediction” (for which the flag is set to indicate 10 a valid input is not being provided to the threat state prediction ANN, so the threat state prediction ANN is forced to use its own prediction): The flag can be also used during deployment, being set to indicate a valid input is being provided where an observation is available, and to indicate a valid input is not being provided where a prediction is required. This can provide an improvement over conventional 15 approaches, as during training the threat state prediction RNN is required to learn to predict the state for varying time-horizons, rather than just a single time-step ahead. The further ANN may be a reachable area prediction ANN arranged to perform reachable area prediction, such that the further threat result estimated by the reachable area prediction ANN is a reachable area of the threat indicated by the sensor 20 data. The further ANN may be a launch point prediction ANN arranged to perform launch point prediction, such that the further threat result estimated by the launch point prediction ANN is a launch point of the threat indicated by the sensor data. The threat estimation system may comprise any combination of one or more 25 ANNs, to provide any combination of one or more of the above functionalities. In accordance with a second embodiment of the invention there is provided a computer program product comprising machine-readable instructions arranged, when executed on a computing device comprising a processor and memory, to provide any 30 of the threat estimation systems described above. In accordance with a third embodiment of the invention there is provided a method of updating any of the threat estimation systems described above, comprising the steps of: training an ANN for a threat estimation system using sensor data and desired 5 threat results for threats indicated by the sensor data; determining the set of weight values in accordance with which the ANN operates; generating an updated set of weight values from the determined set of weight values; 10 providing the updated set of weight values to the threat estimation system; and updating an ANN of the threat estimation system to operate in accordance with the updated set of weight values. The updated set of weight values may just be set of weight values in accordance with which the ANN operates, or may be a subset or a modification 15 thereof. The updated set of weight values may be provided as a data file transmitted to the location of the threat estimation system, or on a storage medium, or in any other desired manner. In accordance with a fourth embodiment of the invention there is provided a 20 method of training a threat state prediction ANN for a threat estimation system as described above, comprising the steps of: _______ . — — •— .- -• -- - — •- ----- -- — - - “ . - — - receiving at least one sample of sensor data, wherein the sample comprises a set of positions of a threat at different times; generating a plurality of samples from the at least one sample, wherein each 25 generated sample comprises a subset of the set of positions of the threat at different times; and training the threat state prediction ANN using the generated plurality of samples. In this way, a threat state prediction ANN with improved performance can be 30 provided, as during training the ANN is required, to learn to predict the state, for varying time-horizons, rather than just a single time-step ahead as with conventional training methods. It will of course be appreciated that features described in relation to one aspect of the present invention may be incorporated into other aspects of the present invention. For example, the method of the invention may incorporate any of the features described with reference to the apparatus of the invention and vice versa. 5 ' • ■ Description of the Drawings Embodiments of the present invention will now be described by way of 10 example only with reference to the accompanying schematic drawings of which: Figure 1 is a schematic diagram of a threat estimation system according to a first embodiment of the invention; Figure 2 shows in more detail the threat classification RNN and novelty detection 15 RNN of the threat estimation system of Figure 1; Figure 3 is a schematic diagram of a training system for the threat estimation system of Figure 1; Figure 4 is a schematic diagram of the workflow of the training by the training system of Figure 3; 20 Figure 5 are graphs representing samples used in the training of the threat state prediction RNNofFigure1; . Figure 6 is a flowchart showing the deployment of an update to the threat estimation system of Figure 1; and Figure 7 is a schematic diagram of a computing device in accordance with an 25 embodiment of the invention. Detailed Description 30 A threat estimation system in accordance with a first embodiment of the invention is now described with reference to Figure 1 to 6. Figure 1 is a schematic diagram of the threat estimation system 100. The threat estimation system 100 comprises a plurality of RNNs, whose training is described in detail later below. The threat estimation system takes as input real-world radar data source 101, 5 i.e. “live” radar data from a radar system. In particular, radar track data from the real-world radar data source 101 is used, giving the latest observed 3D threat position and velocity, range and range rate. The radar track data from the real-world radar data source 101 is passed to a threat classification RNN 111. The threat classification RNN 111 generates an 10 estimated threat type for the threat, i.e. a classification of the type of the threat, based on the behaviour of the threat as shown by the radar track data. The threat type may for example be the type of missile that the threat is. The threat classification RNN 111 is shown in more detail in Figure 2. As will 15 be understood by the skilled person, as the threat classification RNN 111 is an RNN, it comprises a long-short term memory (LSTM) layer 130, which generates a latent space embedding, i.e. the learned set of features of the threat, that provides a static representation of the input radar track data (which is a variable length time-series) that is used by the threat classification RNN 111 to classify the threat type. The latent 20 space embedding output by the LSTM layer 130 is passed to a fully-connected layer 131, whose output is passed to a softmax layer 132, and then to a classifier layer 133, which provide the output of the threat classification RNN 111, i.e. the estimated threat type for the threat. In addition, the latent space embedding output by the LSTM layer 130 of the 25 threat classification RNN 111 is used by a novelty detection RNN 112, which is also shown in Figure 1. The latent space embedding is passed to a one-class support vector machine 140, which uses the latent space embedding to generate a Boolean “is novel?” flag. This flag indicates whether the threat is novel, i.e. dissimilar to types of threat considered during the training of the RNNs of the threat estimation system 100. 30 The use of the shared latent space embedding by both the threat classification RNN 111 and the novelty detection RNN 112 reduces the risk that could otherwise arise that a novelty detection algorithm might distinguish “novelties” based on a set of threat characteristics that do not include the characteristics to which the system performance is sensitive (and so don’t adequately indicate that the system is operating outside of the conditions it was designed for), and / or include characteristics that are irrelevant to system performance (and so provide false positives of “novelties”). 5 The threat classification generated by the threat classification RNN 111 is also passed to a threat state prediction RNN 113, a reachable area prediction RNN 114, and a launch point prediction RNN 115, each of which is also passed the radar track data from the real-world radar data source 101. The threat state prediction RNN 113 generates an estimated future position for the threat, i.e. a 3D trajectory for the threat 10 between the current time and predicted impact time. The reachable area prediction RNN 114 generates an estimated reachable area for the threat, i.e. a 2D polygon representing the area on the surface the threat can reach. The launch point prediction RNN 115 generates an estimated launch point for the threat, i.e. the 3D position from which the threat was launched. 15 . The threat type for the threat generated by the threat classification RNN 111, the “is novel?” flag generated by the novelty detection RNN 112, and the future position for the threat generated by the threat state prediction RNN 113, are each passed to an engagement planner 120. The engagement planner 120 uses these inputs 20 to generate appropriate engagement instructions, i.e. parameters used to control the engagement with the threat, which are passed to a missile launcher 121. .While the ___________ ________ ——-------—~~ r' . • • engagement planner 120 uses outputs from the various RNNs, it is not itself an RNN, but instead uses a bespoke set of logical rules, optimization routines and lookup tables to generate the engagement instructions. 25 The engagement instructions may include an allocation matrix of specific defender missiles to specific threats, firing times / windows for missile, estimated “flyout” times for defender missiles, desired turnover angles for missile soft-launch, or any other appropriate parameters. The engagement planner 120 can also generate sensor priorities that are passed back to the real-world radar data source 101 to control 30 the radar, in particular a prioritized list of threats for the radar to maintain tracks on, and / or improve the resolution of to support the engagement. In particular, the “is novel?” flag generated by the novelty detection RNN 112 is used by the engagement planner 120 to determine whether it is appropriate to use the outputs from the RNNs when generating the engagement instructions, and in particular, in the event that the threat has been flagged as novel by the novelty detection algorithm, the engagement planner 120 uses default values for the engagement instructions, rather than perform threat-specific optimization based on the 5 output of the RNNs. The reason for this is that the flag indicates that the threat is dissimilar to the types of threat considered during the training of the RNNs, and so they are operating outside of the conditions for which they were trained. This is particularly important for systems utilizing ANN components, as the underlying design assumptions made during development of ANN-based systems are more 10 obfuscated compared to those for existing systems based on bespoke algorithms, and generally the behaviour of systems under conditions where design assumptions have been violated is less predictable and robust for ANN-based systems than for bespoke-designed algorithms. The reachable area for the threat generated by the reachable area prediction 15 RNN 114, and the launch point for the threat generated by the launch point prediction RNN 115, are passed to an operator display 122 to be displayed to an operator of the threat estimation system 100. It will be appreciated that the outputs generated by the various RNNs of the threat estimation system 100 could be used in various alternative or additional ways, 20 for example with alternative or additional outputs being used to generate appropriate engagement instructions, and alternative or additional outputs being displayed to an operator. Figure 3 is a schematic diagram of a training system for the threat estimation 25 system 100. The training system 200 takes as input real-world data 201 of radar data, and modelling and simulations data 202, and are used to generate a training dataset 203 of threat radar tracks. The training dataset 203 can then be used to train a set of RNNs 204 corresponding to the RNNs of the threat estimation system 100, in particular, a novelty detection RNN 205, a threat classification RNN 206, a threat 30 state prediction RNN 207, a reachable area prediction RNN 208 and a launch point prediction RNN 209. Figure 4 is a schematic diagram of the workflow of the training by the training system of Figure 3. The workflow 300 begins with customer inputs 301 being provided, consisting of real-world data gathering 310, and the providing of threat models 311, sensor models 312, and scenarios 313. The threat models 311, sensor models 312 and scenarios 313 are used for synthetic data generation 320; this synthetic data may be provided by the customer (i.e. user of the threat estimation 5 system 100), may be generated as part,of the training workflow 330, or a combination. The real-world data 310 and the generated synthetic data 320 are used to generate threat datasets 331. Design work 332 is undertaken to develop an algorithm architecture for the threat estimation system 100, and the threat datasets 331 are then used for ML algorithm training 333 of the RNNs of the training system 200, based on 10 the algorithm architecture design 332, to generate learned parameters 334, using ML training methods which will be well known to the skilled person. Algorithm evaluation 335 is performed, i.e. evaluation of the output of the algorithms using the learned parameters, and the results of this can be fed back into the ML algorithm training 333, and also synthetic data generation 320, to optimise the results of the 15 training. Once a desired amount of training has been performed, the learned parameters 334 can be used to perform system-level integration and testing 336, and if such testing is successfully passed, they can be deployed as an algorithm suite 337, i.e.. used in live customer systems. The process deployment of the is described in more 20 details below. The operation of the threat state prediction RNN 113, and the training of the corresponding threat state prediction RNN 207, is now described in more detail with reference Figure 5. In general, threat state prediction is of particular importance to the 25 planning and execution of an engagement, meaning that its performance can heavily influence the overall system performance. Threat state prediction is used to support firing policy decisions, effector.trajectory planning, sensor scheduling, and many other system functions. The threat state prediction RNN 113, by virtue of its particular implementation, provides improved accuracy of state estimates, particularly for 30 “long” prediction times (more than five prediction steps into the future). Threat state prediction generally requires predicting the value of a single variable S over a series of time steps. Conventionally, when training the true value of the variable at time k, Sk, is used as the RNN input, and the target value, Tk is chosen to be the value of 5 at the next time-step, i.e. Sk+i- During deployment, in order to predict multiple steps into the future, the RNN will be “primed” with the observed states so far (So, Si,..., Sk). Its own predictions, Zk, are then fed back into the RNN with a single step delay, repeatedly, until the desired prediction horizon is reached. 5 In contrast, with the threat state prediction RNN 113 a concept of “observation phase” and “prediction phase” is used, and an “inputs valid?” flag is included in the input to the threat state prediction RNN 113. This flag is used during training, evaluation and deployment of the threat state prediction RNN 113. In addition, the training data for the threat state prediction RNN 113 (or rather the corresponding 10 threat state prediction RNN 207) is modified to derive multiple samples from a single trajectory, where each derived sample has a different ratio of “observations” to “predictions”, as. shown by the samples 401, 402 and 403 of Figure 5. In other Words, from a single trajectory of k time steps, rather than using this to train for a desired result for time k based using the set of observations for all time steps before k, desired 15 results at multiple different time steps n can be trained for (when n may be equal to or less than k), based on different sets of observations prior to n, and not only the set of observations for all time steps before n. During the observation phase, the input to the threat state prediction RNN 113 is the true state Sk, as in conventional systems, and the “inputs valid?” flag is set to be 20 true. Dining the prediction phase, the RNN input is set to null, the target value for the RNN is chosen to be &(r^er than &+ / ),. and the “inputs valid?” flag--18 set-to be -false. In order to predict multiple steps into the future, the threat state prediction RNN 113 is again “primed” with the observed states so far ^o, Si,..., Sk), and then iterated (with null inputs) until the desired prediction horizon is reached. However, this 25 provides a performance improvement over the conventional approach, because during training, the RNN is required to learn to predict the state for varying time-horizons, rather than just a single time-step ahead. The deployment of an update to the threat estimation system 100 is now 30 described with reference to the. flowchart of Figure 6. First, the RNNs of the system are trained on new real-world data (step 501), for example as described above with reference to Figures 3 to 5. This.results in a set of learned parameters 334, which are weight values extracted from the various trained RNNs (step 502), i.e. weight values for their nodes. The extracted weight values are the used to generated an updated set of weight values for the RNNs (step 503). The updated set of weight values may simply be all of the extracted weight values, or a subset of them and / or a modification thereof, as desired. 5 The updated set of weight values is then provided to the threat estimation system 100 (step 504), e.g. as a data file transmitted to the location of the threat estimation system 100, or provided on a storage medium. The updated set of weight values are then used to update the weight values of one or more of the various RNNs of the threat estimation system 100 (step 505), i.e. the threat classification RNN 111, 10 novelty detection RNN 112, threat state prediction RNN 113, reachable area prediction RNN 114, and / or a launch point prediction RNN 115 as appropriate. In this way, the various RNNs of the threat estimation system 100 are updated to operate based on the training of the corresponding RNNs of the training system 200. ' 15 . ■■ Embodiments of the invention include the methods described above performed on a computing device, such as the computing device 800 shown in Figure 7. The Computing device 800 comprises a data interface 801, thought which data can be sent and received, for example over a network or using a data storage medium such as a 20 CD or USB storage device. The computing device 800 further comprises a processor 802 in communication with the data interface 801, and memory 803 in______z . ..... communication with the processor 802. In this way, the computing device 800 can receive data via the data interface 801, and the processor 802 can store the received data in the memory 803, and process it so as to perform the methods described herein. 25 Each device, module, component, machine or function as described in relation to any of the embodiments described herein may comprise a processor and / or processing system or may be comprised in apparatus comprising a processor and / or processing system. One or more aspects of the embodiments described herein comprise processes performed by apparatus. In some embodiments, the apparatus comprises one 30 or more processing systems or processors configured to carry out these processes. In this regard, embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware). Embodiments also extend to computer programs, particularly computer programs on or in a carrier, adapted for putting the above described embodiments into practice. The program may be in the form of non-transitory source code, object code, or in any other non-transitory form suitable for use in the implementation of the 5 processes. The carrier may be any entity or device capable of carrying the program, such as a RAM, a ROM, or an optical memory device, etc. While the present invention has been described and illustrated with reference to particular embodiments, it will be appreciated by those of ordinary skill in the art 10 that the invention lends itself to many different variations not specifically illustrated herein. By way of example only, certain possible variations will now be described. Where in the foregoing description, integers or elements are mentioned which have known, obvious or foreseeable equivalents, then such equivalents are herein incorporated as if individually set forth. Reference should be made to the claims for 15 determining the true scope of the present invention, which should be construed so as to encompass any such equivalents. It will also be appreciated by the reader that integers or features of the invention that are described as preferable, advantageous, convenient or the like are optional and do not limit the scope of the independent : claims. Moreover, it is to be understood that such optional integers or features, whilst 20 of possible benefit in some embodiments of the invention, may not be desirable, and may therefore be absent, .in other embodiments_______ ______ _____

Claims

1. A threat estimation system comprising:a processor, and5 memory,wherein the threat estimation system is arranged, using the processor and memory, to provide:a first artificial neural network, ANN, that operates in accordance with a first set of weight values; and10 an engagement planner;wherein the threat estimation system is arranged to: -receive a source of sensor data;estimate from the sensor data, using the first ANN, a threat result for a threat indicated by the sensor data; and15 determine from the threat result, using the engagement planner, a threat actionto be performed;and wherein the threat estimation system is further arranged to:. receive an updated first set of weight values for the first ANN; andupdate the first ANN to operate in accordance with the updated first set of20 weight values.

2. A threat estimation system as claimed in claim 1, wherein the first ANN is a ; recurrent neural network, RNN.25 3. A threat estimation system as claimed in claim 1 or 2, wherein the sensor data isradar data.

4. A threat estimation system as claimed in any preceding claim, wherein the first ANN is a threat classification ANN arranged to perform threat classification, such that30 the threat result estimated by the threat classification ANN is a Classification of the type of the threat indicated by the sensor data.

5. A threat estimation system as claimed in claim 4, further arranged to:provide a novelty detection ANN that operates in accordance with a novelty detection set of weight values;determine from the threat result estimated by the threat classification ANN if the threat indicated by the sensor data is of a type known to the first ANN; and5 wherein the engagement planner is arranged to determine the threat action to be performed using the determination by the novelty detection ANN.

6. A threat estimation system as claimed in claim 5, wherein the threat estimation system is further arranged to:10 receive an updated novelty detection set of weight values for the noveltydetection ANN; andupdate the novelty detection ANN to operate in accordance with the updated novelty detection set of weight values.15 7. A threat estimation system as claimed in claim 5 or 6, wherein the novelty detectionset of weight values for the novelty detection ANN are determined using a subset of the first set of weight values for the threat classification ANN. '8. A threat estimation system as claimed in claim 7, wherein the threat classification 20 ANN comprises a layer with internal memory states that operates in accordance with a subset of the first set of weight values, and wherein the novelty detection ANN. comprises a layer with internal memory states that operates in accordance with the subset of the first set of weight values.25 9. A threat estimation system as claimed in any of claims 5 to 8, wherein the threatclassification ANN and the novelty detection ANN share a layer with internal memory states.

10. A threat estimation system as claimed in any of claims 5 to 9, further arranged to 30 store at least one predetermined set of default settings, and wherein the engagement - ■ planner is arranged, in the case that the novelty detection ANN-determines that the -threat indicated by the sensor data, is not of a type known to die threat classification ANN, to determine the threat action to be performed using the default settings.

11. A threat estimation system as claimed in any preceding claim, further arranged to: provide a further ANN that operates in accordance with a further set of weight values;5 . estimate from the sensor data, using the further ANN, a further threat result forthe threat indicated by the sensor data; and wherein the engagement planner is arranged to determine the threat action to be performed using the further threat result.10 12. A threat estimation system as claimed in claim 11, further arranged to: ;receive an updated further set of weight values for the further ANN; and update the further ANN to operate in accordance with the updated further set of weight values.15 13. A threat estimation system as claimed in claim 11 or 12, wherein the further ANNis arranged to use the threat result estimated by the first ANN to estimate the further threat result from the sensor data.

14. A threat estimation system as claimed in any of claims 11 to 13, wherein the20 further ANN is a threat state prediction ANN arranged to perform threat state prediction, such that the further threat result estimated by the threat state prediction _ . ANN is a future position of the threat indicated by the sensor data.

15. A threat estimation system as claimed in claim 14, wherein the threat state25 prediction ANN is arranged to take as input a flag indicating if a valid input is being provided, and is arranged:when the flag indicates a valid input is provided, determine the position of the threat at a next time step using sensor data; and ~when the flag indicates a valid input is not provided, determine the position of 30 the threat at a next time step using the position of the threat determined by the threat state prediction ANN at the preceding time step.

16. A threat estimation system as claimed in any of claims 1 l ;to 4¾ Wherein the farther ANN is a reachable area prediction ANN arranged to perform reachable area prediction, such that the further threat result estimated by the reachable area prediction ANN is a reachable area of the threat indicated by the sensor data..5 .

17. A threat estimation system as claimed in any of claims 11 to 13, wherein the further ANN is a launch point prediction ANN arranged to perform launch point prediction, such that the further threat result estimated by the launch point prediction ANN is a launch point of the threat indicated by the sensor data.1018. A computer program product comprising machine-readable instructions arranged, when executed on a computing device comprising a processor and memory, to provide the threat estimation system as claimed in any preceding claim.15 19. A method of updating a threat estimation system as claimed in any of claims 1 to17, comprising the steps of:training an ANN for a threat estimation system using sensor data and desired threat results for threats indicated by the sensor data:;determining the set of weight values in accordance with which the ANN20 operates;generating an updated set of weight values from the determined set of weight values;providing the updated set of weight values to the threat estimation system; and updating an ANN of the threat estimation system to operate in accordance25 with the updated set of weight values.

20. A method of training a threat state prediction ANN for a threat estimation system as claimed in claim 14, comprising the steps of:receiving at least one sample of sensor data, wherein the sample comprises a ~ 30 set of positions of a threat at different times;generating a plurality of samples from the at least one sample, wherein each generated sample comprises a subset of the set of positions of the threat at differenttraining the threat state prediction ANN using the generated plurality of samples.

Images