Computer system and method for automated detection of materials using hyperspectral imaging and artificial intelligence

Abstract

Systems, apparatus and methods for detecting materials in a target area are provided herein. The system includes an imaging device, and a computing device. The imaging device is configured to collect input data including hyperspectral image data of the target area. The computing device is configured to receive the input data. The computing device is further configured to analyze the input data using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output. At least a portion of the analyzing includes pattern of life algorithms. The computing device is further configured to provide the output to a user device.

Description

COMPUTER SYSTEM AND METHOD FOR AUTOMATED DETECTION OF MATERIALS USING HYPERSPECTRAL IMAGING AND ARTIFICIAL INTELLIGENCETechnical Field

[0001] The following relates generally to detection of materials, and more particularly to systems and methods for automated detection of materials using hyperspectral imaging (HSI) and artificial intelligence (Al).Introduction

[0002] Detection of materials, objects, or structures located in an area has various potential applications, such as in geophysical surveys, mineral exploration, environmental monitoring, or reconnaissance operations. In some cases, methods of detection may utilize imaging techniques.

[0003] One imaging technique is electromagnetic imaging (EMI), which uses electromagnetic waves to create images of objects or structures. This technique employs principles of electromagnetism to obtain information about the properties of materials being examined. For example, electromagnetic scanning, involves scanning an area with electromagnetic sensors to detect and map variations in electromagnetic fields. This information could then be used to generate an image of the scanned area. However, EMI equipment and surveys can be expensive to purchase, operate, and maintain. EMI can also require skilled personnel for the imaging and analysis aspects. Further, EMI data is generally complex and can often be misinterpreted.

[0004] Another technique, ground-penetrating radar (GPR), uses radar pulses to image the Earth’s subsurface. GPR is commonly used in various studies to detect objects, voids, or structures beneath the ground without excavating the area. Like EMI though, GPR equipment and surveys can be expensive, and usually require expertise to carry out. GPR data is also quite complex and prone to misinterpretation. GPR signals can also be attenuated or absorbed by some materials, thereby reducing the effectiveness of GPR in certain conditions. This, in combination with complex environments, can lead to poor resolution of images obtained via GPR.

[0005] Furthermore, due to the complexity of data obtained by existing imaging techniques, classical techniques for distinguishing between different objects, substances or materials can be unreliable, cumbersome, and heavily reliant on human judgement.

[0006] For example, current algorithms can only provide coarse estimates of where objects or anomalies of interest may be located. This can lead to several repetitions being required to obtain meaningful data, which can be costly and time-consuming.

[0007] Accordingly, there is a need for an improved system and method for detection of materials that overcomes at least some of the disadvantages of existing systems and methods.

[0008] This background information is provided to reveal information believed by the applicant to be of possible relevance to the present disclosure. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present disclosure.Summary

[0009] A system for detecting materials in a target area is provided. The system includes an imaging device, and a computing device. The imaging device is configured to collect input data including hyperspectral image data of the target area. The computing device is configured to receive the input data. The computing device is further configured to analyze the input data using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output. At least a portion of the analyzing includes pattern of life algorithms. The computing device is further configured to provide the output to a user device.

[0010] In an embodiment, the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

[0011] In an embodiment, the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

[0012] In an embodiment, the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

[0013] In an embodiment, the annotated data describes at least one of: a detected signature; and a detected combination of signatures.

[0014] In an embodiment, the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

[0015] In an embodiment, the imaging device and the computing device are located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

[0016] In an embodiment, the input data further includes electromagnetic image data of the target area.

[0017] In an embodiment, the input data further includes ground-penetrating radar data of the target area.

[0018] In an embodiment, the hyperspectral signature detection model is an anomaly detection model.

[0019] In an embodiment, the anomaly detection model is an unsupervised deep learning model.

[0020] In an embodiment, the unsupervised deep learning model includes an Adversarially Learned Anomaly Detection architecture.

[0021] In an embodiment, the unsupervised deep learning model is a reconstruction-based anomaly detection architecture that evaluates how far a sample is from its reconstruction by a generative adversarial network from an encoded latent from the generative adversarial network.

[0022] In an embodiment, the anomaly detection model receives an abundance map from an unmixing algorithm as input.

[0023] In an embodiment, the anomaly detection model receives a hyperspectral cube including all spectral bands as input.

[0024] In an embodiment, the anomaly detection model receives a hyperspectral cub including only some spectral bands determined by a dimensionality reduction algorithm

[0025] In an embodiment, the anomaly detection model is a supervised learning segmentation model configured to assign a probability of anomaly to each pixel or anomaly type out of a predefined taxonomy to each pixel.

[0026] In an embodiment, the supervised learning segmentation model receives an abundance map as input.

[0027] In an embodiment, the supervised learning segmentation model receives a hyperspectral cube as input.

[0028] A method of detecting materials in a target area is provided. The includes executing via a computer system comprising at least one processor: receiving input data including hyperspectral image data of the target area; analyzing the input data using a hyperspectral signature detection model trained to detect at least one spectral signature indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms; and providing the output to a user device.

[0029] In an embodiment, the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

[0030] In an embodiment, the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

[0031] In an embodiment, the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

[0032] In an embodiment, the generated annotated data describes at least one of: a detected signature; and a detected combination of signatures.

[0033] In an embodiment, the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

[0034] In an embodiment, the computing system is located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

[0035] In an embodiment, the input data further includes electromagnetic image data of the target area.

[0036] In an embodiment, the input data further includes ground-penetrating radar data of the target area.

[0037] An apparatus for detecting materials in a target area is provided. The apparatus includes a network interface, a processor, and a non-transitory computer readable memory. The non-transitory computer readable memory has stored thereon instructions which, when executed by the processor, configure the apparatus to: receive input data including hyperspectral image data of the target area; analyze the input data using a hyperspectral signature detection model trained to detect at least one spectral signature indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms; and provide the output to a user device.

[0038] In an embodiment, the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

[0039] In an embodiment, the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

[0040] In an embodiment, the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

[0041] In an embodiment, the generated annotated data describes at least one of: a detected signature; and a detected combination of signatures.

[0042] In an embodiment, the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

[0043] In an embodiment, the computing system is located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

[0044] In an embodiment, the input data further includes electromagnetic image data of the target area.

[0045] In an embodiment, the input data further includes ground-penetrating radar data of the target area.

[0046] Other aspects and features will become apparent, to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0047] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0048] Figure 1A is a block diagram of an example system for detecting materials in a target area, according to an embodiment;

[0049] Figure 1 B is a block diagram of an example computing device from the system of Figure 1A, according to an embodiment;

[0050] Figure 1 C is a schematic of an example target area from the system of Figure 1A, according to an embodiment;

[0051] Figure 2 is a flowchart of an example method of detecting materials in a target area, according to an embodiment;

[0052] Figure 3 is a block diagram of an example apparatus for detecting materials in a target area, according to an embodiment;

[0053] Figure 4 is a block diagram of an example electronic device, according to an embodiment;

[0054] Figure 5 is a schematic diagram of an example network system, according to an embodiment;

[0055] Figure 6 is an example of an image-based graphical user interface (GUI) output of a system for detecting materials in a target area, according to an embodiment;

[0056] Figure 7 is an example annotated (output) image of a target area generated by the system of Figure 1 , according to an embodiment.Detailed Description

[0057] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0058] As used herein, the term “about” should be read as including variation from the nominal value, for example, a + / -10% variation from the nominal value. It is to be understood that such a variation is always included in a given value provided herein, whether or not it is specifically referred to.

[0059] One or more systems described herein may be implemented in computer programs executing on programmable computers, each comprising at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, and without limitation, the programmable computer may be a programmable logic unit, a mainframe computer, server, and personal computer, cloud-based program or system, laptop, personal data assistance, cellular telephone, smartphone, or tablet device.

[0060] Each program is preferably implemented in a high-level procedural or object-oriented programming and / or scripting language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Eachsuch computer program is preferably stored on a storage media or a device readable by a general or special purpose programmable computer for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein.

[0061] A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present disclosure.

[0062] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0063] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device I article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0064] The following relates generally to detection of materials, and more particularly to systems and methods for automated detection of materials using hyperspectral imaging (HSI) and artificial intelligence (Al).

[0065] To address the various limitations and challenges present in existing detection methods, embodiments disclosed herein describe techniques for automated detection of materials in a target area using hyperspectral imaging (HSI) and artificial intelligence (Al). The input data is then analyzed using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data. The hyperspectral signature detection model includes a machine learning based model configured to receive the input data and generate annotated data as an output. At least aportion of the analyzing is based on pattern of life algorithms. The output is then provided to a user device for further analysis or use.

[0066] An HSI pattern-of-materials (or pattern of life) approach as provided herein is a novel method to find signs of concealed threats in a way that is both revolutionary and complementary to the existing methods of electromagnetic imaging (EMI) and ground-penetrating radar (GPR). Using spatial-spectral patterns allows the system to highlight unusual placements of heavily occluded or visually unrecognizable objects, providing better performance and robustness than RGB object detection. Specifically, spatial-spectral pattern identification underlying the techniques of the present disclosure may be used to detect combinations of materials that if found together may be unusual, hazardous or dangerous. Spatial-spectral patterns may also be used to detect spectrally unexpected materials, such as, for example, a fake rock or a chemical residue, as well as where the earth is undisturbed or has been disturbed. Including traditional spectral methods and considering fusion with other data types offers an enhanced capability. Techniques disclosed herein create new knowledge in the capabilities of a compact shortwave infrared (SWIR) camera and corresponding embedded system processing.

[0067] Further, low size, weight, and power (SWaP) GPUs are just beginning to be used for such applications. By designing a solution around an HSI SWIR camera and a low-power GPU for embedded processing, techniques disclosed herein explicitly support deployment of small, light hyperspectral SWIR payloads on a small unmanned aerial system (UAS) or telescopic pole. The latest research in low-power embedded systems shows successful implementations of Al algorithms on edge devices, from cell phones to autonomous vehicles.

[0068] In an embodiment of the present disclosure, there is shown and described a system for detecting materials in a target area.

[0069] Referring now to Figure 1A, shown therein is a system 100 for detecting materials in a target area 102, according to an embodiment of the present disclosure.

[0070] The system 100 includes a transmitting platform 104 and a receiving platform 106.

[0071] The transmitting platform 104 includes an imaging device 105, a computing device 110, and a transmitter 115.

[0072] In various embodiments, the transmitting platform 104 may be deployed on a low-cost unmanned aerial system (UAS) or telescopic pole with size, weight, and power (SWaP) constraints.

[0073] In various embodiments, such as where the transmitting platform 104 is a UAS or similar device, the transmitting platform 104 may further include an autonomous operation subsystem 140 to enable the transmitting platform 104 to autonomously traverse various landscapes or locations.

[0074] The imaging device 105 is configured to collect input data 118 including hyperspectral image data 122 of the target area 102.

[0075] Hyperspectral image data refers to radiation from the sun that has undergone reflection from various surfaces and is then collected by a sensor. The radiation captured and interpreted by hyperspectral sensors often lies within the visible (VIS) to short wavelength infrared (SWIR) range, which is 400nm-2500nm. This radiation may be represented as a reflectance spectrum, which allows for identification of material types due to the uniqueness of these various spectra.

[0076] Due to the very fine spectral resolution of hyperspectral sensors, images created using a hyperspectral dataset are composed of hundreds of contiguous spectral bands. These images are very detailed and different surfaces are much easier to distinguish as opposed to more traditional color imaging with wider spectral inputs such as regular RGB cameras.

[0077] Hyperspectral data is represented in the form of a data cube. There are a variety of different types of data cubes, depending on the units and what data is being represented. Irradiance cubes contain hyperspectral data in the form of power per unit area per wavelength and contain information about the light source present when the image was taken, such the sun or halogen light bulbs. Reflectance is a unitless representation of the amount of light reflected, regardless of the light source or any atmospheric interference.

[0078] In various embodiments, hyperspectral image data 122 may include shortwave infrared (SWIR) image data.

[0079] The computing device 110 is configured to receive the input data 118.

[0080] In various embodiments, the computing device 110 may be located in close proximity to the imaging device 105, such that the imaging device 105 and computing device 110 may be connected via a wired connection.

[0081] In other embodiments, the imaging device 105 and computing device 110 may be located far apart from one another, such that the imaging device 105 may wirelessly transmit the input data 118 to the computing device 110.

[0082] Similarly, and in some embodiments, the imaging device 105 may provide data to another device (e.g., an intermediary device), after which it is provided to the computing device 110.

[0083] The computing device 110 is further configured to analyze the input data 118 using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data indicative of a potential threat.

[0084] The hyperspectral signature detection model includes a machine learning based model configured to receive the input data 118 and generate annotated data as an output 124. At least a portion of the analyzing is performed using pattern of life (or pattern of materials) algorithms.

[0085] The computing device 110 is further configured to provide the output 124 to a transmitter 115, which transmits the output 124 to the receiving platform 106.

[0086] Output 124 may be transmitted as wireless signal 112, which may include various signal types such as cellular signals, Wi-Fi™, satellite signals, or Bluetooth®.

[0087] The receiving platform 106 includes a receiver 120, a processing device 125, and a user device 130.

[0088] While in some embodiments the receiving platform 106 may be a stationary structure (e.g., a base or lab), in other embodiments the receiving platform 104 may be mobile (e.g., deployed on a vehicle).

[0089] In various embodiments, such as where the receiving platform 106 is a vehicle or similar mobile machine, the receiving platform 106 may further include a control system 135 to allow for the receiving platform 106 to be remotely controlled.

[0090] The wireless signal 112 may be received by the receiver 120.

[0091] The receiver 120 provides the received data 126 to the processing device 125.

[0092] The processing device 125 then processes the received data 126 to generate and provide display data 114 to the user device 130.

[0093] In various embodiments, the user device 130 may be a device that is supervised by a human, in other cases the user device 130 may be an unsupervised system (e.g., an artificial intelligence system) capable of analyzing the output without human intervention.

[0094] The user device 130 may include a display device and an output device, among other sub-devices.

[0095] A display device may include any type of device for presenting visual information. For example, a display device may be a handheld electronic device, a computer monitor, a flat-screen display, a projector or a display panel. Output device may include any type of device for presenting a hard copy of information, such as a printer for example. An output device may also include other types of output devices such as speakers, for example.

[0096] In various embodiments, such as where the receiving platform 106 is a vehicle or similar mobile machine, the processing device 125 may further generate control data 116 for the control system 135 to allow for the receiving platform 106 to be remotely controlled.

[0097] Advantageously, embodiments disclosed herein improve the accuracy of detection of materials. Techniques of the present disclosure may be used to significantly enhance the accuracy of material detection by leveraging hyperspectral imagery, advanced machine learning algorithms, and pattern of life (or pattern of materials) algorithms.

[0098] Referring now to Figure 1 B, shown therein is an example expanded view of the computing device 110 of Figure 1A, according to an embodiment.

[0099] The computing device 110 includes a data storage device 155, a receiving module 160, and an analyzing module 165. The receiving module 160 and analyzing module 165 may be executed by one or more processors in communication with the data storage device 155.

[0100] The receiving module 160 receives input data 118 including hyperspectral image data 122 of a target area 102 (not shown) from a data storage device 155.

[0101] In some embodiments, the computing device 110 may be a system on a chip (SoC), such that all required components for data storage and processing are located on a single integrated circuit.

[0102] In other embodiments, the data storage device 155 may be located on a separate chip from the processing components of the computing device 110.

[0103] The receiving module 160 then provides the input data 118 to an analyzing module 165.

[0104] The analyzing module 165 analyzes the input data 118 using a hyperspectral signature detection model trained to detect at least one spectral signature indicative of a potential threat.

[0105] The hyperspectral signature detection model includes a machine learning based model configured to receive the input data 118 and generate annotated data as an output 124. At least a portion of the analyzing includes using a hyperspectral image anomaly detector 128. The anomaly detector 128 uses pattern of life (or pattern of materials) algorithms.

[0106] In training, the anomaly detector 128 learns the normal “pattern of materials” including (i) types of materials (e.g., anomalous materials), and (ii) spatial layout of materials (e.g., anomalous combinations).

[0107] The anomaly detector 128 is trained and configured to detect areas in the hyperspectral image that deviate from the above normal pattern of materials. Suchdeviation may be taken to indicate a threat or hazard (e.g., explosive device present) present in the environment captured in the field of view of the image.

[0108] The analyzing module 165 then provides the output 124 to the data storage device 155 for further use by the computing device 110.

[0109] In some embodiments, the anomaly detector 128 identifies hyperspectral image data that does not fit the distribution of normal data (i.e. , normal patterns of life or materials).

[0110] In some embodiments, the anomaly detector 128 uses deep, unsupervised learning for anomaly detection.

[0111] In some embodiments, classical detection algorithms such as the Adaptive Cosine Estimator (ACE) may also be used to compare the image pixel derived spectra against a-priori known spectra to identify specific materials.

[0112] In an embodiment, the anomaly detector 128 uses an unsupervised deep learning model.

[0113] In a particular embodiment, the model uses the Adversarially Learned Anomaly Detector (ALAD) architecture by Zenati et al. ALAD is a reconstruction-based anomaly detection architecture that evaluates how far a sample is from its reconstruction by a generative adversarial network (GAN) from an encoded latent, also from the GAN.

[0114] In some embodiments, the input to the model may be an abundance map from an unmixing (classical or AI / ML) algorithm. In some embodiments, the input to the model may be a hyperspectral cube with all spectral bands. In some embodiments, the input to the model may be a hyperspectral cube with fewer spectral bands. The spectral bands may be obtained from a dimensionality reduction algorithm like PCA, autoencoders, or the like.

[0115] At test time or during operational use, the model outputs an anomaly score. The anomaly score quantifies the dissimilarity between feature activations in the discriminator network for the original and reconstruction samples. Note that the anomaly score may also or instead be computed between the input and reconstructed input (as opposed to the feature activations of the discriminator). Several functions (metric, non-metric, etc.) may be used to model the anomaly score. Additionally, an auxiliary classification model pre-trained on data from the same domain may also be used to map the input and reconstructed input to a latent space where their similarity can be measured (i.e. , feature matching or perceptual loss).

[0116] In another embodiment, the anomaly detector 128 may use a supervised learning segmentation model. The model receives an abundance map or hyperspectral cube as input and assigns a probability of anomaly (or anomaly type out of a predefined taxonomy or set of class labels) to each pixel in the input. The model may be trained using a supervised learning technique using a dataset of annotated abundance maps or hyperspectral cubes.

[0117] In some embodiments, the anomaly detector 128 uses adversarially learned anomaly detection (ALAD). ALAD is a reconstruction-based anomaly detection technique that evaluates how far a sample is from its reconstruction by a generative adversarial network (GAN). Normal samples should be accurately reconstructed whereas anomalous samples will likely be poorly reconstructed.

[0118] The below table lists examples of common object detection metrics implemented by the system in an embodiment. The listed metrics are used to evaluate the performance of the anomaly detection method. Although these metrics are not directly used to train the model, they are important in assessing its accuracy and effectiveness. Specifically, these metrics can help determine an optimal threshold for the anomaly score, which differentiates normal from anomalous samples. A higher threshold will result in fewer anomalies being detected but may miss some true anomalies. Conversely, a lower threshold will detect more anomalies but may also incorrectly label normal samples as anomalous. By analyzing these metrics, the threshold can be adjusted to balance the trade-off between false positives and false negatives, thus refining the anomaly detection process.

[0119] Referring now to Figure 1 C, shown therein is example expanded view of the target area 102 of Figure 1A, according to an embodiment.

[0120] In the case of Figure 1 C, target area 102 includes objects 175, 180 and 185. While only three example objects are depicted, it will be understood by those of skill in the art that this is not to be taken as a limitation.

[0121] Object 180 is an anomalous combination of three materials 181 , 182, 183. The presence of the three materials 181 , 182 and 183 in close proximity (e.g., within a proximity threshold of one another) is anomalous (or suspicious). It is possible that each of the materials 181 , 182 and 183 on their own may not be anomalous and that only the combined presence in proximity is anomalous. Such an anomalous combination 180 may include a plurality of materials, objects, chemicals, or ingredients indicative of a potential hazard (e.g., an explosive device).

[0122] The analyzing module 165 is configured to receive hyperspectral image data of target area 102 as input and detect and flag object 180 in the hyperspectral data. The analyzing module 165 may flag the component materials 181 , 182 and 183 individually or the collective object 180. Flagging may include annotating a hyperspectral or optical image with a bounding box or the like enclosing the detected object.

[0123] Object 175 is a material whose presence in the image is anomalous or suspicious (“anomalous material”). An example of an anomalous material may be a fake plant or disturbed earth. Such materials may be indicative of a hazard, such as an explosive device.

[0124] The analyzing module 165 is configured to receive hyperspectral image data of target area 102 as input and detect and flag object 175 in the hyperspectral data. The analyzing module 165 may identify the anomalous material 175.

[0125] Object 185 is a material whose presence is not anomalous (or not suspicious) (“normal material”). While used broadly, a normal material 185 may include any material, object, chemical or compound that is normally or typically found in a particular target area 102, or a material, object, chemical or compound that is not suspicious in any way, nor is it (at least by itself) a potential hazard.

[0126] The analyzing module 165 would not detect object 185 in the hyperspectral image.

[0127] Therefore, in the example target area 102 of Figure 1 C, the analyzing module 165 may detect and flag anomalous combination 180 and anomalous material 175 in the hyperspectral image data of target 102.

[0128] In various embodiments, the system 100 and, in particular, the analyzing module 165, may be refined or trained further using false positive detection data or incorrect detection data. This type of reinforced learning may allow for the system 100 to increase in accuracy over time, thereby providing more reliable results. For example, if the analyzing module identified normal material 185 as anomalous, the hyperspectral image may be tagged (e.g., by a user reviewing the annotated image via a user interface) as a false positive and training sample for retraining of the model.

[0129] In some embodiments, the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

[0130] In various embodiments, a plurality of materials that are present in close proximity to one another, for example, a plurality of materials stacked on top of one another, or a mixture of a plurality of materials, may exhibit a single spectral signature which is a combination of the individual spectral signatures of each material. Such a combined spectral signature may also be indicative of a potential threat based on the types of materials present together.

[0131] In various embodiments, the pattern of life (or pattern of materials) algorithms used by the anomaly detector 128 are used to analyze a spectral signature to determine if the spectral signature may be a combined spectral signature of a combination of materials. This may be particularly true where the combination of materials is a suspicious combination, or indicative of an actual or potential threat.

[0132] In some embodiments, the potential threat includes at least one of a hazard, an explosive device, a disturbed portion of land, and a chemical residue.

[0133] In some embodiments, the pattern of life algorithms used by the anomaly detector 128 include at least one of deep learning object detection algorithms, object tracking algorithms, and pattern matching algorithms.

[0134] In various embodiments, the HSI algorithmic approach builds upon pattern- of-life algorithms used by the anomaly detector 128 that utilize deep learning object detection, tracking, and pattern matching. Pattern of life algorithms may also be referred to as pattern of materials algorithms.

[0135] Such approaches can outperform established HSI analysis algorithms like the Adaptive Coherence Estimator (ACE) in infrared target detection due to the complexity of the decision boundaries they can handle. The non-linear nature of deep learning systems is well suited to HSI data analysis where the use of N-dimensional convolutions performs fast and detailed dimensionality reduction to extract intelligence from high-dimensional spatial-spectral data.

[0136] In various embodiments, pattern of life algorithms may utilize a time series analysis of what changes in a target area. With enough sets of time series data, it becomes possible to associate certain motions with particular activities (e.g., fishing, robotics, etc.). By noticing such patterns, it can be possible to associate human activities with the motion of objects on the earth.

[0137] It will be appreciated that while in some cases the output may be provided to a user device that is supervised by a human, in other cases the user device may be an unsupervised system (e.g., an artificial intelligence system) capable of analyzing the output without human intervention.

[0138] In some embodiments, the generated annotated data describes at least one of a detected signature, and a detected combination of signatures.

[0139] In various embodiments, the system 100 uses deep learning techniques and machine learning algorithms to detect anomalous materials and anomalous mixtures or combinations of materials.

[0140] As particular substances, materials or combinations thereof will exhibit identifiable spectral signatures, embodiments disclosed herein may utilize a database or other record of such spectral signatures in identifying anomalous materials and mixtures. The database may be stored at the computing device 110 (e.g., on data storage device 155) or at a computing device in communication with the computing device 110.

[0141] The system 100 may also be used to monitor if a particular combination of spectral signatures is detected. For example, if it is known that a particular combination of multiple materials found together (e.g., three materials) is indicative of a hazard, then the computing device 110 may flag that combination, or generate an alert to be sent to a user device (e.g., for review by personnel).

[0142] Identification of materials using IR spectroscopy is based on correlating specific absorption bands in the measured spectrum (i.e., the absence of reflection) with the presence of specific molecular level features within the material, which are responsible for the absorption.

[0143] Further, in an embodiment, the computing device 110 may execute algorithms in the anomaly detector 128 that utilize Principal Component Analysis (PCA) to reduce the dimensionality of the hyperspectral image data to a specific number of bands. Increasing the number of bands may decrease the performance by producing ambiguity.

[0144] In some embodiments, the output is at least one of a set of coordinates in the target area, a marker on a map of the target area, and a marker on an image of the target area. The output 124 may be a digital visualization configured for display in a graphical user interface on a user device.

[0145] Referring now to Figure 6, shown therein is an example image-based graphical user interface (GUI) output 600 of a system for detecting materials in a target area, according to an embodiment. The output 600 may be, for example, the output 124 generated by the computing device 110 of Figure 1 .

[0146] In Figure 6, a shortwave infrared (SWIR) image 605 is depicted on the left, while an (inverted) Adaptive Coherence Estimator (ACE) rules-based classification map 610 of the same image 605 is depicted on the right.

[0147] In the map 610, a fake plant 615, fake rocks 620, and a fake explosive device 625 are detected and displayed for further use or analysis.

[0148] In various embodiments, the output 600 may be displayed on a graphical user interface (GUI) of a computing device.

[0149] Referring now to Figure 7, shown therein is an example annotated (output) image 700 of a target area generated by the system 100 of Figure 1 , according to an embodiment.

[0150] As depicted, the annotated image 700 contains annotations 705 and 710. Annotations 705 indicate that an object type is not normal (anomalous material detection), while annotations 710 indicate that a material distribution is not normal (anomalous material distribution detection). Further, annotations 705 contain spectral standards which are used to calibrate a detector. Annotations 810 contain the anomalous metal and plastic components. The image 700 may be displayed in a graphical user interface on user device 130 (e.g., as display data).

[0151] Referring again to Figure 1A, in various embodiments, the output 124 may be a score, wherein a greater score may correspond to a detection being of greater potential hazard. The score may be a numerical score.

[0152] For example, the score may quantify a level of hazard of a detection out of 100. In some cases, scores may be divided into bands where numerical scores within a given band are categorized into the same category. Similarly, the score may include a binary determination of hazard, wherein a value of “1” corresponds to a hazard, while a value of “0” corresponds to no hazard.

[0153] In other examples, the score may include a categorical score. For example, in an embodiment, the score may be assigned from a fixed set of three or more possible categories with each corresponding to a level of potential hazard (e.g., none, low, medium, or high). In some examples, a categorical score may be determined by converting a numerical score to a categorical score, wherein each category corresponds to a range of possible numerical score values.

[0154] In some embodiments, the computing system 110 is located on at least one of an unmanned aerial vehicle (UAV), and a telescopic pole.

[0155] Such a UAV or telescopic pole may be used in circumstances where surveillance or reconnaissance of a target area is needed. For example, a UAV may be sent ahead of a travelling convoy to surveil an area and determine if it is safe for the convoy to proceed.

[0156] In some embodiments, the input data 118 further includes electromagnetic image data of the target area

[0157] Classical algorithm outputs from EMI imagery data may be used as a means to weight the inputs for the subsequent artificial intelligence / machine learning processing.

[0158] In some embodiments, the input data further includes ground-penetrating radar (GPR) data of the target area.

[0159] Classical algorithm outputs from GPR imagery data may also be used as a means to weight the inputs for the subsequent artificial intelligence / machine learning processing.

[0160] Referring now to Figure 2, shown therein is a method 200 of detecting materials in a target area, according to an embodiment of the present disclosure. The method 200 may be encoded as computer-executable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the method 200. In an embodiment, the method 200 is implemented by the computing device 110 of Figure 1.

[0161] At 210, the method 200 includes receiving input data including hyperspectral image data of the target area.

[0162] In various embodiments, hyperspectral image (HSI) data may be used to obtain the spectrum for each pixel in an image of a target area, for purposes including detecting objects, identifying materials, or observing changes.

[0163] HSI data is generally obtained as a set of images, with each image representing a wavelength range (i.e., a spectral band) of the electromagnetic spectrum. Such images may be combined to form a three-dimensional (x, y, A) hyperspectral data cube.

[0164] Similarly, with spatial-spectral (or spatiospectral) scanning a 2-dimensional output (or image) of a target area may be obtained, comprising a spectral wavelength- coded (A) and spatial (x, y) map.

[0165] In the above examples, x and y represent two spatial dimensions of a target area, while A represents the spectral dimension (a range of wavelengths or spectral bands).

[0166] In some embodiments, the hyperspectral image data is a data cube where each cube tile is 32 x 32 pixels by 104 bands.

[0167] Bands affected by atmospheric absorption (40-70) are deleted from each cube tile. 75% of tiles that are labeled as normal-train / validation set are selected randomly for training set and the rest of 25% are selected as validation set.

[0168] In some embodiments, preprocessing is performed on the hyperspectral image data.

[0169] Preprocessing may include dimensionality reduction. Dimensionality reduction may include applying principal component analysis (PCA) techniques or the like to reduce the dimensionality of the hyperspectral data (e.g., reduce the number of bands). In a particular embodiment, PCA is used to reduce the dimensionality from 104 bands to 3 bands.

[0170] In other embodiments, PCA is used to reduce the dimensionality from 104 bands to 10 bands.

[0171] In various embodiments, hyperspectral image data may include shortwave infrared (SWIR) image data.

[0172] In some embodiments, hyperspectral image data may include red-green- blue (RGB)Zvisible light image data.

[0173] In some embodiments, hyperspectral image data may include visible and near-infrared (VNIR) image data.

[0174] At 220, the method 200 further includes analyzing the input data using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms.

[0175] At 230, the method 200 further includes providing the output to a user device.

[0176] Methods described herein provide enhanced situational awareness by identifying hazards or threats invisible to traditional HSI processing algorithms, cueing other sensors to detect potential threats. Embodiments of the present disclosure enable lower false positive rates in complex urban environments, earlier IED detections, and a new capability to keep safe personnel who may be working in potentially hazardous environments.

[0177] In some embodiments, the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

[0178] In various embodiments, a plurality of materials present in close proximity to one another, for example, a plurality of materials stacked on top of one another, or a mixture of a plurality of materials, may exhibit a single spectral signature which is a combination of the individual spectral signatures of each material. Such a combined spectral signature may also be indicative of a potential threat based on the types of materials present together.

[0179] In some embodiments, the potential threat includes at least one of a hazard, an explosive device, a disturbed portion of land, and a chemical residue.

[0180] In some embodiments, the pattern of life algorithms include at least one of deep learning object detection algorithms, object tracking algorithms, and pattern matching algorithms.

[0181] In various embodiments, the HSI algorithmic approach builds upon pattern- of-life algorithms that use deep learning object detection, tracking, and pattern matching.

[0182] Such approaches can outperform established HSI analysis algorithms like the Adaptive Coherence Estimator (ACE) in infrared target detection due to the complexity of the decision boundaries they can handle. The non-linear nature of deep learning systems is well suited to HSI data analysis where the use of N-dimensional convolutions performs fast and detailed dimensionality reduction to extract intelligence from high-dimensional spatial-spectral data.

[0183] In various embodiments, the computing device 110 executes pattern of life algorithms that utilize a time series analysis of what changes in a target area. With enough sets of time series data, it becomes possible to associate certain motions with particular activities (e.g., fishing, robotics, etc.). By noticing such patterns, it can be possible to associate human activities with the motion of objects on the earth.

[0184] It will be appreciated that while in some cases the output 124 may be provided to a user device that is supervised by a human, in other cases the user device may be an unsupervised system (e.g., an artificial intelligence system) capable of analyzing the output without human intervention.

[0185] In some embodiments, the generated annotated data 124 describes at least one of a detected signature, and a detected combination of signatures.

[0186] In various embodiments, deep learning techniques and machine learning algorithms may be used to detect anomalous materials and anomalous mixtures of materials.

[0187] As particular substances, materials or combinations thereof will exhibit identifiable spectral signatures, embodiments disclosed herein may utilize a database or other record of such spectral signatures in identifying anomalous materials and mixtures.

[0188] Techniques disclosed herein may also be used to monitor if a particular combination of spectral signatures is detected. For example, if it is known that a particular combination of three materials found together is indicative of a hazard, then that combination may be flagged, or an alert may be sent to personnel.

[0189] Further, the algorithms may utilize Principal Component Analysis (PCA) to reduce the dimensionality to a specific number of bands, as increasing the number of bands may decrease the performance by producing ambiguity.

[0190] In some embodiments, the output is at least one of a set of coordinates in the target area, a marker on a map of the target area, and a marker on an image of the target area.

[0191] Referring again to Figure 6, shown therein is an example image-based output 600 of a technique for detecting materials in a target area, according to an embodiment. The output 600 may be, for example, the output 124 generated by the computing device 110 of Figure 1

[0192] In Figure 6, a shortwave infrared (SWIR) image 605 is depicted on the left, while an (inverted) Adaptive Coherence Estimator (ACE) rules-based classification map 610 of the same image 605 is depicted on the right.

[0193] In the map 610, a fake plant 615, fake rocks 620, and a fake explosive device 625 are detected and displayed for further use or analysis.

[0194] In various embodiments, the output 600 may be displayed on a graphical user interface (GUI) of a computing device.

[0195] In various embodiments, the output 600 may be a score, such as a numerical score, wherein a greater score may correspond to a detection being of greater potential hazard. For example, the score may quantify the level of hazard of a detection out of 100. Similarly, the score may include a binary determination of hazard, wherein a value of “1” corresponds to a hazard, while a value of “0” corresponds to no hazard.

[0196] In other examples, the score may include a categorical score. For example, in an embodiment, the score may be assigned from a fixed set of three or more possible categories with each corresponding to a level of potential hazard (e.g., none, low,medium, or high). In some examples, a categorical score may be determined by converting a numerical score to a categorical score, wherein each category corresponds to a range of possible numerical score values.

[0197] In some embodiments, the computing system is located on at least one of an unmanned aerial vehicle (UAV), and a telescopic pole.

[0198] Such a UAV or telescopic pole may be used in circumstances where surveillance or reconnaissance of a target area is needed. For example, a UAV may be sent ahead of a travelling convoy to surveil an area and determine if it is safe for the convoy to proceed.

[0199] In some embodiments, the input data further includes electromagnetic image data of the target area.

[0200] Classical algorithm outputs from EMI imagery data may be used as a means to weight the inputs for the subsequent artificial intelligence / machine learning processing.

[0201] In some embodiments, the input data further includes ground-penetrating radar data of the target area.

[0202] Classical algorithm outputs from GPR imagery data may also be used as a means to weight the inputs for the subsequent artificial intelligence / machine learning processing.

[0203] Referring now to Figure 3, shown therein is an apparatus 300 for detecting materials in a target area, according to an embodiment of the present disclosure.

[0204] The apparatus 300 may be located at a node 302 of a network, such as the network 520 of Figure 5.

[0205] The apparatus includes a network interface 305 and processing electronics 310.

[0206] The processing electronics 310 can include a computer processer executing program instructions stored in memory, or other electronics components such as digital circuitry, including for example FPGAs and ASICs.

[0207] The network interface 305 may include an optical communication interface or radio communication interface, such as a transmitter and receiver.

[0208] In various embodiments, the apparatus 300 may further include, without limitation, an imaging assembly 315 (e.g., an imaging device such as a hyperspectral imaging (HSI) camera), a sensor assembly 320 (e.g., motion sensors, object sensors), a power source 325, and a wireless antenna 330 for wireless network communication.

[0209] Sensor assembly 320 may comprise a plurality of sensors for performing different functions. For example, sensor assembly 320 may include, without limitation, a motion sensor, an infrared (IR) sensor, a hyperspectral sensor, and potentially various other types of sensors.

[0210] The apparatus 300 may be a battery-powered device and may include a battery interface for receiving one or more rechargeable batteries at power source 325.

[0211] The wireless antenna 330 may be used to connect to any type of wireless network, including, but not limited to, data-centric wireless networks, voice-centric wireless networks, and dual-mode networks that support both voice and data communications.

[0212] In some embodiments, user-interaction with the apparatus 300 may also be performed through the wireless antenna 330. The network interface 305 and the processing electronics 310 may interact with the wireless antenna 330 to allow for information, such as text, characters, symbols, images, icons, and other items to be displayed or rendered on a separate user computing device.

[0213] The apparatus 300 may include several other functional components, each of which is partially or fully implemented using the underlying network interface 305 and processing electronics 310.

[0214] Referring now to Figure 4, shown therein is an example electronic device 400 that may perform any or all of operations of the above methods and features explicitly or implicitly described herein, according to different embodiments of the present disclosure.

[0215] For example, a computer equipped with network function may be configured as electronic device 400. The electronic device 600 may be used to implement the apparatus 300 of Figure 3, for example.

[0216] As shown, the device includes a processor 410, such as a Central Processing Unit (CPU) or specialized processors such as a Graphics Processing Unit (GPU) or other such processor unit, memory 420, non-transitory mass storage 430, I / O interface 440, network interface 450, and a transceiver 460, all of which are communicatively coupled via bi-directional bus 470.

[0217] According to certain embodiments, any or all of the depicted elements may be utilized, or only a subset of the elements. Further, the device 400 may contain multiple instances of certain elements, such as multiple processors, memories, or transceivers. Also, elements of the hardware device may be directly coupled to other elements without the bi-directional bus.

[0218] Additionally, or alternatively to a processor and memory, other electronics, such as integrated circuits, may be employed for performing the required logical operations.

[0219] The memory 420 may include any type of non-transitory memory such as static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), any combination of such, or the like.

[0220] The mass storage element 430 may include any type of non-transitory storage device, such as a solid-state drive, hard disk drive, a magnetic disk drive, an optical disk drive, USB drive, or any computer program product configured to store data and machine executable program code.

[0221] According to certain embodiments, the memory 420 or mass storage 430 may have recorded thereon statements and instructions executable by the processor 410 for performing any of the aforementioned method operations described above.

[0222] The electronic device 400 may also include an operating system and software components that are executed by the processor 410 and which may be stored in a persistent data storage device such as the memory 420.

[0223] Additional applications may be loaded onto the electronic device 400 through the network interface 450, the I / O interface 440, the transceiver 460, or any other suitable device subsystem.

[0224] In another embodiment of the present disclosure, there is shown and described a network-implemented system for detecting materials in a target area.

[0225] Figure 5 depicts an example network-implemented system 500, according to an embodiment.

[0226] The network-implemented system 500 includes an apparatus 512 which communicates with a plurality of imaging devices 514, a plurality of database devices 516, and a plurality of administrator devices 518 via a network 520. The apparatus 512 also communicates with a plurality of user devices 522. The apparatus 512 may be a purpose-built machine designed specifically for detecting materials in a target area, such as the apparatus 300 of Figure 3, for example.

[0227] Imaging devices 514 may for example include, without limitation, hyperspectral imaging (HSI) devices, electromagnetic imaging (EMI) devices (e.g., electromagnetic scanning devices), or ground-penetrating radar (GPR) imaging devices.

[0228] Imaging devices 514 may be located, for example, on unmanned aerial vehicles (UAVs), telescopic poles, or similar devices used for surveillance or similar purposes.

[0229] The apparatus 512, imaging devices 514, database devices 516, administrator devices 518 and user devices 522 may be a server computer, desktop computer, notebook computer, tablet, PDA, smartphone, or another computing or electronic device.

[0230] The devices 512, 514, 516, 518, 522 may include a connection with the network 520 such as a wired or wireless connection to the Internet. In some cases, the network 520 may include other types of computer or telecommunication networks.

[0231] The devices 512, 514, 516, 518, 522 may include one or more of a memory, a secondary storage device, a processor, an input device, a display device, and an output device. Memory may include random access memory (RAM) or similar types of memory. Also, memory may store one or more applications for execution by processor. Applications may correspond with software modules comprising computer executable instructions to perform processing for the functions described below. Secondary storage device may include a hard disk drive, floppy disk drive, CD drive, DVD drive, Blu-ray drive, or other types of non-volatile data storage.

[0232] Processor may execute applications, computer readable instructions or programs. The applications, computer readable instructions or programs may be stored in memory or in secondary storage or may be received from the Internet or other network 520. Input device may include any device for entering information into device 512, 514, 516, 518, 522. For example, input device may be a keyboard, keypad, cursor-control device, touchscreen, camera, or microphone.

[0233] Display device may include any type of device for presenting visual information. For example, display device may be a computer monitor, a flat-screen display, a projector or a display panel. Output device may include any type of device for presenting a hard copy of information, such as a printer for example. Output device may also include other types of output devices such as speakers, for example.

[0234] In some cases, devices 512, 514, 516, 518, 522 may include multiple of any one or more of processors, applications, software modules, second storage devices, network connections, input devices, output devices, and display devices.

[0235] Although devices 512, 514, 516, 518, 522 are described with various components, one skilled in the art will appreciate that the devices 512, 514, 516, 518, 522 may in some cases contain fewer, additional or different components. In addition, although aspects of an implementation of the devices 512, 514, 516, 518, 522 may be described as being stored in memory, one skilled in the art will appreciate that these aspects can also be stored on or read from other types of computer program products or computer-readable media, such as secondary storage devices, including hard disks, floppy disks, CDs, or DVDs; a carrier wave from the Internet or other network; or otherforms of RAM or ROM. The computer-readable media may include instructions for controlling the devices 512, 514, 516, 518, 522 and / or processor to perform a particular method.

[0236] In the description that follows, devices such as apparatus 512, imaging devices 514, database devices 516, administrator devices 518, and user devices 522 are described performing certain acts. It will be appreciated that any one or more of these devices may perform an act automatically or in response to an interaction by a user of that device. That is, the user of the device may manipulate one or more input devices (e.g. a touchscreen, a mouse, or a button) causing the device to perform the described act. In many cases, this aspect may not be described below, but it will be understood.

[0237] As an example, it is described below that the devices 512, 514, 516, 518, 522 may send information to the apparatus 512. For example, a user using the user device 522 may manipulate one or more input devices (e.g., a mouse and a keyboard) to interact with a user interface displayed on a display of the user device 522. Generally, the device may receive a user interface from the network 520 (e.g., in the form of a webpage). Alternatively, or in addition, a user interface may be stored locally at a device (e.g., a cache of a webpage or a mobile application).

[0238] Apparatus 512 may be configured to receive a plurality of information, from each of the plurality of imaging devices 514, database devices 516, administrator devices 518, and user devices 522. Generally, the information may comprise at least an identifier identifying the device, database, administrator, or user. For example, the information may comprise one or more of a username, e-mail address, password, or social media handle.

[0239] In response to receiving information, the apparatus 512 may store the information in storage database. The storage may correspond with secondary storage of the device 512, 514, 516, 518, 522. Generally, the storage database may be any suitable storage device such as a hard disk drive, a solid state drive, a memory card, or a disk (e.g. CD, DVD, or Blu-ray etc.). Also, the storage database may be locally connected with apparatus 512. In some cases, storage database may be located remotely from apparatus 512 and accessible to apparatus 512 across a network for example. In somecases, storage database may comprise one or more storage devices located at a networked cloud storage provider.

[0240] The imaging device 514 may be associated with an imaging account. Similarly, the database device 516 may be associated with a database account, the administrator device 518 may be associated with an administrator account, and the user device 522 may be associated with a user account. Any suitable mechanism for associating a device with an account is expressly contemplated.

[0241] In some cases, a device may be associated with an account by sending credentials (e.g. a cookie, login, or password etc.) to the apparatus 512. The apparatus 512 may verify the credentials (e.g. determine that the received password matches a password associated with the account). If a device is associated with an account, the apparatus 512 may consider further acts by that device to be associated with that account.

[0242] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art. Elements of each embodiment may be incorporated into other embodiments, for example, configurations discussed in relation to one embodiment, may be applied to other embodiments disclosed herein. Further, it is evident that various modifications and combinations can be made without departing from the invention. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present disclosure.

Claims

Claims:1 . A system for detecting materials in a target area, the system comprising: an imaging device configured to collect input data including hyperspectral image data of the target area; and a computing device configured to: receive the input data; analyze the input data using a hyperspectral signature detection model trained to detect at least one spectral signature in the input data indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms; and provide the output to a user device.

2. The system of claim 1 , wherein the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

3. The system of claim 1 , wherein the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

4. The system of claim 1 , wherein the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

5. The system of claim 1 , wherein the annotated data describes at least one of: a detected signature; and a detected combination of signatures.

6. The system of claim 1 , wherein the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

7. The system of claim 1 , wherein the imaging device and the computing device are located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

8. The system of claim 1 , wherein the input data further includes electromagnetic image data of the target area.

9. The system of claim 1 , wherein the input data further includes ground-penetrating radar data of the target area.

10. The system of claim 1 , wherein the hyperspectral signature detection model is an anomaly detection model.11 . The system of claim 10, wherein the anomaly detection model is an unsupervised deep learning model.

12. The system of claim 11 , wherein the unsupervised deep learning model includes an Adversarially Learned Anomaly Detection architecture.

13. The system of claim 11 , wherein the unsupervised deep learning model is a reconstruction-based anomaly detection architecture that evaluates how far a sample is from its reconstruction by a generative adversarial network from an encoded latent from the generative adversarial network.

14. The system of claim 10, wherein the anomaly detection model receives an abundance map from an unmixing algorithm as input.

15. The system of claim 10, wherein the anomaly detection model receives a hyperspectral cube including all spectral bands as input.

16. The system of claim 10, wherein the anomaly detection model receives a hyperspectral cub including only some spectral bands determined by a dimensionality reduction algorithm17. The system of claim 10, wherein the anomaly detection model is a supervised learning segmentation model configured to assign a probability of anomaly to each pixel or anomaly type out of a predefined taxonomy to each pixel.

18. The system of claim 17, wherein the supervised learning segmentation model receives an abundance map as input.

19. The system of claim 17, wherein the supervised learning segmentation model receives a hyperspectral cube as input.

20. A method of detecting materials in a target area, the method comprising executing via a computer system comprising at least one processor: receiving input data including hyperspectral image data of the target area; analyzing the input data using a hyperspectral signature detection model trained to detect at least one spectral signature indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms; and providing the output to a user device.21 . The method of claim 20, wherein the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.

22. The method of claim 20, wherein the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

23. The method of claim 20, wherein the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

24. The method of claim 20, wherein the generated annotated data describes at least one of: a detected signature; and a detected combination of signatures.

25. The method of claim 20, wherein the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

26. The method of claim 20, wherein the computing system is located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

27. The method of claim 20, wherein the input data further includes electromagnetic image data of the target area.

28. The method of claim 20, wherein the input data further includes ground-penetrating radar data of the target area.

29. An apparatus for detecting materials in a target area, the apparatus comprising: a network interface; a processor;a non-transitory computer readable memory having stored thereon instructions which, when executed by the processor, configure the apparatus to: receive input data including hyperspectral image data of the target area; analyze the input data using a hyperspectral signature detection model trained to detect at least one spectral signature indicative of a potential threat, the hyperspectral signature detection model comprising a machine learning based model configured to receive the input data and generate annotated data as an output, wherein at least a portion of the analyzing includes pattern of life algorithms; and provide the output to a user device.

30. The apparatus of claim 29, wherein the spectrum from at least one pixel in the input data includes a combination of multiple end member spectral signatures.31 . The apparatus of claim 29, wherein the potential threat includes at least one of: a hazard; an explosive device; a disturbed portion of land; and a chemical residue.

32. The apparatus of claim 29, wherein the pattern of life algorithms include at least one of: deep learning object detection algorithms; object tracking algorithms; and pattern matching algorithms.

33. The apparatus of claim 29, wherein the generated annotated data describes at least one of: a detected signature; and a detected combination of signatures.

34. The apparatus of claim 29, wherein the output is at least one of: a set of coordinates in the target area; a marker on a map of the target area; and a marker on an image of the target area.

35. The apparatus of claim 29, wherein the computing system is located on at least one of: an unmanned aerial vehicle (UAV); and a telescopic pole.

36. The apparatus of claim 29, wherein the input data further includes electromagnetic image data of the target area.

37. The apparatus of claim 29, wherein the input data further includes groundpenetrating radar data of the target area.

Images