System and methods for the aerial detection and elimination of landmines and other unexploded secondary ordnance

Abstract

The present invention is specifically directed to an aerial drone apparatus configured to emit a board spectrum of different signal waves in various dynamic frequencies and waveforms transmitted through an array of leveled antennas positioned on an autonomous drone, which can further identify the unique characteristics of buried unexploded ordinance, such as landmines and other secondary objects.

Description

[0001] SYSTEM AND METHODS FOR THE AERIAL DETECTION AND ELIMINATION OF LANDMINES AND OTHER UNEXPLODED SECONDARY ORDNANCE

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This International PCT application claims the benefit of and priority to U.S. Provisional Application No. 63 / 420,556 filed October 29, 2022, the specification, claims and drawings of which are incorporated herein by reference in their entirety.

[0004] TECHNICAL FIELD

[0005] The present invention relates to novel systems, methods, and apparatus for the aerial detection of buried objects, and in particular buried landmines and other unexploded ordinance. BACKGROUND

[0006] The ease of detecting landmines is decreasing, due to two main factors. First, newer mines contain less metal and are often much smaller, thereby becoming more difficult to detect with existing and inexpensive technology. For example, newer antipersonnel mines can range from 6 to 15 cm in diameter. Therefore, detection devices must be extremely accurate in locating the mines. The lack of metal in the landmine further eliminates the efficacy of current, affordable detection devices, namely metal detectors. Second, older landmines have often been in place for extended periods-often so long that any visual indication of its planting has disappeared, and vegetation has grown over its surface. The existence of vegetation over older landmines presents additional problems, including the fact that such vegetation often causes distortion of present detection techniques, since such techniques cannot be practiced immediately above the soil surface. Present methods of detecting landmines include using metal detectors, ground penetrating radar (GPR), infrared sensors (IR), dynamic thermography (DT), and ultra-sound (US). Each of these techniques has both benefits and drawbacks.

[0007] The drawbacks of metal detectors have been previously discussed. Essentially, more and more modern mines are being manufactured with little or no metal. For example, the PMN mine, previously manufactured by the Soviet Union, is enclosed in a thick rubber casing, thereby preventing most metal detecting devices from sensing its presence.

[0008] Ground penetrating radar (GPR) is a type of detection method that actively emits electromagnetic waves and collects reflected signals. The differences in the reflected signals may indicate where landmine may be buried. GPR may be limited by environmental conditions. For example, differences in humidity in soil may result in varying readings, often indicating a landmine when none actually exists. Further, ideal conditions for detecting landmines require not only dry, consistent soil, but also the use of a low frequency signal. Unfortunately, low frequency signals provide poor resolution images in determining where such landmines may be.

[0009] Standard Infrared (IR) detection devices, also referred to as thermal radiation detection devices, typically use electromagnetic, temporal waves to radiate the soil from a stationary platform. Things beneath the surface, such as landmines, are heated through the IR radiation and the temporal signature heated landmines produce. These signatures are generally not sufficient to detect single landmines. IR detection devices that move (i.e., are not stationary) generally need to use spatial electromagnetic waves. Spatial waves may provide better resolution than temporal waves. Infrared detection devices have been known to operate either passively or actively. Passive infrared sensors detect changes in temperature based upon the natural radiation of an object, often because of changing environmental (i.e., weather) conditions. Drawbacks of known passive infrared detection techniques include the slight disparity in temperature that generally occurs through the day between a landmine and the ambient soil it is buried in. Further, results may be greatly skewed by environmental conditions (i.e., a cool wind blowing over warm soil may disrupt readings). Active infrared detection devices utilize infrared radiation to heat bodies in a wall or another medium in order to artificially stimulate their thermal signature to be detected by IR cameras. Active infrared detectors also have drawbacks, including cost, size, and weight.

[0010] Ultrasound (US) detection devices generally emit a high frequency sound (above the audible range) and collect reflections of this sound. A difference between US and GPR methods is that GPR does not cause any physical effects, while US does. As the US sound wave propagates through a medium, the sound wave causes molecules of the medium to oscillate around their equilibrium position. If the medium is entirely homogenous, i.e., no landmines, the US wave will continue propagating. If a different medium is encountered, the US wave is generally reflected and refracted. Thus, when US waves are reflected and refracted back to sensors, it is an indication of a possible landmine location. The main drawback of US detection devices is that US waves tend to greatly attenuate when the medium is air. US detection devices work best when there is little to no air gap between the emitter and the desired medium, i.e., the soil. As it is often difficult and unsafe to have a US detection device in direct contact with the soil, the results of such devices may be somewhat unreliable, due to this distance. It has also been found that ultrasound and audible sound waves may be used to determine where landmines may be located. These waves may be transmitted toward the surface to be evaluated, and the reflected waves collected and analyzed. Studies have shown that when the sound waves encounter a landmine or similar foreign body, they are reflected back at a different rate than waves reflected due to normal soil conditions. By analyzing this data and comparing the differential in reflection rates, possible landmine locations may be ascertained. The device for digitally recording the reflected sound waves from a directional high sensitivity microphone is referred to herein as an “acoustic camera” for simplicity.

[0011] There are further drawbacks associated with each of the discussed detection devices. Generally, their costs are prohibitive, especially to the under-developed countries that most have the need forthem. Further, the majority of devices require direct human operation, thereby placing lives in jeopardy.

[0012] Thus, there is a need for a more reliable, effective, inexpensive, and / or remotely or autonomously operated landmine detection system. There is further a need for landmine detection device that can leverage current advances in machine learning to more accurately classify and differentiate between landmines and similar non-mine objects

[0013] SUMMARY OF INVENTION

[0014] In one aspect, the present invention includes systems, methods, and devices for the aerial detection of buried objects, and in particular unexploded ordnance such as landmines and the like. Specifically, in one embodiment, the inventive technology can be ground or aerial-based, and in a preferred embodiment includes an aerial drone-based technology platform having a antenna array configured to emit a plurality of signal waves across a board spectrum ranging from radio waves to sub-infrared. In this preferred aspect, the antenna array generates multiple signal layers using different waves in various dynamic frequencies which can be transmitted through the array to a target location. The multiple signal layers can penetrate the ground and be reflected back to the aerial drone where they are detected by a receiving antenna and processed, for example through an Al-based system to identify the unique signal characteristics of buried unexploded ordinance, such as landmines.

[0015] In another embodiment, the present invention includes a systems, methods and devices for the aerial detection of buried objects, and in particular the detection and classification of non- metallic objects, and preferably non-metallic (plastic-based) landmines, Improvised Explosive Devices (lEDs), and other various Unexploded Ordinance (UXOs) that are partially or fully buried beneath the ground.

[0016] In another embodiment, the present invention includes a systems, methods and devices for the aerial detection of buried objects, and in particular a novel signal processing system configured to distinguishing between, for example a landmine versus a similar mine-like objects using machine learning and custom neural network models.

[0017] In another embodiment, the present invention includes a systems, methods and devices for the aerial detection of buried objects, and in particular an autonomous aerial drone configured to traverse a pre-selected path and scan a target location and process the returned signal data in realtime without human-intervention.

[0018] In another embodiment, the approaches described herein may use multi-signal detection and machine learning module applications, such as Convolutional neural network (ConvNet) analysis, to analyze an object of interest resolvable by dynamic multi-signal layers applied to transmitted by a transmission array, or other comparable instrument. In a preferred embodiment, the object of interest comprises a landmine, or unexploded ordnance that may further be fully or partially buried or otherwise obscured.

[0019] Additional embodiments of the present invention will be apparent from the specification, figures and claims provided below.

[0020] BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG. 1 shows a flow diagram of a signal transmission system of the invention configured to generate and emit a plurality of signal waves in various dynamic frequencies, which can be transmitted through an array of leveled antennas secured to an autonomous serial drone in one embodiment thereof.

[0022] FIG. 2 shows a flow diagram of a signal processing and display system of the invention configured to detect and process signal waves emitted from the antennas arrays in one embodiment thereof.

[0023] FIG. 3A-D (A-C) demonstrates exemplary user interface during detection of unburied ordinance in sample geographic location; (D) demonstrates detection of exemplary landmines at exact positions (in GPS real-world location) with less than the threshold (< Im + / -) positional errors. FTG. 4 shows a side view of a multi-spectrum aerial-based detection array having an antenna array module responsive to a sensor management module and central processing module in one embodiment thereof.

[0024] FIG. 5 shows a top perspective view of a multi-spectrum aerial-based detection array secured to a casing having an antenna array module responsive to a sensor management module and central processing module in one embodiment thereof.

[0025] FIG. 6 shows a bottom perspective view of a multi-spectrum aerial-based detection array having an antenna array module responsive to a sensor management module and central processing module in one embodiment thereof.

[0026] FIG. 7 shows a bottom perspective view of a multi-spectrum aerial-based detection array having a plurality of antenna array modules secured to a responsive to a sensor management module and central processing module in one embodiment thereof.

[0027] FIG. 8 shows a side perspective view of a multi-spectrum aerial-based detection array having a plurality of antenna array modules secured to protective casing in one embodiment thereof.

[0028] DETAILED DESCRIPTION OF INVENTION

[0029] The present invention includes a variety of aspects, which may be combined in different ways. The following descriptions are provided to list elements and describe some of the embodiments of the present invention. These elements are listed with initial embodiments; however, it should be understood that they may be combined in any manner and in any number to create additional embodiments. The variously described examples and preferred embodiments should not be construed to limit the present invention to only the explicitly described systems, techniques, and applications. Further, this description should be understood to support and encompass descriptions and claims of all the various embodiments, systems, techniques, methods, devices, and applications with any number of the disclosed elements, with each element alone, and also with any and all various permutations and combinations of all elements in this or any subsequent application.

[0030] First Exemplary Embodiment

[0031] Multi-Spectrum Signal Generation and Transmission

[0032] The present invention include an aerial -based detection system to identify buried objects, and in particular landmines and other unexploded ordinance. In a preferred embodiment, the present invention includes a multi -spectrum aerial-based detection system (100), preferably including aerial drone (101), or other autonomous aerial or ground vehicle attached with the antenna arrays (102) that is configured to generate a multiple signal layers (102), generally referred to herein as a signal layer(s), using different waves in different dynamically changing frequency ranges. In this embodiment, the multi-signal layer (102) can generate a waveform that can be focused, for example through beamforming, to a target location using an antenna array (103), sometimes referred to herein as a transmission array (103), secured to the aerial drone (101) of the invention. In a preferred embodiment, the individual signals of the multi-signal layer (102) are generated according to a variable power density and may further be configured to include a penetration focal point of the layer to be created.

[0033] As generally shown in FIG. 1, the system and methods of multi-signal layer (102) generation and transmission includes a signal generator module (104) comprising one or more signal generators configured to generate one or more signal waves that form the multi-signal layer (102). The signal generator module (104) of the invention may include one or more analog signal generators, responsive to a one or more waveformers, amplifiers and filters, configured to generate and dynamically adjust, amplify and / or filter the signal. The signal generator module (104) of the invention may include one or more analog to digital converters to convert analog signals into a digital format that is further responsive to one or more signal processing units, which can include in analog and digital field-programmable gate array (FPGA) processors.

[0034] As shown in Figure 1, the signal generator module (104) of the invention may include a plurality of signal generators, which may preferably include analog or digital signal generators configured to generate one or more signal waves. In a preferred embodiment, the signal generator module (104) of the invention may include a plurality of signal generators, wherein signal generator is configured to generate a different signal, such as radio waves to microwave radiation. In one embodiment, the signal generators (104) of invention generate a plurality of signals that are each transmitted to a frequency mixer (105), which may preferably include an IQ mixer configured to oscillate the signals with the baseband frequency (104a), also referred to herein as the base frequency (105), which than represents the targeted spectrum range of a unique layer of the multisignal layer (102). In this preferred embodiment, the multi-signal layer (102) of the invention includes continuous waves with multiple frequencies ranging from 350 MHz to 7 GHz in adaptive frequency-domain and time-domain signal patterns. As further shown in Figure 1, the plurality of signals, once it has been processed by the frequency mixer (105), and preferably an IQ mixer, one or more filters and or amplifiers are applied to the signals to modulate and / or increase the gains of one or more of the plurality of signals. The amplified and / or modulated signals are then transmitted to the antenna array (103), which can comprise a dedicated array of directional antennas. In this embodiment, the transmission of a plurality of different dynamically modulated signals from the antenna array (103) generates multiple signal layers (102) above the surface, sub-surface and below the surface based on the focal point calibrations directing the signals to a target location.

[0035] In one preferred embodiment, the multiple signal generator (104) of the invention simultaneously generates signals having different nano-second timed frequencies. The generation of such nano-second timed frequencies is continuous such that there is no need to switch frequency sine waves back and forth, which would not provide an effective multi-signal layer (102) wave even in nano-second changes.

[0036] As noted above, in one preferred embodiment, a mixer (105) of the invention functions by mixing the signal waves (low-frequency like 1MHz to 150MHz) with the base frequency (high- frequency, 350 MHz to 7 GHz) to make it transmissible within the surface, sub-surface and deeper in the ground, to create layers using different focal lengths. The base frequencies are dedicated to each antennas on a particularly changing frequency based on a dedicated algorithm.

[0037] Multi-Signal Processing and Display

[0038] The present invention further includes systems, methods and devices to receive and process a plurality of dynamically transmitted signals each having a unique frequency and waveform forming a multi-signal layer (102). As show in Figure 2, an aerial drone (101) may be adapted to include a plurality of reception antennas (108) configured to receive back the reflected signals of the signals layer (102) transmitted by the antenna array (103) In a preferred embodiment, a reception antenna array (108), also generally referred to as a reception array (108), may be secured to an aerial drone or other aerial or ground autonomous vehicle (101) and further be positioned approximately perpendicular or parallel to the transmission array (103) so as to be able to receive signals from the multi-signal layer (2) that are reflected back from the ground to the arial drone (101). The received signals then undergo analog signal processing to further improve the signal to noise ratio of each signal. In a preferred embodiment, the received signals are processed by one or more filters (106) and amplifiers (107), such as a low-noise amplifier, to increase the signal to noise ratio. As further shown in figure 2, in one embodiment, the invention includes a ground plane alignment step (115). In a preferred embodiment, this describes the use of a calibrating algorithm based on the first layer values in respect to the ground reference value. For example, in one embodiment the signals on the ground are pre-referenced and based on that it is being calibrated on the flight due to focal length changes to reference the ground plane topography.

[0039] Again, referring to Figure 2, the processed analog signals are then converted into a digital signal, for example by an analog to an analog digital converter (109) and then further processed by a digital signal processor (110), which may include one or more FPGAs and tensor core processors. Again, as shown in Figure 2, in this preferred embodiment, the plurality of received and processed signals are consolidated into a single solidified signal, for example by the digital signal processor (110). In this embodiment, the transmitted and base frequencies of the signal layers are removed. This solidified signal can then be used to distinguish the unique signatures within the solidified signal using the Al-based learning module. For example, as described above, once the solidified signal has been filtered and processed, it can be separated into the area of interests within the generated layered-signal to be processed through the tensor neural network core.

[0040] In this embodiment, the neural network, and preferably a convolutional neural network (ConvNet), generally referred to as a data acquisition unit (111) facilitates the processing of the solidified signal and its components through a custom supervised machine learning model designed to distinguish between the landmine signatures and the non-landmine false detection data. In another embodiment, the machine learning model of the invention can change the threshold and weights within its hidden layers based on the real-time flight data changes of the aerial drone (101). This gives the machine learning model flexibility to change its hidden layer values while on flight with newly given data.

[0041] In another embodiment, once the solidified signal data have been analyzed by the machine learning model and a classification of whether an object of interest is landmine or not, and aerial drone (101) may further include an onboard computer (112) responsive to one or more sensors that can measure and transmit real-time data of its location, depth, altitude of the drone (101), image data and various sensor outputs through, for example a local network (113) a server user interface (114). Initial studies of the system of the invention have demonstrated a detection and classification accuracy of more than 93% including the false positive detections. As generally outlined above, in one embodiment the inventive technology include systems and methods of applying machine learning to detect and analyze an object of interest using multisignal analysis. In one preferred embodiment, a neural network, such as a multi-layer ConvNet, may be trained via an initial training dataset. In this embodiment, at least one reference dataset may be generated by transmitting, receiving and processing a multi-signal layer (102) a reference sample comprising an object of interest, such as a fully or partially buried landmine or other unexploded. Digital profdes of the object of interest may be generated and transmitted to one or more processors, or other similar data processing device or system, where features of interest that may be indicative of the object of interest, such as unique reflected multi-signal pattern of a landmine are extracted. This extraction may be accomplished in a preferred embodiment by a machine learning system, and more preferably a ConvNet Machine learning Module as generally described herein. These unique reflected multi-signal patterns can be correlated with the object of interest, such as a landmine of interest.

[0042] In one preferred embodiment, a machine learning systems or module of the invention may include a ConvNet featuring extraction module may take a collection of raw or preprocessed (where the preprocessing step may cull multi- antenna array results based on a given predetermined pattern or there threshold) multi- antenna array results and extracts “features,” generally referred to as a “features of interest.” In a preferred embodiment, a feature of interest may be correlated with a landmine, or other unexploded ordinance. These features may typically be extracted via Convolutional Neural Networks (CNNs), but could be extracted by other feature extractors, such as Principal Component Analysis (PCA). The outputs of this module may be the resulting features and optionally the original multi-signal measurements for further processing downstream.

[0043] In another preferred embodiment, a machine learning module may include fusion module that may be optionally used to leverage data and / or meta-information from other sources. The features from a ConvNet may be combined with other measurement or descriptive features through a variety of methods (e.g., a two input Artificial Neural Network, a Random Forest algorithm or Gradient Boosting algorithm for feature selection) producing a new set of feature of interest outputs. In another embodiment, a machine learning module may include one or more classification or classifier modules that assign a predefined label and probability of a class based on the multi-signal measurements. Second Exemplary Embodiment Multi-Spectrum Aerial-Based Detection Array

[0044] As generally show in in Figures 4-8, the present invention includes a multi-spectrum aerialbased detection array (100). In one preferred embodiment, the array device (100) of the invention includes one or more antenna array modules (201) configured to transmit multi-spectrum signal patterns across a defined frequency spectrum. In a preferred embodiment, each antenna array module (201) can include modular elements consisting of a plurality of transmitting & receiving ultra-wideband (UWB) antennas.

[0045] Again, generally referring to Figures 4-8, the multi-spectrum aerial-based detection array

[0046] (200) can include a casing (210) configured to secure and mount the detection array (200) of the invention to an aerial service, such as an aerial drone (101) as described above. The antenna array module (201) of the invention can further be configured to movably adjusted so as to accommodate engaged directionality of the transmitted signal.

[0047] The antenna array module(s) (201) of the invention can be positioned at varied angles relative to the surface to be scanned so as to maximize resolution and penetration of the signal. As shown in Figure 4, in a preferred embodiment, an antenna array module (201) is coupled with a stepper motor (207) through an array mount (208) configured to allow the antenna array module

[0048] (201) to be mechanically adjusted, preferably between angles (+ / - 15 degrees). In a preferred embodiment, the angle or adjustment can be made in real-time, and can further be directed by a user or determined by the processing unit based upon the sensor fusion information received

[0049] As shown in Figures 7-8, a plurality of antenna array modules (201) are positioned at varied angles relative to the surface to be scanned so as to maximize resolution and penetration of the signal. In this embodiment, each antenna array module (201) can be secured to different positions / angles positioned at a different position along a protective array casing (210). Each antenna array module (201) can further be adjusted in one or more independent directions. As shown in Figure 7-8, a plurality of antenna array modules (201) are secured to a protective array casing (210) through a stepper motor (207) coupled with an array mount (208) configured to allow each antenna array module (201) to be independently mechanically adjusted, preferably between angles (+ / - 15 degrees). In a preferred embodiment, the angle or adjustment of each individual antenna array module (201) can be synchronous or asynchronous, and further can be made in realtime. The adjustment each antenna array module (201) can further be directed by a user or determined by the processing unit based upon the sensor fusion information received. In this configuration, the antenna array (201) can transmit distinct signal patterns across a frequency spectrum, preferably between 350 MHz to 7 GHz in both frequency modulation and time-domain. The reception of reflected signals from the surface, subsurface, and beneath facilitates multidimensional spatial resolution imaging. This antenna array (201) configuration is further optimized for the detection of targets such as ordinance which can include, but not be limited to: anti-personnel, anti-tank mines, unexploded ordinances, and explosive remnants of war.

[0050] The multi-spectrum aerial-based detection array (100) further includes a sensor management module (203) responsive to the one or more antenna array modules (201). In a preferred embodiment, the sensor management module (203) is responsive to, and amalgamates the capabilities of an infrared hyperspectral imager (204), a high-definition RGB imager (205), and / or a LIDAR altimetry sensor (206).

[0051] The multi-spectrum aerial -based detection array (100) further includes a central processing module (202) that is responsive to the sensor management module (203) and / or the one or more antenna array modules (201). The central processing module (202) of the invention can further include an interface connector (209), which can allow for both firmware and data interface connections to internal, or external devices or networks and the like. In a preferred embodiment, the central processing module (202) incorporates an adaptive control procedure configured to adjust radiation patterns based on environmental conditions, target region specifications, distance, and interference. The antenna arrays (201), configured to transmit and receive signals across a wide-band spectrum are responsive to the central processing module (202) so as to execute both time and frequency domain analyses. Unique signal patterns, governed by a machine executable program, can consider the distance from the antenna arrays (201) to the ground's surface. The LIDAR altimetry sensor (206) in the sensor fusion unit (203) can assist in determining this distance and 3 dimensional positioning of the target. Generally, the sensors (204, 205, 206) within the sensor management module (203) incorporate and fuse with the antenna arrays (201) signals allowing an Al-bases computer executable program unit within the processing module (202) to have sensor fusion capabilities.

[0052] Again, referring to Figures 4-8, in a preferred embodiment, signals are transmitted at intervals, coordinating the antennas within each array module (201). Individual antennas can be assigned specific frequency bands, enhancing the signal-to-noise ratio and minimizing interference. Received signals pass through analog low noise amplifiers and a digital signal processing unit as described above. Through advanced processing, the system achieves a high- resolution image with minimal ground noise. During operation, scans from the antenna arrays (201) and the sensors (204, 205, 206) within the sensor management module (203) undergo processing using deep neural network models, trained on diverse targets and environmental anomalies. The engine discerns between genuine targets and decoys using recurrent training and feedback loops. It processes scans in real time, relaying a comprehensive report - including depth, terrain vectors, estimated dimensions, and confidence levels - to the user interface promptly.

[0053] Unless otherwise indicated, the method operations and device features disclosed herein involve techniques and apparatus used in signal transmission, reception and processing, software design and programming, and statistics, which are within the skill of the art.

[0054] Unless defined otherwise herein, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Although any methods and materials similar or equivalent to those described herein find use in the practice or testing of the embodiments disclosed herein, some methods and materials are described in detail and represent preferred embodiments of the current inventive technology.

[0055] Any module, unit, component, server, computer, terminal, engine or device exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and / or nonremovable) such as, for example, magnetic disks, optical disks, or tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information, and which can be accessed by an application, module, or both, which specifically includes cloudbased applications. Any such computer storage media may be part of the device or accessible or connectable thereto. Further, unless the context clearly indicates otherwise, any processor or controller set out herein may be implemented as a singular processor or as a plurality of processors. The plurality of processors may be arrayed or distributed, and any processing function referred to herein may be carried out by one or by a plurality of processors, even though a single processor may be exemplified. Any method, application or module herein described may be implemented using computer readable / executable instructions that may be stored or otherwise held by such computer readable media and executed by the one or more processors or through a cloud-based application.

[0056] Numeric ranges are inclusive of the numbers defining the range. It is intended that every maximum numerical limitation given throughout this specification includes every lower numerical limitation, as if such lower numerical limitations were expressly written herein. Every minimum numerical limitation given throughout this specification will include every higher numerical limitation, as if such higher numerical limitations were expressly written herein. Every numerical range given throughout this specification will include every narrower numerical range that falls within such broader numerical range, as if such narrower numerical ranges were all expressly written herein.

[0057] Two primary embodiments of machine learning modules of the invention may include “deep” convolutional neural network (ConvNet) models and a randomized Principal Component Analysis (PCA) random forests model. However, other forms machine learning model may be employed in the context of this disclosure. A random forests model is relatively easy to generate from a training dataset and may employ relatively fewer training set members. A convolutional neural network may be more time-consuming and computationally expensive to generate from a training set, but it tends to be better at accurately classifying features of interest, such as landmines or other secondary objects, such as unexploded ordinance and the like.

[0058] As used herein, a machine learning system or model is a trained computational model that takes a feature of interest, and classifies them as, for example, particular object such as a landmine. Examples of machine learning models include neural networks, including recurrent neural networks and convolutional neural networks; random forests models, including random forests; restricted Boltzmann machines; recurrent tensor networks; and gradient boosted trees. The term “classifier” (or classification model) is sometimes used to describe all forms of classification model including deep learning models (e.g., neural networks having many layers) as well as random forests models. As used herein, a machine learning system may include a deep learning model that may include a function approximation method aiming to develop custom dictionaries configured to achieve a given task, be it classification or dimension reduction. It may be implemented in various forms such as by a neural network (e.g., a convolutional neural network), etc. In general, though not necessarily, it includes multiple layers. Each such layer includes multiple processing nodes and the layers process in sequence, with nodes of layers closer to the model input layer processing before nodes of layers closer to the model output. In various embodiments, one-layer feeds to the next, etc. The output layer may include nodes that represent various classifications. In some embodiments, a deep learning model is a model that takes data with very little preprocessing, although it may be segmented data such as multi-signal artifacts or patterns or other features of interest may be extracted from multi-signal outputs.

[0059] In various embodiments, a deep learning model may have significant depth and can classify a large or heterogeneous array of features of interest, such as landmines or other secondary objects, such as unexploded ordinance. In some contexts, the term “deep” means that model has a plurality of layers of processing nodes that receive values from preceding layers (or as direct inputs) and that output values to succeeding layers (or the final output). Interior nodes are often “hidden” in the sense that their input and output values are not visible outside the model. In various embodiments, the operation of the hidden nodes may not be monitored or recorded during operation. The nodes and connections of a deep learning model can be trained, for example with a “reference” or “additional sample,” and retrained without redesigning their number, arrangement, interface with multi-signal outputs, etc. and yet classify a large heterogeneous range of features of interest, such as landmines or other secondary objects, such as unexploded ordinance that may have readily identifiable characteristics discernable by the multi-signal outputs.

[0060] In various aspects, provided herein are systems and methods for identifying and optionally characterizing a feature of interest, by analyzing the feature of interest from a test sample and thereby generating a test dataset and comparing it to a training dataset generated from a reference sample, and optionally one or more additional samples. A feature of interest in this embodiment may include objects such as landmines or other secondary objects, such as unexploded ordinance and the like.

[0061] Deep convolutional neural networks may include multiple feed forward layers. As known to those of skill in the art, these layers aim to extract relevant features from a signal output, were the features extracted depend on the objective function used fortraining. The convolutional layer's parameters include a set of learnable filters (or kernels), which have a small receptive field, but are applied to the entire signal output in the convolution step. In certain embodiments, during the forward pass, each filter is convolved across the width and height of the signal output, computing a type of dot product between the entries of the filter and the input and producing an activation map associated with that filter. As a result, the network learns filters that activate when they encounter some specific type of feature at some spatial position in the signal output. The resulting activation maps are processed in both standard feed forward fashion and using “skip connections” in conjunction with feed forward output.

[0062] Convolutional networks may include local or global pooling layers, which reduce the dimensionality of the activation maps. They also include various combinations of convolutional, fully connected layers, skip connections, and customized layers, for example squeeze excite, residual blocks, or spatial transformer subnetworks. The neural network may include various combinations of feed forward stacked layers in order to generate feature representations of the signal output data. The specific nature of the estimated features (obtained via supervised or unsupervised training) depends on the objective function, the input data, and the neural network architecture selected. In certain embodiments, the deep learning classification model may employ a tensor neural network core.

[0063] As described herein, any hardware device of the invention can be any kind of device that can be programmed including, for example, any kind of computer including aerial drone, smart mobile devices (watches, phones, tablets, and the like), personal computers, powerful servers or supercomputers, or the like. The device includes one or more processors such as an ASIC or any combination processors, for example, one general purpose processor and two FPGAs. The device may be implemented as a combination of hardware and software, such as an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. In various embodiments, the system includes at least one hardware component and / or at least one software component. The embodiments described herein could be implemented in pure hardware or partly in hardware and partly in software. In some cases, the disclosed embodiments may be implemented on different hardware devices, for example using a plurality of CPUs equipped with GPUs capable of accelerating scientific computation. Each computational element may be implemented as an organized collection of computer data and instructions. In certain embodiments, a data acquisition algorithm and a machine learning model can each be viewed as a form of application software that interfaces with a user and with system software. System software typically interfaces with computer hardware, typically implemented as one or more processors (e.g., CPUs or ASICs as mentioned) and associated memory. In certain embodiments, the system software includes operating system software and / or firmware, as well as any middleware and drivers installed in the system. The system software provides basic non-task-specific functions of the computer. In contrast, the modules and other application software are used to accomplish specific tasks. Each native instruction for a module is stored in a memory device and is represented by a numeric value.

[0064] At one level a computational element is implemented as a set of commands prepared by the programmer / developer. However, the module software that can be executed by the computer hardware is executable code committed to memory using “machine codes” selected from the specific machine language instruction set, or “native instructions,” designed into the hardware processor. The machine language instruction set, or native instruction set, is known to, and essentially built into, the hardware processor(s). This is the “language” by which the system and application software communicates with the hardware processors. Each native instruction is a discrete code that is recognized by the processing architecture and that can specify particular registers for arithmetic, addressing, or control functions; particular memory locations or offsets; and particular addressing modes used to interpret operands. More complex operations are built up by combining these simple native instructions, which are executed sequentially, or as otherwise directed by control flow instructions.

[0065] The inter-relationship between the executable software instructions and the hardware processor may be structural. In other words, the instructions per se may include a series of symbols or numeric values. They do not intrinsically convey any information. It is the processor, which by design was preconfigured to interpret the symbols / numeric values, which imparts meaning to the instructions.

[0066] In certain embodiments, the modules or systems generally used herein may be configured to execute on a single machine at a single location, on multiple machines at a single location, or on multiple machines at multiple locations. When multiple machines are employed, the individual machines may be tailored for their particular tasks. For example, operations requiring large blocks of code and / or significant processing capacity may be implemented on large and / or stationary machines not suitable for mobile or field operations. Such operations may be implemented on hardware remote from the site where the data is processed, for example on a server or server farm connected by a network to a field device, such as an aerial drone that transmits and receives multisignal data or through a cloud-based network.

[0067] A “reference sample or dataset” as used herein is a sample that may be used to train a computer learning systems, such as by generating a training dataset. A “test sample” as used herein is a sample that may be used to generate a test dataset, for example of one or more features of interest, which may be qualitatively and / or quantitatively compared to a training dataset as generally described herein.

[0068] As used herein, a “feature,” “feature of interest” is a feature of a sample that represents a quantifiable and / or observable feature of an object of interest, and preferably a landmine or other unexploded ordinance that is fully or partially buried. In certain embodiments, a “feature of interest” may potentially correlate to a landmine or other unexploded ordinance that is fully or partially buried.. In certain embodiments, a feature of interest is a feature that is discerned from processed multi-signal layers and may be recognized, segmented, and / or classified by a machine learning module. A feature of interest presented above can be used as a separate classification for the machine learning systems described herein. Such systems can classify any of these alone or in combination with other examples.

[0069] The terms “threshold” herein refer to any number that is used as, e.g., a cutoff to classify a sample feature as particular type of feature of object of interest. The threshold may be compared to a measured or calculated value to determine whether the source giving rise to such value suggests that it should be classified in a particular manner. Threshold values can be identified empirically or analytically. The choice of a threshold is dependent on the level of confidence that the user wishes to have to make the classification. Sometimes they are chosen for a particular purpose (e.g., to balance sensitivity and selectivity).

[0070] As used herein, the singular terms “a,” “an,” and “the” include the plural reference unless the context clearly indicates otherwise. The term “or” as used herein, refers to a non-exclusive or, unless otherwise indicated.

[0071] The invention now being generally described will be more readily understood by reference to the following examples, which are included merely for the purposes of illustration of certain embodiments of the embodiments of the present invention. The examples are not intended to limit the invention, as one of skill in the art would recognize from the above teachings and the following examples that other techniques and methods can satisfy the claims and can be employed without departing from the scope of the claimed invention. Indeed, while this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.

[0072] EXAMPLES

[0073] Example 1 : Drone-based Identification Anti-Personnel and Anti-Tank Landmine Replicas.

[0074] Overview: In one preferred embodiment, the Applicant’s demonstrated that the present invention can identify unexploded anti-personal and anti-tank ordinance, including buried and non-metal variants of the same. Specifically, the present inventive technology demonstrated the ability to autonomously detect and identify anti-personnel and anti-tank replica landmines buried under the ground, in real-time, with accurate location of the position of the mine.

[0075] System Configuration: In this embodiment, the EAGLE A7 (EA7) Multicopter Airframe with the Proprietary EA7 sensor package consisting of a custom array of antennas, signal generator and processors, and onboard Al Engine was utilized. A base station with screens, a communication transceiver module, a telemetry unit, and a GNSS-RTK base module.

[0076] Weather conditions: See Table 1 below (4-hour test window): Test Procedure: For the EAGLE A7 system’s test on-site location, the following approach was carried out:

[0077] • Selected a test location and a landmine contaminated area.

[0078] • Set of AP and AT replica mines were buried underground (based on Ukraine’s Landmine Report, Source: HRW, UN).

[0079] . EAGLE A7 system configured for Detection on the selected landmine contaminated area.

[0080] . On EA7 UI (Base station), selected an area for detection, giving the command to take-off . EA7 Scanned the area of interest and sent real-time data to the EA7 UI (Base station)

[0081] . Data of detected landmines analyzed with actual location coordinates

[0082] • Accuracy for identification and location positioning accuracy was determined

[0083] Test Results: See Table 2 below

[0084] EA7 UI Screen: Detected Landmines Positions: The landmines were detected at exact positions (in GPS real-world location) with less than the threshold (< Im + / -) positional errors (See Figure 3) As demonstrated above, the Applicant’s System is capable of Detecting and Identifying broad-range of non-metallic plastic landmine replicas, currently being used in Ukraine (according to HRW and UN / GICHD reports). And It can send real-time flight data and Identified Ordnance Information from the ML engine to the EA7 User Interface software with accurate GPS positioning.

Claims

CLAIMSWhat is claimed is1. A system for the aerial detection of unexploded ordinance comprising:- an land or aerial drone having:- a transmission array responsive to a multi-signal generator module configured to transmit a plurality of signal waves in various dynamic frequencies and waveforms forming a multi-signal layer;- a reception array configured to receive the reflected signals from the multi-signal layer;- a digital signal processor configured to consolidate the reflected signals from the multisignal layer into a single solidified signal; and- a data acquisition unit responsive to the digital signal processor configured to process the solidified signal and its components through a custom machine learning model designed to distinguish an object of interest from secondary objects within a target area.

2. The system of claim 1 , wherein said unexploded ordinance comprises a buried or partially buried landmine.

3. The system of claim 1, wherein said multi-signal layer comprises a waveform that is configured to be beamformed to a target location.

4. The system of claim 1, wherein said multi-signal layer comprises a plurality of signal layers using different waves in different dynamically changing frequency ranges.

5. The system of claim 1, wherein said multi-signal layer is generated using variable power densities and further be configured to include a penetration focal point6. The system of claim 1, wherein said multi-signal generator module comprises one or more analog signal generators that are responsive to a one or more waveformers, amplifiers and / or filters, configured to generate and dynamically adjust, amplify and / or filter the signal.

7. The system of claim 1, wherein said multi-signal generator module comprises one or more analog to digital converters configured to convert analog signals into a digital format that is further responsive to one or more signal processing units.

8. The system of claim 8, wherein said one or more signal processing units comprise one or more field-programmable gate array (FPGA) processors.

9. The system of claim 1, wherein said multi-signal generator module is responsive to one or more frequency mixers.

10. The system of claim 9, wherein said one or more frequency mixers comprises one or more IQ mixers configured to oscillate the signals with a baseband frequency.

11. The system of any of claims 9-10, wherein said a mixer mixes low frequency signal waves with the base frequency.

12. The system of claim 10, wherein said baseband frequency comprises a targeted spectrum range of a unique layer of the multi-signal layer.

13. The system of any of claims 1-11, wherein said multi-signal layer comprises continuous signal waves having multiple frequencies ranging from 350 MHz to 7 GHz in adaptive frequencydomain and time-domain signal patterns..

14. The system of any of claims 1-12, wherein said multi-signal generator module is responsive to one or more filter and / or amplifiers.

15. The system of claim 1, wherein said reception array is positioned approximately perpendicular or parallel to the transmission array.

16. The system of claim 1, wherein the reflected signals are processed by one or more filters and / or amplifiers configured to increase the signal to noise ratio.

17. The system of claim 1, wherein the machine learning model comprises a convolutional neural network (ConvNet).

18. The system of claim 1, and further comprising a ground planning alignment module configured to calibrate the received signals based on first layer values in respect to the ground reference value.

19. The system of claim 1, and further comprising an analog to digital converter.

20. The system of claim 1, and further comprising an onboard computer responsive to one or more sensors configured to measure and transmit real-time data of the aerial drone’s location, depth, altitude, image data and various sensor outputs through a local network to a server user interface.

21. A multi-spectrum aerial-based detection array system comprising:- one or more antenna array modules having:- a transmission array responsive to a multi-signal generator module configured to transmit a plurality of signal waves in various dynamic frequencies and waveforms forming a multi-signal layer;- a reception array configured to receive the reflected signals from the multi-signal layer;- a sensor management module responsive to the one or more antenna array and configured to transmit one or more sensors signals or capture one or more images;- a digital signal processor configured to consolidate the reflected signals from the multisignal layer into a single solidified signal;- a central processing module configured to dynamically adjust, fuse and process the solidified signal and sensor signals and / or images to distinguish an object of interest from secondary objects within a target area.

23. The system of claims 21, wherein the detection array is mounted to an aerial drone or a land- based drone.

23. The system of claims 21, wherein said one or more antenna array modules are dynamically adjustable.

24. The system of claims 21, wherein said one or more antenna array modules are dynamically adjustable via a stepper motor.

26. The system of claims 21, wherein said sensor management module comprises at least one of the following:- a hyperspectral imager;- a high-definition RGB imager; and- a LIDAR altimetry sensor.

25. The system of claims 21, wherein said object of interest comprises unexploded ordinance.

26. The system of claims 25, wherein said unexploded ordinance comprises a buried or partially buried landmine.

26. The system of claims 21, wherein said sensor signals and / or images is selected from: a hyper spectral image, a high-definition RGB image, a LIDAR signal.

27. The system of claims 21, wherein said multi-signal layer comprises a waveform that is configured to be beamformed to a target location.

28. The system of claims 21, wherein said multi-signal layer comprises a plurality of signal layers using different waves in different dynamically changing frequency ranges.

29. The system of claims 21, wherein said multi-signal layer is generated using variable power densities and further be configured to include a penetration focal point30. The system of claims 21, wherein said multi-signal generator module comprises one or more analog signal generators that are responsive to a one or more waveformers, amplifiers and / or filters, configured to generate and dynamically adjust, amplify and / or filter the signal.

31. The system of claims 21, wherein said multi-signal generator module comprises one or more analog to digital converters configured to convert analog signals into a digital format that is further responsive to one or more signal processing units.

32. The system of claims 21, wherein said one or more signal processing units comprise one or more field-programmable gate array (FPGA) processors.

33. The system of claims 21, wherein said multi-signal generator module is responsive to one or more frequency mixers.

34. The system of claims 21, wherein said one or more frequency mixers comprises one or more IQ mixers configured to oscillate the signals with a baseband frequency.

35. The system of claims 33-34, wherein said a mixer mixes low frequency signal waves with the base frequency.

36. The system of claims 35, wherein said baseband frequency comprises a targeted spectrum range of a unique layer of the multi-signal layer.

37. The system of claims 21-37, wherein said multi-signal layer comprises continuous signal waves having multiple frequencies ranging from 350 MHz to 7 GHz in adaptive frequency-domain and time-domain signal patterns.

38. The system of claims 21, wherein said multi-signal layer comprises continuous signal waves having multiple frequencies ranging from 250 MHz to 5 GHz peaks combined in one signal wave and in dynamically changing cosine and square waves.

39. The system of claims 21-38, wherein said multi-signal generator module is responsive to one or more filter and / or amplifiers.

40. The system of claims 21 , and further comprising a ground planning alignment module configured to calibrate the received signals based on first layer values in respect to the ground reference value.

41. The system of claims 21, and further comprising an analog to digital converter.

Images