Visual localization of aerial vehicles using dynamic arbitrary uncertainty

Abstract

The invention discloses visual localization of aerial vehicles using dynamic arbitrary uncertainty. Techniques for positioning a vehicle in real time using dynamic uncertainty estimation are presented. The techniques include obtaining a topographic image captured by a vehicle; the topographic image is transmitted to a trained evidence deep learning neural network subsystem, and a dynamic uncertainty value and a first feature vector are obtained from the trained evidence deep learning neural network subsystem in real time; for each of the plurality of candidate topographic locations, comparing the first feature vector with a respective second feature vector representing the candidate topographic location, obtaining a respective similarity score therefrom; for at least one of the plurality of candidate topographic locations, updating a respective location weight in real time by a recursive Bayesian estimator based on the dynamic uncertainty value and a respective similarity score; estimating the position of the carrier in real time based on the plurality of position weights; and the position of the carrier is provided.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to geolocating aerial vehicles. BACKGROUND

[0002] Many aerial vehicles utilize GPS to obtain a position estimate, which aids in navigation. However, GPS signals are susceptible to interference, for example, by bad actors or spoofing. Relying solely on GPS for positioning can lead to single point of failure for the navigation system, a safety-critical component of the aerial vehicle. GPS-denied positioning aims to localize the aerial vehicle through other means of sensing, without relying on GPS.

[0003] Cameras are a popular means of sensing in GPS-denied positioning due to their ubiquity and low cost. Visual images taken from an aerial vehicle can be compared to existing images of an area and matched to determine the aerial vehicle’s location, a task known as visual position recognition (VPR). Although VPR often refers to matching between images taken from a ground vehicle and pre-existing images taken from the same perspective, VPR can be performed between a variety of perspectives: ground-to-ground, aerial-to-satellite, and even ground-to-satellite. An aerial vehicle can use VPR to localize without GPS by comparing images taken from the aerial vehicle during flight to pre-existing satellite images. However, matching aerial vehicle images to pre-existing satellite images can be challenging due to, for example, variations in camera parameters, lighting, weather, seasonal effects, environmental structure changes, and sensor noise.

[0004] Statistical estimators can combat sensor noise and improve VPR performance by processing images over time. However, statistical estimators, such as particle filters, require uncertainty estimates to well-perform the combination of measurements to cancel out noise. Manual tuning measurement models that provide uncertainty estimates are difficult to calibrate because they rely on many different factors that can be difficult to quantify or explain. Due to their calibration difficulty, manual tuning measurement models typically only provide static uncertainty estimates during operation. This means that these measurement models assume that the uncertainty level of all input measurements is the same over time. Importantly, visual measurements are prone to variable noise levels, for example, because while some images contain salient features that uniquely identify the image capture location, other images contain features that lack visual distinctiveness, and perceptual aliasing can occur with high probability. Static measurement models flatten these differences, assuming that across all image matches, the uncertainty level is the same. As a result, statistical estimators that rely on static uncertainty estimates produce inaccurate suboptimal results.

[0005] Machine learning (ML) algorithms are well suited for pattern matching. ML architectures like convolutional neural networks (CNNs) and vision transformers (ViTs) have succeeded in many computer vision tasks, far outperforming non-ML systems. However, despite the advantages of ML in pattern matching and generalization, appearance variations in input images still produce a level of arbitrary uncertainty in ML image matching.

[0006] However, ML systems often lack the ability to explain their internal workings or to reliably quantify uncertainty. This limits their ability to integrate with other modules, like statistical estimators, in larger decision systems.

[0007] Some attempts to include uncertainty estimation with machine learning, like Bayesian neural networks, ensemble methods, and Monte Carlo dropout methods, involve some form of sampling, which is computationally expensive. Here, sampling refers to passing input data through a machine learning system multiple times. The first type of sampling involves repeatedly perturbing input data and passing it through a machine learning system. The second type of sampling involves passing input data through many slightly different machine learning systems. Both types of sampling consume excessive power, and depending on the specific implementation, can also consume excessive time and / or require excessive processing footprint. Systems deployed in the real world, where uncertainty estimation is most critical, often require algorithms to run in real time on hardware with small footprint, low power consumption. These limitations make the approach of extensive sampling impractical.

[0008] Other attempts to combine uncertainty estimation with machine learning, such as Gaussian neural networks, are inherently inaccurate and too slow for applications like real-time localization of high-speed aerial vehicles. SUMMARY

[0009] According to various embodiments, a method for real-time localization of an airborne vehicle using dynamic uncertainty estimation is proposed. The method includes: acquiring a terrain image captured by the airborne vehicle; passing the terrain image captured by the airborne vehicle to a trained evidence deep learning neural network subsystem to obtain, in real-time, a dynamic uncertainty value and a first feature vector, wherein the trained evidence deep learning neural network subsystem is trained to provide a feature vector corresponding to the input image and an associated dynamic uncertainty value; for each of a plurality of candidate terrain locations, comparing the first feature vector with a corresponding second feature vector representing a candidate terrain location among the plurality of candidate terrain locations to obtain a corresponding similarity score, thereby real-time associating each candidate terrain location among the plurality of candidate terrain locations with a corresponding similarity score; for at least one of the plurality of candidate terrain locations, updating a corresponding location weight in real-time using a recursive Bayesian estimator at least based on the dynamic uncertainty value and the corresponding similarity score to obtain a plurality of location weights; estimating the location of the airborne vehicle in real-time based on the plurality of location weights; and providing the location of the airborne vehicle.

[0010] Various optional features of the above method embodiments include the following. The provision may include providing to a navigation system, wherein the air vehicle navigates based on its position. A corresponding dynamic uncertainty value can be obtained from passing a corresponding terrain image through a trained evidence deep learning neural network subsystem in a single pass. The method may include repeatedly acquiring, passing, comparing, updating, and estimating to update the air vehicle's position in real time. The trained evidence deep learning neural network subsystem may be trained using a training corpus comprising labeled pairs of matching and non-matching images. The trained evidence deep learning neural network subsystem may be trained by performing actions for each corresponding image pair in the training corpus including: individually passing each image of the corresponding image pair through one or more neural networks to obtain corresponding pair feature vectors; calculating a corresponding loss based at least on the corresponding pair feature vectors; and updating the weights of one or more neural networks based on the corresponding loss. Each corresponding second feature vector may represent a corresponding satellite image of a candidate terrain location among a plurality of candidate terrain locations. The weights of each corresponding location may be further updated based on the corresponding uncertainty value of the corresponding second feature vector. The recursive Bayesian estimator may include a particle filter. The update may be performed using a measurement model of the recursive Bayesian estimator.

[0011] According to various embodiments, a system for real-time localization of an airborne vehicle using dynamic uncertainty estimation is proposed. The system includes: a non-transient computer-readable medium containing instructions; and at least one electronic processor that executes the instructions to perform operations including: acquiring a terrain image captured by the airborne vehicle; passing the terrain image captured by the airborne vehicle to a trained evidence-based deep learning neural network subsystem, from which a dynamic uncertainty value and a first feature vector are obtained in real time, wherein the trained evidence-based deep learning neural network subsystem is trained to provide a feature vector corresponding to an input image and an associated dynamic uncertainty value; for each of a plurality of candidate terrain locations, comparing the first feature vector with a corresponding second feature vector representing a candidate terrain location among the plurality of candidate terrain locations, thereby obtaining a corresponding similarity score, whereby each candidate terrain location among the plurality of candidate terrain locations is associated with a corresponding similarity score in real time; for at least one of the plurality of candidate terrain locations, updating a corresponding location weight in real time by a recursive Bayesian estimator at least based on the dynamic uncertainty value and the corresponding similarity score, thereby obtaining a plurality of location weights; estimating the location of the airborne vehicle in real time based on the plurality of location weights; and providing the location of the airborne vehicle.

[0012] Various optional features of the above system embodiments include the following. The provision may include providing to a navigation system, wherein the air vehicle navigates based on its position. A corresponding dynamic uncertainty value can be obtained from passing a corresponding terrain image through a trained evidence deep learning neural network subsystem in a single pass. These actions may also include repeated acquisition, passing, comparison, updating, and estimation to update the air vehicle's position in real time. The trained evidence deep learning neural network subsystem can be trained using a training corpus comprising labeled pairs of matching and non-matching images. The trained evidence deep learning neural network subsystem can be trained by performing actions on each corresponding image pair in the training corpus including: passing each image of the corresponding image pair individually through one or more neural networks to obtain corresponding pair feature vectors; calculating a corresponding loss based at least on the corresponding pair feature vectors; and updating the weights of one or more neural networks based on the corresponding loss. Each corresponding second feature vector may represent a corresponding satellite image of a candidate terrain location among a plurality of candidate terrain locations. The weights of each corresponding location may be further updated based on the corresponding uncertainty value of the corresponding second feature vector. The recursive Bayesian estimator may include a particle filter. The update can be performed using a measurement model of the recursive Bayesian estimator.

[0013] Clause 1. A system (100) for real-time positioning of an airborne vehicle using dynamic uncertainty estimation, the system comprising: a non-transient computer-readable medium containing instructions; and at least one electronic processor that executes the instructions to perform operations including:

[0014] Obtain the terrain image (104) captured by the aerial vehicle (202);

[0015] The terrain image captured by the air vehicle is transmitted (204) to a trained evidence deep learning neural network subsystem (110) to obtain a dynamic uncertainty value and a first feature vector in real time, wherein the trained evidence deep learning neural network subsystem is trained to provide a feature vector corresponding to the input image and an associated dynamic uncertainty value;

[0016] For each of the multiple candidate terrain locations, the first feature vector is compared with the corresponding second feature vector representing the candidate terrain location among the multiple candidate terrain locations (206) to obtain the corresponding similarity score, thereby associating each of the multiple candidate terrain locations with the corresponding similarity score in real time;

[0017] For at least one of the multiple candidate terrain locations, the corresponding location weight is updated in real time (208) by a recursive Bayesian estimator based at least on the dynamic uncertainty value and the corresponding similarity score, thereby obtaining multiple location weights;

[0018] The position (140) of the air vehicle is estimated in real time (210) based on the multiple position weights; and

[0019] Provide the location of the air vehicle (212).

[0020] Clause 2. The system according to Clause 1, wherein the provision includes providing to a navigation system, wherein the air vehicle navigates based on the air vehicle's location.

[0021] Clause 3. The system according to Clause 1, wherein the corresponding dynamic uncertainty value is obtained from the trained evidence deep learning neural network subsystem that transmits the corresponding terrain image once.

[0022] Clause 4. The system according to Clause 1, wherein the action further includes repeating the acquisition, the transmission, the comparison, the update, and the estimation to update the position of the air vehicle in real time.

[0023] Clause 5. The system according to Clause 1, wherein the trained evidence deep learning neural network subsystem is trained using a training corpus comprising labeled pairs of matching and non-matching images.

[0024] Clause 6. The system according to Clause 5, wherein the trained evidence deep learning neural network subsystem is trained by performing actions including the following for each corresponding image pair in the training corpus:

[0025] Each image in a corresponding image pair is individually passed through one or more neural networks to obtain the corresponding pair of feature vectors;

[0026] The corresponding loss is calculated at least based on the corresponding paired feature vectors; and

[0027] The weights of the one or more neural networks are updated based on the corresponding loss.

[0028] Clause 7. The system according to Clause 1, wherein each corresponding second feature vector represents a corresponding satellite image of a candidate terrain location among the plurality of candidate terrain locations.

[0029] Clause 8. The system according to Clause 1, wherein the weight of each corresponding position is also updated based on the corresponding uncertainty value of the corresponding second feature vector.

[0030] Clause 9. The system according to Clause 1, wherein the recursive Bayesian estimator includes a particle filter.

[0031] Clause 10. The system pursuant to Clause 9, wherein the update is performed using a measurement model of the recursive Bayesian estimator.

[0032] Unless otherwise stated or contradicted, the inventors have considered and are capable of combining the above-described elements and the elements in the specification (including multiple dependent combinations). Attached Figure Description

[0033] When considered in conjunction with the accompanying drawings, and referring to the following detailed description of the example, the various features of the example can be more fully understood and better appreciated, wherein:

[0034] Figure 1 These are schematic diagrams of systems for real-time positioning of airborne vehicles using dynamic uncertainty estimation, according to various embodiments; and

[0035] Figure 2 This is a flowchart of a method for real-time positioning of an airborne vehicle using dynamic uncertainty estimation, according to some embodiments. Detailed Implementation

[0036] Reference will now be made in detail to the exemplary embodiments shown in the accompanying drawings. Wherever convenient, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. In the following description, reference is made to the accompanying drawings, which form a part thereof, in which specific exemplary examples in which the invention may be practiced are shown by way of illustration. These examples are described in sufficient detail to enable those skilled in the art to practice the invention, and it should be understood that other examples may be utilized and modifications may be made without departing from the scope of the invention. Therefore, the following description is merely exemplary.

[0037] Evidence-based deep learning (EDL) treats neural network training as evidence acquisition, representing neural network predictions as distributions with mean and variance. Some embodiments use EDL uncertainty estimation, which provides well-calibrated uncertainty from a single pass of input data through the neural network, offering scalable uncertainty estimation without requiring resource-intensive sampling. Some embodiments use a Gaussian loss function to train the neural network, which in turn trains the machine learning subsystem to provide uncertainty estimation with only a single forward pass of input data. Compared to ML-based VPR with manually tuned uncertainty models, some embodiments are expected to provide a 98% reduction in final estimation error.

[0038] Some embodiments use EDL uncertainty estimation for measurements provided to a recurrent Bayesian estimator (e.g., a particle filter) in a visual positioning system. According to various embodiments, aerial imagery taken from an airborne vehicle during deployment is compared in real-time with existing satellite imagery of the aircraft's operational area. According to various embodiments, a neural network equipped with EDL is used to predict the similarity between different regions of the aerial imagery and previously acquired and stored satellite imagery. Therefore, some embodiments improve upon prior art systems that use neural networks to predict only image similarity and rely on statically tuned measurement models to attempt to introduce uncertainty into the particle filter. Due to the complexity involved in manually tuned measurement models, such prior art systems typically assume a static level of uncertainty in the input image, which produces inaccurate results. Some embodiments improve upon such prior art systems by using a neural network that simultaneously predicts both similarity and uncertainty during a single neural network inference process. For example, embodiments using EDL and a recurrent Bayesian estimator for visual positioning provide dynamically generated, accurate arbitrary uncertainty values ​​that take into account input images containing varying noise levels. Therefore, such embodiments provide more accurate positioning results compared to prior art systems that rely on statically tuned uncertainty estimation.

[0039] These and other features and advantages are illustrated and described herein with reference to the accompanying drawings.

[0040] Figure 1This is a schematic diagram of a system 100 for real-time localization of an airborne vehicle using dynamic uncertainty estimation, according to various embodiments. Broadly speaking, system 100 includes a combination of an EDL neural network subsystem 110 and a recurrent Bayesian estimator subsystem 130 (e.g., a particle filter) for visual localization. System 100 can be implemented on an airborne vehicle. Figure 1 As shown, system 100 includes an airborne image sensor 102, a nontransferable electronic prior image storage device 106, and one or more electronic processors.

[0041] As a non-limiting example, the image sensor 102 may be a visible light camera, an infrared camera, or a synthetic aperture radar (SAR) sensor.

[0042] Electronic previously acquired image storage device 106 stores previously acquired images (or feature vectors representing them, as described in detail below with reference to EDL neural network subsystem 110) taken by satellites or aircraft days, weeks, months, or years prior to the deployment of system 100. The previously acquired images depict the terrain on which the airborne vehicle is deployed. Each previously acquired image 108 (or its corresponding feature vector) is stored in association with data indicating its location, such as latitude and longitude coordinates. According to some embodiments, each previously acquired image (or feature vector) is also stored in association with an uncertainty value representing the confidence level of the image's content.

[0043] EDL neural network subsystem 110 may include one or more neural networks. For example, EDL neural network subsystem 110 may include one or two neural networks. If EDL neural network subsystem 110 includes two neural networks, the neural networks may have the same architecture; one network for real-time images and one network for previously acquired images. Each of the one or more neural networks may include a backbone neural network and a head for extracting features from the input image. The backbone neural network may include an image classification network such as ResNet, ViT, or VGG, but without a final classification layer, such that its output is a high-dimensional feature vector rather than an image classification. More generally, the backbone neural network may include convolutional neural network layers, visual transformers, or another architecture for extracting feature vectors from images. The head may include pooling layers, fully connected layers, and activation functions that ultimately produce feature embeddings and uncertainty values ​​as described herein. EDL neural network subsystem 110 may also include implementations of comparison functions, such as Euclidean distance, which, as a non-limiting example, compare feature vectors output by the one or more neural networks.

[0044] Prior to deployment, the EDL neural network subsystem 110 may have been trained using a training corpus comprising labeled pairs of matching and non-matching images, where the labels of the corresponding image pairs indicate whether the images match. The evidence deep learning subsystem 110 is trained using each corresponding image pair from the training corpus by performing the following actions: Each image of each labeled image pair is individually input through one or more neural networks. The neural network(s) generates a corresponding feature vector from each input image. The resulting feature vectors from the labeled image pairs are passed to a loss function that computes a loss. A first non-restricted example loss function may include Euclidean distance, where the loss comprises the computed distance between two vectors in the hyperspace. A second non-restricted example loss function is a triple loss, which compares the baseline feature vector with both the matching and non-matching vectors. The loss function used to train the EDL neural network subsystem 110 may interpret the image feature vectors as a Gaussian distribution to allow the EDL neural network subsystem 110 to generate arbitrary uncertainty in the predictions. The loss is backpropagated through the neural network(s), and the weights of the neural network(s) are updated accordingly based on the loss function. This process can be repeated for each labeled image pair in the training corpus.

[0045] During deployment, system 100 can operate in real time, with each time step including the capture and processing of the real-time image 104 described herein. During deployment, image sensor 102 acquires real-time images 104 of the terrain below the air vehicle, for example, as still images within a video stream captured by image sensor 102. Each real-time image 104 acquired by image sensor 102 is provided to EDL neural network subsystem 110 and compared with one or more previously acquired images (e.g., 104). EDL neural network subsystem 110 outputs the similarity between pairs of images taken at different times (real-time image 104 acquired during air vehicle operation and previously acquired image 108) and at least the uncertainty associated with real-time image 104. According to some embodiments, the EDL neural network subsystem may also output the uncertainty associated with previously acquired image 108 at this time. System 100 may perform multiple such comparisons at each time step, for example, comparing real-time image 104 with each of a plurality of candidate previously acquired images. Candidate previously acquired images may correspond to all images or their feature vectors stored in the electronic previously acquired image storage device 106, or a subset of previously acquired images or their feature vectors. For example, the subset may correspond to images or their feature vectors located within a specified radius of a previously determined position of an airborne vehicle.

[0046] Therefore, each comparison performed by the EDL neural network subsystem 110 between the real-time image 102 and the previously acquired image 104 results in the EDL neural network subsystem 110 producing an associated output 120, which includes at least a predicted similarity (denoted as μ) and a corresponding arbitrary uncertainty (denoted as σ) for the real-time image 104. The similarity μ represents how closely the compared images match, and each arbitrary uncertainty σ represents the confidence level of the content of the corresponding image. The output from the EDL neural network subsystem 110 is provided to the recursive Bayesian estimator subsystem 130.

[0047] As a non-limiting example, the recurrent Bayesian estimator subsystem 130 can be implemented using any of a variety of Bayesian estimators, such as a Kalman filter, Monte Carlo localization, or a particle filter. For illustrative and non-limiting purposes, the recurrent Bayesian estimator subsystem 130 is described with reference to a particle filter. The particle filter reweights its particles using the similarity μ from each comparison of the previously acquired image 108 with the real-time image 104 and the arbitrary uncertainty σ of the real-time image 104. According to some embodiments, the particle filter also reweights using the arbitrary uncertainty associated with the previously acquired image 108. Reweighting can be performed using a Gaussian measurement model. Instead of a statically manually tuned Gaussian measurement model, the measurement model used in the recurrent Bayesian estimator subsystem 130, according to various embodiments, is modulated by a dynamic arbitrary uncertainty output by the EDL neural network subsystem 110, compared to existing systems. The distribution of the dynamic arbitrary uncertainty values ​​is centered at zero, and the standard deviation of the distribution determines the standard deviation of the Gaussian measurement model. Similarity is the raw measurement value provided to the Bayesian estimator. For each previously acquired image 108 compared to the real-time image 104, the recursive Bayesian estimator subsystem 130 outputs a noise-filtered weight corresponding to the previously acquired image 108. Because each previously acquired image 108 (or its feature vector) is stored in the previously acquired image storage device 106 in association with data indicating the location depicted in the image, each weight output by the Bayesian estimator subsystem 110 corresponds to a location.

[0048] System 100 tracks the weights provided by the recursive Bayesian estimator subsystem and generates a real-time predicted location 140 at each time step, such as the resulting latitude and longitude. According to some embodiments, the predicted location may be the location corresponding to the highest weight. According to some embodiments, the predicted location may be a location represented by a weighted average of multiple locations with the highest weight (e.g., the peak of the two-dimensional density of the weights).

[0049] System 100 can be coupled to any of a variety of other systems (such as a system for displaying location or a navigation system) and provide it with a predicted location at each time step.

[0050] Compared to existing visual localization systems that use a vanilla neural network (without learned uncertainty values) combined with a particle filter, the EDL neural network subsystem 110, together with the recurrent Bayesian estimator subsystem 130, significantly improves localization performance for visual localization.

[0051] Note that while System 100 is described with reference to an airborne vehicle, by way of non-limiting example, System 100 can be used to locate any vehicle, such as a land vehicle or an underwater vehicle. For example, System 100 can be used to locate a ground vehicle relative to satellite imagery, such as real-time imagery captured using a ground vehicle or an aircraft that includes the ground vehicle in the field of view of a real-time image.

[0052] Figure 2 This is a flowchart of a method 200 for real-time localization of an aerial vehicle using dynamic uncertainty estimation, according to some embodiments. Method 200 can be used by various vehicles, such as aerial vehicles, land vehicles, water vehicles, or other moving objects capable of acquiring images of their surrounding environment. Method 200 can be implemented using systems such as those referenced herein. Figure 1 The system 100 shown and described. Method 200 can be implemented in real time to provide the actual position of the air vehicle during operation.

[0053] In method 202, method 200 includes acquiring terrain images captured by an aerial vehicle. The images can be obtained using an image sensor (such as a reference sensor). Figure 1 The image is obtained by the image sensor 102 shown and described. As a non-limiting example, the image can be obtained by a visible light camera, an infrared camera, or a synthetic aperture radar.

[0054] In step 204, method 200 includes feeding terrain images captured by an aerial vehicle to a trained EDL neural network subsystem, from which dynamic uncertainty values ​​and a first feature vector are obtained in real time. The trained EDL neural network subsystem may have been previously trained to provide feature vectors corresponding to the input image and associated dynamic uncertainty values, as referenced herein. Figure 1 The EDL neural network subsystem 110 is described.

[0055] In 206, method 200 includes, for each of a plurality of candidate terrain locations, comparing a first feature vector with a corresponding second feature vector representing a candidate terrain location among the plurality of candidate terrain locations, thereby obtaining a corresponding similarity score. References can be made as described herein. Figure 1 The EDL neural network subsystem 110 performs the comparison. From 206, each candidate terrain location is associated with its corresponding similarity score in real time.

[0056] In 208, method 200 includes, for at least one of the multiple candidate terrain locations in 206, updating the corresponding location weights in real time using a recursive Bayesian estimator, based at least on dynamic uncertainty values ​​and corresponding similarity scores. The updates can be as described in the references herein. Figure 1 The recursive Bayesian estimator subsystem 130 is described in the execution. The action of 208 provides multiple location weights, at least one of which is updated at 208.

[0057] In step 210, method 200 includes estimating the position of the airborne vehicle in real time based on multiple position weights. The actions of step 210 can be described as referenced herein. Figure 1 The system 100 is executed as described.

[0058] In step 212, method 200 includes providing the location of the airborne vehicle. This location can be provided to any of a variety of other systems, such as a system that displays the location, or provided to a navigation system.

[0059] Some examples can be executed using a computer program or assembly of programs. A computer program can exist in various forms, including active and inactive. For example, a computer program can exist as: (one or more) software programs consisting of source code, object code, executable code, or other program instructions; (one or more) firmware programs or hardware description language (HDL) files. Any of the above can be embodied in compressed or uncompressed form on a transient or non-transient computer-readable medium, including storage devices and signals. Exemplary computer-readable storage devices include conventional computer system RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, and magnetic disks, optical disks, or magnetic tapes.

[0060] This document describes aspects of the disclosure with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented using computer-readable program instructions executable by an electronic processor.

[0061] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via an electronic processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other equipment to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0062] In this embodiment, the computer-readable program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as C or similar programming languages). The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0063] As used herein, the terms “A or B” and “A and / or B” are intended to include A, B, or {A and B}. Furthermore, the terms “A, B, or C” and “A, B, and / or C” are intended to cover a single item, a pair of items, or all items, namely all of the following: A, B, C, {A and B}, {A and C}, {B and C}, and {A and B and C}. The term “or” as used herein means “and / or”.

[0064] As used herein, phrases such as “at least one of X, Y, and Z,” “one or more of X, Y, or Z,” “at least one or more of X, Y, and Z,” “at least one or more of X, Y, or Z,” “at least one or more of X, Y, and / or Z,” and “at least one of X, Y, and / or Z” are intended to include both single items (e.g., only X, or only Y, or only Z) and multiple items (e.g., {X and Y}, {X and Z}, {Y and Z}, or {X, Y, and Z}). The phrase “at least one of…” and similar phrases are not intended to convey a requirement that every possible item must exist, although every possible item may exist.

[0065] The techniques proposed and claimed herein are referenced and applied to physical objects and concrete examples of a practical nature that clearly improve the field of expertise and are therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to this specification contains one or more elements designated as “means for [performing] [function]” or “steps for [performing] [function]”, those elements shall be interpreted in accordance with 35 U.SC §112(f). However, for any claim containing elements designated in any other manner, those elements shall not be interpreted in accordance with 35 U.SC §112(f).

[0066] While the invention has been described with reference to exemplary examples, those skilled in the art will be able to make various modifications to the described examples without departing from the true spirit and scope. The terminology and descriptions used herein are for illustrative purposes only and are not intended to be limiting. In particular, although the method has been described by way of example, the steps of the method may be performed in a different order or simultaneously than those shown. Those skilled in the art will recognize that these and other variations are possible within the spirit and scope defined in the appended claims and their equivalents.

Claims

1. A method (200) of estimating a location of an aerial vehicle in real-time using dynamic uncertainty, the method comprising: obtaining (202) a terrain image (104) captured by the aerial vehicle; passing (204) the terrain image captured by the aerial vehicle to a trained evidential deep learning neural network subsystem (110) from which a dynamic uncertainty value and a first feature vector are obtained in real-time, wherein the trained evidential deep learning neural network subsystem is trained to provide a feature vector and an associated dynamic uncertainty value corresponding to an input image; for each of a plurality of candidate terrain locations, comparing (206) the first feature vector to a respective second feature vector representing a candidate terrain location of the plurality of candidate terrain locations from which a respective similarity score is obtained, whereby each of the plurality of candidate terrain locations is associated with a respective similarity score in real-time; for at least one of the plurality of candidate terrain locations, updating (208) a respective location weight in real-time based at least on the dynamic uncertainty value and the respective similarity score from which a plurality of location weights are obtained; estimating (210) a location (140) of the aerial vehicle in real-time based on the plurality of location weights; and providing (212) the location of the aerial vehicle.

2. The method of claim 1, wherein the providing comprises providing to a navigation system, wherein the aerial vehicle is navigated based on the location of the aerial vehicle.

3. The method of claim 1, wherein a respective dynamic uncertainty value is obtained from a single pass of a respective terrain image through the trained evidential deep learning neural network subsystem.

4. The method of claim 1, further comprising repeating the obtaining, the passing, the comparing, the updating, and the estimating, whereby the location of the aerial vehicle is updated in real-time.

5. The method of claim 1, wherein the trained evidential deep learning neural network subsystem is trained using a training corpus comprising labeled pairs of matching and non-matching images.

6. The method of claim 5, wherein the trained evidential deep learning neural network subsystem is trained by performing, for each respective pair of images in the training corpus, the acts comprising: passing each image of a respective pair of images individually through one or more neural networks from which respective paired feature vectors are obtained; computing a respective loss based at least on the respective paired feature vectors; and updating weights of the one or more neural networks based on the respective loss.

7. The method of claim 1, wherein each respective second feature vector represents a respective satellite image of a candidate terrain location of the plurality of candidate terrain locations.

8. The method of claim 1, wherein each respective location weight is updated further based on a respective uncertainty value of a respective second feature vector.

9. The method of claim 1, wherein the recursive Bayesian estimator comprises a particle filter.

10. The method of claim 9, wherein the updating is performed using a measurement model of the recursive Bayesian estimator.

Images