Systems, devices, and methods for denied / intermittent GPS navigation
Patent Information
- Application Number
- PCT/US2025/034671
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-06-21
- Filing Date
- 2025-06-20
- Publication Date
- 2026-11-12
AI Technical Summary
Modern aircraft, such as UAVs, face navigation challenges due to GPS jamming, which disrupts the availability of location, velocity, and time data, preventing them from following their flight path and reaching destinations.
A system comprising sensors like cameras, IMU, laser range finders, and GPS receivers generates alternative location information using inertial navigation, image matching, and position filtering to navigate autonomously, even in GPS-denied environments.
Enables aircraft to continue navigating to and identifying targets despite GPS jamming by switching to alternative location information, ensuring reliable flight paths and target engagement.
Smart Images

Figure US2025034671_12112026_PF_FP_ABST
Abstract
Description
[0001] Docket No. AERO-VI2405WO
[0002] PATENT COOPERATION TREATY APPLICATION TITLE: SYSTEMS, DEVICES, AND METHODS FOR DENIED / INTERMITTENT GPS NAVIGATION
[0003] INVENTORS: Steven Quintero and Andrew Richardson
[0004] CROSS-REFERENCE TO RELATED APPLICATION
[0005] This application claims the priority benefit of U.S. Provisional Patent Application Serial Number 63 / 663,040 filed June 21, 2024, which is incorporated herein by reference in its entirety.
[0006] TECHNICAL FIELD
[0007] Embodiments relate generally to flight control systems, and more particularly to Intermittent GPS navigation.
[0008] BACKGROUND
[0009] Modem aircrafts, such as unmanned aerial vehicle (UAV), rely on Global Positioning System (GPS) navigation systems, that are space-based radio-navigation systems using satellites, receivers, and algorithms to determine location, velocity, and time data for travel on land, sea, and air. However, GPS jamming is also used to intermpt and impede the availability of this data to an aircraft so as prevent it from following its flight path and / or finding a destination or target. GPS jamming is relatively uncomplicated technique that simply involves producing an RF signal strong enough to drown out the transmissions from GPS satellites.
[0010] SUMMARY
[0011] A system embodiment disclosed herein comprises: one or more sensors configured to collect sensor data for a first aircraft, the sensor data including at least one of: onboard information, location-based visual information, and multiple GPS signals; and a navigation module configured to detect whether an error has occurred on a GPS signal for the first aircraft based on the sensor data; wherein once the erroris detected, the navigation module is configured to replace the GPS signal with alternative location information such that the first aircraft is configured to continue navigating to a target area, and wherein the alternative location information is generated based on the sensor data.
[0012] In one embodiment, the one or more sensors may comprise at least one of one or more cameras, an inertial measurement unit (IMU), a laser range finder, and multiple GPS receivers. In one embodiment, the navigation module may comprise an inertial navigation system (INS); and wherein the INS may be configured to generate the alternative location information, by tracking movement of the aircraft from a known starting point based on the onboard information configured to be collected by the IMU. In one embodiment, the navigation module may be configured to detect the error by comparing the GPS signal with the alternative location information generated by the INS.
[0013] In one embodiment, the navigation module may comprise an image matching module; wherein the image matching module may be configured to generate the alternative location information, by matching the location-based visual information with a known map; and wherein the location-based visual information may be configured to be collected by at least one of the one or more cameras and the laser range finder. In one embodiment, the image matching module may comprise at least one of a laser terrain matching (LTM) module configured to match a real-time terrain profile collected from the laser range finder with the known map; and a digital scene matching (DSM) module configured to match a real-time image collected from the one or more cameras with the known map. In one embodiment, the navigation module may be configured to detect the error by comparing the GPS signal with the alternative location information generated by the image matching module.
[0014] In one embodiment, the navigation module may comprise a comparison module; and wherein the comparison module may be configured to detect the error by detecting inconsistencies between the multiple GPS signals configured to be received by the multiple GPS receivers).
[0015] AERO-VI2405WOIn one embodiment, the navigation module may comprise a position filter module; and wherein the position filter module may be configured to generate the alternative location information by fusing at least two location information of location information generated based on an image collected from the one or more cameras; location information generated based on a terrain profile collected from the laser range finder; and location information generated based on the onboard information collected from the IMU. In one embodiment, the position filter module may be configured to generate the alternative location information by fusing the at least two location information based on assigned weights.
[0016] In one embodiment, the navigation module may be configured to generate multiple alternative location information based on at least two of an aerial image collected from the one or more cameras; a terrain profile collected from the laser range finder; and the onboard information collected from the IMU; and wherein the navigation module may be configured to transition between the multiple alternative location information.
[0017] In one embodiment, when the navigation module detects that the error on the GPS signal ceases, the navigation module may be configured to revert from using the alternative location information to using the GPS signal. In one embodiment, the one or more sensors may be configured to be included in a second aircraft within a swarm, the swarm comprising the first aircraft and the second aircraft; and wherein the navigation module of the first aircraft may be configured to detect the error based on the sensor data transmitted from the second aircraft.
[0018] A method embodiment disclosed herein comprises: collecting, by one or more sensors, sensor data of an aircraft, the sensor data including at least one of onboard information, location-based visual information, and multiple GPS signals; detecting, by a GPS error detection module of a processor, whether an error has occurred on a GPS signal during navigation based on the sensor data; and replacing, by an alternative global positioning module of the processor, the GPS signal to alternative
[0019] AERO-VI2405WOlocation information once the error is detected such that the aircraft is configured to continue navigating to a target area; wherein the alternative location information is generated from the sensor data.
[0020] In one embodiment, detecting the error may include at least one of: comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the onboard information; comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the location-based visual information; and detecting, by the GPS error detection module, inconsistencies between the multiple GPS signals.
[0021] In one embodiment, the method may further comprise: generating the alternative location information based on the sensor data, wherein generating the alternative location information may include at least one of: generating, by an inertial navigation system (INS) module of the GPS error detection module, the alternative location information based on the onboard information; generating, by an image matching module of the GPS error detection module, the alternative location information based on an aerial image of the location-based visual information; generating, by the image matching module, the alternative location information based on a terrain profile of the location-based visual information; and generating, by a position filter module of the alternative global positioning module, the alternative location information by fusing at least two location information among multiple location information generated based on the onboard information, the aerial image, and the terrain profile.
[0022] In one embodiment, generating the alternative location information may include generating multiple alternative location information based on at least two of: the onboard information, the aerial image, and the terrain profile; and wherein the method may further comprise: transitioning, by the alternative global positioning module, between the multiple alternative location information.
[0023] AERO-VI2405WOIn one embodiment, the method may further comprise: reverting, by the alternative global positioning module, from using the alternative location information to using the GPS signal when the GPS error detection module detects that the error on the GPS signal ceases. In one embodiment, the method may further comprise: receiving, by the processor, the sensor data collected from the one or more sensors included in another aircraft within a swarm.
[0024] A system embodiment disclosed herein comprises: a GPS error detection module configured to detect whether an error has occurred on a GPS signal for an aircraft based on at least one of: a camera, an inertial measurement unit, a laser range finder, and multiple GPS receivers; wherein once the error is detected, the aircraft is configured to replace the GPS signal to alternative location information such that the aircraft is configured to continue navigating to a target area; and wherein the alternative location information is generated based on the camera, the IMU, and a laser range finder.
[0025] BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principals of the invention. Like reference numerals designate corresponding parts throughout the different views. Embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which:
[0027] FIG. 1 depicts a block diagram of a system for navigation in GPS-denied, intermittent, and / or spoofing environments, according to one embodiment;
[0028] FIG. 2 depicts an aircraft system, according to one embodiment;
[0029] FIG. 3 depicts a more detailed view of the aircraft system, according to one embodiment;
[0030] FIG. 4 depicts yet another detailed view of the aircraft system, which is corresponding to a close up M2 of the aircraft system 200 of FIG. 2, according to one embodiment;
[0031] FIG. 5 A depicts a flowchart of a method for navigating to a target area, according to one embodiment;
[0032] AERO-VI2405WOFIG. 5B depicts a flowchart of a method for generating one or more alternative location information, according to one embodiment;
[0033] FIG. 5C depicts a method for detecting an error on a GPS signal, according to one embodiment;
[0034] FIG. 5D depicts a flowchart of a method for engaging a target upon reaching or getting near a target zone;
[0035] FIG. 6 depicts an environment where the disclosed aircraft may operate in, according to one embodiment;
[0036] FIG. 7 illustrates an example top-level functional block diagram of a computing device embodiment;
[0037] FIG. 8 shows a high-level block diagram and process of a computing system for implementing an embodiment of the system and process;
[0038] FIG. 9 shows a block diagram and process of an exemplary system in which an embodiment may be implemented; and
[0039] FIG. 10 depicts a cloud computing environment for implementing an embodiment of the system and process disclosed herein.
[0040] DETAILED DESCRIPTION
[0041] An aircraft being subjected to a GPS jamming attack may be instantly aware that something is wrong, as the system will be unable to produce a geolocation result. The present disclosure includes embodiments of systems and methods for an aircraft to navigate to a target area by overcoming GPS jamming and / or spoofing. The present systems and methods may also be configured to identify a target within that area and engage the target.
[0042] FIG. 1 depicts a diagram of a system for navigation in GPS-denied, intermittent, and / or spoofing environments, according to one embodiment. The disclosed embodiments may use a layered approach to overcome GPS Jamming or Spoofing to autonomously navigate to a target area. With reference to FIG. 1, the system 100 may include a vehicle 110 (e.g., an aircraft system) configured to navigate to a target area for mission and a mission planning device 160 configured to control the actual mission of the vehicle 110. In some embodiments, the system 100 may
[0043] AERO-VI2405WOfurther include at least one additional vehicle 111 (e.g., another aircraft system). In this case, the multiple vehicles 110, 111 may be located within a swarm. The vehicle 110 may include one or more onboard sensors 112 to collect sensor data which may be configured to generate alternative location information of the vehicle 110. The alternative location information generated based on the sensor data may be used to provide GPS spoof detector functionality. The one or more onboard sensors 112 may include at least one of: one or more cameras 114, 116, an inertial measurement unit (IMU) 118, a laser range finder 120, and multiple GPS receivers 122. In some embodiments, the IMU 118 may provide onboard information to track an alternative location information from a known starting point. In some embodiments, the one or more cameras 114, 116 and the laser range finder 120 may provide location-based visual information (e.g., image, video, terrain profile, etc.) to generate an alternative location information through an image matching process. In some embodiments, the multiple GPS receivers 122 may be provided to compare multiple GPS signals received through the multiple GPS receivers 122.
[0044] In one embodiment, a computing module 130 of the vehicle 110 may include a processor 132 with an addressable memory 158 that includes a navigation module 134. The navigation module 134 may receive GPS information of the vehicle 110 from one or more GPS receivers 122 via a GPS module 136 to navigate to the target area. During navigation, the processor 132 may monitor whether there is an error, including GPS jamming or spoofing, via a GPS error detection module, or GPS jam / spoof detection module, 138 and determine if the system 100 can trust the GPS information. The GPS jam and / or spoof detection on this module 138 may be performed based on alternative source of location information of the vehicle 110 generated by at least one of: an inertial navigation system (INS) 140, an image matching module 142, and a comparison module 144. One embodiment may be configured to have all the alternative sources generated in parallel where in another embodiment, the system may rely on one source (in addition to the GPS) and immediately switch another source should any discrepancies be detected.
[0045] Alternatively, the system, to save on processing and computing resources, may run
[0046] AERO-VI2405WOtwo of the alternative sources while comparing those two sources to each other in isolation from the GPS information.
[0047] In some embodiments, the INS 140 may generate alternative location information of the vehicle 110 based on the IMU 118 (e.g., accelerometer, gyroscope, etc.). The onboard information, including vehicle performance parameters (e.g., Aircraft performance parameters) such as position, velocity, and orientation of the vehicle 110, may be dynamically collected at all times using the IMU 118, and the INS 140 may track movement of the vehicle 110 based on this onboard information from a known starting point. Accordingly, the GPS jam / spoof detection module 138 of the system 100 may predict the current location of the vehicle 110 (e.g., aircraft) so that it continuously compares that GPS information with the alternative location information generated based on the collected onboard information. If any spoofing happens, the GPS jam / spoof detection module 138 may immediately make a determination of such event. For example, although the vehicle 110 (e.g., aircraft) is traveling at a higher rate of speed than a speed where the aircraft is able to make a turn within a particular turning radius, spoofed GPS data may falsely indicate that the turn is occurring within the expected flight envelope, the system 100 may be configured to determine the vehicle 110 (e.g., aircraft) does not have the ability to make such a turn and alert the system 100. Accordingly, the aircraft may make adjustments to the flight path or flight pattern before it is too late to make the turn correctly, e.g., not have the ability to turn in time to maintain the flight path.
[0048] In some embodiments, the image matching module 142 may generate alternative location information of the vehicle 110 based on one or more capture devices, where the capture devices may include a first camera 114 (e.g., forward look IR camera), a second camera 116 (e.g., down looking camera), and a laser range finder 120. The image matching module 142 may include at least one of: a laser terrain matching (LTM) module 132, a digital scene matching (DSM) module 134, or other configurations that may provide an absolute positioning fix. The LTM module 132 may utilize the laser range finder 120 to compare or match the real-time terrain profile with a stored or known terrain map. The DSM module 134 may utilize the first
[0049] AERO-VI2405WOcamera 114 and / or the second camera 116 to compare or match the real-time visual or infrared image / video with a stored or known image map. The match updates from the LTM module 132 and DSM module 134 may be continuously fed into the navigation module 134. Based on the match results, the GPS jam / spoof detection module 138 of the navigation module 134 may check against the GPS information to determine if GPS spoofing is detected or occurring.
[0050] In some embodiments, the comparison module 144 may be used to detect GPS spoofing based on multiple GPS receivers 122. Specifically, the disclosed system embodiments may be configured to read in GPS information but have some thresholds for detecting whether any of the readings are getting spoofed, for example, velocity spoofed, position spoofed, etc. To overcome these limits, in some embodiments, the system 100 may use different GPS receivers 122 to determine whether the GPS information is reliable by detection of inconsistencies between multiple GPS signals received by the set of GPS receivers 122.
[0051] When the GPS jam and / or spoof detection device 138 detects that the GPS signal is jammed and / or spoofed, an alternative global positioning module 146 of the system 100 may automatically transition or switch to another means of determining positioning data to replace the GPS data from the GPS module 136 with alternate location information generated based on the sensor data. The alternative global positioning information (alternate location information) may be obtained, during a GPS jamming time, from the inertial navigation system (INS) 140 and / or the image matching module 142, such as the laser terrain matching (LTM) 132, the digital scene matching (DSM) 134, or other configurations that may provide an absolute positioning fix. In some embodiments, the alternative global positioning module 146 may enable a smooth transition between, or back and forth, among all the options available for providing the alternate location information. Based on the alternative location information, a control module 150 (e.g., flight control module), may direct the vehicle 150 to follow a designated path to the target area.
[0052] AERO-VI2405WOAs mentioned above, the LTM module 132 and DSM module 134 updates may be continuously fed into the navigation module 134. In some embodiments, as soon as the system 100 determines an image match (e.g., LTM or DSM match) between real-time sensor data (e.g., current terrain or scene data) to stored or known reference data, the alternative global positioning module 146 of the system 100 may be configured to fuse those sets of information to obtain the combined alternate position information, while checking it against the GPS information to determine if the system 100 can trust the GPS information and therefore detect a GPS spoofing. The combined image information may contain more sufficient, accurate information including real-time information (e.g., weather) and detailed information on the area, and replace the GPS information as an alternative global positioning information.
[0053] In some embodiments, the alternative global positioning module 146 may generate alternative location information based on more than one alternative source of information. Using a position filter 148, based on the weight on the IMU 118 (e.g., accelerometer, gyroscope), the first and second look cameras 114, 116 (e.g., forward look IR camera, down looking camera), the laser range finder 120, the INS 140, and image matching module 142, the alternative global positioning module 146 may provide a single solution of alternative location information. Specifically, the position filter 148 may combine, merge, and / of fuse all or at least two sources of the sensor data together and result in an estimate position of the vehicle 110. The multiple sources of information may be weighted and determined online in real-time, where in one embodiment, data integration process of bringing together information from multiple sources to create a unified dataset may be used. Some examples of data integration techniques may be data blending, data merging, concatenation, consolidation, etc.
[0054] In one embodiment, the laser train matching (LTM) may use the laser range finder 120 and its reading combined with other detailed information (e.g., orientation data, map data) to geolocate itself and create alternative global positioning information. In one embodiment, the digital scene matching (DSM) algorithm may use satellite imagery of the ground, and the laser terrain matching (LTM) may use an
[0055] AERO-VI2405WOelevation map based on the satellite imagery of the ground to create alternative global positioning information.
[0056] In one embodiment, the vehicle 110 may collect the sensor data, such as their operational data, which may correspond to the onboard information and route or location-based visual information, from other vehicle(s) 111 (e.g., other aircraft system) in the swarm or in the area, via a communication module 152. Accordingly, even though GPS data is not being received, communications may be transmitted to and received from other vehicle to substitute for the denied GPS information.
[0057] Multiple vehicles (aircrafts) may communicate with each other using various methods, including mesh networks, cellular communication (4G / 5G), and ad hoc networks (MANETs). These techniques may enable the vehicles to share data (specifically GPS data), coordinate actions, and potentially extend their operational range and reliability.
[0058] The layered approach may allow the vehicle 110 to survive any GPS denying environments that it may encounter. Accordingly, if, during navigation (e.g., during midflight), the vehicle 110 (e.g., aircraft system) hits the GPS jammer and loses location, then the vehicle 110 may automatically switch over to one of these other alternatives and continue navigating to a target area. When GPS jamming and / or spoofing ceases and trusted GPS information becomes available, then the vehicle 110 may automatically fall back or revert from using the alternative location data to using the GPS information.
[0059] Accordingly, the below list may provide a summary of the features disclosed: Laser terrain matching (LTM), Digital scene matching (DTM), smooth transition between options, fall back to GPS when it becomes available, a navigation module always running for obtaining alternate position information, LTM match bring in, DSM match (digital scene matching) bring in, fusing LTM and DSM match together from multiple sources (continuously), checking against GPS to determine whether it may be relied on at that time, spoofing detection, weights between sources of position
[0060] AERO-VI2405WOinformation (e.g., covariance), using metrics to determine if it is being spoofed based on aircraft dynamics and maneuverability to detect.
[0061] In one embodiment, a layered approach may be implemented to overcome GPS Jamming or Spoofing to autonomously navigate to a target area. Accordingly, once a target within the target area is identified, the vehicle 110 (e.g., aircraft system) may continue to go toward the targe by substituting alternative location information, even if the GPS is jammed and the vehicle is no longer able to confirm the exact location using that GPS data.
[0062] In one embodiment, when the vehicle 110 reaches or gets near the defined area of interest, the vehicle may identify one or more targets in the defined area of interest. In the disclosed embodiments, the vehicle 110 (e.g., an aircraft system, such as an unmanned aerial vehicle (UAV)), via a tracker (e.g., Automatic Target Recognition (ATR) package) of a computer vision module 156, may identify one or more targets in the defined area of interest, for example, up to approximately 250+ miles from the starting location. Then, the mission module 154 of the vehicle 110 may use its payload for the mission 124, to engage the one or more targets. The tracker of the computer vision module 156 may detect one or more targets by searching the target area and determining which object matches a preprogrammed set of targets. In one embodiment, during the terminal phase, which starts while the vehicle 110 is in a search mode, the vehicle 110 may reach its designated target area, generate navigation waypoints according to a search pattern, and automatically search this area to find targets according to preloaded models or preloaded targets. In one example, the models may include different vehicle types in them, and whenever the vehicle 110 first detects a vehicle type, it may determine whether the detected vehicle type is a predetermined vehicle type (e.g., truck). The system 100 may use image-based object detection and be configured to enter into the search pattern followed by hitting the target. In one embodiment, the mission planning device 160 may communicate with the vehicle 110 to confirm the engagement of the one or more targets via the communication module 152.
[0063] AERO-VI2405WOFIG. 2 depicts an aircraft system 200, according to one embodiment. The aircraft system 200 may include some or all of the components of the vehicle shown in FIG. 1. With reference to FIG. 2, a first camera 214 of the aircraft system 200 may be positioned on a nose of a fuselage and configured to capture images and / or videos of the scenery for use of providing alternative location information and / or identifying one or more targets. In some embodiments, the first camera 214 may include at least one feature of: a forward look or an IR camera but is not limited thereto. In some embodiments, the first camera 214 may be an integrated forward look IR camera. The aircraft system 200 with an integrated forward look IR camera 214 may be configured to locate and confirm one or more targets via a computer vision module, while continuing navigation to the target area in GPS-jammed and / or spoofed environments by providing alternative location information (e.g., LTM and DSM functionality).
[0064] In some embodiments, the aircraft system 200 may further include a second camera 216 positioned on the bottom of the fuselage. In some embodiments, the second camera 216 may be a down looking camera. The first camera 214 and the second camera 216 may be pointed in different directions to gather information on different parts of the environment. Additionally, the aircraft system 200 may include fully integrated avionics (e.g., computing module equipped with a Graphics Processing Unit (GPU)), with vision-based navigation solution and propulsion pod.
[0065] FIG. 3 depicts a nose of an aircraft system 300 which is a close up Ml of the aircraft system 200 of FIG. 2, according to one embodiment. FIG. 3 illustrates the nose of the fuselage showing the details of where a first camera 314 (e.g., an IR camera) may be installed. With reference to FIG. 3, the nose of the fuselage may include a groove, or hole, 313 configured to receive the IR camera 314. In some embodiments, the groove, or hole, 313 may be extended along a longitudinal axis of the fuselage to accommodate the majority of the IR camera 314, exposing a portion of the IR camera 314 that is configured to capture images (e.g., a front portion). The nose of the fuselage may have a shape that tapers toward the front, and the inner top surface of the groove, or hole, 313 may be longer than the inner bottom surface. That is, the IR camera 314 may be positioned inside the groove, or hole, 313 so that the top
[0066] AERO-VI2405WOsurface of the IR camera 314 is completely covered by the inner top surface of the groove, or hole, 313, while a portion of the bottom surface of the IR camera 314 exposed to outside. The IR camera 314 may continually capture images and / or videos of the scenery for use of identifying one or more targets and / or providing alternative location information (e.g., the LTM and DSM functionality) as mentioned above.
[0067] FIG. 4 depicts a side view of a portion of an aircraft system 400 which may correspond to a close up M2 of the aircraft system 200 of FIG. 2, according to one embodiment. FIG. 4 illustrates the details of the portion where fully integrated computing module, or avionics, 430 (e.g., compute module,) with a down looking camera 416 for vision-based navigation solution and a propulsion pod 408 may be installed. The integrated avionics 430 may include a processor having an addressable memory where the processor may be configured to execute the components for determining whether any spoofing is happening and / or there is a GPS jamming so that the system may remedy and continue to have the ability to determine the location coordinates of the aircraft system 400 even if via an alternative method as disclosed herein.
[0068] FIG. 5 A depicts a flowchart of a method for navigating to a target area, according to one embodiment. With reference to FIG. 5A, the method 450 may include the steps of receiving, by a GPS receiver, GPS information of a vehicle to navigate to a target area (step 452); collecting, by one or more sensors, sensor data for the vehicle (step 453); receiving, by a processor, the sensor data from the one or more sensors (step 454); generating, by a navigation module, one or more alternative location information (step 456); during navigation, detecting, by a GPS error detection module, whether at least one of GPS signal denial jamming, or spoofing has occurred using alternative sources of information (e.g., alternative location information or multiple GPS signals) based on the sensor data (step 458); replacing, by an alternative global positioning module, the GPS signal to the alternative location information once the at least one of GPS signal denial jamming, and / or spoofing is detected (step 460); continuing, by the vehicle, navigation to the target area autonomously using the alternative location information (step 462); transitioning, by
[0069] AERO-VI2405WOthe alternative global positioning module, between multiple alternative location information (step 464); and reverting, by the alternative global positioning module, from using the alternative location information to using the GPS signal when the GPS error detection module detects that the error on the GPS signal ceases (step 466).
[0070] FIG. 5B depicts a method for generating one or more alternative location information, according to one embodiment. With reference to FIG. 5B, the method 470 for generating the alternative location information may include at least one of: generating, by an INS module of the GPS error detection module, the alternative location information based on the onboard information (step 472); generating, by an image matching module of the GPS error detection module, the alternative location information based on an aerial image of the location-based visual information (step 474); generating, by the image matching module, the alternative location information based on a terrain profile of the location-based visual information (step 476); and generating, by a position filter module, the alternative location information by fusing / integrating at least two location information among multiple location information generated based on the onboard information, the aerial image, and the terrain profile (step 478). Thereby combining multiple data sources to create a more comprehensive and useful dataset that is more complete, consistent, and accurate than any single source could provide.
[0071] FIG. 5C depicts a method for detecting an error on a GPS signal, according to one embodiment. With reference to FIG. 5C, the method 480 for detecting an error on a GPS signal may include at least one of: comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the onboard information (step 482); comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the location-based visual information (step 484); and detecting, by the GPS error detection module, inconsistencies between the multiple GPS signals (step 486).
[0072] FIG. 5D depicts a flowchart of a method for engaging a target upon reaching or getting near a target zone, according to one embodiment. With reference to FIG.
[0073] AERO-VI2405WO5D, the method 490 engaging a target upon reaching or getting near the target zone may include: receiving, by the processor, image data of the target area from one or more sensors (step 492); determining, by a computer vision module, which object in the target area matches preprogrammed target or set of targets based on the received image data via a computer vision (step 494); confirming, by the computer vision module, whether the target or set of targets are located in a designated engagement zone (step 496); and engaging, by a mission module, the target or set of targets with a payload (step 498).
[0074] FIG. 6 depicts an environment 495 where the disclosed vehicle (e.g., aircraft system) may operate in. The figure shows a mission planning device, having multi-programmable capability and ability to be used by untrained personal and friendly user interface. The present embodiments disclose a system, device, and method for denied / intermittent GPS navigation where the aircraft may be preprogramed with a set of targets by way of object recognition and given a target zone to head toward and search for the preprogramed set of targets.
[0075] With reference to FIG. 6, vehicles (e.g., multiple aircrafts) may transit to a target area autonomously via a layered GPS-denied capability. That is, once the aircraft starts the transit to a target area, the disclosed autonomous aspect may provide the capability for it to continue on the mission / flight patch and reach the target zone even in the occurrence of a GPS-denied or spoofing event.
[0076] The system may be configured to use alternative sources of information to initially detect whether such an even, e.g., denied / jamming or spoofing, has occurred and in response, use alternative location information to proceed to eliminate preprogrammed targets in area of interest — even absent GPS information. In the event that GPS jamming occurs, the aircrafts may maintain the swam formation and, even absent any communication (comm silent), may reach the desired target zone using the LTM and DSM alternative information. In one embodiment, the LTM and DSM data updates may be continuously fed into or transmitted to the Basic Input / Output System (BIOS) as the BIOS may always be running and soon as the system determines there
[0077] AERO-VI2405WOis an LTM or DSM match, the system fuses those sets of information into the BIOS while checking against the GPS and to determine if the GPS information can be trusted and therefore detect a spoofing. Using a position filter, based on the weight on the accelerometer, gyroscope, down looking camera, image matching may be performed against each other to provide a single solution of determining where every position is. Accordingly, the system may be configured to fuse all the data together which results in an estimate of the aircraft’s position.
[0078] Once in the target zone, the aircraft may automatically and autonomously determine which objects in the target area (on the ground or in the air) match the preprogrammed set of targets, at which point the aircrafts then act as Long-Range Precision Guided Munitions to engage the targets. In one embodiment, the target may be determined based on received images or videos from the same integrated forward looking IR camera that was used for the DSM data collection and locates and confirms targets via computer vision. Once the aircraft reaches the target zone (or target area), the on-board computer vision module may provide a confirmation of the target or set of targets and perform a check to confirm that it is indeed operating in a designated engagement zone (free fire zone).
[0079] The components on the vehicle (e.g., aircraft system) are configured to execute software programs to ensure real-time detection of any GPS jamming, denial, or spoofing via the onboard information being collected via one or more sensors and onboard equipment. For example, engine performance, vibration, temperature, speed and torque, noise, electrical currents, battery power level, and fuel properties may be combined with geospatial positioning information for determining of such occurrences. In one embodiment, information may be collected from other aircraft in the swarm or in the area such as their operational data and route information.
[0080] Accordingly, even though GPS data is not being received, communications may be transmitted to and received from other aircraft to substitute for the denied GPS information.
[0081] AERO-VI2405WOFIG. 7 illustrates an example of a top-level functional block diagram of a computing device embodiment 500. The example operating environment is shown as a computing device 520 comprising a processor 524, such as a central processing unit (CPU), addressable memory 527, an external device interface 526, e.g., an optional universal serial bus port and related processing, and / or an Ethernet port and related processing, and an optional user interface 529, e.g., an array of status lights and one or more toggle switches, and / or a display, and / or a keyboard and / or a pointer-mouse system and / or a touch screen. Optionally, the addressable memory may, for example, be: flash memory, eprom, and / or a disk drive or other hard drive. These elements may be in communication with one another via a data bus 528. In some embodiments, via an operating system 525 such as one supporting a web browser 523 and applications 522, the processor 524 may be configured to execute steps of a process establishing a communication channel and processing according to the embodiments described above.
[0082] FIG. 8 is a high-level block diagram 600 showing a computing system comprising a computer system useful for implementing an embodiment of the system and process, disclosed herein. Embodiments of the system may be implemented in different computing environments. The computer system includes one or more processors 602, and can further include an electronic display device 604 (e.g., for displaying graphics, text, and other data), a main memory 606 (e.g., random access memory (RAM)), storage device 608, a removable storage device 610 (e.g., removable storage drive, a removable memory module, a magnetic tape drive, an optical disk drive, a computer readable medium having stored therein computer software and / or data), user interface device 611 (e.g., keyboard, touch screen, keypad, pointing device), and a communication interface 612 (e.g., modem, a network interface (such as an Ethernet card), a communications port, or a PCMCIA slot and card). The communication interface 612 allows software and data to be transferred between the computer system and external devices. The system further includes a communications infrastructure 614 (e.g., a communications bus, cross-over bar, or network) to which the aforementioned devices / modules are connected as shown.
[0083] AERO-VI2405WOInformation transferred via communications interface 614 may be in the form of signals such as electronic, electromagnetic, optical, or other signals capable of being received by communications interface 614, via a communication link 616 that carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular / mobile phone link, an radio frequency (RF) link, and / or other communication channels. Computer program instructions representing the block diagram and / or flowcharts herein may be loaded onto a computer, programmable data processing apparatus, or processing devices to cause a series of operations performed thereon to produce a computer implemented process.
[0084] Embodiments have been described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments. Each block of such illustrations / diagrams, or combinations thereof, can be implemented by computer program instructions. The computer program instructions when provided to a processor produce a machine, such that the instructions, which execute via the processor, create means for implementing the functions / operations specified in the flowchart and / or block diagram. Each block in the flowchart / block diagrams may represent a hardware and / or software module or logic, implementing embodiments. In alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures, concurrently, etc.
[0085] Computer programs (i.e., computer control logic) are stored in main memory and / or secondary memory. Computer programs may also be received via a communications interface 612. Such computer programs, when executed, enable the computer system to perform the features of the embodiments as discussed herein. In particular, the computer programs, when executed, enable the processor and / or multicore processor to perform the features of the computer system. Such computer programs represent controllers of the computer system.
[0086] FIG. 9 shows a block diagram of an example system 700 in which an embodiment may be implemented. The system 700 includes one or more client devices 701 such as consumer electronics devices, connected to one or more server
[0087] AERO-VI2405WOcomputing systems 730. A server 730 includes a bus 702 or other communication mechanism for communicating information, and a processor (CPU) 704 coupled with the bus 702 for processing information. The server 730 also includes a main memory 706, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 702 for storing information and instructions to be executed by the processor 704. The main memory 706 also may be used for storing temporary variables or other intermediate information during execution or instructions to be executed by the processor 704. The server computer system 730 further includes a read only memory (ROM) 708 or other static storage device coupled to the bus 702 for storing static information and instructions for the processor 704. A storage device 710, such as a magnetic disk or optical disk, is provided and coupled to the bus 702 for storing information and instructions. The bus 702 may contain, for example, thirty-two address lines for addressing video memory or main memory 706. The bus 702 can also include, for example, a 32-bit data bus for transferring data between and among the components, such as the CPU 704, the main memory 706, video memory and the storage 710. Alternatively, multiplex data / address lines may be used instead of separate data and address lines.
[0088] The server 730 may be coupled via the bus 702 to a display 712 for displaying information to a computer user. An input device 714, including alphanumeric and other keys, is coupled to the bus 702 for communicating information and command selections to the processor 704. Another type or user input device comprises cursor control 716, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to the processor 704 and for controlling cursor movement on the display 712.
[0089] According to one embodiment, the functions are performed by the processor 704 executing one or more sequences of one or more instructions contained in the main memory 706. Such instructions may be read into the main memory 706 from another computer-readable medium, such as the storage device 710. Execution of the sequences of instructions contained in the main memory 706 causes the processor 704 to perform the process steps described herein. One or more processors in a multi-
[0090] AERO-VI2405WOprocessing arrangement may also be employed to execute the sequences of instructions contained in the main memory 706. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions to implement the embodiments. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
[0091] The terms "computer program medium," "computer usable medium," "computer readable medium", and "computer program product," are used to generally refer to media such as main memory, secondary memory, removable storage drive, a hard disk installed in hard disk drive, and signals. These computer program products are means for providing software to the computer system. The computer readable medium allows the computer system to read data, instructions, messages or message packets, and other computer readable information from the computer readable medium. The computer readable medium, for example, may include non-volatile memory, such as a floppy disk, ROM, flash memory, disk drive memory, a CD-ROM, and other permanent storage. It is useful, for example, for transporting information, such as data and computer instructions, between computer systems. Furthermore, the computer readable medium may comprise computer readable information in a transitory state medium such as a network link and / or a network interface, including a wired network or a wireless network that allow a computer to read such computer readable information. Computer programs (also called computer control logic) are stored in main memory and / or secondary memory. Computer programs may also be received via a communications interface. Such computer programs, when executed, enable the computer system to perform the features of the embodiments as discussed herein. In particular, the computer programs, when executed, enable the processor multi-core processor to perform the features of the computer system. Accordingly, such computer programs represent controllers of the computer system.
[0092] Generally, the term "computer-readable medium" as used herein refers to any medium that participated in providing instructions to the processor 704 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for
[0093] AERO-VI2405WOexample, optical or magnetic disks, such as the storage device 710. Volatile media includes dynamic memory, such as the main memory 706. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise the bus 702. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0094] Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
[0095] Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to the processor 704 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to the server 730 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector coupled to the bus 702 can receive the data carried in the infrared signal and place the data on the bus 702. The bus 702 carries the data to the main memory 706, from which the processor 704 retrieves and executes the instructions. The instructions received from the main memory 706 may optionally be stored on the storage device 710 either before or after execution by the processor 704.
[0096] The server 730 also includes a communication interface 718 coupled to the bus 702. The communication interface 718 provides a two-way data communication coupling to a network link 720 that is connected to the world wide packet data communication network now commonly referred to as the Internet 728. The Internet 728 uses electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 720 and
[0097] AERO-VI2405WOthrough the communication interface 718, which carry the digital data to and from the server 730, are exemplary forms or carrier waves transporting the information.
[0098] In another embodiment of the server 730, interface 718 is connected to a network 722 via a communication link 720. For example, the communication interface 718 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line, which can comprise part of the network link 720. As another example, the communication interface 718 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, the communication interface 718 sends and receives electrical electromagnetic or optical signals that carry digital data streams representing various types of information.
[0099] The network link 720 typically provides data communication through one or more networks to other data devices. For example, the network link 720 may provide a connection through the local network 722 to a host computer 724 or to data equipment operated by an Internet Service Provider (ISP). The ISP in turn provides data communication services through the Internet 728. The local network 722 and the Internet 728 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 720 and through the communication interface 718, which carry the digital data to and from the server 730, are exemplary forms or carrier waves transporting the information.
[0100] The server 730 can send / receive messages and data, including e-mail, program code, through the network, the network link 720 and the communication interface 718. Further, the communication interface 718 can comprise a USB / Tuner and the network link 720 may be an antenna or cable for connecting the server 730 to a cable provider, satellite provider or other terrestrial transmission system for receiving messages, data and program code from another source.
[0101] AERO-VI2405WOThe example versions of the embodiments described herein may be implemented as logical operations in a distributed processing system such as the system 700 including the servers 730. The logical operations of the embodiments may be implemented as a sequence of steps executing in the server 730, and as interconnected machine modules within the system 700. The implementation is a matter of choice and can depend on performance of the system 700 implementing the embodiments. As such, the logical operations constituting said example versions of the embodiments are referred to for e.g., as operations, steps or modules.
[0102] Similar to a server 730 described above, a client device 701 can include a processor, memory, storage device, display, input device and communication interface (e.g., e-mail interface) for connecting the client device to the Internet 728, the ISP, or LAN 722, for communication with the servers 730. The system 700 can further include computers (e.g., personal computers, computing nodes) 705 operating in the same manner as client devices 701, where a user can utilize one or more computers 705 to manage data in the server 730.
[0103] Referring now to FIG. 10, illustrative cloud computing environment 50 is depicted. As shown, cloud computing environment 50 comprises one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA), smartphone, smart watch, set-top box, video game system, tablet, mobile computing device, or cellular telephone 54A, desktop computer 54B, laptop computer 54C, and / or automobile computer system 54N may communicate. Nodes 10 may communicate with one another. They may be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows cloud computing environment 50 to offer infrastructure, platforms and / or software as services for which a cloud consumer does not need to maintain resources on a local computing device. It is understood that the types of computing devices 54A-N shown in FIG. 8 are intended to be illustrative only and that computing nodes 10 and cloud computing environment 50 can
[0104] AERO-VI2405WOcommunicate with any type of computerized device over any type of network and / or network addressable connection (e.g., using a web browser).
[0105] It is contemplated that various combinations and / or sub-combinations of the specific features and aspects of the above embodiments may be made and still fall within the scope of the invention. Accordingly, it should be understood that various features and aspects of the disclosed embodiments may be combined with or substituted for one another in order to form varying modes of the disclosed invention. Further, it is intended that the scope of the present invention is herein disclosed by way of examples and should not be limited by the particular disclosed embodiments described above.
[0106] AERO-VI2405WO
Claims
CLAIMS:What is claimed is:
1. A system comprising:one or more sensors (112) configured to collect sensor data for a first aircraft (110), the sensor data including at least one of: an onboard information, a location-based visual information, and two or more GPS signals; and a navigation module (134) configured to detect whether an error has occurred on a GPS signal for the first aircraft (110) based on the collected sensor data;wherein once the error is detected, the navigation module (134) is configured to replace the GPS signal with an alternative location information such that the first aircraft (110) is configured to continue navigating to a target area, and wherein the alternative location information is generated based on the sensor data.
2. The system of claim 1, wherein the one or more sensors (112) comprises at least one of: one or more cameras (114, 116), an inertial measurement unit (IMU, 118), a laser range finder (120), and multiple GPS receivers (122).
3. The system of claim 2, wherein the navigation module (134) comprises an inertial navigation system (INS) (140); andwherein the INS (140) is configured to generate the alternative location information, by tracking movement of the first aircraft (110) from a known starting point based on the onboard information configured to be collected by the IMU (118).
4. The system of claim 3, wherein the navigation module (134) is configured to detect the error by comparing the GPS signal with the alternative location information generated by the INS (140).
5. The system of claim 2, wherein the navigation module (134) comprises an image matching module (142);AERO-VI2405WOwherein the image matching module (142) is configured to generate the alternative location information, by matching the location-based visual information with a known map; andwherein the location-based visual information is collected by at least one of the one or more cameras (114, 116) and the laser range finder (120).
6. The system of claim 5, wherein the image matching module (142) comprises at least one ofa laser terrain matching (LTM) module (132) configured to match a real-time terrain profile collected from the laser range finder (120) with the known map; anda digital scene matching (DSM) module (134) configured to match a real-time image collected from the one or more cameras (114, 116) with the known map.
7. The system of claim 5, wherein the navigation module (134) is configured to detect the error by comparing the GPS signal with the alternative location information generated by the image matching module (142).
8. The system of claim 2, wherein the navigation module (134) comprises a comparison module (144); andwherein the comparison module (144) is configured to detect the error by detecting inconsistencies between the multiple GPS signals configured to be received by the multiple GPS receivers (122).
9. The system of claim 2, wherein the navigation module (134) comprises a position filter module (148); andwherein the position filter module (148) is configured to generate the alternative location information by fusing at least two location information oflocation information generated based on an image collected from the one or more cameras (114, 116);AERO-VI2405WOlocation information generated based on a terrain profile collected from the laser range finder (120); andlocation information generated based on the onboard information collected from the IMU (118).
10. The system of claim 9, wherein the position filter module (148) is configured to generate the alternative location information by fusing the at least two location information based on assigned weights.
11. The system of claim 2, wherein the navigation module (134) is configured to generate multiple alternative location information based on at least two of: an aerial image collected from the one or more cameras (114, 116); a terrain profile collected from the laser range finder (120); and the onboard information collected from the IMU (118); and wherein the navigation module (134) is configured to transition between the multiple alternative location information.
12. The system of claim 2, wherein when the navigation module (134) detects that the error on the GPS signal ceases, the navigation module (134) is configured to revert from using the alternative location information to using the GPS signal.
13. The system of claim 3, wherein the one or more sensors (112) is configured to be included in a second aircraft within a swarm, the swarm comprising the first aircraft and the second aircraft; andwherein the navigation module (134) of the first aircraft (110) is configured to detect the error based on the sensor data transmitted from the second aircraft.
14. A method comprising:collecting, by one or more sensors, sensor data of an aircraft, the sensor data comprising at least one of: an onboard information, a location-based visual information, and two or more GPS signals (step 453);AERO-VI2405WOdetecting, by a GPS error detection module of a processor having addressable memory, whether an error has occurred on a GPS signal during navigation based on the sensor data (step 456); andreplacing, by an alternative global positioning module of the processor, the GPS signal to an alternative location information once the error is detected such that the aircraft is configured to continue navigating to a target area (step 458);wherein the alternative location information is generated from the sensor data.
15. The method of claim 14, wherein detecting the error includes at least one of: comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the onboard information (step 482);comparing, by the GPS error detection module, the GPS signal with the alternative location information generated based on the location-based visual information (step 484); anddetecting, by the GPS error detection module, inconsistencies between the multiple GPS signals (step 486).
16. The method of claim 14, further comprising: generating the alternative location information based on the sensor data (step 470);wherein generating the alternative location information comprises at least one of:generating, by an inertial navigation system (INS) module of the GPS error detection module, the alternative location information based on the onboard information (step 472);generating, by an image matching module of the GPS error detection module, the alternative location information based on an aerial image of the location-based visual information (step 474);generating, by the image matching module, the alternative location information based on a terrain profile of the location-based visual information (step 476); andAERO-VI2405WOgenerating, by a position filter module of the alternative global positioning module, the alternative location information by fusing at least two location information among multiple location information generated based on the onboard information, the aerial image, and the terrain profile (step 478).
17. The method of claim 16, wherein generating the alternative location information includes generating multiple alternative location information based on at least two of the onboard information, the aerial image, and the terrain profile; and wherein the method further comprises: transitioning, by the alternative global positioning module, between the multiple alternative location information (step 464).
18. The method of claim 14, further comprising: reverting, by the alternative global positioning module, from using the alternative location information to using the GPS signal when the GPS error detection module detects that the error on the GPS signal ceases (step 466).
19. The method of claim 14, further comprising: receiving, by the processor, the sensor data collected from the one or more sensors included in another aircraft within a swarm (step 492).
20. A system comprising:a GPS error detection module (138) configured to detect whether an error has occurred on a GPS signal for an aircraft (110) based on at least one of: a camera (114, 116), an inertial measurement unit (IMU, 118), a laser range finder (120), and two or more GPS receivers (122);wherein once the error is detected, the aircraft (110) is configured to replace the GPS signal to an alternative location information such that the aircraft (110) is configured to continue navigating to a target area; andAERO-VI2405WOwherein the alternative location information is configured to be generated based on the camera (114, 116), the IMU (118), and the laser range finder (120).AERO-VI2405WO
