System and method for calculating flight controls for vehicle landing

Abstract

The present invention is entitled "System and method for calculating flight control for landing a vehicle". The present invention discloses a method, system and non-transitory computer-readable medium for landing a vehicle. For example, the method may include: receiving a landing zone confirmation from a service before a descent transition point, the landing zone confirmation including landing zone location information and an indication that the landing zone is clear; determining a landing flight path based on the landing zone location information; and when the vehicle begins to descend to the landing zone using the landing flight path: receiving landing zone data from at least one of a radar system, a camera system, an altitude and heading reference system (AHRS), and a GPS system; performing an analysis based on the landing zone data to determine whether an unsafe condition exists; and calculating flight control for the vehicle to continue the descent or modify the descent based on the analysis.

Description

Technical Field

[0001] Various embodiments of the present disclosure relate generally to systems and methods for computing flight controls for vehicle landing, and more particularly, to systems and methods for computing flight controls for vehicle landing using sensor data fusion. Background Art

[0002] The infrastructure and processes of Urban Air Mobility (UAM) may face several challenges. For example, UAM may require a large amount of data collection, communication, processing and reporting to ensure timely, safe and efficient resource allocation for travel in the UAM environment. For example, certification agencies may require operators of UAM vehicles to ensure specific tolerances for vehicle operations, such as, among other things, sufficient vehicle spacing within traffic constraints, landing operations within certain limits, etc. Specifically, due to spatial constraints around the landing area (physical space or restrictions on operations near / above buildings, or due to noise generated by the aircraft), traffic to or from the landing area, etc., it may be a challenge to land safely in the landing area. Therefore, it may be a challenge to communicate with the aircraft when the UAM vehicle approaches the landing area (approaches and attempts to land an available (e.g., unobstructed) landing area) and confirm that the landing is being carried out in a safe manner.

[0003] The present disclosure is directed to overcoming one or more of the challenges described above. Summary of the Invention

[0004] According to certain aspects of the present disclosure, systems and methods for computing flight controls for vehicle landing are disclosed.

[0005] For example, a method may include: receiving a landing zone confirmation from a service before a descent transition point, the landing zone confirmation including landing zone location information and an indication that the landing zone is clear; determining a landing flight path based on the landing zone location information; and when the vehicle begins to descend to the landing zone using the landing flight path: receiving landing zone data from at least one of a radar system, a camera system, an altitude and heading reference system (AHRS), and a GPS system; performing an analysis based on the landing zone data to determine whether an unsafe condition exists; and calculating flight controls for the vehicle to continue the descent or modify the descent based on the analysis.

[0006] A system may include: a memory storing instructions; and a processor executing the instructions to perform a process. The process may include: receiving a landing zone confirmation from a service before a descent transition point, the landing zone confirmation including landing zone location information and an indication that the landing zone is clear; determining a landing flight path based on the landing zone location information; and when the vehicle begins a descent to the landing zone using the landing flight path: receiving landing zone data from at least one of a radar system, a camera system, an altitude and heading reference system (AHRS), and a GPS system; performing an analysis based on the landing zone data to determine whether an unsafe condition exists; and calculating flight controls for the vehicle to continue the descent or modify the descent based on the analysis.

[0007] A non-transitory computer-readable medium may store instructions that, when executed by a processor, cause the processor to perform a method. The method may include: receiving a landing zone confirmation from a service before a descent transition point, the landing zone confirmation including landing zone location information and an indication that the landing zone is clear; determining a landing flight path based on the landing zone location information; and when the vehicle begins a descent to the landing zone using the landing flight path: receiving landing zone data from at least one of a radar system, a camera system, an altitude and heading reference system (AHRS), and a GPS system; performing an analysis based on the landing zone data to determine whether an unsafe condition exists; and calculating flight control for the vehicle to continue the descent or modify the descent based on the analysis.

[0008] Additional objects and advantages of the disclosed embodiments will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the disclosed embodiments.

[0009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and, together with the description, serve to explain the principles of the disclosed embodiments.

[0011] Figure 1 An exemplary environment is shown in which the methods, systems, and other aspects of the present disclosure may be implemented.

[0012] Figure 2 An exemplary system according to one or more embodiments is shown.

[0013] Figure 3A and Figure 3BAn exemplary block diagram of a vehicle of a system according to one or more embodiments is shown.

[0014] Figure 4 and Figure 5 An example scenario of vehicle landing using sensor data fusion is shown in accordance with one or more embodiments.

[0015] Figure 6 A flow chart is shown for computing flight controls for vehicle landing using sensor data fusion, according to one or more embodiments.

[0016] Figure 7 A flow chart is shown for computing flight controls for vehicle landing using sensor data fusion, according to one or more embodiments.

[0017] Figures 8A to 8C A flow chart is shown for computing flight controls for vehicle landing using sensor data fusion, according to one or more embodiments.

[0018] Figure 9 An exemplary system is shown that may perform the techniques presented herein. DETAILED DESCRIPTION

[0019] Various embodiments of the present disclosure generally relate to vehicle landing.

[0020] In general, the present disclosure relates to systems and methods for computing flight controls for vehicle landing using sensor data fusion. For example, a vehicle of the present disclosure may include a multi-sensor and navigation sensor suite that performs one or more of the methods of the present disclosure. The multi-sensor and navigation sensor suite may include a radar (such as a lightweight digital radar or software adjustable radar (SWARS)), an onboard camera (e.g., at least two cameras), an altitude and heading reference system (AHRS), and a GPS system. The multi-sensor and navigation sensor suite may perform a descent cross-check process when descending to a landing zone. The descent cross-check process may: control the SWARS, onboard camera, AHRS, and GPS systems to cross-check potential conflicts, confirm that the landing zone is clear, and determine whether an unsafe condition exists. For example, using the descent cross-check process, the vehicle may: control the AHRS to obtain positioning information and alignment information to confirm the position and alignment with the landing pad beacon, and control the SWARS to obtain altitude and descent rate to confirm the altitude and descent rate (e.g., by performing a profile check process); control the camera to obtain a light pattern or identifier pattern (e.g., by performing a visual confirmation process); control the SWARS to scan and map the landing area to ensure it is clear (e.g., by performing a scan confirmation process to ensure that there are no other vehicles / obstacles parked at / on the landing area); and cross-check for conflicts (if any of the above indicates a potential conflict) or confirm that the landing pad is clear. If no unsafe conditions are detected, the vehicle's multi-sensor and navigation sensor suite may continue the descent cross-check process until the vehicle lands on the landing area. Thus, the multi-sensor and navigation sensor suite can confirm that the landing is proceeding in a safe manner by cross-checking multiple different sensor systems (e.g., radar and camera) and data sources (AHRS and GPS system).

[0021] Although the present disclosure describes systems and methods relating to aircraft, it should be understood that the systems and methods of the present disclosure are applicable to the management of vehicles, including drones, cars, ships, or any other autonomous and / or internet-connected vehicles.

[0022] like Figure 1 As shown, Figure 1 An exemplary environment is shown in which the methods, systems, and other aspects of the present disclosure may be implemented. Figure 1The environment of FIG100 may include an airspace 100 and one or more hub ports 111-117. A hub port, such as any of 111-117, may be a ground facility (e.g., an airport, a vertiport, a heliport, a vertiport pad, a helipad, a temporary landing / takeoff facility, etc.) where aircraft may take off, land, or remain parked. Airspace 100 may accommodate various types of aircraft 131-133 (collectively referred to as "aircraft 131," unless otherwise specified herein) flying at various altitudes and via various routes 141. An aircraft, such as any of aircraft 131a-133b, may be any aerial transportation device or vehicle capable of traveling between two or more hub ports 111-117, such as an airplane, a vertical take-off and landing aircraft (VTOL), a drone, a helicopter, an unmanned aerial vehicle (UAV), a hot air balloon, a military aircraft, etc. Any of the aircraft 131a-133b can be connected to each other and / or to one or more of the hubs 111-117 via a communication network using a vehicle management computer corresponding to each aircraft or each hub. Each vehicle management computer may include a computing device and / or a communication device, as described below. Figure 3A and Figure 3B This is described in more detail in Figure 1 As shown, different types of aircraft sharing airspace 100 are shown, which are distinguished by way of example as model 131 (aircraft 131a and 131b), model 132 (aircraft 132a, 132b, and 132c), and model 133 (aircraft 133a and 133b).

[0023] like Figure 1 As further shown, airspace 100 may have one or more weather constraints 121, spatial restrictions 122 (e.g., buildings), and temporary flight restrictions (TFRs) 123. These are exemplary factors that an aircraft's vehicle management computer may be required to consider and / or analyze in order to derive the safest and most optimal flight trajectory for the aircraft. For example, if the vehicle management computer of an aircraft planning to travel from hub 112 to hub 115 predicts that the aircraft may be affected by adverse weather conditions (such as weather constraint 121) in the airspace, the vehicle management computer may modify the direct path (e.g., route 141 between hubs 112 and 115) by slightly curving away from weather constraint 121 (e.g., circling north) to form a deviated route 142. For example, deviated route 142 may ensure that the aircraft's path and time (e.g., 4D coordinates of the flight trajectory) do not intersect any location and time coordinates (e.g., 4D coordinates of weather constraint 121) of weather constraint 121.

[0024] As another example, the vehicle management computer of aircraft 131b may predict before takeoff that space restrictions 122 caused by buildings will prevent aircraft 131b from flying a direct flight path from hub port 112 to hub port 117, as shown in FIG. Figure 1 In response to this prediction, the vehicle management computer of aircraft 131b may generate a 4D trajectory having a vehicle path that bypasses the three-dimensional zones (e.g., zones including locations and altitudes) associated with those particular buildings. As another example, the vehicle management computer of aircraft 133b may predict before takeoff that some possible 4D trajectories of TFR 123 and another aircraft 132c will obstruct or conflict with the direct flight path of aircraft 133b, such as Figure 1 As shown. In response, the vehicle management computer of aircraft 133b may generate a 4D trajectory having path and time coordinates that do not intersect the 4D coordinates of TFR 123 or the 4D trajectory of other aircraft 132c. In this case, TFR 123 and the risk of collision with another aircraft 132c are examples of dynamic factors that may be active or inactive depending on the planned travel time, the TFR's validity period, and the path and schedule of the other aircraft 132c. As described in these examples, the 4D trajectory derivation process, including any modifications or renegotiations, may be completed before the aircraft's takeoff.

[0025] As another example, the vehicle management computer of aircraft 131b may determine to use one of routes 141 that is set aside for exclusive use or not exclusive use by aircraft 131. Aircraft 131b may generate a 4D trajectory with a vehicle path that follows one of routes 141.

[0026] As indicated above, Figure 1 The present invention is merely an example of an airspace environment including example types of aircraft, hubs, zones, restrictions, and routes. Other examples are possible with respect to the specific details of the aircraft, hubs, zones, restrictions, and routes and may be compared with those provided with respect to the present invention. Figure 1 The content described is different. For example, in addition to those described above, the types of zones and restrictions that may factor into trajectory derivation may include the availability of hub ports, reserved paths or sky lanes (e.g., Route 141), any ground-based obstacles extending outward to a specific altitude level, any known avoidance zones (e.g., noise-sensitive areas), air transportation regulations (e.g., proximity to airports), etc. During the derivation process, any factors that allow the 4D trajectory to be modified from a direct path or shortest path between two hub ports may be considered.

[0027] Figure 2 An exemplary system according to one or more embodiments is shown. Figure 2System 200 shown in FIG. 1 may include one or more aircraft (such as aircraft 131), one or more intrusive aircraft 230, cloud service 205, one or more communication stations 210, and / or one or more ground stations 215. One or more aircraft 131 may travel along a route in route 141 from a first hub (e.g., hub 114) to a second hub (e.g., hub 112). One or more ground stations 215 may be distributed along route 141 (e.g., evenly, based on traffic considerations, etc.) between, near, and / or on hubs (e.g., hubs 111-117). One or more communication stations 210 may be distributed (e.g., evenly, based on traffic considerations, etc.) between, near, and / or on hubs (e.g., hubs 111-117). Some (or all) of one or more ground stations 215 may be paired with one of one or more communication stations 210.

[0028] Each of the one or more ground stations 215 may include a transponder system, a radar system, and / or a data link system.

[0029] The radar system of ground station 215 may include a directional radar system. The directional radar system may be pointed upward (e.g., from the ground toward the sky) and may transmit beam 220 to provide three-dimensional coverage over a portion of route 141. Beam 220 may be a narrow beam. The three-dimensional coverage of beam 220 may be directly above ground station 215 or at various tilt angles (relative to vertical). The directional radar system may detect objects, such as aircraft 131, within the three-dimensional coverage of beam 220. The directional radar system may detect objects through skin detection. If ground station 215 is located on a hub port (such as hub port 112), the directional radar system may transmit beam 225 to provide three-dimensional coverage above hub port 112. Beam 225 may also be tilted at an angle (from vertical) to detect objects arriving at, descending into, and landing on hub port 112. The beams 220 / 225 may be controlled mechanically (by moving the radar system), electronically (eg, a phased array), or by software (eg, a digital phased array "DAPA" radar), or any combination thereof.

[0030] The transponder system of ground station 215 may include an ADS-B and / or Mode S transponder, and / or other transponder systems (collectively, an interrogator system). The interrogator system may have at least one directional antenna. The directional antenna may be aimed at a portion of route 141. For example, aiming at a portion of route 141 may reduce the likelihood of blanketing the ecosystem (e.g., aircraft 131) with interrogation, as would be the case if the interrogator system used an omnidirectional antenna. The directional antenna may be aimed at a specific portion of route 141 by transmitting signals using the same or different beam patterns as beams 220 / 225 discussed above for the radar system. The interrogator system may transmit an interrogation message to an aircraft, such as aircraft 131, within that portion of route 141. The interrogation message may include an identifier for the interrogator system and / or request that an aircraft, such as aircraft 131, transmit an identification message. The interrogator system may receive an identification message from an aircraft, such as aircraft 131. The identification message may include an identifier for the aircraft and / or transponder aircraft data (e.g., speed, position, trajectory, etc.) for the aircraft.

[0031] If the radar system detects an object and the transponder system does not receive a corresponding identification message from the object (or if it does receive an identification message, but the identification message is not a legitimate identification message, e.g., an identifier of an unauthorized aircraft), ground station 215 may determine that the object is an intruder aircraft 230. Ground station 215 may then transmit an intruder alert message to cloud service 205. If the radar system detects an object and the transponder system receives a corresponding identification message from the object, ground station 215 may determine that the object is a legitimate aircraft. Ground station 215 may then transmit a legitimate aircraft message to cloud service 205. Additionally or alternatively, ground station 215 may transmit a detection message based on the detection of the object and whether ground station 215 receives an identification message (a "response message"). Thus, ground station 215 may not determine whether the detected object is an intruder aircraft or a legitimate aircraft, but may instead send a detection message to cloud service 205 for cloud service 205 to determine whether the detected object is an intruder aircraft or a legitimate aircraft.

[0032] The data link system of ground station 215 can communicate with at least one of one or more communication stations 210. Each of the one or more communication stations 210 can communicate with at least one of the one or more ground stations 215 in an area surrounding the communication station 210 to receive data from or transmit data to the one or more ground stations 215. Some or no communication stations 210 may not communicate directly with the ground station 215, but may instead act as relays for other communication stations 210 that are in direct communication with the ground station 215. For example, each ground station 215 can communicate (directly or indirectly) with the nearest communication station 210. Additionally or alternatively, a ground station 215 can communicate with the communication station 210 that has the best signal, bandwidth, etc. for the ground station 215. One or more communication stations 210 can include a wireless communication system to communicate with the data link system of the ground station 215. The wireless communication system may implement cellular communication according to, for example, 3G / 4G / 5G standards. The wireless communication system may enable Wi-Fi communication, Bluetooth communication, or other short-range wireless communication. Additionally or alternatively, one or more communication stations 210 may communicate with one or more of the one or more ground stations 215 based on wired communications (such as Ethernet, fiber optic, etc.).

[0033] For example, ground station 215 may transmit an intrusive alert message or a legitimate aircraft message (and / or a detection message) to communication station 210. Communication station 210 may then relay the intrusive alert message or the legitimate aircraft message (and / or the detection message) to cloud service 205 (directly or indirectly through another communication station 210).

[0034] One or more communication stations 210 may also communicate with one or more aircraft, such as aircraft 131, to receive data from the one or more aircraft and to transmit data to the one or more aircraft. For example, one or more communication stations 210 may relay data between cloud service 205 and a vehicle, such as aircraft 131.

[0035] Cloud service 205 may communicate with one or more communication stations 210 and / or communicate directly (e.g., via satellite communications) with an aircraft, such as aircraft 131. Cloud service 205 may provide instructions, data, and / or warnings to aircraft 131. Cloud service 205 may receive confirmations from aircraft 131, aircraft data from aircraft 131, and / or other information from aircraft 131. For example, cloud service 205 may provide aircraft 131 with weather data, traffic data, landing zone data for hub ports (such as hub ports 111-117), updated obstacle data, flight plan data, and the like. Cloud service 205 may also provide software as a service (SaaS) to aircraft 131 to perform various software functions, such as navigation services, flight management system (FMS) services, and the like, in accordance with a service contract, API requests from aircraft 131, and the like.

[0036] Figure 3A and Figure 3B An exemplary block diagram of a vehicle of a system according to one or more embodiments is shown. Figure 3A and Figure 3B Block diagrams 300A and 300B may be shown for vehicles, such as aircraft 131-133, respectively. Generally speaking, block diagram 300A may illustrate the systems, information / data, and communications between these systems for a piloted or semi-autonomous vehicle, while block diagram 300B may illustrate the systems, information / data, and communications between these systems for a fully autonomous vehicle. Aircraft 131 may be either a piloted or semi-autonomous vehicle and / or a fully autonomous vehicle.

[0037] Block diagram 300A of aircraft 131 may include a vehicle management computer 302 and electrical, mechanical, and / or software systems (collectively, the "vehicle system"). The vehicle system may include: one or more displays 304; a communication system 306; one or more transponders 308; a pilot / user interface 324 for receiving and transmitting information from a pilot and / or user 310 of the aircraft 131; edge sensors 312 on the structure 346 of the aircraft 131 (such as doors, seats, tires, etc.); a power system 378 for providing power to an actuation system 360; a camera 316; a GPS system 354; an onboard vehicle navigation system 314; a flight control computer 370; and / or one or more data storage systems. The vehicle management computer 302 and the vehicle system may be connected via one or a combination of wired or wireless communication interfaces, such as TCP / IP communication over Wi-Fi or Ethernet (with or without a switch), RS-422, ARINC-429, or other communication standards (with or without a protocol switch as needed).

[0038] The vehicle management computer 302 may include at least a network interface, a processor, and a memory, each of which may be coupled to one another via a bus or indirectly coupled to one another via a wired or wireless connection (e.g., Wi-Fi, Ethernet, parallel or serial ATA, etc.). The memory may store a vehicle management program, and the processor may execute the vehicle management program. The vehicle management program may include a weather program 322, a detection / sensing and avoidance aid (D / S&A) program 334, a flight routing program 344, a vehicle status / health program 352, a communication program 368, a flight control program 370, and / or a vertiport status program 372 (collectively, "subroutines"). According to the program code of the vehicle management program, the vehicle management program may obtain input from the subroutines and send output to the subroutines to manage the aircraft 131. According to the program code of the vehicle management program, the vehicle management program may also obtain input from the vehicle system and output instructions / data to the vehicle system.

[0039] The vehicle management computer 302 may transmit instructions / data / graphical user interfaces to the one or more displays 304 and / or the pilot / user interface 324. The one or more displays 304 and / or the pilot / user interface 324 may receive user input and transmit the user input to the vehicle management computer 302.

[0040] Communication system 306 may include various data link systems (e.g., satellite communication systems), cellular communication systems (e.g., LTE, 4G, 5G, etc.), radio communication systems (e.g., HF, VHF, etc.), and / or wireless local area network communication systems (e.g., Wi-Fi, Bluetooth, etc.). Communication system 306 may enable communication between aircraft 131 and external networks, services, and cloud services 205 discussed above according to communication procedures 368. Examples of external networks may include wide area networks (such as the Internet). Examples of services may include weather information services 318, traffic information services, etc.

[0041] One or more transponders 308 may include an interrogator system. The interrogator system of aircraft 131 may be an ADS-B, Mode S transponder, and / or other transponder system. The interrogator system may have an omnidirectional antenna and / or a directional antenna (interrogator system antenna). The interrogator system antenna may transmit / receive signals to transmit / receive interrogation messages and transmit / receive identification messages. For example, in response to receiving an interrogation message, the interrogator system may obtain an identifier of aircraft 131 and / or transponder aircraft data of aircraft 131 (e.g., speed, position, trajectory, etc.), for example, from an onboard vehicle navigation system 314; and transmit an identification message. Conversely, the interrogator system may transmit an interrogation message to a nearby aircraft; and receive an identification message. One or more transponders 308 may send a message to the vehicle management computer 302 to report interrogation messages and / or identification messages received / transmitted to it from other aircraft and / or ground station 215. As discussed above, the interrogation message may include an identifier of the interrogator system (in this case, aircraft 131 ), a request for nearby aircraft to transmit identification messages, and / or transponder aircraft data (other than that described above) of aircraft 131 (e.g., speed, position, trajectory, etc.); the identification message may include an identifier of aircraft 131 and / or transponder aircraft data of aircraft 131 .

[0042] Edge sensors 312 on structure 346 of aircraft 131 may be sensors for detecting various environmental and / or system status information. For example, some edge sensors in edge sensors 312 may monitor discrete signals, such as edge sensors on seats (e.g., occupied or unoccupied), doors (e.g., closed or not closed), etc. of aircraft 131. Some edge sensors in edge sensors 312 may monitor continuous signals, such as edge sensors on tires (e.g., tire pressure), brakes (e.g., engaged or not engaged, amount of wear, etc.), passenger cabin (e.g., cabin air pressure, air composition, temperature, etc.), support structures (e.g., deformation, strain, etc.), etc. of aircraft 131. Edge sensors 312 may transmit edge sensor data to vehicle management computer 302 to report discrete and / or continuous signals.

[0043] The power system 378 may include one or more battery systems, fuel cell systems, and / or other chemical power systems to power the actuation system 360 and / or the vehicle system in general. In one aspect of the present disclosure, the power system 378 may be a battery pack. The power system 378 may have various sensors to detect one or more of temperature, remaining fuel / charge, discharge rate, etc. (collectively referred to as power system data 348). The power system 378 may transmit the power system data 348 to the vehicle management computer 302 so that the power system status 350 (or battery pack status) can be monitored by the vehicle status / health program 352.

[0044] The actuation system 360 may include motors, engines, and / or propellers for generating thrust, lift, and / or directional force for the aircraft 131; flaps or other surface controls for enhancing the thrust, lift, and / or directional force of the aircraft 131; and / or aircraft mechanical systems (e.g., for deploying landing gear, windshield wipers, signal lights, etc.). Based on the flight control program 370, the vehicle management computer 302 may control the actuation system 360 by transmitting instructions, and the actuation system 360 may transmit feedback / current status of the actuation system 360 (which may be referred to as actuation system data) to the vehicle management computer 302.

[0045] The camera 316 may include an inferred or optical camera, LIDAR, or other visual imaging system to record the interior or exterior environment of the aircraft 131. The camera 316 may obtain inferred images; optical images; and / or LIDAR point cloud data, or any combination thereof (collectively, "imaging data"). The LIDAR point cloud data may include the coordinates of each data point received by the LIDAR (which may include, for example, location, intensity, time information, etc.). The camera 316 and / or the vehicle management computer 302 may include machine vision functionality. The machine vision functionality may process the obtained imaging data to detect objects, the location of the detected objects, the speed / velocity (relative and / or absolute) of the detected objects, the size and / or shape of the detected objects, etc. (collectively, "machine vision output"). For example, the machine vision functionality may be used to image a landing zone to confirm that the landing zone is clear / unobstructed (landing zone (LZ) status 362). Additionally or alternatively, the machine vision functionality may determine whether the physical environment surrounding the aircraft 131 and / or on / near the route 141 (e.g., buildings, structures, cranes, etc.) is or will be within a safe flight envelope for the aircraft 131 (e.g., based on the position, speed, and flight plan of the aircraft 131). The imaging data and / or machine vision output may be referred to as "imaging output data." The camera 316 may transmit the imaging data and / or the machine vision output of the machine vision functionality to the vehicle management computer 302. The camera 316 may determine whether a detected element in the physical environment is known or unknown based on the obstacle data stored in the obstacle database 356, such as by determining the location of the detected object and determining whether an obstacle in the obstacle database has the same location (or is within a defined distance range). The imaging output data may include any obstacles that are determined not to be in the obstacle data in the obstacle database 356 (unknown obstacle information).

[0046] The GPS system 354 may include one or more global navigation satellite (GNSS) receivers. The GNSS receivers may receive signals from the Global Positioning System (GPS) developed by the United States, the Global Navigation Satellite System (GLONASS) developed by Russia, the Galileo system developed by the European Union, and / or the BeiDou system developed by China, or other global or regional satellite navigation systems. The GNSS receivers may determine positioning information for the aircraft 131. The positioning information may include information regarding one or more of the following: the vehicle's position (e.g., latitude and longitude, or Cartesian coordinates), altitude, speed, heading, or trajectory. The GPS system 354 may transmit the positioning information to the onboard vehicle navigation system 314 and / or the vehicle management computer 302.

[0047] The onboard vehicle navigation system 314 may include one or more radars, one or more magnetometers, an attitude and heading reference system (AHRS), and / or one or more air data modules. The one or more radars may be weather radars for scanning the weather and / or DAPA radars (omnidirectional and / or directional) for scanning terrain / surfaces / objects / obstacles. The one or more radars (collectively referred to as "radar systems") may obtain radar information. This radar information may include information about local weather and terrain / surfaces / objects / obstacles (e.g., the aircraft or obstacles and their associated positions / movements). The one or more magnetometers may measure magnetic fields to obtain orientation information of the aircraft 131. The AHRS may include sensors (e.g., three sensors on three axes) to obtain attitude information of the aircraft 131. The attitude information may include the roll, pitch, and yaw of the aircraft 131. The air data module may sense external air pressure to obtain airspeed information of the aircraft 131. The radar information, orientation information, attitude information, airspeed information, and / or positioning information (collectively referred to as navigation information) may be transmitted to the vehicle management computer 302.

[0048] Weather program 322 may use communication system 306 to transmit and / or receive weather information from one or more of weather information services 318. For example, weather program 322 may obtain local weather information from weather radar and onboard vehicle navigation system 314 (such as an air data module). Weather program 322 may also transmit a request for weather information 320. For example, the request may be for weather information 320 (route weather information) along route 141 of aircraft 131. Route weather information may include information about the external environment of aircraft 131, such as precipitation, wind, turbulence, storms, cloud cover, visibility, etc., along / near the flight path, at the destination and / or departure location (e.g., one of hubs 111-117), or for the general area surrounding the flight path, destination location, and / or departure location. One or more of weather information services 318 may transmit a response including the route weather information. Additionally or alternatively, one or more of weather information services 318 may transmit update messages to aircraft 131 that include route weather information and / or updates to route weather information.

[0049] D / S&A program 334 may use one or more transponders 308 and / or pilot / user interface 324 to detect and avoid objects that may pose a potential threat to aircraft 131. For example, pilot / user interface 324 may receive user input (or radar / imaging detection) from a pilot and / or user of vehicle 310 indicating detection of an object; pilot / user interface 324 (or radar / imaging detection) may transmit the user input (or radar or imaging information) to vehicle management computer 302; vehicle management computer 302 may invoke D / S&A program 334 to execute object detection process 328 to determine whether the detected object is a non-cooperative object 332 (e.g., an aircraft that is not participating in transponder communications); optionally, vehicle management computer 302 may determine the position, velocity, trajectory (non-cooperative object information) of non-cooperative object 332, such as by radar tracking or image tracking; in response to determining that the object is a non-cooperative object 332, vehicle management computer 302 may determine a course of action, such as instructing flight control program 370 to avoid non-cooperative object 332. As another example, one or more transponders 308 may detect an intruding aircraft (such as intruding aircraft 230) based on an identification message from the intruding aircraft; one or more transponders 308 may transmit a message to vehicle management computer 302, the message including the identification message from the intruding aircraft; vehicle management computer 302 may extract an identifier and / or transponder aircraft data from the identification message to obtain an identifier and / or speed, position, trajectory, etc. of the intruding aircraft; vehicle management computer 302 may invoke D / S&A program 334 to perform position detection process 326 to determine whether the detected object is a cooperative object 330 and its position, speed, heading, trajectory, etc.; in response to determining that the object is a cooperative object 330, vehicle management computer 302 may determine a course of action, such as instructing flight control program 370 to avoid the cooperative object 330. For example, the course of action may be different or the same for non-cooperative and cooperative objects 330 / 332, depending on regulatory and / or scenario-based rules.

[0050] Flight routing program 344 may use communication system 306 to generate / receive flight plan information 338 and receive system vehicle information 336 from cloud service 205. Flight plan information 338 may include a departure location (e.g., one of hub ports 111-117), a destination location (e.g., one of hub ports 111-117), intermediate locations (if any) between the departure and destination locations (e.g., waypoints or one or more of hub ports 111-117), and / or one or more routes 141 to be used (or not used). System vehicle information 336 may include other aircraft positioning information relative to aircraft 131 (referred to as "receiving aircraft 131" for reference). For example, other aircraft positioning information may include positioning information of other aircraft. The other aircraft may include: all aircraft 131-133 and / or intruding aircraft 230; aircraft 131-133 and / or intruding aircraft 230 within a threshold distance of the receiving aircraft 131; aircraft 131-133 and / or intruding aircraft 230 using the same route 141 as the receiving aircraft (or will use the same route 141 or cross the same route 141); and / or aircraft 131-133 and / or intruding aircraft 230 within the same geographic area (e.g., a city, town, metropolitan area, or a subdivision thereof) of the receiving aircraft.

[0051] Flight routing program 344 may determine or receive planned flight path 340. Flight routing program 344 may receive planned flight path 340 from another aircraft 131 or cloud service 205 (or other services, such as an operations service for aircraft 131). Flight routing program 344 may determine planned flight path 340 using various planning algorithms (e.g., a flight planning service on or off aircraft 131), aircraft constraints of aircraft 131 (e.g., cruise speed, maximum speed, maximum / minimum altitude, maximum range, etc.), and / or external constraints (e.g., restricted airspace, noise reduction zones, etc.). Based on flight plan information 338 and / or system vehicle information 336, the planned / received flight path may include a flight trajectory with 4D coordinates, a waypoint-based flight path, any suitable flight path for aircraft 131, or any combination thereof. The 4D coordinates may include 3D coordinates of the flight path in space (e.g., latitude, longitude, and altitude) and time.

[0052] The flight path selection program 344 may determine the unplanned flight path 342 based on the planned flight path 340 and unplanned event triggers and using various planning algorithms, aircraft constraints of the aircraft 131, and / or external constraints. The vehicle management computer 302 may determine the unplanned event triggers based on data / information received by the vehicle management computer 302 from other vehicle systems or from the cloud service 205. The unplanned event triggers may include one or a combination of the following: (1) an emergency landing, as indicated by the vehicle status / health program 352 discussed below, or indicated by user input to one or more displays 304 and / or pilot / user interface 324; (2) an intruding aircraft 230, cooperating object 330, or non-cooperating object 332 that encroaches on the safe flight envelope of the aircraft 131; (3) a weather change indicated by the route weather information (or an update thereof); (4) a machine vision output indicating that a portion of the physical environment may be or will be within the safe flight envelope of the aircraft 131; and / or (5) a machine vision output indicating that the landing area is blocked.

[0053] Unplanned flight path 342 / planned flight path 340 and other aircraft positioning information may be collectively referred to as flight plan data.

[0054] Vehicle status / health program 352 can monitor the status / health of vehicle systems and, based on the monitored status / health, perform actions, such as periodically reporting status / health, indicating an emergency situation, etc. The vehicle can obtain edge sensor data and power system data 348. Vehicle status / health program 352 can process edge sensor data and power system data 348 to determine the status of power system 378 and various structures and systems monitored by edge sensors 312, and / or track the health of power system 378 and the structures and systems monitored by edge sensors 312. For example, vehicle status / health program 352 can obtain power system data 348, determine battery status 350, and, based thereon, perform actions, such as reducing consumption of non-essential systems, reporting battery status, etc. Vehicle status / health program 352 can determine an emergency landing condition based on one or more of power system 378 and the structures and systems monitored by edge sensors 312 having a condition indicating that power system 378 and the structures and systems monitored by edge sensors 312 have failed or will soon fail. Additionally, the vehicle status / health program 352 can transmit status / health data to the cloud service 205 as a status / health message (or as part of other messages to the cloud service). The status / health data can include actuation system data, all edge sensor data and / or power system data (portions thereof), summaries of the edge sensor data and power system data, and / or system status indicators based on the edge sensor data and power system data (e.g., operating normally, wear reduced, inoperable, etc.).

[0055] Flight control programs 370 may control actuation system 360 based on unplanned flight path 342 / planned flight path 340, other aircraft position information, control laws 358, navigation rules 374, and / or user input (e.g., from the pilot if aircraft 131 is a piloted or semi-autonomous vehicle). Flight control programs 370 may receive planned flight path 340 / unplanned flight path 342 and / or user input (collectively, a "route") and, based on control laws 358 and navigation rules 374, determine inputs to actuation system 360 to change the speed, heading, and attitude of aircraft 131 to match the route. Control laws 358 may specify a range of possible actions for actuation system 360 and map the inputs to the range of actions to achieve the route through, for example, the physics of flight of aircraft 131. Navigation rules 374 may dictate acceptable actions based on location, waypoints, portions of the flight path, the environment, etc. (collectively, a "situation"). For example, the navigation rules 374 may indicate minimum / maximum altitudes, minimum / maximum speeds, minimum separation distances, headings or ranges of acceptable headings, etc. for a given situation.

[0056] Vertiport status procedures 372 may control aircraft 131 during takeoff (by executing takeoff procedure 364) and during landing (by executing landing procedure 366). Takeoff procedure 364 may determine whether the landing area from which aircraft 131 will depart and the flight environment during ascent are clear (e.g., based on control laws 358, navigation rules 374, imaging data, obstacle data, unplanned flight path 342 / planned flight path 340, other aircraft positioning information, user input, etc.), and control the aircraft or guide the pilot through the ascent (e.g., based on control laws 358, navigation rules 374, imaging data, obstacle data, flight plan data, user input, etc.). Landing procedure 366 may determine the landing zone where aircraft 131 will land and whether the flight environment during descent is clear (e.g., based on control laws 358, navigation rules 374, imaging data, obstacle data, flight plan data, user input, landing zone status, etc.), and control the aircraft or guide the pilot to complete the descent (e.g., based on control laws 358, navigation rules 374, imaging data, obstacle data, flight plan data, user input, landing zone status, etc.).

[0057] The one or more data storage systems may store data / information received, generated, or obtained on the aircraft. The one or more data storage systems may also store software for one or more computers on the aircraft.

[0058] Block diagram 300B may be the same as block diagram 300A, except that block diagram 300B may omit the pilot / user interface 324 and / or the one or more displays 304 and include a vehicle position / velocity / altitude system 376. The vehicle position / velocity / altitude system 376 may or may not include the onboard vehicle navigation system 314 and / or the GPS system 354 discussed above. In the event that the vehicle position / velocity / altitude system 376 does not include the onboard vehicle navigation system 314 and / or the GPS system 354, the vehicle position / velocity / altitude system 376 may obtain navigation information from the cloud service 205.

[0059] In one aspect of the present disclosure, the landing process 366 of the convertiport state procedure 372 may include: receiving a landing zone confirmation message from the cloud service 205 before the descent transition point; determining a landing flight path based on the landing zone location information, for example; and initiating a descent to the landing zone using the landing flight path. The landing process 366 may also include: receiving landing zone data from at least one of a radar system, a camera system, or an instant messaging system during the descent; performing an analysis based on the landing zone data to determine whether an unsafe condition exists; and calculating flight control for the vehicle to continue or modify the descent based on the analysis.

[0060] For example, Figure 4 and Figure 5 An exemplary scenario of using sensor data fusion to calculate flight control for vehicle landing according to one or more embodiments may be shown. Figure 4 In FIG. 4 , a vehicle 101 (e.g., one of the aircraft 131) on route 141 to hub port 112 may receive a landing zone confirmation message before, at, or after descent transition point 405. For example, in one aspect of the present disclosure, vehicle 101 may receive a landing zone confirmation message at time t0 before descent transition point 405; vehicle 101 may receive a landing zone confirmation at time t1 at descent transition point 405; and vehicle 101 may receive a landing zone confirmation message at time t2, t3, or t4 after descent transition point 405. The landing zone confirmation message may include landing zone location information and an indication of whether the landing zone is clear or a change from an initial landing zone to another landing zone. The landing zone location information may include GPS coordinates of the landing zone. While descent transition point 405 is shown as coinciding with an edge of beam 225 of directional radar system of ground station 215 of central port 112 (or landing zone 112A of multiple landing zones 112A through 112N of central port 112), descent transition point 405 may be located outside beam 225, closer to central port 112, or based on central port conditions (e.g., clearance of obstacles).

[0061] With or without a landing zone confirmation message, vehicle 101 may determine a landing flight path from route 141 (or from outside route 141) to descent transition point 405 and then to the landing zone (e.g., based on landing zone location information and / or based on control laws 358, navigation rules 374, imaging data, obstacle data, flight plan data, user input, landing zone status, etc.). Vehicle 101 may then begin descending to the landing zone using the landing flight path.

[0062] The landing process 366 may also include, during the descent, confirming position and alignment, locking onto an approach beacon, and performing a descent cross-check process to determine if an unsafe condition exists. The landing process 366 may also include, during the descent, calculating flight controls for the vehicle to continue or modify the descent based on the output of the descent cross-check process.

[0063] To confirm the position and alignment, the landing process 366 may control the radar system to receive a signal from the landing pad beacon and / or lock onto a signal from the landing pad beacon; analyze the signal to determine whether the vehicle 101 has selected the correct landing zone (e.g., corresponding to the landing zone in the landing zone confirmation message); obtain position and / or orientation information from, for example, the cloud service 205 or an onboard system (e.g., AHRS, radar, camera, GPS, etc.); compare the position and / or orientation information with the expected navigation data and / or radar data to confirm the proper alignment and position for the approach; and, in response to determining that the correct landing zone has been selected and / or the comparison indicates the proper alignment and position for the approach, then confirm that the vehicle 101 can proceed with the landing approach. The landing pad beacon may be the same directional radar system as the ground station 215 at the hub port 112, or the landing pad beacon may be a different directional radar system at the hub port 112. The expected navigation data and / or radar data may be landing zone specific information for acceptable approach positions and orientations stored on the vehicle 101, received from the cloud service 205, or received in a landing zone confirmation message. For example, the landing zone beacon may be a DAPA radar that is specifically programmed to transmit a coded signal (e.g., a uniquely coded signal) to the vehicle 101 to indicate the landing pad 112A.

[0064] Obtaining position and / or orientation information may include requesting and receiving position and / or orientation information from, for example, a cloud service 205 or an onboard system; and determining one or a combination of: (1) a distance from a landing zone beacon; (2) a relative orientation from the landing zone beacon; and / or (3) a position and / or altitude relative to the landing zone beacon. The position and / or orientation information may include position information, attitude information, airspeed information, and / or positioning information of navigation information to indicate the position (e.g., GPS coordinates), altitude, orientation, velocity (descent rate and / or other velocity vector components), airspeed, and / or orientation of the vehicle 101. The position and / or orientation information may also include altitude (e.g., based on ranging to the landing zone) and descent rate (e.g., a time derivative of altitude), and / or altitude and descent rate based on radar signals from a radar system.

[0065] To perform the descent cross-check process, the vehicle 101 may receive landing zone data from at least one of a radar system, a camera system, or an instant messaging system; and perform an analysis based on the landing zone data to determine if an unsafe condition exists.

[0066] For example, the vehicle 101 may periodically (e.g., at predetermined intervals) or continuously receive landing zone data and perform analysis. Figure 4 In , the vehicle 101 may never determine the unsafe condition, so the vehicle continues to descend until landing at time t5; meanwhile, in Figure 5 , the vehicle 101 may determine an unsafe condition at time t4 and perform a maneuver to modify the descent, and perform subsequent vehicle operations at time t5. For example, calculating flight controls for the vehicle to continue the descent or modify the descent may include: continuing the descent if the unsafe condition does not exist, and modifying the descent if the unsafe condition does exist. Modifying the descent may include one or more of: reducing the descent rate, performing a maneuver to a holding area or an alternative landing area, or reattempting the descent. For example, reducing the descent rate may reduce the descent rate to greater than zero, zero, or less than zero (e.g., the vehicle is at an ascending altitude). For example, the vehicle 101 may invoke the flight control program 370 to control the actuation system 360 to follow the descent flight path (if the unsafe condition does not exist), or to control the actuation system 360 to modify the descent (e.g., reducing the rate or descent, moving to a holding area, etc.).

[0067] Specifically, the vehicle 101 may (in accordance with the descent cross-check process and landing process 366): receive descent data from one or more of a plurality of systems (which may correspond to the landing zone data described above); perform an analysis of the descent data to determine whether an unsafe condition exists (e.g., determine whether the analysis indicates whether an unsafe condition exists); in response to determining that the analysis indicates an unsafe condition, perform a maneuver to a holding area, an alternative landing zone, or reattempt the landing zone; in response to determining that the analysis does not indicate an unsafe condition, determine whether the landing is complete; in response to determining that the landing is not complete, receive more descent data and perform the analysis again; and in response to determining that the landing is complete, transmit a success message to the cloud service 205.

[0068] The descent cross-check process may perform one or more different cross-check processes, such as a first process, a second process, and / or a third process. The descent cross-check process may perform the first process, the second process, and / or the third process in parallel or sequentially, or the descent cross-check process may perform only one, two, or all of the first process, the second process, or the third process. The first process may be an independent process for receiving descent data and performing analysis; the second process may be a first sequential process for receiving descent data and performing analysis; and the third process may be a second sequential process for receiving descent data and performing analysis.

[0069] For example, the independent process may separately control the first sensor system and the second sensor system to detect an unsafe condition by independently executing the first sensor process and the second sensor process; the first sequential process may execute the first sensor process and then, in certain circumstances, execute the second sensor process; the second sequential process may execute the second sensor process and then, in certain circumstances, execute the first sensor process.

[0070] The first sensor process may perform a profile check process and one of a scan confirmation process or a visual confirmation process; the second sensor process may perform the other of the scan confirmation process or the visual confirmation process. The scan confirmation process may control the radar system of the vehicle 101 to obtain radar data from (the radar information of) the radar system of the vehicle 101. The visual confirmation process may control the camera 316 to obtain imaging output data from the vehicle 101 (camera 316). The profile check process may: control the onboard vehicle navigation system 314 to obtain navigation data from (the navigation information of) the onboard vehicle navigation system 314 of the vehicle 101 and / or control the radar system of the vehicle 101 to obtain radar data from (the radar information of) the radar system of the vehicle 101; and determine whether the descent remains within the descent profile.

[0071] With respect to the profile check process, the profile check process may also obtain navigation data and radar data, as discussed above with respect to locking onto a landing zone beacon signal. The profile check process may: obtain navigation data and / or radar data; compare the navigation data to the radar data to check whether the reports of each data type are identical to each other or within a threshold similarity; and compare the navigation data and / or radar data to a data set expected by the profile. Based on previous readings of the navigation data and radar data, the data set expected by the profile may include the same data types as the navigation data and radar data (e.g., position, altitude, descent rate, etc.) and be adjusted for expected changes in the data (e.g., based on velocity, acceleration, rotation, etc. and aerodynamics). Based on the comparison, the profile check process may determine whether there is a substantial change from the current navigation data and / or radar data to the data set expected by the profile. For example, a substantial change may be a position / altitude / descent rate that is greater than a threshold amount from the position / altitude / descent rate of the data set expected by the profile. In response to detecting a substantial change, the profile check process may determine an unsafe condition.

[0072] Additionally or alternatively, the data anticipated by the profile may include descent conditions. The descent conditions may include one or a combination of the following: (1) a flight envelope condition (e.g., a three-dimensional volume within which the vehicle 101 must remain), (2) a descent rate condition (e.g., a descent rate must be less than a predetermined value, and the predetermined value may vary based on the altitude of the vehicle, such as the descent rate must slow as the altitude decreases); and / or (3) a heading, pitch, and / or roll condition (e.g., the heading / pitch / roll of the vehicle 101 must be between a first predetermined degree and a second predetermined degree). The profile checking process may determine whether one or more of the above conditions are met; and in response to determining that the one or more conditions are not met, determine an unsafe condition.

[0073] Those skilled in the art will recognize that the comparison and / or condition indicating a substantial change can be varied and determined in a variety of ways, and typically the comparison and / or condition is to confirm that the vehicle 101 is descending as expected. For example, the comparison / condition may determine whether a gust of wind has increased the rate of descent by more than a threshold amount, determine whether the vehicle 101 is drifting away from the landing zone, etc.

[0074] With respect to the scan confirmation process, the scan confirmation process may control the radar system to scan and map the landing zone and include landing zone mapping data in the radar data. The landing zone mapping data may include radar mapping data indicating the two-dimensional area / three-dimensional structure of the landing zone (e.g., multiple radar readings from different sections of the scan of the landing zone, the radar readings indicating the surface of the landing zone and / or the location of obstacles on the landing zone). The scan confirmation process may compare the landing zone mapping data with expected landing zone mapping data (which may be included in the scan expectation data set). When the landing zone is clear, the expected landing zone mapping data may indicate the two-dimensional area / three-dimensional structure of the landing zone. The expected landing zone mapping data may be provided by the cloud service 205 in the route to the hub 112 or in the landing zone confirmation message. If the cloud service 205 becomes aware of a change in the two-dimensional area / three-dimensional structure of the landing zone, the expected landing zone mapping data may be updated by the cloud service 205, and the cloud service 205 may provide the vehicle 101 with the most recent two-dimensional area / three-dimensional structure of the landing zone. Alternatively, the expected landing zone mapping data may be based on the first / most recent reading of the landing zone mapping data, or a reading between the first and most recent readings, or any combination thereof. The scan confirmation process may determine, based on the comparison, whether there is a substantial change from the current reading of the landing zone mapping data to the expected landing zone mapping data. For example, a substantial change may be a segment of the two-dimensional area / three-dimensional structure (e.g., a grouping of radar readings) of the current reading of the landing zone mapping data that is greater than a threshold distance from a point in the two-dimensional area / three-dimensional structure of the expected landing zone mapping data. Those skilled in the art will recognize that the comparison indicating a substantial change can be varied and determined in a variety of ways, and generally speaking, the purpose of the scanning and mapping described herein is to determine whether an obstacle has entered the landing zone of the vehicle 101. If the scan confirmation process determines that there is a substantial change, the scan confirmation process may determine that an obstacle (or potential conflict) is present.

[0075] With respect to the visual confirmation process, the imaging output data may include imaging data and / or machine vision output, where the imaging data may include one or more of an inferred image; an optical image; and / or LIDAR point cloud data. In one aspect of the present disclosure, the visual confirmation process may obtain the imaging data and / or machine vision output and analyze the imaging data and / or machine vision output to determine whether an expected symbol or representation (visually expected data) is present. The expected symbol or representation may be a light pattern on the landing area, or an identifier pattern on the landing area. The landing areas of the hub ports 111-117 may have the same or different (e.g., unique) light patterns or identifier patterns. The light pattern may be a visible or inferred light pattern from a light source (e.g., a light / LED, lamp, etc.) on the landing area. The identifier pattern may be a visible or inferred pattern applied, for example, by paint or other material, on the surface of the landing area of ​​the hub ports 111-117. To determine whether the expected symbol or representation is present, the visual confirmation process may analyze the imaging data and / or machine vision output to determine whether a pattern similar to the light pattern or identifier pattern is present in the imaging data or machine vision output. For example, the visual confirmation process may use a machine learning algorithm or a pattern detection algorithm to detect a pattern and match the pattern to a light pattern or identifier pattern of the landing zone of the central harbor 112. If the expected symbol or representation is determined to be present in the imaging data or machine vision output, the visual confirmation process may determine that the landing zone is clear; otherwise, the visual confirmation process may determine that an obstacle (or potential conflict) exists.

[0076] In another aspect of the present disclosure, the visual confirmation process may also, independently or as part of the above process, control the camera to scan and map the landing zone using the vehicle's LIDAR system and include the LIDAR landing zone data in the imaging data. The LIDAR landing zone data may include LIDAR point cloud data that indicates the two-dimensional area / three-dimensional structure of the landing zone (e.g., multiple LIDAR point readings from different sections of the scan of the landing zone, the LIDAR point readings indicating the surface of the landing zone and / or the location of obstacles on the landing zone). The visual confirmation process may compare the LIDAR landing zone data with expected LIDAR landing zone data (which may be included in the visual expectation dataset). The expected LIDAR landing zone data may indicate the two-dimensional area / three-dimensional structure of the landing zone. The expected LIDAR landing zone data may be provided by the cloud service 205 in the route to the central port 112 or in the landing zone confirmation message. If cloud service 205 becomes aware of a change in the 2D area / 3D structure of the landing zone, the expected LIDAR landing zone data may be updated by cloud service 205, and cloud service 205 may provide the vehicle 101 with the most recent 2D area / 3D structure of the landing zone. Alternatively, the expected LIDAR landing zone data may be based on the first / most recent reading of the expected LIDAR landing zone data, or a reading between the first and most recent readings, or any combination thereof. The visual confirmation process may determine, based on this comparison, whether there has been a substantial change from the current reading of the LIDAR landing zone data to the expected LIDAR landing zone data. For example, a substantial change may be a segment of the 2D area / 3D structure of the current reading of the LIDAR landing zone data (e.g., a grouping of LIDAR point readings) that is greater than a threshold distance from a point in the 2D area / 3D structure of the expected LIDAR landing zone data. Those skilled in the art will recognize that the comparison indicating a substantial change can be varied and determined in a variety of ways, and generally speaking, the scanning and mapping performed by the LIDAR system described herein is intended to determine whether an obstacle has entered the landing zone of the vehicle 101. If the visual confirmation process determines that a substantial change exists, the visual confirmation process may determine that an obstacle (or potential conflict) exists.

[0077] In the case of a separate process, the descending cross-check process may: use either the first process or the second process (or, in the case of a non-separate process, use both) to determine whether an obstacle exists to determine the obstacle exists; and in response to determining that the obstacle exists, determine that an unsafe condition exists. In the case of a first sequential process, the descending cross-check process may: use the first process to determine whether a potential conflict exists (e.g., the altitude process or the vision process determines that an object exists as a potential conflict); in response to the first process determining the potential conflict (e.g., the altitude process or the vision process determines that an object exists), the second process may confirm whether the potential conflict is an obstacle (e.g., using the other of the altitude process or the vision process to determine that an object exists); and in response to confirming that the potential conflict is an obstacle, determine the unsafe condition. In the case of a second sequential process, the descending cross-check process may: use the second process to determine whether a potential conflict exists (e.g., the altitude process or the vision process determines that an object exists); in response to the second process determining the potential conflict (e.g., the altitude process or the vision process determines that an object exists), the first process may confirm whether the potential conflict is an obstacle (e.g., using the other of the altitude process or the vision process to determine that an object exists); and in response to confirming that the potential conflict is an obstacle, determine the unsafe condition.

[0078] For example, the vehicle 101 may be equipped with a multi-sensor and navigation sensor suite. The multi-sensor and navigation sensor suite may include a software adjustable radar (SWARS), an onboard camera (e.g., at least two cameras), an altitude and heading reference system (AHRS), and a GPS system. The SWARS may perform a variety of functions, including target detection and tracking, ground mapping, and radar altimetry. The onboard camera may include or be supported by an onboard image processor that analyzes imaging data to detect obstacles and / or patterns on the landing zone. The AHRS may determine the attitude and heading (e.g., roll, pitch, heading) of the vehicle 101. The GPS system may determine the position (e.g., GPS coordinates) of the vehicle 101. The multi-sensor and navigation sensor suite may: receive a landing zone confirmation message (indicating the landing zone location and whether the landing zone is clear, occupied, or changed from the initial landing pad to the backup landing pad) from the cloud service 205 before the vehicle 101 approaches the descent transition point 405; receive final confirmation of a clear landing zone (e.g., another landing zone confirmation message) from the cloud service 205 before or at the descent transition point 405; determine a descent flight path by loading the landing zone location (e.g., GPS location) into the flight path selection program 344; and begin descent using the descent flight path. During descent, the vehicle may: control the SWARS to lock onto the landing pad beacon; and perform a descent cross-check procedure to cross-check for potential conflicts or confirm that the landing zone is clear. Using the descent cross-check process, the vehicle 101 may: control the AHRS to obtain positioning information and alignment information to confirm the position and alignment with the landing pad beacon, and control the SWARS to obtain altitude and descent rate to confirm the altitude and descent rate (e.g., by performing a profile check process); control the camera to obtain a light pattern or identifier pattern (e.g., by performing a visual confirmation process); control the SWARS to scan and map the landing area to ensure it is clear (e.g., by performing a scan confirmation process to ensure that no other vehicles / obstacles are parked at / on the landing area); and cross-check for conflicts (if any of the above indicates a potential conflict) or confirm that the landing pad is clear. The vehicle 101 may continue the descent cross-check process until the vehicle 101 lands on the landing area.

[0079] For example, in the above example, if the camera cannot acquire the light pattern or the identifier pattern, the descent cross-check process may determine the potential conflict by controlling the SWARS to scan and map the landing zone and cross-check the potential conflict to confirm that the potential conflict is an obstacle (or obtain the most recent output of the scan confirmation process). If the potential conflict is confirmed to be an obstacle, the descent cross-check process may determine an unsafe condition and abort the descent to the landing zone.

[0080] In another aspect of the present disclosure, if the vehicle 101 receives a message from the cloud service 205 indicating that another vehicle or an obstacle is present at the landing zone, the descent cross-check process may determine an unsafe condition (and therefore halt the descent to the landing zone). The vehicle 101 may confirm this information by performing one or both of a scanning confirmation process and a visual confirmation process. The vehicle 101 may halt the descent (or even not initiate the descent) with or without confirming the information.

[0081] If no unsafe conditions are detected, the vehicle's multi-sensor and navigation sensor suite can continue the descent cross-check process until the vehicle touches down on the landing zone. Thus, the multi-sensor and navigation sensor suite can confirm that the landing is proceeding in a safe manner by cross-checking multiple different sensor systems (e.g., radar and cameras) and data sources (AHRS and GPS systems).

[0082] Figure 6 A flowchart for calculating flight controls for vehicle landing using sensor data fusion is shown, according to one or more embodiments. Flowchart 600 may illustrate a landing procedure for a vehicle, such as aircraft 131. Flowchart 600 may be performed semi-autonomously or fully autonomously by aircraft 131.

[0083] The aircraft 131 may begin the process of flowchart 600 to communicate with the service while in transit (block 605). For example, the aircraft 131 may transmit current location information to the cloud service 205 and receive an initial landing zone confirmation message and / or weather information, as described above with respect to Figure 3A and Figure 3B discussed.

[0084] The aircraft 131 may continue this process to receive a landing zone confirmation message from the service before the descent transition point of the landing zone (block 610). For example, the aircraft 131 may receive a final confirmation of a clear landing zone, as described above with respect to Figure 3A and Figure 3B discussed.

[0085] The aircraft 131 may continue this process to determine a landing flight path (block 615). For example, the aircraft 131 may input the landing area location (e.g., GPS location) into the flight path selection program 344 to determine the landing flight path, as described above with respect to Figure 3A and Figure 3B discussed.

[0086] The aircraft 131 may continue the process to determine whether to initiate the landing process (block 620). For example, the aircraft 131 may determine whether the aircraft 131 has reached the descent transition point 405 (e.g., by determining whether the GPS location of the aircraft 131 is within a threshold distance from the location of the descent transition point 405), as described above with respect to Figure 3A and Figure 3B discussed.

[0087] In response to determining not to initiate the landing process (block 620: No), the aircraft 131 may continue the process to perform a maneuver to a holding area, an alternative landing area, or reattempt the landing area (block 625). The aircraft 131 may continue the process to determine a landing flight path (block 615), for example, to the same landing area or a different landing area.

[0088] In response to determining to begin the landing process (block 620: yes), the aircraft 131 may continue the process to begin the descent (block 630). The aircraft 131 may continue the process to perform a descent cross-check process (block 635). For example, the aircraft 131 may control the multi-sensor and navigation sensor and perform a scan confirmation process, a visual confirmation process, and / or a profile check process, as described above with respect to Figure 3A and Figure 3B discussed.

[0089] Figure 7 FIG2 shows a flow chart for calculating flight control for vehicle landing using sensor data fusion according to one or more embodiments. Flowchart 700 may illustrate a descent cross-check landing process for a vehicle (such as aircraft 131), as described above in FIG2. Figure 6 Flowchart 700 may be performed by aircraft 131 semi-autonomously or fully autonomously.

[0090] Aircraft 131 may begin the process of flowchart 700 to confirm position and alignment (block 705). For example, aircraft 131 may perform a profile check process as described above with respect to Figure 3A and Figure 3B discussed.

[0091] The aircraft 131 may continue this process to lock onto the approach beacon (block 710). For example, the aircraft 131 may control the radar system to lock onto the signal from the landing zone beacon, as described above with respect to Figure 3A and Figure 3B discussed.

[0092] The aircraft 131 may continue the process to receive descent data from one or more of the plurality of systems (block 715). For example, the aircraft 131 may perform one or more of a scan confirmation process, a visual confirmation process, and a profile check process to obtain descent data, as described above with respect to FIG. Figure 3A and Figure 3B discussed.

[0093] The aircraft 131 may continue the process to perform analysis of the descent data (block 720). For example, the aircraft 131 may continue to perform one or more of a scan confirmation process, a visual confirmation process, and a profile check process to determine potential conflicts or unsafe conditions, as described above with respect to Figure 3A and Figure 3B discussed.

[0094] The aircraft 131 may continue the process to determine whether the analysis results indicate an unsafe condition (block 725). For example, if the descent cross-check process determines that the potential conflict is an obstacle or if the profile check process determines an unsafe condition, the aircraft 131 may determine an unsafe condition, as described above with respect to Figure 3A and Figure 3B discussed.

[0095] In response to determining that the analysis indicates an unsafe condition (block 725 : YES), the aircraft 131 may continue the process to perform a maneuver to a holding area, an alternative landing area, or reattempt the landing area (block 730 ).

[0096] In response to determining that the analysis results do not indicate an unsafe condition (block 725: No), the aircraft 131 may continue the process to determine whether landing is complete (block 735). For example, the aircraft 131 may determine whether the aircraft 131 has landed, such as detecting wheels on the ground.

[0097] In response to determining that landing was not complete (block 735 : NO), the aircraft 131 may continue the process to receive more descent data and perform the analysis again (blocks 715 and 720 , respectively).

[0098] In response to determining that the landing is complete (block 735: YES), the aircraft 131 may continue the process to transmit a success message to the service (block 740). For example, the aircraft 131 may transmit a success message to indicate that the aircraft 131 is using the landing zone.

[0099] Figures 8A to 8C Flowcharts 800A through 800C illustrate a method for calculating flight control for vehicle landing using sensor data fusion according to one or more embodiments. Figure 7 Flowcharts 800A to 800C may be performed semi-autonomously or fully autonomously by the aircraft 131.

[0100] Flowchart 800A may correspond to a separate process for receiving descent data and performing analysis; flowchart 800B may correspond to a first sequential process for receiving descent data and performing analysis; and flowchart 800C may correspond to a second sequential process for receiving descent data and performing analysis. For example, the separate processes may independently control a first sensor system and a second sensor system to detect an unsafe condition by independently executing a first sensor process and a second sensor process. The first sequential process may execute the first sensor process and then, under certain circumstances, execute the second sensor process; the second sequential process may execute the second sensor process and then, under certain circumstances, execute the first sensor process. The first sensor process may correspond to blocks 805 through 830, and the second sensor process may correspond to blocks 835 through 850.

[0101] The aircraft 131 may begin the process of flowchart 800A to control a first sensor system to obtain first data (block 805). For example, the aircraft 131 may control a navigation system to obtain navigation information, a radar system to obtain radar data, and / or a camera system to obtain imaging output data, as described above with respect to FIG. Figure 3A and Figure 3B discussed.

[0102] Aircraft 131 may continue the process to (in parallel, simultaneously, sequentially, alternatively, etc.): (1) compare the first data to the descent profile (block 810) and / or (2) compare the first data to the expected first data (block 815). For example, aircraft 131 may: (1) perform a profile check process to compare the first data to the descent profile, and (2) perform a scan confirmation process or a visual confirmation process to compare the first data to the expected first data, as described above with respect to Figure 3A and Figure 3B discussed.

[0103] In the event that the first data is compared to the descent profile, the aircraft 131 may continue the process to determine whether the comparison indicates that the first data exceeds a threshold value of the descent profile (block 820). For example, the aircraft 131 may determine that one or more conditions are not satisfied (e.g., the aircraft 131 is outside the three-dimensional volume of flight envelope conditions), as described above with respect to Figure 3A and Figure 3B In response to determining that the comparison does not indicate that the first data exceeds the threshold value of the descent profile (block 820: No), the aircraft 131 may continue the process to control the first sensor system to obtain more first data (block 805). In response to determining that the comparison indicates that the first data does exceed the threshold value of the descent profile (block 820: Yes), the aircraft 131 may continue the process to determine an unsafe condition (block 825).

[0104] In the event that the first data is compared to the expected first data, the aircraft 131 may continue the process to determine whether the comparison indicates that the first data exceeds a threshold for the expected first data (block 830). For example, if the scanning confirmation process or the visual confirmation process determines that a substantial change has occurred, the aircraft 131 may determine a potential conflict, as described above with respect to Figure 3A and Figure 3B In response to determining that the comparison indicates that the first data does not exceed the threshold value of the expected first data (block 830: No), the aircraft 131 may continue the process to control the first sensor system to obtain more first data (block 805). In response to determining that the comparison indicates that the first data exceeds the threshold value of the expected first data (block 830: Yes), the aircraft 131 may continue the process to determine an unsafe condition (block 850).

[0105] Separately, the aircraft 131 may also independently initiate the process of flowchart 800A to control a second sensor system to obtain second data (block 835). For example, the aircraft 131 may control a radar system to obtain radar data or control a camera system to obtain imaging output data to obtain the second data, as described above with respect to FIG. Figure 3A and Figure 3B discussed.

[0106] The aircraft 131 may continue the process of comparing the second data with the expected second data (block 840). For example, the aircraft 131 may perform another of the scanning confirmation process or the visual confirmation process, as described above with respect to Figure 3A and Figure 3B discussed.

[0107] The aircraft 131 may continue the process to determine whether the comparison indicates that the second data exceeds the threshold of the expected second data (block 845). For example, if the scanning confirmation process or the visual confirmation process determines that a substantial change has occurred, the aircraft 131 may determine a potential conflict, as described above with respect to Figure 3A and Figure 3B In response to determining that the comparison indicates that the second data does not exceed the threshold value for the expected second data (block 845: No), aircraft 131 may continue the process to control the second sensor system to obtain more second data (block 835). In response to determining that the comparison indicates that the second data exceeds the threshold value for the expected second data (block 845: Yes), aircraft 131 may continue the process to determine an unsafe condition (block 850).

[0108] Flowchart 800B may be the same as flowchart 800A, except that aircraft 131 may perform the first sensor process, and in response to determining that the comparison indicates that the first data exceeds the threshold value of the expected first data (block 830: yes), aircraft 131 may continue the process to control the second sensor system to obtain the second data (block 835), thereby performing the second sensor process.

[0109] Flowchart 800C may be the same as flowchart 800A, except that aircraft 131 may perform the second sensor process, and in response to determining that the comparison indicates that the second data does exceed the threshold for expected second data (block 845: yes), aircraft 131 may continue the process to control the first sensor system to obtain the first data (block 805), thereby performing the first sensor process.

[0110] Figure 9 An exemplary system is shown that may perform the techniques presented herein. Figure 9 9 is a simplified functional block diagram of a computer according to an exemplary embodiment of the present disclosure, which computer can be configured to perform the technology described herein. Specifically, a computer (or "platform", because it may not be a single physical computer infrastructure) may include a data communication interface 960 for packet data communication. The platform may also include a central processing unit ("CPU") 920 in the form of one or more processors for executing program instructions. The platform may include an internal communication bus 910, and the platform may also include program storage devices and / or data storage devices for various data files to be processed and / or transmitted by the platform, such as ROM 930 and RAM 940, although the system 900 can receive programming and data via network communication. The system 900 may also include input and output ports 950 to be connected to input and output devices such as keyboards, mice, touch screens, monitors, displays, etc. Of course, various system functions can be implemented in a distributed manner on multiple similar platforms to distribute processing loads. Alternatively, the system can be implemented by the appropriate programming of a computer hardware platform.

[0111] The general discussion of the present disclosure provides a brief overall description of the suitable computing environment that can realize the present disclosure. In one embodiment, any one of the disclosed system, method and / or graphical user interface can be performed or realized by a computing system consistent with or similar to the computing system shown and / or explained in the present disclosure. Although not necessary, various aspects of the present disclosure are described in the context of computer-executable instructions, such as by a data processing device, the routine performed by a server computer, a wireless device and / or a personal computer. Those skilled in the art will appreciate that various aspects of the present disclosure can be put into practice using other communications, data processing or computer system configurations, including internet devices, handheld devices (including personal digital assistants ("PDAs")), wearable computers, various cellular phones or mobile phones (including voice over IP ("VoIP") phones), dumb terminals, media players, gaming devices, virtual reality devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, set-top boxes, network PCs, microcomputers, mainframe computers, etc. In fact, the terms "computer", "server" etc. are generally used interchangeably herein and refer to any of the above-mentioned devices and systems and any data processors.

[0112] Aspects of the present disclosure may be implemented in a special-purpose computer and / or data processor that is specifically programmed, configured, and / or constructed to perform one or more computer-executable instructions described in detail herein. Although aspects of the present disclosure, such as certain functions, are described as being performed only on a single device, the present disclosure may also be practiced in a distributed environment where functions or modules are shared between different processing devices linked by a communication network, such as a local area network ("LAN"), a wide area network ("WAN"), and / or the Internet. Similarly, the technology presented herein as involving multiple devices may be implemented in a single device. In a distributed computing environment, program modules may be located in a local memory storage device and / or a remote memory storage device.

[0113] Aspects of the present disclosure may be stored and / or distributed on non-transitory computer-readable media, including magnetic or optically readable computer disks, hard-wired or pre-programmed chips (e.g., EEPROM semiconductor chips), nanotechnology memories, biological memories, or other data storage media. Alternatively, computer-implemented instructions, data structures, screen displays, and other data under various aspects of the present disclosure may be distributed over a period of time on propagation signals on a propagation medium (e.g., one or more electromagnetic waves, acoustic waves, etc.) over the Internet and / or over other networks (including wireless networks), and / or they may be provided over any analog or digital network (packet switching, circuit switching, or other schemes).

[0114] The procedural aspects of the technology can be considered as a "product" or "article of manufacture," typically in the form of executable code and / or associated data, which is carried or embodied in a type of machine-readable medium. "Storage" type media include any or all tangible memories of a computer, processor, etc., or its associated modules, such as various semiconductor memories, tape drives, disk drives, etc., which can readily provide non-transitory storage for software programming. All or part of the software can sometimes be communicated via the Internet or various other telecommunications networks. For example, such communication can enable software to be loaded from one computer or processor to another, such as from a management server or host of a mobile communication network to a server's computer platform and / or from a server to a mobile device. Therefore, another type of medium that can carry software elements includes optical waves, radio waves, and electromagnetic waves, such as those used on physical interfaces between local devices, through wired and optical ground networks, and through various air links. Physical elements that carry such waves, such as wired or wireless links, optical links, etc., can also be considered as media that carry software. As used herein, unless restricted to non-transitory, tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0115] The terms used above should be interpreted in their broadest reasonable manner, even when used in conjunction with certain specific exemplary embodiments of the present disclosure. Indeed, certain terms may even be emphasized above; however, any term intended to be interpreted in any limited manner will be explicitly and specifically defined in this detailed description. The foregoing general and specific embodiments are merely exemplary and illustrative and are not intended to limit the features protected by the claims.

[0116] As used herein, the terms "comprises," "includes," "has," "contains" or variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements may not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0117] In this disclosure, relative terms such as, for example, "about," "substantially," "generally," and "approximately" are used to indicate a possible variation of ±10% from the specified value.

[0118] The term “exemplary” is used in the sense of an “example” rather than an “ideal.” As used herein, the singular forms “a,” “an,” and “the” include plural references unless the context dictates otherwise.

[0119] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

Claims

1. A method for calculating flight control for a landing vehicle, the method comprising: receiving a landing zone confirmation from the service prior to the descent transition point, the landing zone confirmation including landing zone location information and an indication that the landing zone is clear; determining a landing flight path based on the landing area position information; When the vehicle begins descending to the landing zone using the landing flight path: receiving landing zone data from at least one of a radar system, a camera system, an altitude and heading reference system, and a global navigation satellite system receiver; performing an analysis based on the landing zone data to determine if an unsafe condition exists, wherein receiving the landing zone data and performing the analysis based on the landing zone data comprises: performing a descent cross-check process, wherein the descent cross-check process performs a first sensor process and a second sensor process in parallel or sequentially, wherein: the first sensor process performing a profile checking process and the first of a scanning confirmation process or a visual confirmation process, the second sensor process performs a second of the scanning confirmation process or the visual confirmation process; and The descent cross-check process determines whether an unsafe condition exists; and Flight controls for the vehicle to continue the descent or to modify the descent are calculated based on the analysis.

2. The method of claim 1 , wherein calculating the flight control for the vehicle to continue the descent or modify the descent comprises: continuing the descent if an unsafe condition does not exist, and modifying the descent if the unsafe condition exists, wherein modifying the descent includes one or more of: reducing the descent rate, performing a maneuver to a holding area or an alternative landing area, or reattempting the descent.

3. The method of claim 2, wherein the descent cross-check process determines that an unsafe condition exists in response to: The configuration file inspection process determines that an unsafe condition exists, The first of the scanning confirmation process or the visual confirmation process determines a potential conflict, and the second of the scanning confirmation process or the visual confirmation process confirms the potential conflict as the unsafe condition, or The second one of the scanning confirmation process or the visual confirmation process determines the potential conflict, and the first one of the scanning confirmation process or the visual confirmation process confirms the potential conflict as the unsafe condition.

Images