System and method for active light-based access positioning of aircraft in GPS refright environment

Abstract

Systems and methods are disclosed that provide guidance to assist an eVTOL aircraft in landing and takeoff operations at a landing location in a GPS rejection environment. An exemplary system includes an aircraft, the aircraft includes a camera configured to generate an image based on information transmitted by a plurality of light sources located near a landing surface for the aircraft and a controller circuit configured to receive the generated image and determine a position and orientation of the aircraft based on the received image, wherein the light sources are arranged in a predetermined pattern on the landing surface, and wherein a characteristic of light emitted from each of the light sources is modulated with respect to time.

Description

[0001] Cross - reference to related applications

[0002] This disclosure claims priority to U.S. Patent Application No. 18 / 451,055 (Attorney Docket No.: 16163.0047-00000), entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHT BASED PRECISION LOCALIZATION OF AIRCRAFTS IN GPS-DENIED ENVIRONMENTS", filed on Aug. 16, 2023, which in turn claims priority to U.S. Provisional Patent Application No. 63 / 420,616 (Attorney Docket No.: 16163.6001-00000), entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHTBASED PRECISION LOCALIZATION OF AIRCRAFTS IN GPS-DENIED ENVIRONMENTS", filed on Oct. 30, 2022, and U.S. Provisional Patent Application No. 63 / 381,571 (Attorney Docket No.: 16163.6001-01000), entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHT BASED PRECISION LOCALIZATION OFAIRCRAFTS IN GPS-DENIED ENVIRONMENTS", filed on Oct. 31, 2022. The contents of these patent applications are incorporated herein by reference in their entirety for all purposes.

[0003] The invention described in this patent application and its various embodiments were at least in part made in support of the Department of Defense (Contract No.: FA8649-22-P-0797). To correct a clerical error, the word "FA8649-21-P-0038" in two previously filed provisional patent applications (Serial Nos. 63 / 420,616 and 63 / 381,571) should be replaced, and is hereby replaced with "FA8649-22-P-0797". The United States Federal Government may retain certain rights in this invention. TECHNICAL FIELD

[0004] This disclosure relates generally to the field of powered aerial vehicles. More specifically and without limitation, this disclosure relates to electric vertical takeoff and landing (eVTOL) aerial vehicles and methods for providing high-accuracy, high-reliability, active-light based landing and takeoff positioning guidance therefor. Certain aspects of this disclosure relate generally to precision landing and takeoff systems that can be used in other types of aircraft but provide particular advantages in aerial vehicles. Summary of the Invention

[0005] Embodiments of the present disclosure generally relate to the field of electric vertical takeoff and landing (eVTOL) aerial vehicles. Moreover, and without limitation, the present disclosure relates to systems and methods for providing guidance to assist an eVTOL aerial vehicle in landing and takeoff operations at a landing location in a GPS-denied environment or in an area where GPS is attenuated and has limited accuracy. The present disclosure further relates to methods for providing landing and takeoff guidance and estimating the pose of an aerial vehicle relative to a landing surface. The method may include utilizing a dynamic distribution combination (constellation) of infrared or visible spectrum reference light sources located at known fixed positions around a designated landing site. As the vehicle approaches the landing site, these light sources are viewed by an on-board camera. The pattern from the light sources projected onto the camera image plane can be used to reliably calculate the camera pose (position and orientation) to achieve an appropriate level of accuracy required for precise eVTOL landing.

[0006] One aspect of the present disclosure relates to a system for a landing surface of an aerial vehicle. The landing surface may include a plurality of light sources arranged in a pre-determined pattern, wherein the characteristics of the light emitted from each of the light sources are configured to be modulated relative to time.

[0007] Another aspect of the present disclosure relates to an aerial vehicle including a camera configured to generate an image based on information transmitted by a plurality of light sources located near a landing surface for controlling the vehicle; and a controller circuit configured to receive the generated image and determine the position and orientation of the aerial vehicle based on the received image. The light sources are arranged in a pre-determined pattern on the landing surface, and wherein the characteristics of the light emitted from each of the light sources are modulated relative to time.

[0008] Yet another aspect of the present disclosure relates to a system including a plurality of light sources arranged at a landing surface for an aerial vehicle, the arrangement of the light sources defining a set of intersecting virtual lines, the light sources being arranged on each virtual line, wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0009] Another aspect of the present disclosure relates to a method for estimating the pose of an aerial vehicle. The method may include: providing a landing surface including light sources arranged in a predetermined pattern; modulating the characteristics of the light emitted from the light sources over time; using a camera mounted on the aerial vehicle to receive an input signal associated with the light emitted from the light sources; generating an image of the light sources based on the received input signal; and determining the position and orientation of the aerial vehicle based on the image. Determining the position and orientation of the aerial vehicle includes detecting at least one of the light sources in the image, determining which one of the light sources arranged in the predetermined pattern the detected light source is, and determining the position and orientation of the aerial vehicle based on the determination of which one of the light sources arranged in the predetermined pattern the detected light source is.

[0010] Another aspect of the present disclosure relates to a computer-implemented system for estimating the pose of an aerial vehicle. The system may include a landing surface and at least one processor, the landing surface including light sources arranged in a predetermined pattern. The processor may be configured to: modulate the characteristics of the light emitted from the light sources over time; activate a camera mounted on the aerial vehicle to receive an input signal associated with the light emitted from the light sources; cause the camera to generate an image of the light sources based on the received input signal; and determine the position and orientation of the aerial vehicle based on the generated image. Determining the position and orientation includes: detecting at least one of the light sources in the image; determining which one of the light sources arranged in the predetermined pattern the detected light source is; and determining the position and orientation of the aerial vehicle based on the determination of which one of the light sources arranged in the predetermined pattern the detected light source is.

[0011] Another aspect of the present disclosure relates to a computer-implemented method for estimating the pose of an aerial vehicle, the method including the following operations performed by at least one processor: modulating the characteristics of the light emitted from light sources arranged in a predetermined pattern on a landing surface for an aerial vehicle over time; activating a camera mounted on the aerial vehicle to enable receipt of an input signal associated with the light emitted from the light sources; causing the camera to generate an image of the light sources based on the received input signal; and determining the position and orientation of the aerial vehicle based on the image. Determining the position and orientation includes: detecting at least one of the light sources in the image; determining which one of the light sources arranged in the predetermined pattern the detected light source is; and determining the position and orientation of the aerial vehicle based on the determination of which one of the light sources arranged in the predetermined pattern the detected light source is.

[0012] Another aspect of the present disclosure relates to a non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a device to cause the device to perform a method. The method may include: modulating over time a characteristic of light emitted from light sources arranged in a predetermined pattern on a landing surface for an aerial vehicle; activating a camera mounted on the aerial vehicle to enable receipt of an input signal associated with the light emitted from the light sources; causing the camera to generate an image of the light sources based on the received input signal; and determining a position and orientation of the aerial vehicle based on the image. Determining the position and orientation includes: detecting at least one of the light sources in the image; determining which one of the at least one of the light sources detected is one of the light sources arranged in the predetermined pattern; and determining the position and orientation of the aerial vehicle based on the determination of which one of the at least one of the light sources detected is one of the light sources arranged in the predetermined pattern.

[0013] Another aspect of the present disclosure relates to an aerial vehicle. The aerial vehicle may include: a camera configured to generate an image based on information received from a plurality of light sources located on a landing surface for the aerial vehicle; and a processor associated with the camera. The processor may be configured to receive the image and be configured to perform the following operations: using a detection algorithm to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light detectable by the camera; associating positions of the detected light sources represented in the image with corresponding positions of the light sources on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode; performing one or more association algorithms and generating a confidence score for the association in the first operation mode; performing one or more tracking algorithms in the second operation mode based on the confidence score obtained from the first operation mode; and determining one of the position or orientation of the aerial vehicle based on the performed association.

[0014] Another aspect of the present disclosure relates to a method of operating an aerial vehicle. The method may include: generating an image using a camera based on information received from a plurality of light sources located on a landing surface for the aerial vehicle; using a detection algorithm to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light detectable by the camera; associating positions of the detected light sources represented in the image with corresponding positions of the light sources on the landing surface, wherein performing the association includes a first operation mode and a second operation mode, wherein the first operation mode includes performing one or more association algorithms and generating a confidence score for the association; the second operation mode includes performing one or more tracking algorithms based on the confidence score obtained from the first operation mode; and determining one of the position or orientation of the aerial vehicle based on the performed association.

[0015] Another aspect of the present disclosure relates to a navigation system for an aerial vehicle. The navigation system may include: a camera configured to generate an image based on information received from a plurality of light sources arranged in a predetermined pattern on a landing surface for the aerial vehicle; a processor associated with the camera and configured to receive the image and perform the following operations: activate the camera mounted on the aerial vehicle using the processor to enable receipt of an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface for the aerial vehicle, the light having a characteristic that is modulated with respect to time; enable the camera to generate at least two images of the light source based on the received input signal; use a detection algorithm to detect the light source in the at least two images; associate a position in the image representing the detected light source with a corresponding position of the light source on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode; perform one or more association algorithms in the first operation mode; perform one or more tracking algorithms in the second operation mode based on results obtained from the first operation mode; and determine one of a position or an orientation of the aerial vehicle based on the performed association.

[0016] Another aspect of the present disclosure relates to a system. The system may include: a landing surface for an aerial vehicle; and a plurality of light sources arranged in a predetermined pattern, wherein a characteristic of light emitted from each of the light sources is configured to be modulated with respect to time, wherein the plurality of light sources includes line light sources and point light sources, and wherein the landing surface includes a portable landing surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1A A schematic diagram illustrating a conventional instrument landing system (ILS) that provides horizontal and vertical guidance for guiding an aircraft along a runway.

[0018] Figure 1B and Figure 1C Illustrate a localizer and a glide slope signal transmission that assist a pilot in horizontal and vertical guidance for landing, respectively.

[0019] Figure 2 A schematic diagram illustrating an exemplary landing / takeoff approach using optical navigation in a GPS-denied environment in accordance with the disclosed embodiments.

[0020] Figure 3 Illustrate an exemplary precision landing and takeoff system and a data communication path between an aircraft, a vertical takeoff and landing airport, and a control unit in accordance with the disclosed embodiments.

[0021] Figure 4A and Figure 4BExemplary landing surfaces or vertical takeoff and landing (VTOL) airports and exemplary light sources consistent with some of the disclosed embodiments are illustrated, respectively.

[0022] Figure 5 An exemplary focal plane array (FPA) camera image of a light source of an exemplary vertical takeoff and landing airport consistent with some of the disclosed embodiments is illustrated.

[0023] Figure 6 An exemplary overview of the algorithms and data pipeline during the operation of an accurate landing and takeoff system consistent with some of the disclosed embodiments is illustrated.

[0024] Figure 7 An exemplary detection algorithm for identifying the position of a light source in an image captured by a camera consistent with some of the disclosed embodiments is illustrated.

[0025] Figure 8 A data plot illustrating a comparison of signals before and after signal processing using a band - pass filter consistent with some of the disclosed embodiments is illustrated.

[0026] Figure 9 A data plot illustrating a comparison of signals before and after discrete Fourier transform (DFT) calculation consistent with some of the disclosed embodiments is illustrated.

[0027] Figure 10A and Figure 10B Data plots respectively showing the effects of the number of light sources on the positioning accuracy in the horizontal and vertical directions consistent with some of the disclosed embodiments are illustrated.

[0028] Figure 11A and Figure 11B Data plots respectively showing the effects of the size of the distribution combination of light sources on the positioning accuracy in the horizontal and vertical directions consistent with some of the disclosed embodiments are illustrated.

[0029] Figure 12A and Figure 12B Data plots respectively showing the effects of centroid error on the algorithm robustness in the horizontal and vertical directions consistent with some of the disclosed embodiments are illustrated.

[0030] Figure 12C and Figure 12D Data plots respectively showing the horizontal positioning error and vertical positioning error of the landing trajectory during the simulated approach of an aerial vehicle consistent with some of the disclosed embodiments are illustrated.

[0031] Figure 13A , Figure 13B , Figure 13C and Figure 13DIllustrates an exemplary matching and association process using the thin plate spline algorithm consistent with some of the disclosed embodiments.

[0032] Figure 14A Illustrates an exemplary camera image with detected points consistent with some of the disclosed embodiments.

[0033] Figure 14B Illustrates a schematic diagram of detected points normalized within a predefined space consistent with some of the disclosed embodiments.

[0034] Figure 14C Illustrates an exemplary Hough transform space consistent with some of the disclosed embodiments that includes a mapping of lines passing through Figure 4B a given point in

[0035] Figure 14D Illustrates a schematic diagram of lines formed by discretization of points in the Hough space through Figure 14C points in

[0036] Figure 14E Illustrates an exemplary Hough transform space with line optimization consistent with some of the disclosed embodiments.

[0037] Figure 14F Illustrates an exemplary k - means clustering representation of optimized lines in Figure 14E consistent with some of the disclosed embodiments.

[0038] Figure 14G Illustrates an exemplary representation of mapped points in a two - dimensional space consistent with some of the disclosed embodiments.

[0039] Figure 14H Illustrates an exemplary projection of points in Figure 14G onto an integer grid space consistent with some of the disclosed embodiments.

[0040] Figure 14J Illustrates an exemplary detection of an associated annotation with a light source based on information from Figure 14H consistent with some of the disclosed embodiments.

[0041] Figure 15A Illustrates an exemplary image showing the shift between frame points and corresponding newly predicted positions using homography consistent with some of the disclosed embodiments.

[0042] Figure 15B Illustrates an exemplary image generated by identifying corresponding detections of known associations for tracking by using a nearest - neighbor - based search consistent with some of the disclosed embodiments.

[0043] Figure 16AIllustrates an exemplary distributed combined linear pattern of light sources consistent with some of the disclosed embodiments.

[0044] Figure 16B Illustrates an exemplary distributed combined star pattern of light sources consistent with some of the disclosed embodiments.

[0045] Figure 17A and Figure 17B Illustrates an exemplary distributed combined pattern of light sources consistent with some of the disclosed embodiments.

[0046] Figure 18A and Figure 18B Illustrates an exemplary set of lines consistent with some of the disclosed embodiments using the Random Sample Consensus (RANSAC) sampling method.

[0047] Figure 18C Illustrates the visualization of a process for calculating inliers after finding a suitable candidate set of lines consistent with some of the disclosed embodiments.

[0048] Figure 18D Illustrates an example of using the angular cross ratio to determine the identity of lines in a distributed combination consistent with some of the disclosed embodiments.

[0049] Figure 19 Illustrates an exemplary voting scheme for using the linear cross ratio to determine the identity of points consistent with some of the disclosed embodiments.

[0050] Figure 20A Is a flowchart illustrating an exemplary method for pose estimation using a data association algorithm consistent with some of the disclosed embodiments.

[0051] Figure 20B Illustrates an exemplary drawing showing a circular trajectory of a simulation flying around a distributed combination consistent with some of the disclosed embodiments.

[0052] Figure 21 Illustrates data plots 2100A and 2100B respectively indicating altitude estimates and corresponding errors for the flight trajectory shown in FIG. 20 consistent with some of the disclosed embodiments.

[0053] Figure 22 Illustrates data plots 2200A and 2200B respectively indicating north estimates and ground truth values as well as corresponding errors for the flight trajectory shown in FIG. 20 consistent with some of the disclosed embodiments.

[0054] Figure 23 Illustrates data plots 2300A and 2300B respectively indicating east estimates and ground truth values as well as corresponding errors for the flight trajectory shown in FIG. 20 consistent with some of the disclosed embodiments.

[0055] Figure 24A Schematic diagram of an exemplary random dot marker consistent with some of the disclosed embodiments.

[0056] Figure 24B and Figure 24C Exemplary point recognition method using local likelihood alignment hashing (LLAH) algorithm consistent with some of the disclosed embodiments.

[0057] Figure 24D Exemplary discretization method of cross - ratio for creating a discretized cross - ratio sequence consistent with some of the disclosed embodiments.

[0058] Figure 24E Exemplary hash table for key - point registration consistent with some of the disclosed embodiments.

[0059] Figure 25 Exemplary area cross - ratio calculation for a subgroup of coplanar light consistent with some of the disclosed embodiments.

[0060] Figure 26A , Figure 26B and Figure 26C Exemplary waveform representing intensity modulation and camera shutter speed operation for encoding / decoding information algorithms consistent with some of the disclosed embodiments.

[0061] Figure 27A , Figure 27B and Figure 27C Exemplary modulation scheme for data transmission using a light source consistent with some of the disclosed embodiments.

[0062] Figure 28 Flowchart of an example method for data association synthesis consistent with some of the disclosed embodiments.

[0063] Figure 29 Flowchart of an example method for data association synthesis consistent with some of the disclosed embodiments.

[0064] Figure 30 Schematic illustration of an exemplary arrangement of light in a line shape in a distribution combination of a light source consistent with some of the disclosed embodiments.

[0065] Figure 31A Schematic illustration of an exemplary data encoding scheme consistent with some of the disclosed embodiments.

[0066] Figure 31B Schematic illustration of an exemplary encoding scheme for data transmission using a combination of linear - shaped light and a point source consistent with some of the disclosed embodiments.

[0067] Figure 31C A flowchart illustrating an exemplary method for pose estimation using a line light source consistent with some of the disclosed embodiments.

[0068] Figure 32A And Figure 32B Is a schematic illustration of an exemplary GPS multipath error consistent with some of the disclosed embodiments.

[0069] Figure 33 Is a schematic illustration of an exemplary pipeline for data enhancement configured to use GPS to enhance a Precision Landing and Takeoff (PLaTO) system consistent with some of the disclosed embodiments.

[0070] Figure 34 Is a schematic illustration of an exemplary pipeline for data enhancement configured to use a PLaTO system to enhance GPS consistent with some of the disclosed embodiments.

[0071] Figure 35 Is a schematic illustration of an exemplary pipeline for data enhancement configured to use a PLaTO system to enhance an INS consistent with some of the disclosed embodiments.

[0072] Figure 36 Is a schematic illustration of an exemplary pipeline for data enhancement configured to use an optical positioning system utilizing an Extended Kalman Filter (EKF) to enhance an INS consistent with some of the disclosed embodiments.

[0073] Figure 37 Illustrates an exemplary system consistent with some of the disclosed embodiments showing the integration of a PLaTO system with an aircraft to support manned or unmanned flight.

[0074] Figure 38 Illustrates a data plot consistent with some of the disclosed embodiments indicating the altitude of an aircraft as a function of horizontal distance when the system is used to enhance an INS.

[0075] Figure 39 Illustrates a data plot consistent with some of the disclosed embodiments indicating the height above ground level versus time and the corresponding error when the system is used to enhance an INS.

[0076] Figure 40 Illustrates a data plot consistent with some of the disclosed embodiments indicating the east estimate versus time and the corresponding error when the system is used to enhance an INS.

[0077] Figure 41 Illustrates a data plot consistent with some of the disclosed embodiments indicating the north estimate versus time and the corresponding error when the system is used to enhance an INS.

[0078] Figure 42 is a flowchart illustrating an exemplary method for determining the position and orientation of an aerial vehicle in accordance with some of the disclosed embodiments.

[0079] Figure 43A and Figure 43B illustrates an exemplary quickly deployable distribution combination of light sources in accordance with some of the disclosed embodiments.

[0080] Figure 44 shows an exemplary VTOL aircraft in accordance with the disclosed embodiments.

[0081] Figure 45 shows an exemplary VTOL aircraft in accordance with the disclosed embodiments.

[0082] Figure 46 shows an exemplary top plan view of a VTOL aircraft in accordance with the disclosed embodiments.

[0083] Figure 47 shows an exemplary propeller rotation of a VTOL aircraft in accordance with the disclosed embodiments.

[0084] Figure 48 shows an exemplary power connection in a VTOL aircraft in accordance with the disclosed embodiments.

[0085] Figure 49 shows an exemplary architecture of an electric propulsion unit in accordance with the disclosed embodiments.

[0086] Figure 50 shows an exemplary top plan view of a VTOL aircraft in accordance with the disclosed embodiments.

[0087] Figure 51 shows a data plot of position errors in the X, Y, and Z directions as a function of distance from a distribution combination of light sources obtained from a field data set in accordance with the disclosed embodiments. Detailed Description

[0088] The present disclosure relates to components of an electric vertical takeoff and landing (eVTOL) aircraft primarily for use in an unconventional aircraft. For example, the eVTOL aircraft of the present disclosure may be intended for frequent (e.g., more than 50 flights per weekday), short-duration flights (e.g., less than 100 miles per flight) over, into, and out of populated areas. The aircraft may be intended to carry a desired 4 to 6 passengers or commuters with a low-noise and low-vibration experience. Accordingly, it may be required that their components be configured and designed to withstand frequent use without wear, that they generate less heat and vibration, and that the aircraft include mechanisms for effectively controlling and managing the heat or vibration generated by the components. Additionally, it may be expected that several of these aircraft operate close to each other over crowded metropolitan areas. Accordingly, it may be required that their components be configured and designed to generate a low level of noise both inside and outside the aircraft, and be configured and designed with various safety and backup mechanisms. For example, for safety reasons, it may be required that the aircraft be propelled by a distributed propulsion system, avoid the risk of single-point failure, and that they be capable of conventional takeoff and landing on a runway. Also, compared to traditional airport runways, it may be required that the aircraft be able to safely vertically take off and land from a relatively restricted space (e.g., a vertical takeoff and landing airport, a helipad, or a taxiway) to the relatively restricted space while transporting approximately 4 to 6 passengers or commuters with accompanying luggage. These usage requirements may impose design constraints on the aircraft size, weight, operating efficiency (e.g., drag, energy use), which may affect the design and configuration of the aircraft components.

[0089] The disclosed embodiments provide new and improved configurations of aircraft components not observed in conventional aircraft, and / or identified design criteria for components different from those of conventional aircraft. Such alternative configurations and design criteria, combined to address the drawbacks and challenges of conventional components, result in the embodiments disclosed herein for various configurations and designs of eVTOL aircraft components.

[0090] In some embodiments, the eVTOL aircraft of the present disclosure can be designed to be capable of both vertical and conventional takeoff and landing, where a distributed electric propulsion system enables vertical flight, forward flight, and transition. Thrust can be generated by supplying high-voltage electrical power to the electric motors of the distributed electric propulsion system, each of which can convert the high-voltage electrical power into mechanical shaft power to rotate the propellers. Embodiments disclosed herein can relate to optimizing the energy density of the electric propulsion system. Embodiments can include electric motors connected to an on-board electrical power source, which can include a device capable of storing energy, such as a battery or capacitor, or can include one or more systems for harnessing or generating electrical power, such as a fuel-powered generator or a solar panel array. Some disclosed embodiments provide a reduction in weight and space of components in the aircraft, thereby improving aircraft efficiency and performance. Given the concern for safety in passenger transportation, the disclosed embodiments implement new and improved safety protocols and system redundancy in the event of a failure to minimize any single point of failure in the aircraft propulsion system. Some disclosed embodiments also provide new and improved methods to meet aviation and transportation laws and regulations. For example, the Federal Aviation Administration in the United States enforces federal laws and regulations that require safety components such as fire barriers adjacent to engines that use more than a threshold amount of oil or other flammable materials.

[0091] In a preferred embodiment, the distributed electric propulsion system can include twelve electric motors, which can be mounted on booms at the front and rear of the main wing of the aircraft. The front electric motors can be capable of tilting between a horizontal orientation position (e.g., to generate forward thrust) and a vertical orientation position (e.g., to generate vertical lift) during flight. In terms of the direction of propeller rotation, the front electric motors can have a clockwise type or a counterclockwise type. The rear electric motors can be fixed in a vertical orientation position (e.g., to generate vertical lift). In terms of the direction of propeller rotation, these rear electric motors can also have a clockwise type or a counterclockwise type. In some embodiments, the aircraft can have various combinations of front and rear electric motors. For example, the aircraft can have six front electric motors and six rear electric motors, four front electric motors and four rear electric motors, or any other combination of front and rear motors, including embodiments where the number of front and rear electric motors is not equal. In some embodiments, the aircraft can have four front and rear propellers, where at least four of these propellers include tiltable propellers.

[0092] In a preferred embodiment, for vertical takeoff and landing (VTOL) missions, the front electric engine and the rear electric engine can provide vertical thrust during takeoff and landing. During the flight phase when the aircraft is in the forward flight mode, the front electric engine can provide horizontal thrust, while the propellers of the rear electric engine can be retracted in a fixed position to minimize drag. The rear electric engine can be actively retracted using position monitoring. The transition from vertical flight to horizontal flight and vice versa can be achieved via a tilt propeller subsystem. The tilt propeller subsystem can redirect the thrust between the primary vertical direction during the vertical flight mode and the primary horizontal direction during the forward flight mode. A variable pitch mechanism can change the collective blade angle of the propeller hub assembly of the front electric engine for operation during the hover phase, transition phase, and cruise phase.

[0093] In some embodiments, for conventional takeoff and landing (CTOL) missions, the front electric engine can provide horizontal thrust for wing-borne takeoff, cruise, and landing. In some embodiments, the rear electric engine may not be used to generate thrust during CTOL missions and the rear propellers can be retracted in place.

[0094] Example embodiments are described herein with reference to the accompanying drawings. The drawings are not necessarily drawn to scale. While examples and features of the disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. Further, the words "comprising," "having," "containing," and "including," and other similar forms are intended to be equivalent in meaning and open-ended, as the one or more items after any of these words are not meant to be an exhaustive listing of such one or more items or meant to be limited to only the listed one or more items. It should also be noted that, as used herein and in the appended claims, the singular forms "a / an" and "the" include plural referents unless the context clearly dictates otherwise.

[0095] Throughout this disclosure, reference is made to "the disclosed embodiments," which refers to examples of the inventive ideas, concepts, and / or manifestations described herein. Many related and unrelated embodiments are described throughout this disclosure. The fact that some "disclosed embodiments" are described as exhibiting a feature or characteristic does not mean that other disclosed embodiments must share this feature or characteristic.

[0096] Embodiments described herein include a computer-readable medium (e.g., a non-transitory computer-readable medium) that contains instructions which, when executed by at least one processor, cause the at least one processor to perform a method or a set of operations. The non-transitory computer-readable medium can be any medium that can store data in any memory in a manner readable by any computing device having a processor to implement a method or any other instructions stored in a memory. The non-transitory computer-readable medium can be implemented to include any combination of software, firmware, and hardware. The software can preferably be implemented as an application program tangibly embodied on a program storage unit or a computer-readable medium composed of parts or by a combination of certain devices and / or equipment. The application program can be uploaded to and executed by a machine including any suitable architecture. Preferably, the machine can be implemented on a computer platform having hardware such as one or more central processing units (“CPUs”), a memory, and an input / output interface. The computer platform can also include an operating system and microinstruction code. The various processes and functions described in this disclosure can be part of the microinstruction code or part of the application program or any combination thereof, which can be executed by the CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units can be connected to the computer platform, such as additional data storage units and printing units. Furthermore, the non-transitory computer-readable medium can be any computer-readable medium other than a transitory propagated signal.

[0097] The memory can include any mechanism for storing electronic data or instructions, including random access memory (RAM), read-only memory (ROM), hard disks, optical disks, magnetic media, flash memory, other permanent, fixed, volatile, or non-volatile memories. The memory can include one or more separate storage devices capable of storing data structures, instructions, or any other data in juxtaposition or distribution. The memory can further include a memory portion containing instructions for execution by the processor. The memory can also be used as a working memory device for the processor or as a temporary storage device.

[0098] Some embodiments can relate to at least one processor. The “at least one processor” can constitute any physical device or group of devices having circuitry for performing logical operations on one or more inputs. For example, the at least one processor can include one or more integrated circuits (ICs), including application-specific integrated circuits (ASICs), microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a server, a virtual server, or other circuitry suitable for executing instructions or performing logical operations. The instructions executed by the at least one processor can, for example, be pre-loaded into a memory integrated with or embedded in the controller, or can be stored in a separate memory.

[0099] In some embodiments, at least one processor may include more than one processor. Each processor may have a similar construction, or the processors may have different constructions that are electrically connected or disconnected from each other. For example, a processor may be a separate circuit or integrated in a single circuit. When more than one processor is used, the processors may be configured to operate independently or collaboratively. The processors may be electrically coupled, magnetically coupled, optically coupled, acoustically coupled, mechanically coupled, or coupled by other means that allow them to interact.

[0100] As used herein, unless otherwise expressly stated, the term "or" encompasses all possible combinations, unless infeasible. For example, if it is stated that a component may include A or B, then unless otherwise expressly stated or infeasible, the component may include A, or B, or A and B. As a second example, if it is stated that a component may include A, B, or C, then unless otherwise expressly stated or infeasible, the component may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0101] In the following description, various working examples are provided for illustrative purposes. However, it should be understood that the present disclosure may be practiced without one or more of these details. Reference will now be made in detail to non-limiting examples of the present disclosure, which are illustrated in the accompanying drawings. The examples are described below with reference to the drawings, in which like reference numerals refer to like elements. When like reference numerals are shown, the corresponding description is not repeated, and the interested reader may refer to the previously discussed drawings for a description of the like elements.

[0102] Various embodiments are described herein with reference to systems, methods, devices, or computer-readable media. The purpose is to disclose one and disclose all. For example, it should be understood that the disclosure of the computer-readable media herein also constitutes the disclosure of methods implemented by the computer-readable media and systems and devices for implementing those methods via, for example, at least one processor. It should be understood that this form of the present disclosure is only for the convenience of discussion, and within the scope contemplated by the present disclosure, one or more aspects of one embodiment herein may be combined with one or more aspects of other embodiments herein.

[0103] Consistent with the present disclosure, some implementations may relate to a network. The network may constitute any combination or type of physical and / or wireless computer networking arrangements for exchanging data. For example, the network can be the Internet, a private data network, a virtual private network using a public network, a Wi-Fi network, a mesh network, a local area network (LAN), a wide area network (WAN), and / or other suitable connections and combinations that enable information exchange among various components of the system. In some embodiments, the network may include one or more physical links for exchanging data, such as Ethernet, coaxial cable, twisted pair cable, optical fiber, or any other suitable physical medium for exchanging data. The network may also include a public wired network and / or a wireless cellular network. The network can be a secure network or an insecure network. In other embodiments, one or more components of the system may communicate directly through a dedicated communication network. The direct communication may use any suitable technology, including, for example, BLUETOOTH TM , BLUETOOTHLE TM (BLE), Wi-Fi, near field communication (NFC), or other suitable communication methods that provide a medium for exchanging data and / or information between separate entities.

[0104] Reference will now be made in detail to example embodiments, which are illustrated by way of example in the accompanying drawings. The following description refers to the accompanying drawings, where the same numbers in different drawings represent the same or similar elements, unless otherwise indicated. The implementations set forth in the following description of the example embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with aspects related to the subject matter recited in the appended claims. Without limiting the scope of the present disclosure, some embodiments may be described in the context of providing systems and methods in an electric vertical takeoff and landing (eVTOL) aircraft or an aerial vehicle. However, the present disclosure is not limited thereto. Other types of aerial vehicles (such as, but not limited to, unmanned aerial vehicles (UAVs), manned aerial vehicles, conventional vertical takeoff and landing (VTOL) aircraft, hybrid VTOLs, and other aerial vehicles) may utilize the systems and methods disclosed herein.

[0105] Advanced Air Mobility (AAM) is an emerging field of aeronautics that involves using small aircraft for daily transportation and other services, and many AAM aircraft are expected to take off and land on new infrastructure called vertiports. As described herein, a vertiport is a landing location or landing surface on which an aerial vehicle such as an eVTOL lands or takes off. In some embodiments, a vertiport may also be referred to as a vertiplex or a vertistop. The location of a vertiport can be determined based on many factors, including but not limited to physical obstacles, federal and state or local regulatory restrictions, surrounding uses, and the like. Physical obstacles can be fixed, anticipated, moving, or temporary obstacles. Examples of anticipated physical obstacles may be adjacent properties that have the development rights for a 40-story building but are currently vacant land. Some examples of physical obstacles may include nearby high-rise buildings, antennas, towers (communication towers and water towers), trees, power lines, utility poles, billboards, land use planning for the vertiport location, rights of landowners, and the like.

[0106] In some cases, regulatory restrictions may include both the current land use designation of the vertiport location and the rights of the property owner. By way of example, in an air rights transaction or transfer of development rights, an owner may sell their right to build in the space above their property to a buyer who wants to construct a larger building than they would otherwise be permitted to build. For example, if a hangar operator sells the air rights above their hangar, a proposed vertiport terminal that would extend into that space may not be built without the approval of the owner of the air rights. Height zones are geographical areas where the maximum building height is restricted, and this should also be considered when siting a vertiport. Physical considerations, including physical obstacles, can be weighed and balanced in view of anticipated future development patterns and the vision of the jurisdiction as it seeks to accommodate population migration, increases or decreases in density, and development, such as the current trend towards mixed-use communities where residential and commercial buildings are closer to each other.

[0107] Moving or temporary physical obstacles include structures that are of a changing or temporary nature. Moving or temporary physical obstacles can include both planned and anticipated considerations. Planned considerations are those that involve processes in which the vertical takeoff and landing airport operator may have an opportunity to provide input, while anticipated considerations are those that occur without or with little prior notice but are likely to occur during the life cycle of the vertical takeoff and landing airport. Some examples of temporary structures may include temporary vertical takeoff and landing airports, construction cranes, flying debris, construction staging, noise, lightning protection equipment, non-acoustic interference factors, electrostatic discharges, urban wind shadows, or future local land use. While these considerations reflect temporary and potentially insignificant events during the operational life of the vertical takeoff and landing airport, they are still worthy of consideration to support safe and efficient operations. Additionally, the vertical takeoff and landing airport siting decision may also be influenced by the anticipated frequency of certain temporary considerations. For example, locating a vertical takeoff and landing airport near tall trees increases the likelihood of regular debris entering the movement area of the vertical takeoff and landing airport, including at some point in the future when the trees may grow to penetrate the airspace of the vertical takeoff and landing airport and pose a hazard to the general aviation airspace.

[0108] When selecting the location of a vertical takeoff and landing (VTOL) airport and designing its operations, it may be crucial to consider the surrounding area. Surrounding uses encompass considerations that arise outside the boundaries of the VTOL airport property but within the local vicinity. These considerations can impact the VTOL airport during site selection, design, or operations and may also change over the life cycle of the VTOL airport. The VTOL airport can also affect the surrounding area and modify these considerations. Some examples of surrounding uses that can influence the site selection of a VTOL airport include critical infrastructure, local fire stations, subway or bus stops, local land use, distance to maintenance or repair facilities, downwind of wind farms, etc. In some cases, the surrounding uses may be affected by the VTOL airport. Some examples of this scenario include schools in the nearby area, properties beneath approach and departure paths, noise-sensitive areas, visual interference (e.g., reflectivity of solar panels), zoos, protected wildlife habitats, privacy of VTOL airport neighbors, etc. The proximity of the VTOL airport to existing infrastructure can be a major site selection factor. Infrastructure considerations include current local land use (e.g., schools, hospitals, parks, or other noise-sensitive areas), emergency response (e.g., fire stations), and direct connections to other transportation options (i.e., multimodality). For early VTOL airport site selection, proximity to these types of existing infrastructure can enable timely development and operations by reducing the lead time for development of these secondary criteria (e.g., land use designated for transportation). On the other hand, if the VTOL airport is sited too close to other types of infrastructure, flight operations may be hindered. For example, proximity to a wind turbine farm may limit approach and departure paths and cause interference with the air flow, which can impede safe flight operations. There may be several other factors when designing or configuring a VTOL airport, including but not limited to aircraft performance in the VTOL airport environment, passenger comfort, economic considerations (such as development costs, maintenance costs, and revenue generation), environmental considerations, airspace considerations, demand considerations, emergency considerations, communication and data management, safety considerations, security and usability, automation, etc.

[0109] For use in urban air mobility, landing and taking off an eVTOL aircraft in an urban environment may require highly accurate and highly integrity positioning capable of operating in a GNSS-challenged environment. As used herein, a GPS denied or GPS-challenged environment is an environment that lacks reliable access to Global Positioning System (GPS) or Global Navigation Satellite System (GNSS) signals. In a GPS denied environment, GPS signals may be attenuated, interrupted, denied, jammed, hacked, or simply disabled due to multipath effects or blockage of satellite signals. Satellite signals may be denied in difficult environments due to a lack of a clear line of sight path between the satellite and the user antenna. Signals may be interrupted or attenuated due to adverse weather conditions, low or poor visibility, high density of tall buildings in an urban environment, adverse or uncooperative landing conditions, etc.

[0110] An aerial vehicle (e.g., Figure 3The aerial vehicle 310) can include an electric-powered aerial vehicle or an eVTOL vehicle. An eVTOL vehicle that can be used for on-demand urban air transportation services can provide an alternative mode of transportation in urban environments with low direct operating costs, low noise, and zero tailpipe emissions. Additionally, an eVTOL aircraft provides an alternative form of air transportation that is guaranteed to be versatile (able to take off and land vertically from a basic landing area), economical (reducing procurement and operating costs), easy to use (enabling operators with little or no flight training or experience to operate), and safe (designed to better withstand failure modes). In this regard, the development of distributed electric propulsion (DEP) can enable inexpensive, quiet, and reliable short-range VTOL aircraft. The use of DEP can provide significant flexibility, which may allow for new aircraft configurations, architectures, and control methods. Additionally, electric propulsion is scale-free in terms of motor power, enabling highly similar levels of weight and efficiency on a large scale. For example, redundant DEP can be used to improve fault tolerance and flight safety. The use of electric motors can also improve safety on the ground through noise reduction, heat dissipation, and the potential for toxic fumes, as well as by shutting down rotating propellers or rotors before passengers enter or exit. Additionally, although the propellers or ducted fans of DEP still generate noise (the noise level and frequency will depend on tip speed, disk loading, and other design parameters), by combining DEP configurations (with multiple smaller rotors directly driven by electric motors), tip speed limits, removing or minimizing engine / turbine / gear noise sources, and potentially using fixed wings for efficient forward flight, the eVTOL is expected to have a modified or reduced noise signature compared to a conventional helicopter of similar size, with a target noise reduction of 15 dB or more. Battery-powered eVTOL aircraft can also have a reduced environmental impact with zero operating emissions. Additionally, the use of DEP instead of complex shafts, cross-couplings, and gear assemblies is expected to reduce both procurement, maintenance, and operating costs. When extended range is required, the aircraft can be designed with a hybrid electric propulsion system that can take advantage of operating a smaller engine at peak efficiency. Operating the hybrid unit engine at idle or off during takeoff and landing can further reduce the aircraft's noise signature at lower altitudes.

[0111] Figure 1A A schematic diagram illustrating a conventional instrument landing system (ILS) that provides horizontal and vertical guidance for guiding an aircraft along a runway is shown. The ILS is a standard precision landing aid that is used to provide accurate azimuth and descent guidance signals for guiding an aircraft to land on a runway under normal or adverse weather conditions. The ILS can include three subsystems, namely, the localizer, the glide slope, and the marker beacon. As Figure 1B shown, the localizer provides horizontal guidance for an approaching aircraft, as Figure 1CAs shown, the glide slope provides vertical guidance for the approaching aircraft, and the marker beacon provides range information during the approach. In some cases, the marker beacon may be replaced by a distance measuring equipment (DME). In addition, the ILS may further include high-intensity lighting at the end of the runway to assist the pilot in locating the runway and transitioning from the approach to a visual landing.

[0112] In the ILS system, as Figure 1B and Figure 1C shown, two or more radio frequencies (RFs) are broadcast, one of which is spatially offset from the other. The spatially offset RF signals repeat in the horizontal and vertical directions. In some cases, a signal sensor associated with the aircraft may measure the intensities of the two signals. If one signal is greater than the other, the aircraft is off-center from the configured landing trajectory. Based on this information, the pilot can correct the heading to continuously align with the centerline and thus navigate along the glide slope to the runway. Although ILS and associated systems are maturely applied to conventional passenger aircraft, they may not be suitable for eVTOL aerial vehicles in urban environments due to non-standard flight approach trajectories, interference with nearby ILS systems or all buildings, or the large footprint of the ILS.

[0113] Now refer to Figure 2 , which illustrates a schematic diagram of an exemplary landing / takeoff approach using optical navigation in a GPS-denied environment consistent with some embodiments of the present disclosure. Figure 2 shows an example view 200 of an eVTOL aircraft approaching a landing site or landing position having an active marker in its camera field of view in a GPS-denied environment such as an urban environment with a high density of tall structures. An exemplary aerial vehicle 210 (such as an eVTOL) may approach a vertical takeoff and landing airport 220 using one or more approach paths 260. One of the several advantages of using an eVTOL is that, unlike the conventional course-based landing approach in an ILS system, the aerial vehicle can approach the vertical takeoff and landing airport from any direction. By way of example, view 200 illustrates another approach direction (approach 2). In some embodiments, as shown, the approach angle may vary between 7° and 9°. The aerial vehicle 210 may include a camera having a field of view 270 ( Figure 2 not illustrated in, but discussed in a later section).

[0114] Now refer to Figure 3, which illustrates an exemplary precision landing and takeoff system and data communication system consistent with some embodiments of the present disclosure. As described herein, a precision landing and takeoff system refers to an optically-navigated eVTOL positioning system operating in a GPS-denied environment. As previously discussed, eVTOL aircraft (such as aircraft 310) can be used in urban air mobility (UAM) applications that give rise to commercial passenger services such as air taxis, or in public service applications such as firefighting, medical aid delivery, emergency search and rescue operations, disaster relief operations, law enforcement, and the like. In some embodiments, the aircraft can be autonomous, i.e., unmanned.

[0115] The precision landing and takeoff system can include an aircraft 310 that includes an airborne optical detection device 315, a vertical takeoff and landing airport 320 that includes markers 350, and a ground control unit 330. The optical detection device 315 can include a camera 311 and a processor 312. In some embodiments, the optical detection device 315, the ground control unit 330, and one or more vertical takeoff and landing airports 320 can communicate wirelessly with each other during the landing or takeoff operation of the eVTOL aircraft 310. The communication between the optical detection device 315, the ground control unit 330, and one or more vertical takeoff and landing airports 320 can include receiving and transmitting data or information associated with providing landing or takeoff guidance to the aircraft 310.

[0116] The precision landing and takeoff system can include one or more vertical takeoff and landing airports 320 (also illustrated as vertical takeoff and landing airport 420 in FIG. 4). In some embodiments, the vertical takeoff and landing airport 320 or vertical takeoff and landing airport 420 can include a landing surface or landing position for an eVTOL aircraft (such as aircraft 310). The vertical takeoff and landing airport 320 can include a plurality of light sources arranged in a predetermined pattern, wherein the characteristics of the light emitted from each of these light sources are configured to be modulated relative to time. Each vertical takeoff and landing airport can include a combination of markers 350 or a dynamic distribution of active light sources. As used herein, an active light source (ALS) refers to a light source whose characteristics can be modulated over time. For example, the intensity of the light emitted from an active light source can be modulated over time. Other characteristics that can be modulated over time include, but are not limited to, the frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light. Examples of active light sources can include, but are not limited to, light-emitting diodes (LEDs).

[0117] In some embodiments, the landing and takeoff area of the vertical takeoff and landing airport 320 can be rectangular, circular, triangular, substantially rectangular, substantially circular, or substantially triangular, or a combination thereof, or other suitable shapes. As used herein, the landing and takeoff area refers to the area on which an aerial vehicle (e.g., aerial vehicle 310) of the vertical takeoff and landing airport can land or take off / ascend. In some embodiments, the active light sources can be arranged in a predetermined pattern that resembles the shape of the landing and takeoff area, such that the active light sources define the boundary of the landing and takeoff area. In some embodiments, the predetermined pattern of the active light sources that define the boundary of the landing and takeoff area can include low-intensity light sources that are within the field of view of a camera when the aerial vehicle approaches the landing target to assist landing or takeoff.

[0118] In some embodiments, the active light sources can be arranged in a substantially axisymmetric shape (such as circular, square, or rectangular). In some embodiments, the active light sources can be arranged in an oval, triangular, trapezoidal, or other shape. In some embodiments, the active light sources can be spaced equally or unequally in an axisymmetric shape. In an unequally spaced arrangement in an axisymmetric shape, the distance between adjacent active light sources may be non-uniform. In some embodiments, the active light sources can be arranged in a grid-based pattern, where the light sources are evenly spaced on the landing platform. Other arrangements (such as suitable arrangements) are also possible.

[0119] In some embodiments, the active light sources can be arranged in an asymmetric shape to maximize the detectability, distinctiveness, or anti-spoofing and anti-jamming capabilities of the active light sources associated with their respective vertical takeoff and landing airports. Figure 5 Examples of asymmetric shapes that illustrate the arrangement of the active light sources are discussed later.

[0120] In some embodiments, the marker 350 can include a distributed combination of infrared (IR) or visible-spectrum reference light sources (e.g., active light sources) located at known positions distributed throughout the vertical takeoff and landing airport 320. In this regard, the precise landing and takeoff system can be referred to as an active fiducial light pattern localization (AFLPL) system. Some of the several advantages of using active light sources as markers in vertical takeoff and landing airports for optical navigation of eVTOL aerial vehicles include:

[0121] i. Authentication and security - One or more characteristics of the light emitted can be modulated to achieve authentication and improve security.

[0122] ii. Can be easily distinguished from the surrounding environment, thereby allowing visual clutter in urban environments to be reduced or eliminated.

[0123] iii. Enhanced Range, Detectability, and Data Transmission - One or more characteristics can be modulated to enhance range, detection, and data transmission, including communicating messages to aircraft or enabling low-bandwidth communication.

[0124] iv. Day / Night Capability - The aerial vehicle can operate during both day and night.

[0125] v. Anti-Jamming and Anti-Spoofing - The optically-based AFLPL method is inherently more resistant to jamming and spoofing than radio frequency (RF) signals. The modulation capabilities of AFLPL can further allow for the implementation of authentication codes.

[0126] vi. No Major Regulatory Hurdles - The AFLPL method may provide a simpler path to obtaining FAA certification, thereby enhancing the acceptability and feasibility of implementing active light sources in public service applications. Additionally, unlike RF-based ranging and communication methods, AFLPL will not require an FCC allocation of the RF spectrum, which has limited availability.

[0127] Furthermore, in military applications, AFLPL narrowband near-IR illumination can provide a lower detection level than RF emissions.

[0128] vii. High Positioning and Guidance Accuracy - The positioning accuracy compared to GPS or GNSS may be equal to or higher in the vicinity of a vertical takeoff and landing (VTOL) airport, specifically in GPS-challenged or GPS-denied environments. The position accuracy error may be less than 1 m within 100 m of a VTOL airport and decreases as the vehicle approaches the VTOL airport. In some cases, the position accuracy error may be less than 10 cm within 10 m of a VTOL airport.

[0129] viii. High Redundancy and High Reliability - Active light sources can be a relatively low-cost option and have no weight in the air,

[0130] thus allowing multiple light sources to be employed at a VTOL airport to provide uninterrupted guidance.

[0131] ix. Enhanced Visibility - The infrared (IR) wavelength of the emitted light can enhance visibility and detectability under adverse weather conditions such as fog, rain, storms, lightning, etc.

[0132] x. Low Implementation Complexity - Minimal infrastructure may be required at a VTOL airport. Visible and infrared light sources can be integrated into the VTOL airport structure and recessed into the landing field with relatively low technical complexity and cost.

[0133] In some embodiments, the positions of the active light sources in the distribution combination in a vertical takeoff and landing airport can be designed to provide optimized positioning across the entire landing trajectory. In some embodiments, the distribution combination can include a first set of light sources arranged in a first predetermined pattern, and each light source in the first set of light sources is configured to be within the field of view of a camera associated with an aerial vehicle when the aerial vehicle is at a first distance from the landing surface. The first set of light sources can include light sources with higher intensity and located at a greater distance from the landing target, which can improve performance when the aerial vehicle is at a relatively long distance from the vertical takeoff and landing airport. The first set of light sources can be within the camera's field of view at a greater distance and outside the camera's field of view when the aerial vehicle approaches the vertical takeoff and landing airport or the landing target. The distribution combination can further include a second set of light sources arranged in a second predetermined pattern, and each light source in the second set of light sources is configured to be within the field of view of the camera when the aerial vehicle is at a second distance from the landing surface. The second set of light sources can include light sources with lower intensity and located within a smaller distance, such that when the aerial vehicle is in its final approach or within a predetermined approach distance, the second set of light sources remains within the camera's field of view. The intensity of the light sources in the second set can be different from that of the first set of light sources to avoid interfering with the detection of the first set of light sources from a greater distance. In some embodiments, the intensity of the first set of light sources can be higher than that of the second set of light sources. In some embodiments, the area covered by the first set of light sources can be larger than the area covered by the second set of light sources. In other words, compared with the second set of light sources, the first set of light sources can be distributed over a larger area so that only the second set of light sources can be detected when the aerial vehicle is within a predetermined approach distance.

[0134] In some embodiments, the light sources in the distribution combination can be arranged to maximize the detectability of each position by maximizing the spacing between each light source. In some embodiments, the light sources can be arranged to maximize the ability to identify a light source from a plurality of light sources, such as by minimizing the symmetry of the arrangement pattern. In some additional embodiments, the predetermined pattern of the light sources can be associated with the landing surface. For example, the distribution combination can include a pattern that can be uniquely identified for each vertical takeoff and landing airport, such that the vertical takeoff and landing airport can be identified based on the arrangement pattern of the light sources in the vertical takeoff and landing airport.

[0135] In some embodiments, additionally or alternatively, the active light source (e.g., the reference point) may be located on the ground, on top of a building, or on other objects along a common flight path. Some possible locations include, but are not limited to, on a road, on top of a light pole, on top of a building, on top of an antenna, or other tall structures. This will allow for accurate position information to be obtained throughout the flight rather than only when visible on the landing surface or landing field. In addition to this, the same algorithm may be used to calculate the position, but since the light source pattern is spread over a much larger area, the system can operate throughout the duration of the flight or for most of the flight duration rather than only when approaching the landing field near the end.

[0136] In some embodiments, the size and dimensions of one or more vertical takeoff and landing (VTOL) airports may vary. When the aerial vehicle is far from the VTOL airport, the size of the VTOL airport can determine the degree of performance. While the reference pattern within the distribution combination may not need to be the same for each VTOL airport, it may be necessary to know in advance the layout or arrangement of the reference light markers. However, in some embodiments, the light source may be configured to transmit information related to its own position, in which case it may not be necessary to know in advance the position or arrangement of the light source. In some embodiments, the distribution combination pattern or arrangement may be stored in the system's database or server. The information stored in the database may be accessible and updatable in real time or based on input from the user.

[0137] In some embodiments, one or more characteristics of the emitted light may be modulated to transmit information. The modulation may be performed by one or more methods, including but not limited to switching on or off, frequency modulation, amplitude modulation, duty cycle modulation, synchronization options, etc. The information transmitted may include the identification of the VTOL airport, the position of the light source, the identification of the light source, the operating status of the VTOL airport. In some embodiments, the information transmitted may include an encoded signal to authenticate the landing surface or VTOL airport. It should be understood that a combination of modulation methods and the information transmitted by modulation may be appropriately applied. For example, the frequency of the light emission from the light source may be modulated to transmit information associated with the identification of the light source, and the duty cycle may be modulated to transmit information associated with the identification of the VTOL airport. In another example, the frequency of the light emission from the light source may be modulated to transmit information associated with the identification of the VTOL airport. In some embodiments, the frequency may be modulated to indicate the operating status of the VTOL airport, such as normal operation, abnormal operation, maintenance, etc.

[0138] In some embodiments, the wavelength of the light emitted from one or more light sources may be determined based on several factors, including but not limited to maximizing the difference between the emitted light and the background light to improve detectability, minimizing the absorption of the emitted light by the atmosphere or weather effects, maximizing the sensitivity detected by the camera, or reducing visible light pollution around the landing target, etc.

[0139] In some embodiments, the wavelength of the light emitted from one or more light sources is in the range of 800 nm to 1550 nm. In some embodiments, the wavelength of the light emitted from one or more light sources is in the range of 800 nm to 850 nm. In a preferred embodiment, the wavelength of the light emitted from one or more light sources is 810 nm. In some embodiments, the wavelength of the light emitted from one or more light sources is 1310 nm. In some embodiments, the wavelength of the light emitted from one or more light sources is 1550 nm.

[0140] In some embodiments, as shown in FIG. Figure 4A one or more light sources 450 may be recessed relative to the landing surface of the vertical takeoff and landing airport 420. Figure 4B Exemplary light sources 450 are illustrated in FIG. In some embodiments, one or more light sources may be recessed, protruding, or coplanar with the landing surface. In some embodiments, the light source 450 may include a protective cover or encapsulation cover to prevent the entry of moisture, dust, or other particles that may affect the performance of the light source. In some embodiments, the protective cover may be configured to transmit substantially all of the light emitted from the light source such that the absorption by the protective cover is negligible or non-existent.

[0141] In some embodiments, each light source may further include an optical sensor configured to detect a portion of the light emitted from at least one other light source among the light sources. This may require synchronizing the camera capture rate with the modulation of the light source to reduce errors that may occur in the case of capturing images during transitions (discussed in a later section). In some embodiments, the landing surface or the vertical takeoff and landing airport may further include a controller circuit configured to operate the light source. In this context, operating the light source may include activating, deactivating, or modulating the characteristics of the light source by, for example, adjusting the electrical signal applied to the corresponding light source. The controller circuit may further include a time management circuit, a power management circuit, a sequencing circuit, etc. In some embodiments, one or more processors may be configured to remotely control the operation of the light source.

[0142] In some embodiments, the wavelength of the emitted light may be modulated to increase the detectability or range of the light source, or to transmit data from a corresponding reference point. The inventors recognize that while the wavelength can be adjusted, it may have a negative impact on the simplicity and feasibility of the system. For example, to detect wavelength changes, a hyperspectral camera may be required. Although such cameras exist and are commercially available, they may be complex, less reliable, and computationally intensive. Additionally, since the wavelength of an LED is mainly determined by the material composition and the transmission medium, such light sources may be experimental, less reliable, and expensive.

[0143] In some embodiments, the vertical takeoff and landing airfield 320 (or vertical takeoff and landing airfield 420) can be a portable landing surface. A portable vertical takeoff and landing airfield (not illustrated herein) can include a deployable landing pad, fabric, or tarp. This can be particularly useful in situations where landing is required at a non-cooperative location with limited or temporary landing infrastructure, such as military operations, firefighting efforts, disaster relief operations, medical aid distribution operations, and the like. In some embodiments, the portable vertical takeoff and landing airfield can include battery-powered active light sources incorporated therein such that they can be remotely activated, deactivated, or modulated.

[0144] In some embodiments, the vertical takeoff and landing airfield can include a plurality of landing surfaces, wherein each landing surface includes a plurality of light sources arranged in a predetermined pattern, and wherein the characteristics of the light emitted from each of these light sources are configured to be modulated with respect to time.

[0145] In some embodiments, one or more landing surfaces of the vertical takeoff and landing airfield can be horizontally displaced from each other, for example, in a vertical takeoff and landing airfield hub, or a vertical takeoff and landing complex, or a large area including a plurality of vertical takeoff and landing airfields. The horizontally displaced vertical takeoff and landing airfields can be coplanar or substantially coplanar. As used herein, the term "coplanar" or "substantially coplanar" means that the landing surfaces of the vertical takeoff and landing airfield are on the ground, similar to a car parked in a parking lot. In some embodiments, one or more vertical takeoff and landing airfields can be vertically displaced from each other such that they are non-coplanar, for example, in a vertical takeoff and landing airfield hangar including multiple levels of vertical takeoff and landing airfields. In some embodiments, one or more vertical takeoff and landing airfields can be horizontally and vertically displaced from each other such that they are offset from each other in the horizontal and vertical axes, thereby allowing for a higher density of vertical takeoff and landing airfields.

[0146] Return reference Figure 3 , the aerial vehicle 310 can include an optical detection device 315, which further includes a camera 311 and a processor 312. The camera 311 can be configured to generate an image based on information transmitted by a light source (e.g., active light source 350) located near the landing surface for the aerial vehicle. For example, the camera 311 can be configured to capture an image of the light emitted by the light source. The processor 312 can include controller circuitry configured to receive the generated image and determine the position and orientation of the aerial vehicle based on the received image.

[0147] In some embodiments, camera 311 may include a color, monochrome, or hyperspectral camera. Camera 311 may be mounted on an aerial vehicle 310 such that camera 311 may provide a plan view of a light source on a landing surface. A plan view of the light source may be desirable when the aerial vehicle is within a landing distance (where the vertical descent is from a height of 50 ft above ground level (AGL) to 0 AGL), or when the aerial vehicle is taking off. In some embodiments, camera 311 may be mounted on an aerial vehicle 310 such that camera 311 may provide a forward view of a light source on a landing surface. This may be desirable for maximizing visibility during approach. In some embodiments, one or more cameras may be mounted on an aerial vehicle to capture multiple frames or views from different angles during landing or takeoff.

[0148] In some embodiments, camera 311 may always be on but be activated to capture images and / or report measurements when a fiducial point or an active light source on the landing surface is detected. Alternatively, camera 311 may be turned on or activated to save power when the aerial vehicle is within a detection distance. In alternative embodiments, camera 311 may be turned on or activated within a predetermined duration, at a predetermined time, or by an activation signal from an external processor (e.g., a flight control computer or a ground control unit 330), or by an operator of the aerial vehicle. In some embodiments, camera 311 may be configured to be activated after the aerial vehicle is within a predetermined distance from the landing surface. The predetermined distance may be based on several factors, including but not limited to weather conditions, landing surface conditions, etc. In a preferred embodiment, the predetermined distance may be 500 m or less.

[0149] In some embodiments, camera 311 may include an optical filter that is configured to permit a wavelength range of light emitted from each of these light sources. In other words, the optical filter of camera 311 may be configured to reject wavelengths that are significantly different from a reference transmission wavelength. As used herein, the reference transmission wavelength refers to the wavelength or wavelength range of light emitted by one or more fiducial markers (e.g., an active light source on the landing surface). For example, if the active light source is configured to emit light at a wavelength of 810 nm, the optical filter may permit a wavelength range of 808 nm to 812 nm and reject wavelengths outside the allowable transmission range. In some embodiments, the sensitivity of the optical detection of camera 311 may be adjusted to filter incoming wavelengths.

[0150] In some embodiments, the permitted wavelength range is in the range of 800 nm to 850 nm. In a preferred embodiment, the permitted wavelength range is approximately 810 nm. In some embodiments, the permitted wavelength range is approximately 1310 nm. In some embodiments, the permitted wavelength range is approximately 1550 nm. In some embodiments, the optical filter may be configured to permit wavelengths corresponding to the emitted light. As used herein, the term "about" refers to an approximation such that the permitted wavelength range is within ±2 nm or less. The optical filter may be a low-pass filter, a high-pass filter, or a band-pass filter. Based on the detected light source, the camera 311 may generate an FPA image, as Figure 5 shown.

[0151] Now refer to Figure 6 , which illustrates an exemplary overview of the algorithm and data pipeline 600 during the operation of an accurate landing and takeoff system consistent with some embodiments of the present disclosure. As shown, a camera (e.g., camera 311) may be configured to receive an optical signal from a light source on a vertical takeoff and landing airport as well as background information such as scene information. The camera including the optical filter may be configured to output a series of still images, or a video stream, or an FPA image. Based on the generated output signal, the processor may be configured to execute one or more algorithms to detect, associate, estimate the pose of an aerial vehicle, and decode the information in the encoded signal from the active light source. The detection algorithm allows the identification of the position of the active light source in the image frame captured by the camera. The association algorithm allows the association or mapping of the identified active light source in the image to the corresponding active light source on the landing surface. The pose estimation algorithm allows the determination of the pose of the aerial vehicle based on the associated active light source.

[0152] Detection

[0153] Identifying the position of the active light source in the image frame may include distinguishing the received signal from the background noise signal. This may be done using background subtraction and threshold operations. If the active light source is modulated in such a way that it is fully on in one frame and fully off in another frame, the frame in which the light source is fully off may be used as the background image to remove the background from the fully on image by subtraction. An exemplary subtraction algorithm is provided here. It should be understood that other suitable subtraction and thresholding techniques may be used to identify the position of the active light source in the image.

[0154] By way of example, if I_n is an MxN matrix of pixel values corresponding to the nth image, then I_diff = abs(I_n - I_(n-1)) is the difference in pixel values of consecutive images. The resulting difference frame (I_diff) can be thresholded to generate a mask that identifies what pixels in the original image correspond to the active light source. Thus, I_mask = where (I_diff > threshold). In the case where the active light source is identified in the image, the position can be calculated via centroid mathematics that can calculate the position of the light source with sub-pixel accuracy.

[0155] Centroid = ∑(Pi * Xi) / ∑Pi where P i is the value of the ith pixel indicated by the mask, and X is the position (xy pair) of that pixel. Additionally or alternatively, image filtering techniques such as temporal filtering and spatial filtering can be used to detect and locate the active light source in the image.

[0156] In some embodiments, improving the detection or localization of the position of the active light source in the image frame may include performing a registration method. This may be desirable for images where the background is moving rapidly due to the movement of the camera (such as a camera mounted on a moving aerial vehicle). The active light source may change position between the on-frame and the off-frame in the image. In this case, it may be necessary to shift the image in order to align between frames, thereby aligning the background for subtraction. To achieve this, one of several techniques including feature matching, translational matching, or current state estimation can be used. In feature matching, features common to each frame can be identified and their positions in each frame can be determined. The image can be translated and / or distorted to align the features. In translational matching, for small changes, two images can be shifted one pixel at a time to determine the position where the background most closely matches. In current state estimation, if the position and rotational speed of the camera are known or can be estimated from previous images, the image distortion required to align each frame can be estimated. Additionally, registration can be performed globally or locally on one or more regions of interest around the active light source. It should be understood that other techniques for improving detection can be employed as alternatives or in combination with the techniques described herein.

[0157] In some embodiments, identifying the location of an active light source in an image frame or improving detection may include tracking the location of the identified active light source to reduce the computational load of subsequent calculations. To adequately predict the location of the active light source, certain measurements of velocity or time change may be useful. This may be obtained by using the aircraft position and velocity from an external system such as GPS or an inertial navigation system (INS), using an internal estimate of the aircraft position and velocity derived from the change in position over time, or using the change in pixel position over time such as tracking the change in the active light source between frames and extrapolating in time. Tracking may be used to reduce the computational load by calculating the region of interest for detection rather than the entire image, to improve accuracy by providing an estimate for registration, or to calculate additional information such as velocity or acceleration that may be reported to other devices on the aircraft.

[0158] Figure 7 Exemplary detection algorithm 700 for identifying the location of a light source in an image captured by a camera, consistent with some embodiments of the present disclosure, is illustrated. As shown, the camera may receive a noise signal and generate an output that is typically in the form of an image or image stream. The image may be processed using an application of a bandpass filter, discrete Fourier transform (DFT) calculations on the filtered image, a low-pass filter, and thresholding. Figure 8 and Figure 9 A data plot is illustrated that compares the signals before and after signal processing using a bandpass filter and DFT calculations, respectively, consistent with some embodiments of the present disclosure. In some embodiments, one or more characteristics of the active light source may be modulated relative to time to improve the detection of the active light source in the captured image.

[0159] Data association

[0160] As used in the context of the present disclosure, data association refers to the process of matching detected points in a camera image with known points in a database of known positions of active light sources, optical markers, or fiducial points on a landing surface. Some prior art techniques for data association may include modulating one or more light sources to convey a unique identifier of the light source and determining a position based on the light modulation. However, such methods may have challenges such as, but not limited to, inaccurate identification due to cross-signaling, poor signal-to-noise ratio (SNR), high background noise, etc. As previously mentioned and as disclosed in some embodiments of the present disclosure, one or more data association algorithms may be executed to map detected points in a camera image to known positions of active light sources. The selection of a data association algorithm may depend on several factors, including but not limited to the existence of an acceptable data association, the reliability, accuracy, robustness, etc. of the obtained association results. As an example, after finding an acceptable and correct association, points may be tracked between images, for example, by a point tracking method (discussed in detail later). As another example, although a grid association algorithm may not depend on an initial association, the algorithm may not always produce a solution. Thus, it may be necessary to execute two or more data association algorithms in parallel or sequentially to establish an acceptable data association and to correlate each identified fiducial point in a camera image with the three-dimensional (3D) position of the identified fiducial point. Some aspects of the present disclosure relate to methods and systems for data association and their advantages.

[0161] a. Iterative closest point algorithm

[0162] In some embodiments, an iterative closest point (ICP) algorithm may be used. The algorithm may include the following steps.

[0163] 1. Given a position estimate, estimate where the ALS will be located in the image.

[0164] 2. For each detected point, find the closest (geometric distance) estimated point.

[0165] 3. Sum the distances between each pair of points (measured and estimated).

[0166] 4. Determine a small perturbation (x-y translation and rotation) of the estimated points that reduces the error distance calculated in step 3.

[0167] 5. Update the estimated position of the points based on the perturbation calculated in step 4.

[0168] 6. Return to step 2 until the error converges.

[0169] 7. After the solution has converged, obtain the resulting association from step 2 (nearest neighbor).

[0170] Although simple to implement, ICP can be sensitive to inaccuracies in the initial estimate of the pose. In some cases where the step size may be too small, it may take a long time to converge and the computational cost may be high.

[0171] b. Thin Plate Spline Robust Point Matching Algorithm

[0172] A spline is a numerical function defined by polynomial functions. Spline functions have a high degree of smoothness at the locations where the polynomial segments are joined (called knots). Feature-based methods for non-rigid registration may face challenges associated with the correspondence of points between two or more sets of features. In this context, the correspondence between two sets of features refers to the association between each identified fiducial point in a 2D image and the 3D position of a fiducial point (e.g., an active light source on the ground). The framework of non-rigid point matching or robust point matching (RPM) algorithms can be extended to include spline-based deformations, and specifically thin plate splines. Some methods that solve both the correspondence and the transformation include ICP (discussed previously). The ICP algorithm uses the nearest neighbor relationship to assign binary correspondences at each step. This estimate of the correspondence can be used to optimize the transformation and vice versa. Although the ICP algorithm is simple and fast and can guarantee convergence to a local minimum, this may not be sufficient, especially when the deformation is large. In addition, the correspondence deteriorates rapidly with outliers, making the ICP algorithm inadequate. In spline theory, generating a smooth interpolation of the spatial mapping that adheres to two sets of landmark points is a common problem. This is because once non-rigidity is allowed, there are an infinite number of ways to map one set of points to another. Smoothness constraints are desirable because they prevent mappings that are too arbitrary or are outliers. In other words, the behavior of the mapping can be controlled by selecting a specific smoothness factor based on prior knowledge.

[0173] Now refer to Figures 13A to 13D , which illustrates the correspondence and matching process using the thin plate spline robust point matching algorithm consistent with some of the disclosed embodiments.

[0174] Figure 13A shows a coordinate space including a normalized grid sorted according to the sorting of the ground active light sources. As Figure 13A shown, the exemplary normalized grid contains 25 points in the format of a 5x5 array. Each point in the normalized grid is numbered from 0 to 24 according to the sorting of the light on the ground. In some embodiments, the normalized grid may be rotated based on the heading of the aircraft approaching the landing surface. The coordinate space further shows the normalized detected points received from the detection algorithm. The normalized detected points are indicated using a "+" mark. The normalized detected points may overlap with the normalized grid points. The circular boundary indicates the radius of the potential matches to be evaluated using robust point matching.

[0175] The Thin Plate Spline Robust Point Matching (TPS-RPM) algorithm may include performing distance-based point matching and association for a larger search area. Figure 13B A coordinate space including normalized grid points and the positions of detected points after thin plate spline deformation is shown (e.g., as Figure 13D shown). After thin plate spline deformation, a line may be drawn to connect the normalized grid points with the corresponding normalized detected points (if any).

[0176] The TPS-RPM algorithm may further include determining a TPS distortion that can be used to reduce the error for associated points. The algorithm may further include reducing the size of the search area and iteratively performing the TPS distortion until the normalized detected points and the normalized grid points converge. Figure 13C The association of the normalized grid points connected to the normalized detected points is shown, indicated by the line connecting the two.

[0177] In some embodiments, given the iterative nature of the TPS-RPM algorithm, using a normalized initial guess to match a known pattern to detected points may take an extremely long time. Thus, to improve the overall performance of the algorithm, particularly when the aircraft is approaching a vertical takeoff and landing airport where there are large changes between frames, the TPS-RPM algorithm may operate in two modes. The first mode will use a normalized initial guess to estimate the pose of the system. Based on a confidence metric in the estimated pose, other association algorithms (such as ICP, cross ratio, grid association, etc.) may be used to supplement or assist by seeding the TPS-RPM with the current pose of the aircraft, and using this current pose to project the known pattern into the camera frame and using this projection as our initial guess before we normalize. In effect, this approach can greatly reduce the number of iterations required to converge to a solution, thus allowing the use of TPS-RPM at full rate.

[0178] c. Grid-based association algorithm

[0179] Now refer to Figure 14A, which represents an exemplary camera image 1400 with detected points (indicated by a square grid) consistent with the disclosed embodiments. The camera image 1400 can be an image captured by a camera mounted on an aerial vehicle approaching closer to a landing surface and at a non-zero angle relative to a plane perpendicular to the landing surface. In some embodiments, the image 1400 may include any number of detected points in an arrayed or non-arrayed arrangement. It should be understood that although the image 1400 illustrates a 5x5 grid of detected points, other grid patterns or random patterns including any number of detected points may also be used. In some embodiments, the detected fiducial points or points may be numbered in a known, distinguishable order. For example, the detected points in the image 1400 may be numbered from 0 to 24 moving from the top left to the top right starting from the top row, and the left-to-right numbering is repeated for each row. It should be understood that the numbering order should in no way be construed as limiting. The numbering format is exemplary and non-restrictive.

[0180] Figure 14B Represents a normalized image space 1410 that includes the normalized detected points of the image 1400. In some embodiments, the detected points may be normalized such that the detected points fit within a predefined image space, for example, -100 to +100 arbitrary units (a.u.), as Figure 14B shown. In some embodiments, normalization may include constructing a transformation matrix using the mean and variance of the points, centering the detected point means, and making the variance of the detected points equal to one (value 1). Some advantages of normalizing the detected points within the normalized image space are that all points are within a unit distance and there is no offset for these points. Additionally, any processing (e.g., data manipulation) performed on one or more points may equally affect the remaining points.

[0181] After the normalization of the detected points, as Figure 14B shown by the normalized image space 1410 of, lines passing through multiple detected points in the normalized image space 1410 can be identified by performing a Hough transform technique. As mentioned herein, the Hough transform technique can be used to isolate features of a specific shape within an image that includes multiple points in a parametric form, such as lines, circles, ellipses, etc. In line detection using the Hough transform, each input measurement (e.g., coordinate point) indicates its contribution to a globally consistent solution (e.g., the physical line that produced that image point). As an example, when fitting a set of line segments to a set of discrete image points (e.g., pixel positions), the lack of knowledge about the desired number of line segments may not constrain the possible solutions for fitting line segments through the image points.

[0182] In the context of image analysis, the coordinates (i.e., x, y) of points on an edge segment in an image are known and thus act as constants in the parametric line equation xcosθ + ysinθ = r, where r and θ are unknown variables. Plotting the possible (r, θ) values defined by each (x, y) point in the Cartesian image space maps to a curve (i.e., a sine curve) in the polar coordinate Hough parameter space, as illustrated by the Hough transform image 1420 in Figure 14C This point-to-curve transformation is the Hough transform for lines. When viewed in the Hough parameter space, collinear points in the Cartesian image space produce curves that intersect at a common (r, θ) point.

[0183] The Hough transform can be used to identify one or more parameters of a curve that fits a set of given points. In some cases, the Hough transform can also help determine what the features are (i.e., to detect features with a parametric description) and how many features are present in the image. The curves generated by collinear points in the gradient image intersect at the peak (r, θ) in the Hough transform space. These intersection points characterize the line segments of the original image. An extractor mechanism can be employed to extract local maxima (e.g., intersection points) from the accumulator array. For example, one method among several methods can include applying thresholding and refinement to isolated clusters of local maxima in the accumulator array image or the Hough transform image 1420. In the context of the Hough transform, thresholding refers to setting a predefined limit for the maxima in the accumulator array, and the values in the accumulator array may be equal to or greater than the predefined maximum.

[0184] In some embodiments, identifying a line that passes through multiple detected points in the normalized image space 1410 can include, among other steps, using the Hough transform to map all lines that pass through a given point in a single sine curve in the Hough transform image 1420, discretizing the Hough space into multiple bins 1424, and incrementing a bin 1424 by one if the sine curve passes through the bin 1424. Each detected point in the normalized image space 1410 is transformed into the Hough space, and if a bin 1424 in the Hough space has a value higher than a predetermined threshold, that point in the Hough space will map to a line that passes through at least that many points in the image when inverse Hough transformed. As an example, in a 5x5 grid array of detected points shown in the normalized image space 1410, the bin threshold can be set to 4. Points in the Hough space represented by bins 1424 with a bin threshold of 4 will map to lines that pass through at least four points in the normalized image space 1410.

[0185] In some embodiments, the data association algorithm can further include optimizing the lines by, for example, culling or removing lines with a poor fit to the detected points. The discretization in the Hough space may produce lines (such as Figure 14DIn the image space 14130 shown), these lines may deviate slightly from the best line fit through a set of detected points. Alternatively or additionally, discretization may also result in multiple lines that are close to each other. The data association algorithm may include optimizing a set of best fit lines by eliminating lines with poor fits. The steps for optimization may include selecting a line from a set of lines and drawing the line in the image space, determining all points located within a predefined distance of the line, performing simple linear regression to define the line that most closely fits these points, and removing other lines located within a certain distance in the Hough transform space. These steps may be repeated any number of times as needed.

[0186] Figure 14E Represents the Hough transform image 1440 after the line optimization steps discussed above with reference to Figure 14D In the Hough transform image 1440, the clusters 1444 correspond to the lines that exist after the optimization steps. One or more cycles of line optimization may be performed iteratively as needed.

[0187] In some embodiments, associating each identified fiducial point in the image with a point in the 2D image of the detected points may include inverse Houghing, i.e., mapping the points from the polar coordinate image space (e.g., the Hough transform space) to a regular grid (e.g., the Cartesian coordinate image space). Doing so may allow for the identification of missing points as well as points that are not included on the grid. Inverse Houghing may include the steps of identifying groups of parallel lines by identifying groups of all Hough points with similar theta (θ) values, and using a clustering algorithm (such as k-means clustering) to identify groups of lines with similar theta (θ) values. Figure 14F Illustrates an exemplary k-means clustering representation 1450 of the optimized lines in Figure 14E It should be understood that other clustering algorithms may also be used. In the representation 1450, clusters of lines with similar slope (or theta) values may be formed, and the distribution of the points in the representation 1450 indicates the range of slopes of the lines in the clusters. Thus, the representation 1450 illustrates four groups of substantially parallel lines with one-dimensional (θ)-based clustering. For example, the first group of substantially parallel lines may be represented as having slopes in the range of 0 to 1 radian, the second group of substantially parallel lines may be represented as having slopes in the range of 1 to 2 radians, the third group of substantially parallel lines may be represented as having slopes in the range of 2 to 2.5 radians, and the fourth group of substantially parallel lines may be represented as having slopes in the range of 2.5 to 3 radians.

[0188] From the representation 1450, two points from the group with the most points and two lines with different theta values may be selected. The intersection of the four lines in the image space may be used to generate four points that form a rectangular box, as Figure 14GThe representation is shown in 1460. The vertices of the rectangular box can be labeled 0, 1, 2, and 3 (starting from the upper right vertex and counterclockwise to the lower right vertex). In some embodiments, a homography matrix can be calculated that moves points in the image space to an integer grid, preferably a square grid. In the context of the present disclosure, a homography is a transformation that occurs between two planes. In other words, the homography is a mapping between two planar projections of an image using a transformation matrix that allows shifting one view to another by multiplying the homography matrix with points in the view to find their corresponding positions in another view of the same scene. When it comes to the present disclosure, at least four points may be required to calculate the homography because the homography matrix has 8 free variables (each point can contain x and y, for a total of 8 equations).

[0189] After calculating the homography, the calculated homography can be used to map all points to the integer grid. In some embodiments, the mapped points can be rescaled such that the minimum and maximum values lie Figure 14H at the edges of the integer grid 1470 as shown.

[0190] One way among several ways to determine a successful mapping through an association algorithm is to determine whether each point in the image 1460 is mapped to a separate and discrete position on the square integer grid 1470. Each reference position on the square integer grid (e.g., the integer grid 1470) can be labeled or numbered using reference characters (e.g., numbers, alphanumerics, letters, or other suitable characters) based on a predefined sequence. The mapping from the points in the image 1460 to the points on the integer grid 1470 can indicate the distance or "offset" between the reference position of the integer grid 1470 and the mapped point. The offset can be presented in arbitrary units or can indicate the actual offset distance between the reference position and the mapped point.

[0191] In some embodiments, the association of the identified fiducial points in the image with the 3D positions of the fiducial points may further include excluding error detection and out-of-range detection. Error detection can include detecting light sources that do not conform to or are not recognized as valid light sources, such as but not limited to reflections from objects, transient light sources with similar characteristics, etc. Excluding error detection can include culling points that are at a distance greater than a predetermined threshold offset distance from the reference position. In some embodiments, the predetermined threshold offset distance can be an absolute integer value, or a fraction, or a percentage of the distance between two adjacent reference positions, or other numbers.

[0192] Additionally, in some embodiments, data associations that produce a mapped integer grid (e.g., having a mapped integer grid 1470) may be culled based on the number of identified error detections. For example, if the number of error detections exceeds a predetermined threshold number of error detections, the data association may be culled, resulting in no association at all, thereby making the data association algorithm a reliable data association algorithm. Figure 14J An annotated detection image 1480 representing detected light sources based on data associations.

[0193] d. Point tracking algorithm

[0194] Now refer to Figure 15A , which illustrates an exemplary image consistent with the disclosed embodiments showing the shift between frame points and corresponding newly predicted positions using a homography for point tracking. Point tracking can be used to generate associations for a new set of detected points. One way among several ways to perform point tracking may include propagating existing associations from one image frame to subsequent image frames based on a previous set of associated points and corresponding pose estimates. In some embodiments, point tracking may include, but is not limited to, extracting corresponding features between two groups of frames, using the corresponding features to calculate a homography, applying the homography to previously known associations, and using a nearest neighbor type search to map the associations to new detections.

[0195] In some embodiments, point tracking may include extracting features or unique features from a previous group of frames and identifying corresponding features in the current group of frames. In this context, the previous group of frames and the current group of frames may refer to the (n - 1)th group of frames and the nth group of frames, where n is an integer. When the aerial vehicle is at position (p - 1) at a given time (t - 1), the previous group of frames (the (n - 1)th group of frames) may include a plurality of pixels of an image captured by a camera (e.g., a camera mounted on the aerial vehicle), and when the aerial vehicle is at position (p0) at a given time (t0), the current group of frames (n0) may include a plurality of pixels of an image captured by the camera. As used herein, the current group of frames refers to the subsequent group of frames immediately following the previous group of frames such that there is no group of frames in between.

[0196] In some embodiments, two sets of features extracted from two groups of frames may be used to determine a homography matrix that may be configured to transform any point in one image to a corresponding point in another image. The determined homography may be configured to shift points from the previous group of frames to predicted positions in the subsequent group of frames.

[0197] In some embodiments, point tracking can be implemented as local association tracking by applying the determined homography to the previously associated detected points in the previous frame group. Alternatively or additionally, point tracking can be determined as pose-based tracking. Exemplary pose-based tracking techniques can include using a previously known position estimate, using pose estimation to project known fiducial points into the image frame, and applying a homography to the projected points as a prediction of where those points will be located in the current frame.

[0198] In some embodiments, as Figure 15B shown, a nearest neighbor search step can be used to find correspondences between the detected points from the detection step and the predicted points from the tracking step so that an associated identification can be found for each detection. Image 1520 represents an exemplary image generated by using a nearest neighbor-based search to identify corresponding detections of known associations to tracking, consistent with the disclosed embodiments.

[0199] In the context of the present disclosure and in optical signal processing, temporal filtering refers to isolating the frequency components of a temporal sequence of images into a specific frequency band or range. The filter used for temporal filtering can have any canonical form, e.g., finite impulse response or infinite impulse response. Spatial filtering refers to the process of changing the nature of an optical image by selectively removing specific spatial frequencies that make up the object. In spatial filtering techniques, the Fourier transform of the input function can be manipulated by the filter. The spatial filter can be a convolutional filter (where the kernel moves over the image) or some other form of spatially oriented filter. As an example, Gaussian blur can be applied to an image to remove high spatial frequencies.

[0200] As previously described, the Active Fiducial Light Pattern Localization (AFLPL) method (also referred to herein as Precise Landing and Takeoff (PLaTO)) presents significant advantages for eVTOL aircraft positioning during the approach and landing phases when GPS is challenged, impaired, or completely unavailable. However, to best utilize AFLPL, an understanding of its potential limitations and methods for mitigating these limitations must be addressed and developed. A brief description of the limitations and mitigation strategies is provided herein.

[0201] i. Geometric constraints — The main geometric limitation for AFLPL is the accuracy decay that occurs as the range to the illumination distribution combination increases. For a fixed focal length camera, the maximum useful horizontal range is approximately 100 times the size of the illumination distribution combination. The second geometric limitation lies in the camera field of view (FOV) constraint that limits the approach trajectory to the landing site. This limitation can be mitigated by using multiple cameras with different viewing directions and FOVs. Given the small size, light weight, and low power usage of cameras, two or three cameras can be used with a minimal impact on aircraft cost and performance.

[0202] ii. Detection and SNR constraints — For visible and near-infrared systems, the background light from the sun and other artificial sources may be significant compared to the intensity of the proposed distributed combined illumination. The AFLPL light source needs to have sufficient brightness, modulation ability, and a combination of different wavelengths to enable detection from a cluttered and backlit but mainly static background. By employing a combination of visible, near-infrared, and long-wavelength infrared sources and detectors, a high percentage of GPS-denied landing scenarios can be achieved.

[0203] iii. Data association — The perspective-n-point algorithm requires the positions of the distributed combined light sources to be known, and the detection of point sources in the focal plane of the camera to be associated with each of the distributed combined light sources. This data association problem is further complicated as point sources leave the field of view due to camera motion or occlusion. This problem has been well studied and is fundamental to many computer vision applications, and there are many standard algorithms that have been developed to solve the association problem, such as RANSAC. It is important to note that the association with fiducial points composed of active point sources is much less complex than the association of visual features from unstructured passive imagery. In the AFLPL method, we are dealing with a small number of different point sources with a high detection probability under favorable placement. Additionally, association can be facilitated by modulating each fiducial source over time using unique codes, or via spectral matching by making each source emit light of a different wavelength.

[0204] Simulation studies and associated analyses have been carried out by exploring the accuracy of the positioning solutions produced by various perspective-n-point (PnP) algorithms under various operating conditions, including the number of distributed combined light sources, the physical size of the distribution combination, and the imaging quality of the camera system, to verify the feasibility of the AFLPL method. Two methods were used to model the fiducial-based positioning of an eVTOL aircraft. Equations were implemented for modeling a pinhole camera with a 4k imager (3840x2160 resolution) and a 90-degree field-of-view lens. Also implemented were the aircraft trajectory corresponding to the landing profile shown and the illumination distribution combination configuration. These models were used to examine the sensitivity of the positioning accuracy to variations in the number of distributed combined light sources, the distribution combination size, and camera imaging errors, as discussed in references Figure 2 shown, Figure 10A , Figure 10B , Figure 11A , Figure 11B , Figure 12A and Figure 12B .

[0205] Now refer to Figure 10A and Figure 10B, which illustrates simulated data plots showing the effect of the size of the distribution combination on the positioning accuracy in the horizontal and vertical directions, respectively, in accordance with some embodiments of the present disclosure. To study the effect of the number of distribution combination points on the positioning accuracy, the eVTOL camera is positioned at discrete positions along Figure 2 the landing profile. At each position, an embodiment of the proposed system is used to calculate the position and altitude of the camera. To generate the Figure 10A and Figure 10B plots, the number of light sources is varied. For each number of light sources in each range, the light sources are evenly distributed in a rectangular volume of 40m x 40m x 20m. 1000 randomly positioned light configurations are used to calculate the mean positioning error. The camera position is compared with the true camera position, and the horizontal and vertical components of the positioning error are calculated and averaged over 1000 runs to generate the plots. The positions of the light points in the image frame are rounded to the nearest pixel. As shown, the positioning error decreases with the number of light sources imaged. If at least 15 sources are used, the positioning error is below 10m within 1200m and drops to below 5m within 500m. When the aircraft approaches the landing target, the positioning error drops to the centimeter level range.

[0206] Figure 11A and Figure 11B illustrate simulated data plots showing the effect of the size of the distribution combination of light sources on the positioning accuracy in the horizontal and vertical directions, respectively, in accordance with some embodiments of the present disclosure. The simulated data plots are generated by varying the horizontal size of the distribution combination of light sources while keeping the number of light sources and the vertical size of the distribution combination constant. The vertical dimension of the distribution combination is 20m, and 20 light sources are used. For each size of the distribution combination at each camera position, 20 light sources are evenly distributed in the distribution combination volume. One thousand randomly positioned light configurations are used to calculate the mean positioning error. The camera position is compared with the true camera position, and the horizontal and vertical components of the positioning error are calculated and averaged over 1000 runs to generate the plots. As shown, the positioning error decreases with the size of the distribution combination. If a horizontal baseline of at least 30m is used, the positioning error is below 10m within 1200m and drops to below 1m within 500m. When the aircraft approaches the landing target, the positioning error drops to the centimeter level range.

[0207] Figure 12A and Figure 12B illustrate simulated data plots showing the effect of centroid error on the algorithm robustness in the horizontal and vertical directions, respectively, in accordance with some embodiments of the present disclosure. When the light points are imaged by the camera, small errors can be introduced by the optics and the image sensor. Figure 12A and Figure 12BThe plots shown depict the effect of these errors on the positioning accuracy. These plots were generated by rounding the floating-point pixel positions of the light sources (their true centroid positions on the image plane) to the nearest integer (with an average error of 0.3 pixels) and then adding different amounts of normally distributed error with a mean of 0, 1, 2, or 3 pixels. This was done while keeping the number of light sources (20) and the size of the distribution combination (40x40x20m) constant within each range. 1000 randomly positioned light configurations with added imaging error were used to calculate the mean positioning error. The camera positions were compared with the true camera positions, and the horizontal and vertical components of the positioning error were calculated and averaged over 1000 runs to generate the plots. As shown, the positioning error increases with the error in the centroid position. If a horizontal baseline of at least 30m is used, the positioning error is below 10m within a range of 1200m and drops to below 1m within a range of 500m. When the aircraft approaches the landing target, the positioning error drops to the centimeter level range. The cases with errors of 0.3 pixels and 1.3 pixels meet the horizontal error requirements.

[0208] Now refer to Figure 12C and Figure 12D , which illustrate simulation data plots showing the horizontal positioning error and the vertical positioning error of the landing trajectory during the simulated approach for an aircraft, respectively, in accordance with some embodiments of the present disclosure. The algorithm was tested using an eVTOL simulation environment. The positioning error increases during the transition from the forward camera to the nadir camera. We expect that this transition error can be resolved by running the nadir positioning algorithm before the transition and fusing the results with those from the forward camera, rather than making a sudden switch between the two.

[0209] Optical distribution combination design

[0210] In some embodiments, the accuracy of data association may be affected by distortion of the pattern of a light source due to the viewing perspective. For example, the pattern of a light source viewed directly above the landing surface may have a different shape compared to when the pattern of the light source is viewed from a shallow viewing angle. This difference in the shape of the pattern of the light source based on the viewing angle may be referred to as "viewpoint transformation". Under viewpoint transformation, many recognition features may be lost. Therefore, it may be necessary to design a reference pattern for the active light source and develop a corresponding data association algorithm for identifying reference pattern points in an image captured by a camera mounted on an aerial vehicle. There is a further need to create a software pipeline for generating highly accurate pose estimates using multiple batches of aerial images of the reference pattern of the active light source laid out on the ground. In some situations (such as in a dense urban environment or adverse weather conditions), obtaining an estimate of the position or orientation of an aircraft may be challenging. In such cases, performing data association and subsequent pose estimation without prior knowledge or estimation of the aircraft position and / or orientation may further exacerbate the problem. Therefore, there is a further need to design a reference pattern and a data association algorithm with the ability to perform data association and subsequent pose estimation without additional sensors or measurement mechanisms based on isolated snapshots of the reference pattern of the light source taken from arbitrary viewpoints. The proposed reference pattern design and data association algorithm are designed to address one or more of the challenges identified herein. In some embodiments of the present disclosure, the proposed reference pattern design and data association algorithm may also allow the pose estimation pipeline to continue to function despite obstacles or malfunctions that prevent some distributed combination of light sources in the reference pattern from being observed or when additional ambient light sources are visible in the camera image.

[0211] Many geometric properties that exist in three-dimensional space are inconsistent when mapped to two-dimensional space under projective transformation. For example, length, area, centroid, and parallelism in a camera image all depend on the position and orientation of the camera relative to the subject of the image. However, the cross-ratio is constant regardless of the viewing angle from which the camera image is taken and serves as the main principle for the reference distribution combination design. In some embodiments, the reference distribution combination design utilizes the projective invariant property of the cross-ratio. The cross-ratio is a viewing angle invariant property that can be used for accurate data association. As used herein, "cross-ratio" refers to the ratio of four values each calculated according to a unique subgroup of features, where the product of two of these values is divided by the product of the other two values. For example, a linear cross-ratio is the ratio of ratios of lengths between collinear points, and an angular cross-ratio is the ratio of ratios of angles between intersecting lines.

[0212] Now referring to Figure 16A , which illustrates an exemplary line having four points A, B, C, and D located at different distances from each other, the linear cross-ratio can be calculated as follows:

[0213]

[0214] The cross - ratio is a constant value for line segments that is independent of the viewing angle. The selection of the line segments used in this calculation may result in different cross - ratios, but they remain the same regardless of the viewing perspective. The cross - ratio is invariant to the viewing angle. For example, for a given set of four points on a line, a total of six cross - ratios can be calculated. The six cross - ratio values can be used to calculate a single invariant that is unique to the spatial distribution of the four points on the line.

[0215] Now refer to Figure 16B , which illustrates an exemplary distribution combination star pattern of light sources consistent with the disclosed embodiments. The light sources can be positioned along line segments A - A, B - B, C - C, and D - D arranged in a star configuration such that different angles can be formed by the intersecting lines. In such a configuration, the angular cross - ratio can be calculated as follows:

[0216]

[0217] or calculated as:

[0218]

[0219] In some embodiments, the landing surface may include a distribution combination of light sources. A plurality of light sources can be arranged at the landing surface for an aerial vehicle, and the arrangement of the light sources defines a set of intersecting virtual lines, with the light sources arranged on each virtual line, where the distance between adjacent light sources on each virtual line is non - uniform, as Figure 17A shown, which illustrates a design 1700 of a distribution combination of active light sources invariant to the viewing angle consistent with the embodiments of the present disclosure. As Figure 17A shown, the geometric features of the distribution combination design 1700 include four lines (AOA, OB, OC, and OD) that intersect at a single point O and are labeled as lines 1702, 1704, 1706, and 1708. Each of lines 1704, 1706, and 1708 can be a virtual line connecting five light sources B1 - B5, C1 - C5, and D1 - D5 respectively. Line 1702 can connect ten light sources A1 - A10. The different lines in the distribution combination and the light sources belonging to these lines can be detected by Random Sample Consensus (RANSAC) or other means. Given the set of lines, there can be two distinct angular cross - ratios, one angular cross - ratio calculated using the lines ABCD in order, and another angular cross - ratio calculated using the order BCDA. Since line AOA can be distinguished from the other lines by the number of lights (i.e., there are ten lights compared to five lights on OB, OC, and OD), the correct identification of each of the other lines can be determined by calculating the cross - ratio obtained using three lines in clockwise or counter - clockwise order from line OA.

[0220] To associate the light detected in the camera image with the specific light in a reference distribution (e.g., distribution combination design 1700), a data association algorithm (such as grid association, ICP, thin plate spline, or point tracking, or a combination thereof) may be configured to fit lines to the cloud of detected points in the camera image, identify the specific lines in the distribution combination, and identify the specific points within each identified line. The data association algorithm may include one or more of the following steps.

[0221] In some embodiments, the data association algorithm may include the step of using RANSAC to determine a best-fit set of four lines having a single intersection point and a correct angle crossing ratio among the point cloud detected in the camera image. Figure 18A And Figure 18B Some examples of potential RANSAC sampling methods such as random pair sampling and k-nearest neighbor (k-NN) sampling are illustrated.

[0222] In some embodiments, using the RANSAC sampling method may include the following steps: (a) Sample four pairs of points from the point cloud of the detected light and draw a line through each pair to form four lines. The sampling of the point pairs may be done randomly or pairs may be drawn from a set of k-nearest neighbors to increase the likelihood of finding point pairs that lie on the same line in the distribution combination. The type of sampling used for RANSAC (random pair or k-NN sampling) may depend on how efficiently a K-dimensional (KD) tree may be used to compute a set of k-nearest neighbors as compared to randomly sampling point pairs. (b) Compute the intersection point represented by the least squares solution of the system of equations describing the set of four lines sampled in step (a). If the error associated with this solution is higher than a predetermined threshold, i.e., the points that best fit the set of four lines are far from each of these lines, then the set of lines does not have a close enough intersection point and the algorithm may return to step (a). Step (c) includes, when determining that the set of lines intersects at a single point, computing the angle crossing ratio of the set of lines by starting from an arbitrary line and including the other three lines in a clockwise or counterclockwise order. As previously described, there may be only two possible angle crossing ratios within the distribution combination design 1700. Thus, if the computed crossing ratio falls outside the predetermined error threshold of the expected crossing ratio, the algorithm may return to step (a). In step (d), when determining the set of four lines that intersect at a single point and match the expected angle crossing ratio of the distribution combination, determine the number of points that are inliers to the set of lines, as Figure 18C shown. If the number of inlier points is greater than the current best inlier count, then the current set of lines may become the best line set and the current inlier count may become the new best inlier count. Step (d) may further include assigning each inlier point to the line in the image that is closest to it.

[0223] In some embodiments, the data association algorithm may further include the step of using a known angular crossing ratio designed by a distribution combination to determine the identity of each line in the set of lines obtained during determining the best fit and the correct angular crossing ratio of four lines. The step of using a known angular crossing ratio designed by a distribution combination to determine the identity of each of these lines in the set of lines may include performing line fitting for each of the four lines in the best line set using the inlier points assigned to each line in step (d) above. Based on the line fitting, the line with the largest number of inlier points may be identified as the baseline (AOA) of the distribution combination design 1700 and the angular crossing ratio obtained using the lines sorted clockwise or counterclockwise from the distribution combination baseline is calculated. This crossing ratio will match one of two possible known distribution combination angular crossing ratios, and thus, the set of lines used to calculate this crossing ratio should be identified as the lines for the pre-calculated crossing ratio for matching, as Figure 18D shown.

[0224] In some embodiments, the data association algorithm may further include the step of using a known linear crossing ratio designed by a distribution combination for each line in the distribution combination to vote for each point in line 1900 and determine the identity of each point, as Figure 19 shown. Line 1900 may connect at least four light sources. In other words, line 1900 may include at least four points. In some embodiments, line 1900 may include five or more points, six or more points, seven or more points, eight or more points, nine or more points, ten or more points, or any suitable number of points. As Figure 19 shown, line 1900 connects six points (points 1 - 6). For each line, the points may be sorted in order of distance from the distribution combination vertex or cut point (e.g., point 1 in line 1900). For each subgroup of four points on line 1900, the linear crossing ratio may be calculated and compared with the pre-calculated known linear crossing ratio of the distribution combination line. Figure 19 Table 1910 of known crossing ratios and Table 1920 of calculated crossing ratios for subgroups of four points are illustrated. The table 1910 of known crossing ratios may include the crossing ratios for subgroups of four points on each of the lines 1702, 1704, 1706, or 1708. The table 1920 of estimated or calculated crossing ratios may include the crossing ratios for subgroups of four points on line 1900.

[0225] When comparing a known crossing ratio with a calculated crossing ratio, a vote table 1930 may be generated. A vote table is a grid of rows and columns that includes vote counts. If the calculated linear crossing ratio (e.g., the crossing ratio of line 1900) is within a predetermined threshold of any of the known crossing ratios of lines 1702, 1704, 1706, or 1708, each point used to calculate the crossing ratio receives a vote for its corresponding point in the group for the known crossing ratio.

[0226] In some embodiments, a vote intensity may be calculated for each fiducial point. As used herein, "vote intensity" is the ratio of the vote count for the candidate point with the highest number of votes to the vote count for the candidate point with the second highest number of votes. If the vote intensity for a fiducial point is greater than a predetermined threshold of the vote intensity, the identity of the candidate point that received the highest number of votes may be assigned to that fiducial point. As an example, the known crossing ratio of line B1B2B3B4 is 1.35, which is closest in value to the calculated crossing ratio of 1.36 for line 1900 connecting points 1, 3, 4, and 5 (labeled as line 1345 in table 1920). If the difference between the known crossing ratio and the calculated crossing ratio (1.35 - 1.36 = -0.1) is within a predetermined threshold difference, each point used to calculate the crossing ratio (e.g., points 1, 3, 4, and 5) may receive a vote for its corresponding point in the group for the known crossing ratio. Corresponding points refer to the position of points on a line relative to a vertex. Point B1 on line 1704 corresponds to point 1 on line 1900, point B2 on line 1704 corresponds to point 2 on line 1900, point B3 on line 1704 corresponds to point 3 on line 1900, point B4 on line 1704 corresponds to point 4 on line 1900, and point B5 on line 1704 corresponds to point 5 on line 1900.

[0227] Return reference Figure 17B , which illustrates an exemplary distribution combination design 1750 of light sources. An equal number of light sources (e.g., light sources 1751, 1753, and 1755) may be arranged on each of the virtual lines 1752, 1754, 1756, 1758, and 1760, which intersect at point 1780. Multiple light sources may be arranged at a landing surface for an aerial vehicle, and the arrangement of the light sources defines a set of intersecting virtual lines, with the light sources arranged on each virtual line, where the distance between adjacent light sources on each virtual line is non-uniform, as Figure 17B shown, which illustrates a perspective-invariant distribution combination design 1750 of active light sources consistent with an embodiment of the present disclosure. It should be understood that although only three light sources are marked on line 1752, there may be more light sources on each line (indicated as small circles, in Figure 17Bnot marked). The linear crossing ratio for each virtual line can be independent of the viewing angle. It should be understood that although each virtual line is illustrated as including an equal number of light sources, other combinations and arrangements are also possible. For example, five light sources can be arranged on one virtual line, and six light sources can be arranged on an adjacent virtual line.

[0228] Each of the lines 1752, 1754, 1756, 1758, and 1760 can be a virtual line connecting light sources. For example, virtual line 1752 can connect at least light sources 1751, 1753, and 1755. All virtual lines can intersect at the virtual intersection point 1780. In this case, the crossing ratio of the angles formed by the intersecting lines is invariant in the projection. The different lines in the distribution combination and the light sources belonging to these lines can be detected by RANSAC or other means. The intersection point can be determined by the intersection of multiple lines. The crossing ratios for (A, B, C, D) and (B, C, D, E) can be used to determine the ordering of the angular distribution of the lines. The ordering of points along a separate line can also be calculated using the linear crossing ratio, as previously referenced Figure 17A as described.

[0229] In some embodiments, each active light source (e.g., Figure 3 light source 350) can be configured to self-identify via a modulation scheme. For example, the light source can be modulated to transmit data. In some embodiments, the data can include an encoded authentication or identification signal that can be used to identify the location of the active light source, or to identify the landing surface, etc., when received by an optical detector or camera 311.

[0230] Quickly deployable optical distribution combination

[0231] In some embodiments, the landing surface can include a portable landing surface. As Figure 43A and Figure 43B shown, a portable vertical takeoff and landing airport can include a re-deployable landing pad, fabric, or tarp, or a quickly deployable landing surface 4310 that includes a plurality of light sources 4325. In some embodiments, the quickly deployable landing surface 4310 can include a distribution combination of IR light sources. In some embodiments, the quickly deployable landing surface 4310 can include a combination of point light sources and line light sources (referenced Figure 30 and described in FIG. 31). This can be particularly useful in situations where landing is required at non-cooperative locations with limited or temporary landing infrastructure, such as military operations, firefighting efforts, disaster relief operations, medical aid distribution operations, etc. In some embodiments, a portable vertical takeoff and landing airport can include battery-powered active light sources incorporated therein such that they can be remotely activated, deactivated, or modulated.

[0232] In some embodiments, for example, the rapidly deployable landing surface 4310 may include a distribution combination of light sources placed on a roll-up mat that can be carried in a backpack. The rapidly deployable landing surface 4310 may be deployed at a specially arranged landing site in an emergency situation, an adverse situation, a rescue operation, etc. In Figure 43B it, the rapidly deployable landing surface 4340 may be a flexible mesh or network structure that includes light sources 4325 woven or clamped therein.

[0233] In some embodiments, estimating the pose of an aircraft based on images captured by a camera mounted on the aircraft may include transmitting the distribution combination configuration of light sources 4325 disposed on or in the rapidly deployable landing surface 4310 or 4340 to an on-board processor associated with the aircraft. The distribution combination configuration may be determined by automatically ranging between the light sources using ultra-wideband (UWB) signals to calibrate the relative positions of the light sources. As used herein, ultra-wideband signals may be used to transmit information across a wide bandwidth (>500 MHz). This allows a large amount of signal energy to be transmitted without interfering with conventional narrowband and carrier transmissions in the same frequency band.

[0234] In some embodiments, the light sources 4325 may be woven onto the rapidly deployable landing surface 4340 (e.g., a mesh or flexible network structure) that may be deployed at a specially arranged landing site. The landing site may be uneven, contoured, or non-planar, and the rapidly deployable landing surface 4310 or 4340 may conform to the landing site. The arrangement of the light sources 4325 may be at known positions on the rapidly deployable landing surface 4340, and thus, the approximate configuration or position of the light distribution combination may be known. In some embodiments, multiple overhead images, for example, obtained by the aircraft, are used to calibrate the light positions. Alternatively or additionally, automatic ranging between the lights using UWB signals may be performed to calibrate the relative positions of the lights and thus learn the distribution combination configuration.

[0235] Automatic generation of optical distribution combination patterns

[0236] In some embodiments, the light distribution combination pattern may be automatically generated to maximize one or more properties of the distribution combination. For example, using a predefined metric, the distribution combination may be designed to maximize the ability to discriminate between the lights for the purposes of detection and data association, thereby effectively maximizing the correctness and robustness of the data association process. One possible metric is the variance of the linear or angular crossing ratios between multiple lines.

[0237] Incorporating uncertainty into optical position estimation

[0238] As previously described, the data association step involves associating the position of a light source in the image plane with the physical light position on the ground. In some cases, the association of the light source between the image and the ground may pose challenges because there may be uncertainties in the physical light position due to measurement errors, or uncertainties in the light image position (in the image plane) due to imaging errors. In some embodiments, incorporating the uncertainty information from these position measurements into the data association algorithm enables the calculation of the probability of a correct match, thereby allowing the use of confidence metrics in the data association algorithm and facilitating the decision-making process.

[0239] Pose recovery / pose estimation

[0240] In some embodiments, determining the position and orientation of an aerial vehicle may include detecting at least one light source among the light sources in the image, determining which one of the at least one light source detected is among the light sources arranged in a predetermined pattern, and determining the position and orientation of the aerial vehicle based on the determination of which one of the at least one light source detected is among the light sources arranged in a predetermined pattern. In a preferred embodiment, a positioning algorithm (e.g., PnP) may require detecting at least four light sources (and determining their pixel positions) in the image plane, and correctly associating those light source pixel positions with their corresponding physical light sources arranged in a predetermined pattern (and known positions) on the ground to generate an estimate of the pose of the camera and / or the aerial vehicle (e.g., camera 311, aerial vehicle 310).

[0241] In another preferred embodiment, the positioning algorithm may require detecting at least five light sources in the image plane, and correctly associating those light source pixel positions with their corresponding physical light sources arranged in a predetermined pattern. Generally, the accuracy of the pose estimation of the camera and / or the aerial vehicle increases as the number of light sources detected in the image plane and correctly associated with their corresponding physical light source positions in a predetermined pattern increases, as previously referenced Figure 10A and Figure 10B discussed.

[0242] In some embodiments, determining the position and orientation of an aerial vehicle may include detecting three light sources in the image plane and correctly associating the three light sources with their corresponding physical light source positions in a predetermined pattern. Standard triangulation methods can be used to identify two potential positions with three points. In the case of a landing surface and a vertical takeoff and landing airport, one of the two potential solutions may be eliminated because it would be below the landing surface, thereby potentially narrowing the solution to one position.

[0243] In some embodiments, if orientation information such as yaw, pitch, and roll or a 3D gravity vector is known, determining the position of an aerial vehicle may include detecting two light sources in an image plane and correctly correlating the two light sources with their corresponding physical light source positions in a pre-determined pattern. In some embodiments, if altitude information of the aircraft is known, determining the position and / or orientation of an aerial vehicle may include detecting one light source in an image plane and correctly correlating the one light source with the corresponding physical light source position in a pre-determined pattern.

[0244] In some embodiments, one or more light sources in a distributed combination design (e.g., design 1700 or 175) may not be observable and cannot be accurately identified due to adverse conditions such as, but not limited to, inclement weather, dense urban environments, poor light transmittance, additional ambient light, etc. In such scenarios, data association and pose estimation may still be required for an aerial vehicle approaching or taking off from a landing surface. The proposed pose estimation algorithms and systems address some of the challenges mentioned above.

[0245] Now refer to Figure 20A , which is a process flow diagram of an exemplary method 2000 for estimating the pose of an aerial vehicle consistent with embodiments of the present disclosure. Method 200 may be performed using a pose estimation algorithm in a precision landing and takeoff system and a data communication system (e.g., Figure 3 's PLaTO system 300). For example, a processor (e.g., Figure 3 's processor 312) may be configured to execute the pose estimation algorithm and may be programmed to implement the steps of the pose estimation algorithm. It should be understood that the steps performed in method 2000 may be reordered, added, removed, or edited as needed. The computer-implemented pose estimation algorithm may enable system 300 (including a camera, a light detector, a microprocessor, a memory, or a storage device mounted on the aircraft) to perform the following steps of pose estimation method 2000.

[0246] In step 2010, a camera (e.g., camera 311 of system 300) is configured to capture a continuous stream of images. The camera application programming interface (API) may be configured to receive a continuous stream of camera images at a frame rate of 100 frames per second (fps). The camera API may be further configured to transmit three consecutive frames of the camera images to the light detector. In some embodiments, the reference light of the distributed combination design may blink at a certain frequency such that each distributed combination light is on in at least one of the three frames and off in at least one frame.

[0247] In step 2020, the light detector is configured to generate an output including the pixel positions of detected light sources based on the frames received from the camera in step 2010. Generating the output includes constructing maximum and minimum images consisting of the respective maximum and minimum gray intensities at each pixel between three images, and subtracting the minimum image from the maximum image to remove any ambient background light. Due to the flickering on and off of the light, the distribution combination forms a high contrast with the background and is easily detectable in the minimum-maximum image. Subtracting the minimum image from the maximum image includes subtracting the intensity of each pixel in the minimum image from the intensity of each corresponding pixel in the maximum image. In some embodiments, the flickering may include adjusting the intensity of the light between a "minimum" intensity and a "maximum" intensity. The minimum intensity may include zero (light off) or any intensity below the maximum intensity of the light source such that the difference between the maximum and minimum values can be discerned by the detector.

[0248] In step 2030, based on the pixel positions of the detected light in the minimum-maximum image received from the light detector, a data association algorithm is used to identify which distribution combination of light corresponds to the detected light source. The data association algorithm performed in this step can be the algorithm previously referenced Figures 18A to 18D and Figure 19 described, or other suitable data association algorithms. In some embodiments, determining the best-fit set of four lines with a single intersection point and correct angle cross ratio among the point clouds detected in the camera image using the RANSAC sampling method can be performed in parallel to improve efficiency and reduce the time required for RANSAC to determine a high-confidence set of lines.

[0249] In step 2040, the pose estimator API is configured to receive the pixel positions of the identified distribution combination points in the camera image and perform an iterative Perspective-n-Point (PnP) algorithm to produce a pose estimate of the camera in the distribution combination coordinate system.

[0250] The pose estimation pipeline is configured to run end-to-end in less than 30 milliseconds (i.e., at a frequency of >33Hz), thereby allowing real-time pose measurements to be generated when acquiring camera images at 100fps. During real-time operation, the camera API can be configured to run separately from the data association and pose estimation APIs such that the next batch of images can be acquired while performing the light detection, data association, and pose estimation algorithms on the current set of images, thus making the pipeline more efficient.

[0251] Example - Simulation and Hardware Testing of a Reference Distribution Combination, Data Association, and Pose Estimation Pipeline

[0252] The above-mentioned reference distribution combination, data association algorithm, and pose estimation pipeline have been tested in simulation and on hardware. The hardware results were obtained by running the pose estimation pipeline on an Intel NUC installed on a hexacopter, which used a camera equipped with an infrared (IR) filter to image IR light sources laid out on the ground in the shape of the reference distribution combination. Several types of trajectories were flown around the distribution combination at distances up to 200 meters to test the data association algorithm and the robustness of the pose estimation pipeline to camera viewpoints. Using data from Figure 20B The results for one of the tested trajectories shown will compare the position estimates calculated by the pose pipeline with the ground truth measurements provided by a real-time kinematic (RTK) GPS sensor mounted on the hexacopter. As shown in Figure 21 , Figure 22 and Figure 23 , the accuracy of the pose pipeline is within a few percent of the distance of the camera from the distribution combination, which is typical for the other tested trajectories.

[0253] Figure 21 Illustrates the altitude estimate 2100A and the corresponding error 2100B plotted using RTK GPS ground truth data for the flight trajectory shown in Figure 20B . Figure 22 Illustrates the north estimate plot 2200A and the corresponding error plot 2200B as a function of the ground truth for the flight trajectory shown in Figure 20B . Figure 23 Illustrates the east estimate plot 2300A and the corresponding error plot 2300B as a function of the ground truth for the flight trajectory shown in Figure 20B .

[0254] Now refer to Figure 24A , which illustrates an exemplary random distribution combination pattern of active light sources on the ground consistent with some of the disclosed embodiments. In some embodiments, the distribution combination design 2400 may include randomly arranged light sources on the ground. The randomly placed light sources can be IR light sources. In the context of an automated landing (such as the landing of an eVTOL aerial vehicle), a dense urban environment or obstructed visibility due to adverse weather and low light conditions can pose significant challenges to safe landing and takeoff operations.

[0255] As Figure 24AAs shown, the random dot distribution combination design 2400 can be based on a reference marker called a random dot marker (RDM), which implements a local likelihood alignment hashing (LLAH) algorithm to identify randomly placed dots. Due to the clustering nature of the method used to compute descriptors, the LLAH algorithm is a naturally robust algorithm against occlusion. In the context of eVTOL landing and takeoff operations in a dense environment, occlusion may occur due to failed lights, covered lights, or some lights temporarily moving out of sight as the aircraft approaches the landing surface. In addition, the distribution combination placement on the landing field may be practical because the dots are randomly placed and not limited to square tags, or rectangular tags, or any specific shape.

[0256] In some embodiments, the LLAH random dot identification method may include steps of distribution combination design, key point registration, and key point retrieval. Designing the distribution combination may include creating a random dot marker by generating random x and y coordinates for N dots to fit within a selected marker size. In some embodiments, newly generated dots that overlap with existing dots may be culled or excluded from consideration. Due to the sensitivity of the cross ratio equation to dot positions, the random distribution of dots results in a naturally unique cross ratio. While other reference markers may be limited to a certain shape, the random dot marker can adopt any shape as long as the dots are arranged on a planar surface.

[0257] In some embodiments, key point registration in the LLAH algorithm may benefit from the cross ratio to ensure invariance under perspective transformation. While affine invariants can be used because they use fewer feature points to compute descriptors, using the cross ratio can offer several advantages over affine invariants. For example, in this context, a low angle approaching the landing field can result in severe perspective distortion of the points and may require not assuming an affine transformation for local point clusters. In addition, since the number of reference points (e.g., IR light sources on the ground) may be relatively small, affine invariance may be excessive or even inefficient in some cases.

[0258] In some embodiments, to uniquely identify dots using the LLAH algorithm, descriptors can be computed for each key point in the distribution combination. As used herein, a descriptor is a sequence of discretized cross ratios. Figure 24B An exemplary distribution combination including key point p and n nearest points is illustrated. Computing the descriptor may include determining the n nearest neighbors for each key point p in the distribution combination, and selecting a combination of m points from the n nearest neighbors, as Figure 24B shown. Among the selected m points, a combination of five points can be used to compute the cross ratio. As implemented herein, key point p can be one of the five points used to compute the cross ratio.

[0259] Figure 24C Illustrates calculating cross-ratios using combinations of five points out of m = 7 points to create a discretized sequence of cross-ratios. In some embodiments, the cross-ratios can be discretized, and the sequence of discretized cross-ratios from each combination can be used as a descriptor for the key point p. In this method, the boundaries for discretization can be selected by calculating all cross-ratio combinations for the distribution combinations, sorting them, and dividing them among a selected number of bins. The upper and lower bounds of the bins can be defined by the cross-ratios with the lowest and highest values in that bin. This method of discretizing these cross-ratios is because for a single distribution combination, the values of the cross-ratios are not represented by a uniform distribution. Generally, there are many cross-ratios with low values (less than 20), and very few with high values (> 100). Therefore, when discretizing the lower values, a finer range may be required.

[0260] Now refer to Figure 24D , which illustrates the discretization of cross-ratios consistent with some of the disclosed embodiments to create a discretized sequence of cross-ratios. In some embodiments, a sequence of discretized cross-ratios is used as a descriptor instead of the cross-ratio values. As Figure 24D shown, two sequences of cross-ratios (e.g., group 1 and group 2) can be generated. After discretizing the cross-ratios, group 1 and 2 have exactly the same cross-ratio values. However, the sequence in which the values appear may be unique. For example, although the discretized cross-ratios for group 1 (0, 2, 0, 3) and the discretized cross-ratios for group 2 (2, 3, 0, 0) are equal, the sequences of cross-ratios in these groups are different. To make the sequences easy to replicate during retrieval, the nearest neighbors of each key point are sorted in clockwise order before calculating any combinations or descriptors. Each descriptor can have a dimension of mC5, and each point will have nCm descriptors. As Figure 24E shown, a hash index can be calculated from each descriptor, and the key point ID and the tagger ID can be saved at the hash index along with the descriptor.

[0261] In some embodiments, the LLAH algorithm may further include key point retrieval. In order to perform marker retrieval using matching, a descriptor may be calculated for each detected point in the image using the same method as previously described. However, because the orientation of the distribution combination in the camera view may be different from the orientation during the registration step, simply sorting these points in clockwise order before calculating the sequence may not produce accurate results. The first point among the sorted nearest neighbors in the saved distribution combination may be different from the first point in the live distribution combination. Therefore, all n clockwise orders will be calculated and used for voting. For example, if the original clockwise order of the points is a, b, c, d, e, f, then the live retrieval must also calculate descriptors for b, c, d, e, f, a, etc.

[0262] The descriptor can be used to compute the hash index needed to reference the table. A vote can be cast for each keypoint ID and marker ID candidate found in the table. For each marker candidate, a homography is then computed using RANSAC to confirm the match. In this case, there is only one marker candidate, so once the votes for each keypoint and / or multiple keypoints identified exceed a predetermined threshold, we can compute the homography. At this point, we have identified our keypoints.

[0263] In some embodiments, all points (e.g., active light sources) may be non-coplanar or non-colinear. In such cases, an area cross ratio (ACR) may be used. For example, using any five arbitrary points, the ACR may be calculated based on the ratio of the areas of the triangles defined by three of the five points, as follows:

[0264]

[0265] Where Z0, Z1, Z2, Z3 and Z4 are arbitrary points forming a triangle, such as Figure 25 shown.

[0266] As shown above, this equation can be used to determine an example cross ratio value. Similar to the linear cross ratio, six unique area cross ratios can be determined for a given distribution combination of five non-colinear and non-coplanar points. A single invariant value can be calculated based on the six (6) area cross ratios.

[0267] In some embodiments, an area intersection ratio algorithm may be used for data association. The algorithm comprises the following steps:

[0268] 1. Given a set of points to be associated, select 5 unassociated or arbitrary points.

[0269] 2. Calculate the single constant value mentioned above.

[0270] 3. Find the invariant in the pre-computed table that most closely matches the computed value.

[0271] 4. Follow the associated entry to a second pre-computed table that is unique for that single invariant value.

[0272] 5. Use the cross-ratio value computed in step 2 to find a set of values that most closely match to determine the ordering of these points.

[0273] 6. Based on the ordering of the points used in steps 1 and 5, the identities of all 5 points can be determined.

[0274] 7. Remove these 5 points from consideration and repeat at step 1.

[0275] 8. If there are fewer than 5 points, previously identified points can be used to complete a set of 5 points.

[0276] In some embodiments, in a table matching technique for data association, a pre-computed table can be formed by enumerating all possible groups of 5 points in a 3D distribution combination with a known geometry. The j-invariant area cross-ratios generated when each point in each group of 5 points is selected as the center point can be computed. These values form a table where the rows correspond to the IDs of the center points and the columns correspond to a set of 4 non-center points in a group of 5 points. It should be understood that some cells in the table are empty because a single point cannot be both the center point and a non-center point in a group of 5 points. One method for completing the data association process can include searching the table and matching the j-invariant values computed from the observed 2D image points with those pre-computed values in the table. By leveraging the already matched points in the table traversal algorithm, the possible j-invariant values that are potential matches at a given time step in the table can be reduced. This enables us to effectively solve for the best match in a method similar to region growing, where we first find and identify a single group of 5 points and then incrementally increase the group we consider, adding one point at a time. Additionally, when determining the threshold for matching j-invariant values, uncertainty propagation techniques can be used to account for pixel and distribution combination calibration uncertainties.

[0277] In some embodiments, the Hungarian association matrix or the Munkres algorithm can be used to perform data association. In the Hungarian association matrix technique, an exemplary 5x5 matrix of fractions can be used, as shown in Table 1 below.

[0278]

[0279] Table 1. 5x5 matrix of fractions

[0280] A large number, such as 1.0, can be subtracted from all the numbers to convert to costs and obtain the results shown in Table 2 below.

[0281]

[0282] Table 2.

[0283] In the next step, the minimum cost can be subtracted from all rows, as shown in Table 3 below.

[0284]

[0285] Table 3.

[0286] In the next step, the minimum cost can be subtracted from all columns, as shown in Table 4 below.

[0287]

[0288] Table 4.

[0289] In the next step, as few row or column lines as possible can be drawn to connect all the 0 values in Table 4. Find the minimum value among the non-zero elements of the matrix shown in Table 5 below.

[0290]

[0291] Table 5.

[0292] In the next step, subtract the minimum value (e.g., 15 in Table 5) from all the unconnected elements, and keep the rest. The resulting table is shown below as Table 6.

[0293]

[0294] Table 6.

[0295] In the next step, identify the points in the table. If a point is missing, repeat the steps in Table 5.

[0296]

[0297] Table 7.

[0298] In a pose recovery or pose estimation algorithm, the physical pose of an aerial vehicle can be estimated based on the identified points and the points associated in the data association step described above. In some embodiments, a Perspective-n-Point (PnP) algorithm can be used to calculate the relative pose (attitude and position) of the camera with respect to the landing surface. Various implementations of the PnP algorithm are available (e.g., such as in the OpenCV library). The accuracy of PnP calculations can be improved when the distributed combination points are not coplanar. In some cases, the vertical takeoff and landing airport distributed combination may be coplanar, so various PnP methods can be explored for their robustness to coplanarity. Compared with the detection and association steps, PnP calculations are relatively fast and are less likely to limit the speed of the pose recovery pipeline (detection, association, pose recovery). PnP will produce a camera pose estimate at the frame rate of the camera. Since the pose of the camera relative to the aircraft is known, the pose of the aircraft relative to the landing location can be calculated based on the camera pose information.

[0299] The PnP technique works by minimizing the reprojection error of 2D points observed in the image with respect to 3D points reprojected into the image, while optimizing the 3D pose (position and orientation) of the camera. There are various algorithms to solve this optimization and it can be solved by non-linear least squares techniques.

[0300] In addition to geometry-based methods, it may also be necessary to use methods such as an Extended Kalman Filter (EKF) to fuse the PnP type pose solution with IMU information. This fusion method can provide several advantages, including enabling the calculation of the pose solution at an update rate higher than the update rate of the on-board camera, and providing increased redundancy and robustness to the pose recovery process, thus allowing outliers in the PnP solution to be rejected in a mathematically rigorous manner.

[0301] In some embodiments, a tightly coupled Moving Horizon Estimation (MHE) formulation can be used, where the raw camera imagery and Inertial Measurement Unit (IMU) information are processed simultaneously. The MHE formulation method is less sensitive to non-linearity and is therefore more accurate than the loosely coupled EKF method, while having the same advantages.

[0302] In some embodiments, other image-based navigation aids are used to enhance the accuracy of the system, such as using visual odometry, optical flow, or using intermediate homography information derived from the images generated by the camera. Measurements of the rate of change of the aircraft's speed and attitude can be fused with the pose estimate (i.e., using a Kalman filter or other sensor fusion techniques) to improve the overall accuracy of the system.

[0303] In some embodiments, one or more characteristics of light emitted from an active light source can be used to improve detection via background subtraction. Now refer to Figures 26A to 26C, which illustrates an exemplary waveform consistent with some embodiments of the present disclosure representing intensity modulation and camera shutter speed operations for encoding / decoding information algorithms.

[0304] For background subtraction, it may be necessary to have one frame with light turned on at 100% intensity and another frame with light completely off or at 0% intensity. The maximum difference in the intensity of light emitted from the light source allows subtracting each pixel between the two frames, thereby removing all constant light sources. One way among several ways to achieve this is to switch the light on and off at 1 / 2 the shutter speed of the camera, as Figure 26A shown. In Figure 26A , Tm indicates the modulation period, and Ts indicates the shutter period. However, such techniques may be limited by timing alignment problems. For example, when the camera sampling rate (also referred to herein as the camera capture rate) aligns with the transition of the active light source, as Figure 26B shown. In such cases, since the camera exposure is not instantaneous, the camera may capture 1 / 2 on 1 / 2 off, resulting in 50% pixel values. This not only reduces the total signal intensity but may also affect the background subtraction algorithm because each frame will be at the same 50% value, thereby potentially making the reference point undetectable.

[0305] To mitigate problems associated with the timing alignment between the camera capture rate and the transition of the active light source, the camera capture speed can be synchronized with the modulation frequency of the light source. However, there may be several problems associated with the synchronization process, such as all light having to be synchronized together so that the camera can also be synchronized, and synchronizing to a pulse can be complex because a reference point must be identified before synchronization can occur, etc. Although it can be done, it may add significant complexity to the algorithm.

[0306] In some embodiments, the timing alignment between the camera capture rate and the transition of the active light source can be overcome by flashing at a rate different from the shutter speed, as Figure 26C shown. Although this overcomes the problem of 50% pixel values for each frame, it does not completely remove partial frames and may introduce two sequential frames at the same exposure rate (in Figure 26C(shown as two consecutive dark bars). One way to mitigate the problem of identical exposures may be to increase the sampling size from 2 images to 3 images. In such cases, for background subtraction, instead of subtracting frame 2 from frame 1, we take the maximum and minimum pixel values for a given pixel location in all three images and subtract them. However, for a given set of 3 images, one image may be at 100% intensity, while another image may be at 0% intensity, but the third frame may have a partial exposure, or two consecutive frames may have the same exposure rate. By setting the flash frequency to a non-multiple of the shutter speed, we can mathematically prove that each batch of three images will have at least 1 image on and one image off.

[0307] In some embodiments, the capture rate of the camera is at least 100 frames per second. In some embodiments, the adjustment of the capture rate of the camera is based on the modulation of the light source. The adjustment of the capture rate may include synchronizing the capture rate of the camera with the modulation rate of the light source. In some embodiments, the flash rate of the light source is approximately 30 Hz. In some embodiments, the flash rate of the light source may be adjusted based on the capture rate of the camera.

[0308] In some embodiments, the controller is configured to adjust the capture rate of the camera based on the modulation of the light source. The controller may be further configured to adjust the flash rate of the light source based on the capture rate of the camera. The controller may be further configured to adjust the bit transfer rate of the camera, where the bit transfer rate of the camera is 10 Hz or higher.

[0309] In some embodiments, the controller is further configured to generate an output signal including information associated with the position and orientation of the aerial vehicle based on the output signal from the camera. The information associated with the position of the aerial vehicle may include the GPS coordinates of the aerial vehicle. The controller may be further configured to transmit the information associated with the position and orientation of the aerial vehicle to an external processor.

[0310] In some embodiments, one or more characteristics of the light emitted from the active light source may be used to transmit information associated with the light source or the landing surface associated with the light source. Conventional techniques for data transmission may have several disadvantages, including but not limited to the possible need for a common clock signal to trigger sampling, the binary values of on = 1 and off = 0 may be insufficient, and clockless transmission modes require synchronous sampling, etc.

[0311] In some embodiments, the duty cycle of the active light source may be adjusted. Example bit-by-bit transmission mode, where 1 indicates a 70% duty cycle, and 0 indicates a 30% duty cycle. But for the reference point, it alternates the duty cycle at a rate lower than the flash rate of the light source. For example, the binary value 9 can be represented as four bits 1001, as Figure 27AAs shown. Once the reference point is identified by the camera, the intensity of the light can be averaged over multiple samples to calculate the average duty cycle for the period and the bit value assigned to the period, as Figure 27B shown. The transmission and averaging can be varied to fit the requirements for the transmission rate and robustness to noise. Alternatively, the same averaging technique can be used to calculate the average duty cycle, and 1 or 0 can be represented by the rising edge or the falling edge, as Figure 27C shown.

[0312] In some embodiments, the camera capture rate and the flashing frequency of the active light source can be synchronized to reduce the errors that would occur in the case of capturing an image during a transition, as Figure 26B shown. In some embodiments, each active light source can be synchronized by a synchronization pulse sent via a connected wired network. In some embodiments, the synchronization pulse can be sent wirelessly via RF transmission. The frequency of the synchronization pulse can be appropriately selected. In some embodiments, the synchronization can be performed via an optical sensor associated with each active light source. In such cases, each active light source needs to be able to detect the light transmission from at least one other light source. Since each light source is synchronized with its adjacent light sources, all active light sources can be synchronized.

[0313] In some embodiments, if the active light sources are synchronized, the camera can be synchronized with the active light sources by a synchronization pulse sent wirelessly via RF transmission. In some embodiments, an on-board processor (e.g., processor 312) can adjust the frame rate of the camera based on the quality of the reference detection.

[0314] Associative synthesis

[0315] As previously described, more than one algorithm can be used for data association to associate each identified reference point in the two-dimensional image with the three-dimensional position of the reference point on the ground. Some of those algorithms include ICP, TPS-RPM, point tracking, linear cross ratio, angular cross ratio, and grid association. Each data association algorithm may provide several advantages when used alone, but may also pose challenges. Due to their different advantages and applicability, it may be necessary to combine two or more of these data association algorithms to produce a more reliable and robust association for use in precise landing and takeoff operations of an eVTOL in a GPS-denied environment.

[0316] Now refer to Figure 28 , which is a flowchart of an example method 2800 for data association synthesis consistent with the disclosed embodiments. Figure 28The exemplary method shown includes: using a camera mounted on an aircraft to capture an image of light sources (reference points) of a vertical takeoff and landing airport laid out on the ground; determining the pixel positions of each reference point in the image (detection step); associating each identified reference point in the two-dimensional image with the three-dimensional position of the reference point on the ground (data association step); determining the orientation and position (pose estimation step) of the aircraft based on the association and validating the pose; performing one or more sampling algorithms (e.g., RANSAC) to remove outliers; and generating the pose of the aircraft by performing a perspective-n-point transformation and applying a pose filter.

[0317] In some embodiments, the data association synthesis pipeline may include performing one or more data association algorithms to generate an association between the identified reference points in the 2D image and the 3D positions of the reference points on the ground. In a preferred embodiment, two or more data association algorithms may be executed to generate the association. The generated associations may be compiled, for example, in a data storage server or memory to form an aggregated data list or a larger set of information related to the generated associations.

[0318] In some embodiments, for each data association algorithm executed, a PnP algorithm may also be executed to accurately determine the pose (attitude and orientation) generated based on the association. Additionally or alternatively, for each data association algorithm executed, the determined pose may be verified by, for example, confirming that the position of the aircraft is above the ground, confirming that the aircraft is within a certain distance of an assigned vertical takeoff and landing airport or landing surface, and confirming that the aircraft is pointing in the correct general direction. Once the determined pose is verified, the associated information may be added to the aggregated data list.

[0319] The data association synthesis pipeline may further include identifying all unique points from the aggregated data list to form a second aggregated data list and performing a sampling method (e.g., RANSAC sampling) or a similar algorithm to remove outliers. A third aggregated data list including the association information after removing the outliers may form a final aggregated data list, which may be used to perform a full PnP transformation to generate a pose estimate of the aircraft. In some embodiments, one or more pose filters (such as but not limited to a Kalman filter, an extended Kalman filter, or other suitable pose filters) may be used to generate a pose estimate of the aircraft.

[0320] Now refer to Figure 29 , which is a flowchart of an example method 2900 for data association synthesis consistent with the disclosed embodiments. Figure 28The exemplary method shown includes: using a camera mounted on an aircraft to capture an image of light sources (reference points) of a vertical takeoff and landing airport laid out on the ground; determining the pixel positions of each reference point in the image (detection step); associating each identified reference point in the two-dimensional image with the three-dimensional positions of the reference points on the ground (data association step); determining the orientation and position (pose estimation step) of the aircraft based on this association and validating the pose; performing one or more sampling algorithms (e.g., RANSAC) to reject outliers; and generating the pose of the aircraft by performing a perspective-n-point transformation and applying a pose filter.

[0321] Compared to method 2800, in some embodiments, each data association algorithm performed can be considered an independent sensor, as Figure 29 shown. For each data association algorithm performed, a full PnP transformation and RANSAC sampling can be performed before combining all measurements using a pose filter (such as a Kalman filter).

[0322] Data association algorithms (such as grid association and TPS-RPM) are computationally intensive and iterative in nature, respectively. To overcome these and other problems, an association pipeline with dual-mode operation can be implemented. The two modes in the dual-mode operation can be a "lost in space" mode and a "tracking" mode.

[0323] The "lost in space" mode may be useful when no prior information is available, such as during the first flight of an aerial vehicle, or for newly installed light sources, connection loss, or loss of attitude information due to transmission challenges. The lost in space mode can operate at a lower rate and can run the grid association and TPS-RPM algorithms to associate the detected light with known patterns, thereby providing a set of associated points to a pose estimation (PnP) algorithm configured to generate a pose estimate. Based on the confidence level of the association provided by the "lost in space" mode, the processor can be configured to switch to the tracking mode, which operates at a higher rate than the lost in space mode.

[0324] The tracking mode may be useful when prior information is available or obtained from the lost in space mode. In the tracking mode, tracking algorithms (such as but not limited to local associated point tracking, pose-based point tracking, and ICP) can be seeded with the previously computed pose and associations. These algorithms can generate associations that can be provided individually or as a synthetic superset to the pose estimation. Such pose estimates can be output by the system at a higher rate.

[0325] To further improve the associated throughput, the lost space association algorithm may be executed in parallel after switching to the tracking mode to provide calibration. Since the association algorithm does not depend on the previous state and does not accumulate errors, the overall pose estimation can be improved. Based on the confidence metric of the system, a threshold can be determined. If the confidence metric exceeds a predetermined threshold, the system may switch to the lost space mode until better pose measurements are obtained from the sensors. In some embodiments, the switching between the lost space mode and the tracking mode can be performed automatically or autonomously by the system. However, in some embodiments, the switching can be performed manually through user intervention or user input. Therefore, it may be beneficial to provide an association pipeline with dual-mode operation that allows switching between modes based on the available information or the associated throughput.

[0326] Linear light source

[0327] As previously discussed, an active light source in a vertical takeoff and landing airport or fiducial points laid out on the ground can be a point source, e.g., an LED. Point sources of light such as LEDs may be easy to install and modulate, however, in some cases, it may be necessary to enhance the overall signal-to-noise ratio of the optical signal originating from the light source on the ground. While the amount of light emitted from the light source can be enhanced by installing more LEDs, point sources cannot emit light spread over a large area, making them insufficient for high SNR applications. Therefore, systems and methods may be needed to enhance the signal strength and SNR of the light received from the light source and by the optical detector.

[0328] See Figure 30 , which is a schematic illustration of an exemplary arrangement of line-shaped light in a distribution combination of light sources consistent with the disclosed embodiments. The landing surface 3000 may include a plurality of point sources 3030 and a plurality of line sources 3020. In some embodiments, as an alternative or addition to the point sources 3030, the landing surface 3000 may include line light sources. In some embodiments, the landing surface 3000 may be a rectangular, square, triangular, circular, or elliptical landing area, or any other suitable shape. In some embodiments, the line light sources 3020 may be arranged along all sides of the landing surface 3000 (e.g., a landing surface having a rectangular or square shape). Each side of the landing surface 3000 may include a plurality of collinear line segments, each line segment including a line light source. In some embodiments, adjacent line light sources of the plurality of line segments may be separated by point sources. In some embodiments, no light source may be provided between adjacent line light sources of the plurality of line segments such that a discontinuous row of line sources may be formed along one side of the landing surface 3000.

[0329] Using a line light source can provide several advantages, including but not limited to a higher signal-to-noise ratio due to the larger spread of light generated from a line source compared to a point source, a more robust line detection algorithm, a higher data throughput from the line light source compared to a point source, compatibility with a range of algorithms, a simple and reliable coding scheme, etc.

[0330] One of the several benefits of employing a line light source in the distribution combination of light sources on the ground includes compatibility with detection algorithms, data association algorithms, PnP transforms, and pose recovery or pose estimation algorithms as well as data coding. For example, background subtraction techniques, point detection algorithms (if using a combination of point sources and line sources), and line detection after background subtraction can be unchanged and transferred directly from point source detection algorithms.

[0331] Figure 31A is a schematic illustration of an exemplary data coding scheme 3100 using a line light source as shown in the landing surface 3000, which is consistent with the disclosed embodiments. Each of these edges can be segmented into multiple collinear line segments without affecting the detection or association algorithms. In some embodiments, each line segment can be used to represent a single data bit. As an example, the landing surface 3000 (each landing surface includes four edges with four line segments) can be configured to transmit 16 bits of data per transmission cycle. It should be understood that the number of line segments can vary as needed.

[0332] Figure 31A exemplary coding schemes such as on / off schemes are illustrated. In an on / off coding scheme, since line detection and association only apply to a single illuminated segment on each line, coding can be performed by stopping the blinking of the line segment. Additionally, at least one line segment may need to blink during the transmission of any binary representation for pose recovery. Alternatively, multiple fixed segments that are always blinking can be used for pose recovery but can be excluded from the transmission. As an example, in Figure 31A the four-segment coding shown, without affecting the integrity of pose recovery, the segment closest to the corner of each line can always be toggled so that the internal segments can be used to transmit data. The encoded binary bit shown in the exemplary scheme 3100 represents the value 13. In some embodiments, the blinking of the line segment can be programmed to represent a predefined value associated with the identification of the landing surface.

[0333] Now refer to Figure 31B, which illustrates an exemplary coding scheme 3150 for data transmission using a combination of linear-shaped light and point sources, consistent with some of the disclosed embodiments. An exemplary combined landing surface such as landing surface 3110 may include a plurality of line light sources 3120 forming the edges of a pattern (e.g., rectangle, square, triangle, etc.), and a plurality of point light sources 3125 distributed within the area defined by the line light sources 3120. In some embodiments, the point light sources 3125 may have a pre-determined pattern distribution with known positions, or a randomly generated pattern distribution with unknown positions of the point light sources.

[0334] In some embodiments, data transmission between the landing surface and an aerial vehicle configured to land on or take off from the landing surface may include providing a coding scheme. The coding scheme 3150 may include marking the intersection points of the line light sources 3120. The marking scheme may include using heading information from an INS or one or more previous iterations to mark these points in a pre-determined known order. As an example, in Figure 31B the illustrated rectangular pattern, the northeast may be marked as "0", the southeast may be marked as "1", the southwest may be marked as "2", and the northwest may be marked as "3". As another example, the intersection points may be marked based on the number of nearby point light sources. The corner with the fewest number of nearby point light sources may be marked as "0", and the corner with the most number of nearby point light sources may be marked as "3". It should be understood that although only two marking schemes are discussed, other suitable marking schemes may be applied.

[0335] Data transmission may further include projecting the light sources onto a normalized grid 3130. One way among several ways of projecting the light sources includes calculating and applying a homography matrix to remove the distortion associated with the projection. The normalized grid 3130 may be divided into a plurality of pre-determined subspaces 3140. Although the normalized grid 3130 is shown as being divided into nine subspaces 3140, the normalized grid 3130 may be divided into any number of subspaces based on the number of point light sources, the density of point light sources, the area of point light sources, or as needed. For each subspace, if a point light source is detected (i.e., activated or turned on), it may be marked as "1", and if no point light source is detected, it may be marked as "0". The labels of each subspace may be combined in a pre-determined order to generate a binary value that may be configured to represent or identify the landing space. As an example, as shown in the normalized grid 3130, the labels of the nine subspaces 3140 may be combined to form the binary value 100010011, which represents the number 275. In some embodiments, the number 275 may represent an identification associated with the landing surface, or the spatial orientation of the landing surface, or an authentication code that may be used to verify the user identity or aircraft identification, etc.

[0336] Figure 31C A flowchart of an exemplary method 3160 for pose estimation using a line light source, in accordance with some of the disclosed embodiments, is illustrated. Method 3160 may be performed in combination with or in place of the method for pose estimation using a point light source (as previously described).

[0337] Method 3160 includes, but is not limited to, the following steps: receiving at least two images from a camera mounted on an aerial vehicle; performing background subtraction on the received images; performing line detection using a line detection algorithm such as the Hough transform, a line detection filter, etc.; determining the positions where the lines intersect; using a marking scheme to mark each intersection point; performing a PnP transformation algorithm using the marked points; and selecting a valid pose from the results. The steps represented by the shaded blocks (such as background subtraction, connected components, centroid, association, and PnP) are processes that exist in the pose estimation method using a point light source. One or more steps for pose estimation using a point light source may be additionally or alternatively used for pose estimation using a line light source.

[0338] In some embodiments, the marking scheme may include using heading information from an INS or one or more previous iterations to mark the points in a pre-determined known order. As an example, in the Figure 31B rectangular pattern shown, the northeast may be marked as "0", the southeast may be marked as "1", the southwest may be marked as "2", and the northwest may be marked as "3". As another example, the intersection points may be marked based on the number of nearby point light sources. The corner with the least number of nearby point light sources may be marked as "0", and the corner with the most number of nearby point light sources may be marked as "3".

[0339] Data enhancement

[0340] In an urban environment, GPS signals may be delayed, blocked, distorted, or completely undetectable due to reflections or obstructions from densely located structures such as high-rise buildings, towers, etc. GPS signals may be reflected by buildings, walls, vehicles, and in some cases even the ground. It is well known that glass, metal, and wet surfaces are strong reflectors of light. These reflected signals can interfere with the reception of the signals directly received from the satellites and may be received via multiple paths, for example, by reflections from other surfaces and structures in the vicinity of the aircraft. This phenomenon is called multipath interference or multipath effect, as Figure 32B shown. However, in some cases, the direct signal from a GPS satellite (e.g., GPS satellite 3220) may be blocked or obstructed by, for example, a high-rise building 3210, and only the reflected signal is received. This phenomenon is called non-line-of-sight (NLOS) reception, as Figure 32A shown.

[0341] In the context of the present disclosure, multipath effects can pose several challenges and can be more troublesome than NLOS reception, because measurements from GPS signals can be not only distorted but also undetected. In some cases, a single signal may be received twice, or the signal may be significantly delayed, which can directly affect the time-of-arrival (TOA) calculations required to generate a position. These effects can be exacerbated during landing and takeoff positions when the aircraft is closer to the landing surface and at an altitude where signals can be blocked, reflected, or distorted by surrounding structures. The proposed Precise Landing and Takeoff (PLaTO) system and method can be used to weed out one or more of these effects to improve the overall accuracy of position measurements, as disclosed in some embodiments herein.

[0342] a. Using GPS to Enhance PLaTO

[0343] As previously described, the algorithms used in PLaTO can be iterative in nature, and thus, the convergence time and signal latency can be reduced by providing an initial guess. In some existing systems and methods, if an initial guess is not available, performing one or more steps of the algorithm can take several milliseconds, or even seconds, making the algorithm inefficient and negatively affecting data throughput. However, in some embodiments, GPS signals can be used to reduce the commute time when in a "lost in space" mode by seeding the algorithm with an initial guess for the current position of the aircraft. Even if the guess is inaccurate, it can speed up the algorithm by reducing the number of iterations required to achieve convergence.

[0344] Now refer to Figure 33 , which illustrates a data enhancement pipeline 3300 configured to enhance the accuracy or velocity of aircraft horizontal positioning by combining GPS and inertial navigation system (INS) measurements, consistent with the disclosed embodiments. In practice, the aircraft can be configured to use some form of GPS positioning, and information from the INS may also be available.

[0345] The data augmentation pipeline 3300 may include receiving information associated with the position of an aircraft based on GPS signals from one or more GPS satellites. In some embodiments, the position information may include the position coordinates of the aircraft. The pipeline 3300 may further include receiving information associated with the position of a landing surface or a vertical takeoff and landing airport, which information may already be present in a database. In some embodiments, the position of the landing surface may initially be determined based on GPS signals and stored in an accessible database for later use. The pipeline 3300 may further include receiving information associated with the attitude (pose, position, and orientation) of the aircraft based on INS measurements. Using information related to the position of the aircraft in 3D space, the position of the landing surface in 3D space, and the orientation of a camera configured to determine the attitude of the aircraft may allow for the determination of the region of interest 3350. In this context, a "region of interest" is an area designated as a landing space or a vertical takeoff and landing airport for an eVTOL aircraft. In some embodiments, the region 3355 (indicated by the pixelated area surrounding the region of interest 3350) may be excluded from consideration to reduce processing time. In some embodiments, as previously described, the GPS information associated with the position of the aircraft may not have to be accurate, and even an initial guess may help reduce processing time.

[0346] b. Using PLaTO to augment GPS measurements

[0347] Now refer to Figure 34 , which illustrates a pipeline 3400 for data augmentation configured to use information from a PLaTO system to augment GPS measurements, consistent with the disclosed embodiments.

[0348] As previously mentioned, multipath effects in dense urban environments create several challenges associated with receiving GPS signals and thus negatively impact the accuracy of position measurements. In some embodiments, a PLaTO system may be used to help mitigate some of these issues to improve the overall accuracy of position measurements made by GPS.

[0349] In some embodiments, the pipeline 3400 may include using position measurements from a PLaTO system to bound the position estimate provided by GPS signals. In some cases, GPS signals may be affected by multipath effects, NLOS reception, or both, and thus, GPS signals may only provide a position estimate but not an accurate measurement. For a reliable GPS signal, at least five satellites may be required. In some scenarios where there are more than five satellites, if the use of one or more satellites results in a position reference outside the bounds of the PLaTO estimate, the use of one or more satellites may be discarded. As Figure 34As shown, if the number of satellites (N) detected by the pipeline 3400 is less than the minimum number of satellites required to generate a reliable GPS signal, one or more satellites can be verified. If the number of satellites exceeds the minimum number of satellites, one or more satellites can be excluded, and the GPS signal can be calculated.

[0350] In some embodiments, if GPS satellites cannot be verified, they are considered unable to provide a position estimate or are underperforming satellites and can be excluded from consideration. In such cases, the remaining satellites can be used to calculate the position. In some embodiments, verifying existing satellites can include estimating the GPS signals from available GPS satellites, determining for each satellite whether the error between the PLaTO signal and the GPS signal is less than a predefined threshold error limit, and calculating the position based on the signals received from the satellites after the determination. If the error is greater than the predefined threshold limit, the signals from the satellite and the satellite can be excluded from consideration.

[0351] In some embodiments, based on a given position measurement from PLaTO, the positions of the individual satellites can be determined. Information associated with the positions of the individual satellites can be used to eliminate multipath effects or synthesize satellite signals. One or more of the following steps can be performed to achieve the elimination of multipath effects.

[0352] i. Provide multiple satellite measurements such as time difference of arrival (TDOA) and position measurements from the PLaTO system

[0353] ii. Calculate the expected TDOA signal for each satellite given the position reported by the PLaTO system

[0354] iii. Compare the measured TDOA signal with the expected TDOA signal and exclude outliers that fall outside a given range

[0355] iv. Use one or more expected TDOAs to calculate the position to allow GPS to calculate the position using fewer satellites than normally required.

[0356] In some embodiments, based on the position measurement from PLaTO and fewer than the minimum number of GPS satellites (e.g., two satellites), two measured position probability distributions can be combined with a Kalman filter to reduce the measurement error to below the measurement error that either system can produce individually.

[0357] c. Use PLaTO to enhance INS measurements

[0358] In the context of the present disclosure, using the last known position and extrapolating it based on INS measurements is referred to as dead reckoning navigation. This method can be useful for short distances, but for longer distances, the drift of the INS sensors will result in large errors accumulating over a long period of time. This may be due to the fact that the INS directly measures acceleration and generates velocity and position estimates by integrating the acceleration over time. Therefore, over time, small offsets in acceleration may accumulate into large position errors. In some embodiments, measurements from PLaTO can be used to enhance INS measurements, as Figure 35 shown.

[0359] In some embodiments, an absolute correction of the position of the aircraft over time can be obtained from the PLaTO system. This can be achieved by fusing the position and velocity estimates from the PLaTO system and the INS measurements to generate a high-fidelity measurement. One or more camera images from the PLaTO system can be used to perform visual odometry to enhance the INS measurements. In the context of the present disclosure, "visual odometry" is similar to the point tracking described in the data association algorithm above, but the end result is a change in position between frames rather than a change in position. One advantage among several advantages of visual odometry is that it does not require fiducial points to be present in the frame to work, which means it can be used even when the landing surface or vertical takeoff and landing airport is not in the nearby area during normal flight of the aircraft. The visual odometry algorithm may include the following steps:

[0360] 1. Provide images (a) and (b) captured at times t a and t b respectively;

[0361] 2. Identify distinctive features in images (a) and (b);

[0362] 3. Use the nearest neighbor or a suitable algorithm to correlate the features in image (a) with the features in image (b); and

[0363] 4. Estimate the change in the position of the camera between image (a) and image (b).

[0364] In some embodiments, the result of the visual odometry algorithm can be used to correct the drift that occurs during the time range between t a and t b when the pictures are captured. In some embodiments, the difference between t a and t b is one, such that at t a and t bThe images captured at the location are consecutive images. The change in position can be similarly integrated over time to generate the relative position starting from the starting point. Another advantage of visual odometry may be that it is less prone to drift effects because it is related to the visual range of the camera and the environment, rather than to acceleration measurements.

[0365] d. INS assisted by an optical positioning system

[0366] Now refer to Figure 36 which is an exemplary pipeline 3600 for a data augmentation method consistent with some of the disclosed embodiments. Pipeline 3600 represents a method of using a fixed-lag smoother algorithm to assist in INS measurements performed by an optical positioning system. In some embodiments, an optical positioning system using a fixed-lag smoother algorithm can be used in combination with INS on an aircraft to provide a more accurate position estimate than using either alone. Standard filtering implementations would utilize an Extended Kalman Filter (EKF) to fuse the optical positioning pose solution with the inertial navigation system data. However, as an alternative to the EKF, a pose graph optimization method (such as, for example, a fixed-lag smoother) can be used to allow the use of sensor data across a time window (rather than just instantaneously) to calculate the optimal position estimate. In some embodiments, using a fixed-lag smoother algorithm can improve the accuracy of position estimates at higher frequencies (e.g., 500 Hz).

[0367] e. Integrating PLaTO with an aircraft

[0368] Now refer to Figure 37 which illustrates an exemplary system 3700 consistent with some of the disclosed embodiments showing the integration of a PLaTO system with an aircraft to support manned or unmanned flight. The electric propulsion system for an eVTOL can include an electric engine that provides mechanical shaft power to a propeller assembly to generate thrust. In some embodiments, the electric engine of the electric propulsion system can include a high-voltage power system that supplies high-voltage power to the electric engine and / or a low-voltage system that supplies low-voltage DC power to the electric engine. Some embodiments can include an electric engine that digitally communicates with a flight control system ("FCS") including a flight control computer ("FCC") 3750, which can send signals to and receive signals from the electric engine, the signals including command and response data or status. Some embodiments can include an electric engine capable of receiving operating parameters from the FCC and transmitting operating parameters to the FCC, the operating parameters including speed, voltage, current, torque, temperature, vibration, propeller position, and any other values of operating parameters.

[0369] In some embodiments, a flight control system may include a system of devices capable of communicating with an electric motor to send and receive analog / digital signals to and from the electric motor and control the thrust of a tilt propeller that can be redirected between a primary vertical direction during vertical flight mode and a primary horizontal direction during forward flight mode. In some embodiments, the system may be referred to as a tilt propeller system (“TPS”) and may be capable of relaying and orienting additional features of an electric propulsion system.

[0370] In some embodiments, system 3700 may communicate measured pose (position and orientation) to FCC 3750. In some embodiments, FCC 3750 may fuse estimated poses from other sources such as GPS 3710, INS 3720, altimeter 3710, PLaTO 3740, etc. to generate an optimal estimate of the pose of the aircraft. This may be performed using a variety of sensor fusion techniques such as Kalman filters, extended Kalman filters, fixed lag smoothers, or other methods for performing sensor fusion.

[0371] In a manned aircraft, the final position estimate of the aircraft may be used to provide visual feedback to the pilot. In an unmanned aircraft, the final position estimate may be used to calculate flight control commands such as motor commands, flight surface controls 3770, or other control signals for maneuvering the aircraft in flight.

[0372] Example—EKF Utilizes Position Data from PLaTO

[0373] Figures 38 to 41 Illustrates real-time flight test results from testing with an extended Kalman filter utilizing position data from the PLaTO system. Figure 38 Illustrates a comparison of altitude estimates plotted as a function of the horizontal distance as measured by RTK-GPS ground truth data and EKF. As Figure 38 shown, the EKF line (dashed) closely adheres to the RTK line (solid) for the most part. Figure 39 Illustrates a comparison of the altitude above ground of the aircraft as measured by EKF and RTK-GPS ground truth data and the corresponding error. As Figure 39 shown, the EKF line (dashed) closely adheres to the RTK line (solid) for the most part. Figure 40 Illustrates a plot of east estimate and the corresponding error as a function of ground truth. As Figure 40 shown, the EKF line (dashed) closely adheres to the RTK line (solid) for the most part. Figure 41 Illustrates a plot of north estimate and the corresponding error as a function of ground truth. As Figure 41 shown, the EKF line (dashed) closely adheres to the RTK line (solid) for the most part.

[0374] Now refer to Figure 42 , which is a flowchart illustrating an example method 4200 for estimating the pose of an aerial vehicle consistent with some embodiments of the present disclosure. Method 4200 may be implemented using computing devices and systems such as those disclosed herein. In some embodiments, method 4200 may be performed by at least one processor of a computer-implemented system. The corresponding steps and operations of these components for method 4200 are described below. It should be understood that the components and operations may be combined, modified, and / or rearranged according to the application and system embodiments.

[0375] As Figure 42 shown, at step 4210, a landing surface including light sources arranged in a predetermined pattern may be provided. The landing surface may be a vertical takeoff and landing airport for an eVTOL aerial vehicle. Each of these light sources may be an active light source configured to emit light, and one or more characteristics of the emitted light may be modulated over time. The light sources may be arranged in a predetermined pattern, where the position of each light source is a known position. The distribution combination design of the arranged light sources may include an arrangement of light sources defining a set of intersecting virtual lines, with the light sources arranged on each virtual line, where the distance between adjacent light sources on each virtual line is non-uniform.

[0376] At step 4220, one or more characteristics of the light sources on the landing surface may be modulated over time. The characteristics of the light sources may include the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light. Modulating one or more characteristics of the emitted light may be configured to identify the landing surface, identify the light sources, identify the positions of the light sources, identify the operating state of the landing surface, or encode a signal for authenticating the landing surface. The characteristics of the light sources may be modulated by a controller on the landing surface.

[0377] At step 4230, a camera mounted on the aerial vehicle may receive an input signal associated with the light emitted from the light sources. The camera is mounted on the aerial vehicle at a known position and in a known orientation. The camera may use an optical filter or lens to permit a certain range of wavelengths.

[0378] At step 4240, based on the received input signal, the camera may generate an output in the form of a static image, a series of static images, or a stream video of information captured from the landing surface and the surrounding environment.

[0379] At step 4250, the processor may determine the position and orientation of the aerial vehicle based on information from an image captured by a camera. Determining the position and orientation of the aerial vehicle may include detecting at least one light source in the image, determining which of the light sources arranged in a predetermined pattern the detected light source is, and determining the position and orientation of the aerial vehicle based on the determination of which of the light sources arranged in a predetermined pattern the detected light source is. The processor may be configured to perform one or more algorithms to estimate the pose of the aerial vehicle based on the information received from the camera.

[0380] Figure 44 is an illustration of a perspective view of an exemplary VTOL aircraft consistent with the disclosed embodiments. Figure 45 is another illustration of a perspective view of an exemplary VTOL aircraft in an alternative configuration consistent with the embodiments of the present disclosure. Figure 44 and Figure 45 illustrate VTOL aircraft 4400, 4500 in cruise configuration and vertical takeoff, landing, and hover configuration (also referred to herein as "lift" configuration), respectively, consistent with the embodiments of the present disclosure. Elements corresponding to Figure 44 and Figure 45 may have similar reference numerals and refer to like elements of aircraft 4400, 4500. Aircraft 4400, 4500 may include fuselages 4402, 4502, wings 4404, 4504 mounted to fuselages 4402, 4502, and one or more rear stabilizers 4406, 4506 mounted to the rear of fuselages 4402, 4502. A plurality of lift propellers 4412, 4512 may be mounted to wings 4404, 4504 and may be configured to provide lift for vertical takeoff, landing, and hovering. A plurality of tilt propellers 4414, 4514 may be mounted to wings 4404, 4504 and may be tiltable between a lift configuration and a cruise configuration, in which lift configuration the plurality of tilt propellers provide a portion of the lift required for vertical takeoff, landing, and hovering, as Figure 45 shown, and in which cruise configuration the plurality of tilt propellers provide forward thrust to aircraft 4400 for horizontal flight, as Figure 44 shown. As used herein, the tilt propeller lift configuration refers to any tilt propeller orientation in which the tilt propeller thrust primarily provides lift to the aircraft, and the tilt propeller cruise configuration refers to any tilt propeller orientation in which the tilt propeller thrust primarily provides forward thrust to the aircraft.

[0381] In some embodiments, the lift propellers 4412, 4512 may be configured to provide only lift, where all horizontal thrust is provided by the tilt propellers. Accordingly, the lift propellers 4412, 4512 may be configured to have fixed positions and may generate thrust only during the takeoff, landing, and hover phases of flight. Meanwhile, the tilt propellers 4414, 4514 may be tilted upward into a lift configuration in which the thrust from the propellers 4414, 4514 is directed downward to provide additional lift.

[0382] For forward flight, the tilt propellers 4414, 4514 may be tilted from their lift configuration to their cruise configuration. In other words, the orientation of the tilt propellers 4414, 4514 may vary from an orientation in which the tilt propeller thrust is directed downward (to provide lift during vertical takeoff, landing, and hover) to an orientation in which the tilt propeller thrust is directed rearward (to provide forward thrust to the aircraft 4400, 4500). The tilt propeller assembly for a particular electric engine may tilt about a rotational axis defined by the mounting points that connect the pylon and the electric engine. When the aircraft 4400, 4500 are in full forward flight, the lift may be provided entirely by the wings 4404, 4504. Meanwhile, in the cruise configuration, the lift propellers 4412, 4512 may be shut off. The blades 4420, 4520 of the lift propellers 4412, 4512 may be held in a low drag position for aircraft cruise. In some embodiments, the lift propellers 4412, 4512 may each have two blades 4420, 4520 that may be locked for cruise in a minimum drag position in which one blade is directly in front of the other, as shown in FIG. 474. In some embodiments, the lift propellers 4412, 4512 have more than two blades. In some embodiments, the tilt propellers 4414, 4514 may include more blades 4416, 4516 than the lift propellers 4412, 4512. For example, as Figure 44 and Figure 45 shown, the lift propellers 4412, 4512 may each include, for example, two blades, while the tilt propellers 4414, 4514 may each include more blades, such as the five blades shown. In some embodiments, each of the tilt propellers 4414, 4514 may have from 2 to 5 blades, and possibly more, depending on the design considerations and requirements of the aircraft.

[0383] In some embodiments, the aircraft may include a single wing 4404, 4504 on each side of the fuselage 4402, 4502 (or a single wing extending across the entire aircraft). At least a portion of the lift propellers 4412, 4512 may be located behind the wings 4404, 4504, and at least a portion of the tilt propellers 4414, 4514 may be located in front of the wings 4404, 4504. In some embodiments, all of the lift propellers 4412, 4512 may be located behind the wings 4404, 4504, and all of the tilt propellers 4414, 4514 may be located in front of the wings 4404, 4504. According to some embodiments, all of the lift propellers 4412, 4512 and tilt propellers 4414, 4514 may be mounted to the wings - that is, no lift propellers or tilt propellers may be mounted to the fuselage. In some embodiments, the lift propellers 4412, 4512 may all be located behind the wings 4404, 4504, and the tilt propellers 4414, 4514 may all be located in front of the wings 4404, 4504. According to some embodiments, all of the lift propellers 4412, 4512 and tilt propellers 4414, 4514 may be positioned inside the ends of the wings 4404, 4504.

[0384] In some embodiments, the lift propellers 4412, 4512 and tilt propellers 4414, 4514 may be mounted to the wings 4404, 4504 via struts 4422, 4522. The struts 4422, 4522 may be mounted below the wings 4404, 4504, on top of the wings, and / or may be integrated into the wing profile. In some embodiments, the lift propellers 4412, 4512 and tilt propellers 4414, 4514 may be mounted directly to the wings 4404, 4504. In some embodiments, one lift propeller 4412, 4512 and one tilt propeller 4414, 4514 may be mounted to each strut 4422, 4522. The lift propellers 4412, 4512 may be mounted at the rear ends of the struts 4422, 4522, and the tilt propellers 4414, 4514 may be mounted at the front ends of the struts 4422, 4522. In some embodiments, the lift propellers 4412, 4512 may be mounted on the struts 4422, 4522 in a fixed position. In some embodiments, the tilt propellers 4414, 4514 may be mounted to the front ends of the struts 4422, 4522 via hinges. The tilt propellers 4414, 4514 may be mounted to the struts 4422, 4522 such that the tilt propellers 4414, 4514 are aligned with the bodies of the struts 4422, 4522 when in their cruise configuration, forming a continuous extension of the front ends of the struts 4422, 4522 that minimizes the drag for forward flight.

[0385] In some embodiments, the aircraft 4400, 4500 can include, for example, one wing on each side of the fuselage 4402, 4502 or a single wing that extends across the aircraft. According to some embodiments, at least one of the wings 4404, 4504 is a high wing mounted to the upper side of the fuselage 4402, 4502. According to some embodiments, the wings include control surfaces, such as flaps and / or ailerons. According to some embodiments, the wings 4404, 4504 may have been designed with a profile that reduces drag during forward flight. In some embodiments, the wingtip profile can be curved and / or tapered to minimize drag.

[0386] In some embodiments, the rear stabilizers 4406, 4506 include control surfaces, such as one or more rudders, one or more elevators, and / or one or more combined rudder-elevators. The wings can have any suitable design. In some embodiments, the wings have a tapered leading edge.

[0387] In some embodiments, the lift propellers 4412, 4512 or tilt propellers 4414, 4514 can be deflected relative to at least one other lift propeller 4412, 4512 or tilt propeller 4414, 4514. As used herein, deflection refers to the relative orientation of the axis of rotation of the lift propeller / tilt propeller about a line parallel to the forward-backward direction, similar to the roll degree of freedom of an aircraft. Deflection of the lift propellers and / or tilt propellers can help minimize damage caused by propeller burst by orienting the plane of rotation of the lift propeller / tilt propeller disk (the blades plus the hub to which the blades are mounted) so as not to intersect critical parts of the aircraft (such areas of the fuselage where personnel may be located, critical flight control systems, batteries, adjacent propellers, etc.) or other propeller disks, and can provide enhanced yaw control during flight.

[0388] Figure 46 is an illustration of a top plan view of an exemplary VTOL aircraft consistent with embodiments of the present disclosure. The aircraft 4600 shown in the figure can be respectively at Figure 44 and Figure 45Top plan views of the aircraft 4400, 4500 shown. As discussed herein, the aircraft 4600 may include twelve electric propulsion systems distributed across the aircraft 4600. In some embodiments, the distribution of the electric propulsion systems may include six forward electric propulsion systems 4614 and six aft electric propulsion systems 4612 mounted on struts at the front and rear of the main wing 4604 of the aircraft 4600. In some embodiments, the length from the wing 4604 to the rear end of the strut 4624 to the lift propeller 4612 may include the rear end lengths of similar struts 4624 across multiple rear ends of the strut. In some embodiments, the length of the rear end of the strut may vary across the exemplary six rear ends of the strut. For example, each rear end of the strut 4624 may include a different length from the wing 4604 to the lift propeller 4612, or a subgroup of the rear ends of the strut may be similar in length. In some embodiments, the front end of the strut 4622 may include various lengths from the wing 4604 to the tilt propeller 4614 across the front end of the strut. For example, as Figure 46 shown, the length from the tilt propeller 4614 closest to the fuselage to the front end of the strut 4622 of the wing 4604 may include a greater length than the length from the wing 4604 to the front end of the strut 4622 of the tilt propeller 4614 furthest from the fuselage. Some embodiments may include front ends of struts having similar lengths across the exemplary six front ends of the strut, or any other length distribution of the front ends of the struts from the wing 4604 to the tilt propeller 4614. Some embodiments may include an aircraft 4600 having eight electric propulsion systems, where the eight electric propulsion systems have four forward electric propulsion systems 4614 and four aft electric propulsion systems 4612, or any other distribution of forward and aft electric propulsion systems, including embodiments where the number of forward electric propulsion systems 4614 is less than or greater than the number of aft electric propulsion systems 4612. Additionally, Figure 46 depicts an exemplary embodiment of a VTOL aircraft 4600 having forward propellers 4614 in a horizontal orientation for horizontal flight and aft propeller blades 4620 in a stowed position for the forward flight phase.

[0389] As disclosed herein, the forward and aft electric propulsion systems may be of the clockwise (CW) type or the counterclockwise (CCW) type. Some embodiments may include various forward electric propulsion systems having a mixture of both CW and CCW types. In some embodiments, the aft electric propulsion systems may have a mixture of CW and CCW type systems among the aft electric propulsion systems.

[0390] Figure 47is a schematic illustration of an exemplary propeller rotation of a VTOL aircraft consistent with the disclosed embodiments. The aircraft 4700 shown in the figure may be a top plan view of the aircraft 4400, 4500, and 4600 shown in FIGS. 1, Figure 2 and Figure 3 respectively. The aircraft 4700 may include six front electric propulsion systems, where three of these front electric propulsion systems have a CW type 4724 and the remaining three front electric propulsion systems have a CCW type. In some embodiments, three rear electric propulsion systems may have a CCW type 4728 while the remaining three rear electric propulsion systems may have a CW type 4730. Some embodiments may include an aircraft 4700 having four front electric propulsion systems and four rear electric propulsion systems, where the four front electric propulsion systems and the four rear electric propulsion systems each have two CW types and two CCW types. In some embodiments, the propellers may rotate in opposite directions relative to adjacent propellers to cancel out the torque steer experienced by the fuselage or wings of the aircraft generated by the rotation of the propellers. In some embodiments, the difference in the direction of rotation may be achieved using the direction of rotation of the engines. In other embodiments, the engines may all rotate in the same direction and a gear arrangement may be used to achieve different propeller rotation directions.

[0391] Some embodiments may include an aircraft 4700 having front electric propulsion systems and rear electric propulsion systems, where the number of CW type 4724 and CCW type 4726 is not equal among the front electric propulsion systems, among the rear electric propulsion systems, or among the front electric propulsion systems and the rear electric propulsion systems.

[0392] Figure 48FIG. 0 is a schematic illustration of an exemplary power connection in a VTOL aircraft consistent with the disclosed embodiments. The VTOL aircraft may have various power systems connected to diagonally opposed electric propulsion systems. In some embodiments, the power system may include a high-voltage power system. Some embodiments may include a high-voltage power system connected to an electric engine via a high-voltage channel. In some embodiments, the aircraft 4800 may include six power systems, which include batteries 4826, 4828, 4830, 4832, 4834, and 4836 stored within the wings 4870 of the aircraft 4800. In some embodiments, the aircraft 4800 may include six front electric propulsion systems having six electric engines 4802, 4804, 4806, 4808, 4810, and 4812 and six rear electric propulsion systems having six electric engines 4814, 4816, 4818, 4820, 4822, and 4824. In some embodiments, the batteries may be connected to diagonally opposed electric engines. In such a configuration, the first power system 4826 may provide power to the electric engine 4802 via a power connection channel 4838 and to the electric engine 4824 via a power connection channel 4840. In some embodiments, the first power system 4826 may also be paired with the fourth power system 4832 via a power connection channel 4842 having a fuse to prevent excessive current from flowing through the power systems 4826 and 4832. Further for this embodiment, the VTOL aircraft 4800 may include a second power system 4828 paired with the fifth power system 4834 via a power connection channel 4848 having a fuse, and may provide power to the electric engines 4810 and 4816 via power connection channels 4844 and 4846, respectively. In some embodiments, the third power system 4830 may be paired with the sixth power system 4836 via a power connection channel 4854 having a fuse, and may provide power to the electric engines 4806 and 4820 via power connection channels 4850 and 4852, respectively. The fourth power system 4832 may also provide power to the electric engines 4808 and 4818 via power connection channels 4856 and 4858, respectively. The fifth power system 4834 may also provide power to the electric engines 4804 and 4822 via power connection channels 4860 and 4862, respectively. The sixth power system 4836 may also provide power to the electric engines 4812 and 4814 via power connection channels 4864 and 4866, respectively.

[0393] As disclosed herein, an electric propulsion system may include an electric engine connected to a high-voltage power system (such as a battery) located within an aircraft via a high-voltage channel or a power connection channel. Some embodiments may include various batteries stored within an aircraft wing, which has a high-voltage channel that travels through the aircraft (including the wing and the boom) to the electric propulsion system. In some embodiments, multiple high-voltage power systems may be used to create an electric propulsion system with multiple high-voltage power sources to avoid the risk of single-point failure. In some embodiments, the aircraft may include multiple electric propulsion systems, which may be wired in a certain pattern to various batteries or power sources stored throughout the aircraft. It should be recognized that such configurations may be beneficial in avoiding the risk of single-point failure, where the failure of one battery or power source may cause a portion of the aircraft to be unable to maintain the thrust required to continue flying or to perform a controlled landing. For example, if a VTOL has two front electric propulsion systems and two rear propulsion systems, the front electric propulsion system and the rear propulsion system on opposite sides of the VTOL aircraft may be connected to the same high-voltage power system. In such a configuration, if one high-voltage power system fails, the front electric propulsion system and the rear propulsion system on opposite sides of the VTOL aircraft will remain operational and may provide a more balanced flight or landing compared to the front electric propulsion system and the rear propulsion system that fail on the same side of the VTOL aircraft. Some embodiments may include four front electric propulsion systems and four rear electric propulsion systems, where diagonally opposite electric engines are connected to a common battery or power source. Some embodiments may include various configurations of electric engines electrically connected to a high-voltage power system such that the risk of single-point failure is avoided in the event of a power source failure, and the flight phase during which the failure occurs may continue, or the aircraft may perform an alternative flight phase in response to the failure.

[0394] As discussed above, an electric propulsion system may include an electric engine that provides mechanical shaft power to a propeller assembly to generate thrust. In some embodiments, the electric engine of the electric propulsion system may include a high-voltage power system that supplies high-voltage power to the electric engine and / or a low-voltage system that supplies low-voltage DC power to the electric engine. Some embodiments may include an electric engine that digitally communicates with a flight control system ("FCS") including a flight control computer ("FCC"), which may send signals to and receive signals from the electric engine, the signals including commands and response data or status. Some embodiments may include an electric engine capable of receiving operating parameters from the FCC and transmitting operating parameters to the FCC, the operating parameters including speed, voltage, current, torque, temperature, vibration, propeller position, and any other values of operating parameters.

[0395] In some embodiments, a flight control system may include a system that is capable of communicating with an electric engine to send and receive analog / discrete signals to and from the electric engine and control a device that is capable of redirecting the thrust of a tilt propeller between a primary vertical direction during a vertical flight mode and a primary horizontal direction during a forward flight mode. In some embodiments, the system may be referred to as a tilt propeller system (“TPS”) and may be capable of transmitting and orienting additional features of an electric propulsion system.

[0396] Figure 49 A block diagram illustrating an exemplary architecture and design of an electric propulsion unit 4900 consistent with the disclosed embodiments. In some embodiments, an electric propulsion system 4902 may include an electric engine subsystem 4904 that may supply torque via an axial propeller subsystem 4906 to generate thrust of the electric propulsion system 4902. Some embodiments may include an electric engine subsystem 4904 that receives low voltage DC (LV DC) power from a low voltage system (LVS) 4908. Some embodiments may include an electric engine subsystem 4904 that receives high voltage (HV) power from a high voltage power system (HVPS) 4910 that includes at least one battery or another device capable of storing energy. In some embodiments, the high voltage power system may include more than one battery or another device capable of storing energy that supplies high voltage power to the electric engine subsystem 4904. It should be appreciated that such a configuration may be advantageous as there is no risk of a single point of failure where a single battery failure causes the electric propulsion system 4902 to fail.

[0397] Some embodiments may include an electric propulsion system 4902 that includes an electric engine subsystem 4904 that receives signals from and sends signals to a flight control system 4912. In some embodiments, the flight control system 4912 may include a flight control computer that is capable of using Controller Area Network (“CAN”) data bus signals to send commands to and receive status and data from the electric engine subsystem 4904. It should be understood that while CAN data bus signals are used between the flight control computer and the electric engine, some embodiments may include any form of communication capable of sending and receiving data from the flight control computer to the electric engine. In some embodiments, the flight control system 4912 may also include a tilt propeller system (“TPS”) 4914 that is capable of sending analog discrete data to and receiving analog discrete data from the electric engine subsystem 4904 of the tilt propeller. The tilt propeller system 4914 may include devices that are capable of transmitting operating parameters to the electric engine subsystem 4904 and articulating the orientation of the propeller subsystem 4906 to redirect the thrust of the tilt propeller during various stages of flight using mechanical components such as gearbox assemblies, linear actuators, and any other configured components to change the orientation of the propeller subsystem 4906.

[0398] As discussed throughout, an exemplary VTOL aircraft may have various types of electric propulsion systems that include tilt propellers and lift propellers that include a front electric engine capable of tilting during various stages of flight and a rear electric engine that remains in one orientation and may be active only during certain stages of flight (i.e., takeoff, landing, and hover).

[0399] In some embodiments, the flight control system may include a system capable of controlling control surfaces and their associated actuators in an exemplary VTOL aircraft. Figure 50 is an illustration of a top plan view of an exemplary VTOL aircraft 5000 that is consistent with embodiments of the present disclosure. The aircraft 5000 shown in the figure may be the top plan view of the aircraft 4400, 4500 shown respectively in Figure 44 and Figure 45 Shown. In some embodiments, the aircraft 5000 may be similar to Figure 46Aircraft 4600. In aircraft 5000, in addition to the propeller blades discussed previously, the control surfaces may also include a flaperon 5072 and a rudder elevator 5074. The flaperon 5072 may combine the functions of one or more flaps, one or more ailerons, and / or one or more spoilers. The rudder elevator 5074 may combine the functions of one or more rudders and / or one or more elevators. In some embodiments, the control surfaces may include, for example, flaps, ailerons, spoilers, rudders, or elevators. In aircraft 5000, in addition to the electric propulsion system discussed previously, the actuators may also include control surface actuators (CSA) associated with, for example, the flaperon 5072 and the rudder elevator 5074.

[0400] Working Example - Landing Using an IR Random Dot Marker

[0401] The random dot marker has proven to be robust against occlusion and reliable. The algorithm used in this example does not rely solely on frame-by-frame point tracking to identify points at shallow viewing angles, but rather the method has been improved for nearest neighbor and descriptor calculations to ensure that the marker can be redetected even if tracking fails.

[0402] Optical detection : The light source flashes at a frequency of 1 / 3 of the camera frame rate, and the frames are processed in batches of three. The maximum and minimum gray intensities at each pixel between the three images are determined to construct maximum and minimum images. Next, the minimum image is subtracted from the maximum image to remove any ambient background IR light. In the minimum-maximum image, the distribution combination will contrast sharply with the background, and then the pixel positions of the light can be used for both keypoint registration and retrieval.

[0403] Key point registration:Before live pose estimation can be performed, the distribution combination must be registered using known light positions and IDs. To identify points using LLAH, multiple "descriptors" are calculated for each key point. To calculate the descriptor, LLAH finds the n nearest neighbors for each key point. During this step, the distribution combination is rescaled to have a 1:1 length and width ratio. Otherwise, the nearest neighbors may change under drastic view transformations. Since the cross ratio is order-dependent, the nearest neighbors are sorted according to their clockwise position relative to the key point. Then, a combination of m points is selected from the sorted nearest neighbors. Among these m points, the cross ratio is calculated using a combination of 4 points and the key point. The cross ratio for each of the 4-point combinations is calculated, discretized, and stored in the order in which they are calculated. Finally, the descriptor is stored starting with the lowest discretized cross ratio value in the sequence, while maintaining the original order. Overall, each descriptor will have a dimension of mC4, and each point will have nCm descriptors. A hash index is calculated based on each descriptor, and the key point ID is saved at this index along with the associated descriptor.

[0404] Key point retrieval : During live processing, matching or tracking can be used to retrieve the tagger. If all key points were identified in the previous frame, the algorithm defaults to basic point tracking between the previous frame and the current frame. Otherwise, the algorithm will attempt both matching and tracking and use the result from the method with more identified points. To use matching for tagger retrieval, the same method as for registration is used to calculate descriptors for each detected point in the image. Thereafter, the descriptors can be used to calculate the hash index. At this index, a vote is taken for each key point ID candidate with a matching descriptor. The key point is identified by the ID with the most votes above a specific threshold, as long as that ID has not been used to identify another point. After all key points have been processed during the matching process, the homography between the live frame and the points from the known distribution combination is then calculated to confirm the match and identify points that were not assigned an ID during the matching.

[0405] Pose estimation : If at least 20 out of 25 points are successfully identified in a frame, the iterative perspective-n-point (PNP) pose estimation of OpenCV is used to determine the pose of the aircraft relative to the distribution combination. If the change in roll between the previous frame and the current frame is no more than 2 degrees, the pose estimation is accepted. Adding this constraint ensures that impossible PNP poses are eliminated, as the aircraft should not be able to rotate 2 degrees within the time of a single frame.

[0406] Result: Obtaining live data by using a helicopter to approach a randomly placed distribution combination during the daytime. Detecting IR LEDs during the daytime may be more difficult than at night due to interference from external reflections. The algorithm is capable of identifying the light at a distance of approximately 200 m from the distribution combination. The current Python implementation of this data association algorithm takes 923 ms to retrieve frames using matching and 0.371 ms to retrieve frames using tracking. For this data set, 21% of these frames are retrieved purely through tracking, while the rest require the matching process. The mean position errors in each direction are -1.46 m, -0.41 m, and -0.39 m, with standard deviations of 0.35 m, 0.34 m, and 0.22 m, as Figure 51 shown.

[0407] Embodiments of the present disclosure may be further described with respect to the following clauses:

[0408] 1. A system, comprising:

[0409] A landing surface for an aerial vehicle, the landing surface comprising:

[0410] A plurality of light sources arranged in a predetermined pattern, wherein the characteristics of the light emitted from each of the light sources are configured to be modulated with respect to time.

[0411] 2. The system according to clause 1, wherein the light sources include a first group of light sources arranged in a first predetermined pattern, and wherein each light source in the first group of light sources is configured to be within the field of view of a camera associated with the aerial vehicle when the aerial vehicle is at a first distance from the landing surface.

[0412] 3. The system according to clause 2, wherein the light sources include a second group of light sources arranged in a second predetermined pattern, and wherein each light source in the second group of light sources is configured to be within the field of view of the camera when the aerial vehicle is at a second distance from the landing surface.

[0413] 4. The system according to clause 3, wherein each light source in the first group of light sources is configured to be outside the field of view of the camera when the aerial vehicle is at the second distance from the landing surface.

[0414] 5. The system according to clause 3 or 4, wherein the area covered by the first group of light sources is larger than the area covered by the second group of light sources.

[0415] 6. The system according to any one of clauses 3 to 5, wherein the intensity of the first group of light sources is configured to be higher than the intensity of the second group of light sources.

[0416] 7. The system according to any one of clauses 1 to 6, wherein the predetermined pattern of the light source is associated with the landing surface.

[0417] 8. The system according to any one of clauses 1 to 7, wherein the light source is arranged such that one of the light sources in the predetermined pattern is uniquely identifiable.

[0418] 9. The system according to any one of clauses 1 to 8, wherein the modulation of the characteristics of the emitted light is configured to identify the landing surface.

[0419] 10. The system according to any one of clauses 1 to 8, wherein the modulation of the characteristics of the emitted light is configured to identify one of the light sources.

[0420] 11. The system according to any one of clauses 1 to 8, wherein the modulation of the characteristics of the emitted light is configured to identify the position of one of the light sources.

[0421] 12. The system according to any one of clauses 1 to 8, wherein the modulation of the characteristics of the emitted light is configured to identify the state of the landing surface.

[0422] 13. The system according to clause 1 of any one of claims 1 to 9, wherein the modulation of the characteristics of the emitted light is configured to encode a signal for authenticating the landing surface.

[0423] 14. The system according to any one of clauses 1 to 13, wherein the modulation of the characteristics of the emitted light includes modulation of the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light.

[0424] 15. The system according to any one of clauses 1 to 14, wherein the wavelength of the emitted light from the light source is in the range of 800 nm to 850 nm.

[0425] 16. The system according to clause 15, wherein the wavelength of the emitted light is approximately 810 nm.

[0426] 17. The system according to clause 15, wherein the wavelength of the emitted light is approximately 1310 nm.

[0427] 18. The system according to clause 15, wherein the wavelength of the emitted light is approximately 1550 nm.

[0428] 19. The system according to any one of clauses 1 to 18, wherein the landing surface is a portable landing surface including a deployable landing pad, fabric, or tarp.

[0429] 20. The system according to any one of clauses 1 to 19, further comprising a controller circuit configured to operate the light source.

[0430] 21. The system according to any one of clauses 1 to 20, wherein each light source in the light source is recessed relative to the landing surface.

[0431] 22. The system according to any one of clauses 1 to 21, wherein each light source in the light source includes an optical sensor configured to detect a portion of the light emitted from at least one other light source in the light source.

[0432] 23. The system according to any one of clauses 1 to 22, further comprising a plurality of landing surfaces, wherein each landing surface includes a plurality of light sources arranged in a predetermined pattern, and wherein the characteristics of the light emitted from each light source in the light source are configured to be modulated relative to time.

[0433] 24. The system according to clause 23, wherein the landing surfaces are horizontally displaced from each other.

[0434] 25. The system according to clause 23, wherein the landing surfaces are vertically displaced from each other.

[0435] 26. The system according to clause 23, wherein the landing surfaces are horizontally and vertically displaced from each other.

[0436] 27. A system, comprising:

[0437] An aerial vehicle, comprising:

[0438] A camera configured to generate an image based on information transmitted by a plurality of light sources located near a landing surface for the aerial vehicle; and

[0439] A controller circuit configured to:

[0440] Receive the generated image; and

[0441] Determine the position and orientation of the aerial vehicle based on the received image,

[0442] wherein the light sources are arranged in a predetermined pattern on the landing surface, and wherein the characteristics of the light emitted from each light source are modulated relative to time.

[0443] 28. The system according to clause 27, wherein the camera is configured to provide a planar view of the light sources on the landing surface.

[0444] 29. The system according to clause 27, wherein the camera is configured to provide a forward view of the light source on the landing surface.

[0445] 30. The system according to any one of clauses 27 to 29, wherein the camera includes an optical filter configured to permit the wavelength range of the light emitted from each of the light sources.

[0446] 31. The system according to clause 30, wherein the permitted wavelength range is between 800 nm and 850 nm.

[0447] 32. The system according to clause 30, wherein the permitted wavelength range is approximately 810 nm.

[0448] 33. The system according to clause 30, wherein the permitted wavelength range is approximately 1310 nm.

[0449] 34. The system according to clause 30, wherein the permitted wavelength range is approximately 1550 nm.

[0450] 35. The system according to any one of clauses 30 to 34, wherein the optical filter includes a band-pass filter configured to permit the wavelength range of the light emitted from each of the light sources.

[0451] 36. The system according to any one of clauses 27 to 35, wherein the controller is further configured to adjust the capture rate of the camera based on the modulation rate of the light source.

[0452] 37. The system according to clause 36, wherein the adjustment of the capture rate includes synchronization of the capture rate of the camera with the modulation rate of the light source.

[0453] 38. The system according to clause 36 or 37, wherein the capture rate of the camera is at least 100 frames per second (Hz).

[0454] 39. The system according to any one of clauses 36 to 38, wherein the controller is further configured to adjust the blink rate of the light source based on the capture rate.

[0455] 40. The system according to clause 39, wherein the blink rate of the light source is 30 Hz.

[0456] 41. The system according to any one of clauses 36 to 40, wherein the controller is further configured to adjust the bit transmission rate of the camera.

[0457] 42. The system according to clause 41, wherein the bit transfer rate of the camera is 10 Hz or higher.

[0458] 43. The system according to any one of clauses 36 to 42, wherein the controller is further configured to transmit synchronization pulses to synchronize the capture rate of the camera with the modulation rate of the light source.

[0459] 44. The system according to any one of clauses 27 to 43, wherein the modulation of the characteristics of the light emitted from each of the light sources includes modulation of the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light.

[0460] 45. The system according to any one of clauses 27 to 44, wherein the camera is configured to be activated based on an activation signal from an external processor associated with the aerial vehicle, an operator of the aerial vehicle, or the controller.

[0461] 46. The system according to any one of clauses 27 to 45, wherein the controller is further configured to generate an output signal that includes information associated with the position and orientation of the aerial vehicle.

[0462] 47. The system according to clause 46, wherein the information associated with the position of the aerial vehicle includes the global positioning system (GPS) coordinates of the aerial vehicle.

[0463] 48. The system according to clause 46 or 47, wherein the information associated with the orientation of the aerial vehicle includes the orientation of the aerial vehicle relative to the landing surface.

[0464] 49. The system according to any one of clauses 46 to 48, wherein the controller is further configured to transmit the information associated with the position and orientation of the aerial vehicle to the external processor.

[0465] 50. The system according to any one of clauses 27 to 49, wherein the camera is a color, monochrome, or hyperspectral camera.

[0466] 51. The system according to any one of clauses 27 to 50, wherein the camera is configured to be activated after the aerial vehicle is within a predetermined distance from the landing surface.

[0467] 52. The system according to clause 51, wherein the predetermined distance is 500 m or less.

[0468] 53. A system comprising:

[0469] A plurality of light sources, which are arranged at a landing surface for an aerial vehicle, the arrangement of the light sources defining a set of intersecting virtual lines, the light sources being arranged on each virtual line, wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0470] 54. The system according to clause 53, wherein an equal number of light sources are arranged on each virtual line.

[0471] 55. The system according to clause 53 or 54, wherein the linear intersection ratio for each virtual line is independent of the viewing angle.

[0472] 56. The system according to any one of clauses 53 to 55, wherein the intersecting virtual lines define a plurality of regions, and wherein the area intersection ratio for each region is independent of the viewing angle.

[0473] 57. A method for estimating the pose of an aerial vehicle, comprising:

[0474] Providing a landing surface including light sources arranged in a predetermined pattern;

[0475] Modulating the characteristics of the light emitted from the light sources over time;

[0476] Using a camera mounted on the aerial vehicle to receive an input signal associated with the light emitted from the light sources;

[0477] Generating an image of the light sources based on the received input signal;

[0478] Determining the position and orientation of the aerial vehicle based on the image, wherein determining the position and the orientation of the aerial vehicle includes:

[0479] Detecting at least one of the light sources in the image;

[0480] Determining which one of the at least one of the light sources detected is a light source arranged in the predetermined pattern; and

[0481] Determining the position and the orientation of the aerial vehicle based on the determination of which one of the at least one of the light sources detected is a light source arranged in the predetermined pattern.

[0482] 58. The method according to clause 57, further comprising encoding information associated with at least one of the light sources in the characteristics of the modulated light emitted from the light sources.

[0483] 59. The method according to clause 57 or 58, wherein determining which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source includes using a processor to decode the encoded information associated with the light source.

[0484] 60. The method according to any one of clauses 57 to 59, wherein detecting the at least one light source among the light sources in the image further includes background subtraction and thresholding.

[0485] 61. The method according to any one of clauses 57 to 60, wherein detecting the at least one light source among the light sources in the image further includes image filtering techniques.

[0486] 62. The method according to clause 61, wherein the image filtering includes temporal filtering, spatial filtering, or a combination thereof.

[0487] 63. The method according to any one of clauses 57 to 62, which further includes storing information associated with the predetermined pattern in a database.

[0488] 64. The method according to clause 63, wherein determining which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source includes matching points in the image with points in the database.

[0489] 65. The method according to any one of clauses 57 to 64, wherein determining which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source further includes calculating a cross ratio of the positions of the corresponding light sources.

[0490] 66. The method according to any one of clauses 57 to 65, wherein the predetermined pattern includes an arrangement of the light sources defining a set of intersecting virtual lines, the light sources are arranged on each virtual line, and wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0491] 67. The method according to clause 66, wherein the linear cross ratio for each virtual line is independent of the viewing angle.

[0492] 68. The method according to clause 66 or 67, wherein the intersecting virtual lines define a plurality of regions, and wherein the area cross ratio for each region is independent of the viewing angle.

[0493] 69. The method according to any one of clauses 57 to 68, wherein modulating the characteristics of the emitted light includes modulating the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light.

[0494] 70. The method according to any one of clauses 57 to 69, wherein determining the position and the orientation of the aerial vehicle comprises:

[0495] Detecting at least four light sources in the image;

[0496] Determining which of the at least four light sources the detected light sources are that are arranged in the predetermined pattern; and

[0497] Determining the position and the orientation of the aerial vehicle based on the determination of which of the at least four light sources the detected light sources are that are arranged in the predetermined pattern.

[0498] 71. The method according to any one of clauses 57 to 69, wherein determining the position and the orientation of the aerial vehicle comprises:

[0499] Detecting at least five light sources in the image;

[0500] Determining which of the at least five light sources the detected light sources are that are arranged in the predetermined pattern; and

[0501] Determining the position and the orientation of the aerial vehicle based on the determination of which of the at least five light sources the detected light sources are that are arranged in the predetermined pattern.

[0502] 72. A computer-implemented system for estimating the pose of an aerial vehicle, the system comprising:

[0503] A landing surface including light sources arranged in a predetermined pattern; and

[0504] At least one processor configured to:

[0505] Modulate, relative to time, the characteristics of the light emitted from the light sources;

[0506] Activate a camera mounted on the aerial vehicle to receive an input signal associated with the light emitted from the light sources;

[0507] Cause the camera to generate an image of the light sources based on the received input signal;

[0508] Determine the position and orientation of the aerial vehicle based on the generated image, wherein determining the position and the orientation comprises:

[0509] Detecting at least one of the light sources in the image;

[0510] Determining which one of the at least one of the light sources the detected light source is that is arranged in the predetermined pattern; and

[0511] Determine the position and orientation of the aerial vehicle based on the determination of which one of the at least one light source among the light sources arranged in the predetermined pattern is detected.

[0512] 73. A computer-implemented method for estimating the pose of an aerial vehicle, the method comprising the following operations performed by at least one processor:

[0513] Modulate the characteristics of light emitted from a light source arranged in a predetermined pattern on a landing surface for the aerial vehicle with respect to time;

[0514] Activate a camera mounted on the aerial vehicle to enable receiving an input signal associated with the light emitted from the light source;

[0515] Enable the camera to generate an image of the light source based on the received input signal;

[0516] Determine the position and orientation of the aerial vehicle based on the image, wherein determining the position and the orientation

[0517] Comprise:

[0518] Detect at least one light source among the light sources in the image;

[0519] Determine which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source; and

[0520] Determine the position and orientation of the aerial vehicle based on the determination of which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source.

[0521] 74. The computer-implemented method according to clause 73, wherein determining the position and orientation of the aerial vehicle comprises:

[0522] Detect at least four light sources in the image;

[0523] Determine which of the at least four light sources among the light sources arranged in the predetermined pattern are the detected light sources; and

[0524] Determine the position and orientation of the aerial vehicle based on the determination of which of the at least four light sources among the light sources arranged in the predetermined pattern are the detected light sources.

[0525] 75. The computer-implemented method according to clause 73 or 74, wherein determining the position and orientation of the aerial vehicle comprises:

[0526] Detect at least five light sources in the image;

[0527] Determine which of the at least five light sources detected are the light sources arranged in the predetermined pattern; and

[0528] Determine the position and orientation of the aerial vehicle based on the determination of which of the at least five light sources detected are the light sources arranged in the predetermined pattern.

[0529] 76. A non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a device to cause the device to perform a method, the method comprising:

[0530] Modulate, relative to time, a characteristic of light emitted from light sources arranged in a predetermined pattern on a landing surface for an aerial vehicle;

[0531] Activate a camera mounted on the aerial vehicle to enable receipt of an input signal associated with the light emitted from the light sources;

[0532] Cause the camera to generate an image of the light sources based on the received input signal;

[0533] Determine the position and orientation of the aerial vehicle based on the image, wherein determining the position and the orientation

[0534] Comprises:

[0535] Detect at least one of the light sources in the image;

[0536] Determine which one of the at least one of the light sources detected is the light source arranged in the predetermined pattern; and

[0537] Determine the position and orientation of the aerial vehicle based on the determination of which one of the at least one of the light sources detected is the light source arranged in the predetermined pattern.

[0538] 77. The non-transitory computer-readable medium according to clause 76, wherein the set of instructions executable by the at least one processor of the device causes the device to determine the position and orientation of the aerial vehicle, and wherein determining the position and orientation of the aerial vehicle comprises:

[0539] Detect at least four light sources in the image;

[0540] Determine which of the at least four light sources detected are the light sources arranged in the predetermined pattern; and

[0541] Determining the position and orientation of the aerial vehicle based on a determination of which of the at least four light sources detected are the light sources arranged in the predetermined pattern.

[0542] 78. The non-transitory computer-readable medium according to clause 76, wherein the set of instructions executable by the at least one processor of the device causes the device to determine the position and orientation of the aerial vehicle, and wherein determining the position and orientation of the aerial vehicle comprises:

[0543] Detecting at least five light sources in the image;

[0544] Determining which of the at least five light sources the detected light sources are that are arranged in the predetermined pattern; and

[0545] Determining the position and orientation of the aerial vehicle based on a determination of which of the at least five light sources the detected light sources are that are arranged in the predetermined pattern.

[0546] 79. A computer-implemented method for locating a light source for a landing surface of an aerial vehicle, the method comprising the following operations performed by at least one processor:

[0547] Activating a camera mounted on the aerial vehicle to enable receipt of an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface for the aerial vehicle, the light having a characteristic that is modulated with respect to time;

[0548] Enabling the camera to generate at least two images of the light source based on the received input signal;

[0549] Enabling a detector to detect at least

[0550] one of the light sources in the at least two images using a detection algorithm, wherein the detection algorithm comprises the following steps:

[0551] Using a subtraction algorithm to determine the difference in pixel intensity values of the at least two images;

[0552] Applying a predetermined threshold pixel intensity to the difference frame to generate a mask;

[0553] Using the mask to identify the positions of the pixels in the image representing the light source; and

[0554] Based on the positions of the pixels in the image, using a centroid algorithm to calculate the position of the light source on the landing surface.

[0555] 80. The computer-implemented method according to clause 79, wherein the detection algorithm further includes generating the difference frame based on the determined difference in pixel intensity values of the at least two images.

[0556] 81. The computer-implemented method according to clause 79 or 80, wherein the property of the light is modulated to fully activate the light source in the first of the at least two images and to fully deactivate the light source in the second of the at least two images.

[0557] 82. The computer-implemented method according to clause 81, wherein the first image includes a signal image and the second image includes a background image.

[0558] 83. The computer-implemented method according to clause 81 or 82, wherein the first image and the second image include consecutive images.

[0559] 84. The computer-implemented method according to any one of clauses 81 to 83, wherein the detection algorithm further includes performing image registration to achieve background subtraction by aligning the first image and the second image.

[0560] 85. The computer-implemented method according to clause 84, wherein the image registration is performed using techniques including feature matching, translation matching, current state estimation, or a combination thereof.

[0561] 86. The computer-implemented method according to any one of clauses 79 to 85, wherein the detection algorithm further includes tracking the identified position of the pixel representing the light source by extrapolation based on a combination of information about the speed of the aerial vehicle and the time elapsed between capturing the at least two images.

[0562] 87. The computer-implemented method according to any one of clauses 79 to 86, wherein the aerial vehicle includes an electric vertical takeoff and landing aircraft.

[0563] 88. A computer-implemented method of mapping a position in an image to a position on a landing surface for an aerial vehicle, the method comprising the following operations performed by at least one processor:

[0564] Using a detection algorithm to detect a light source in the image, the light source being arranged on the landing surface and configured to emit light detectable by a camera mounted on the aerial vehicle;

[0565] Using a first association algorithm to associate the position representing the detected light source in the image with the corresponding position of the light source on the landing surface, wherein the association algorithm includes the following steps:

[0566] Normalize the position representing the detected light source in the image in the Cartesian coordinate space;

[0567] Transform the normalized position into a curve in the polar coordinate space, where the collinearly normalized positions in the Cartesian coordinate space form curves intersecting at a common point in the polar coordinate space;

[0568] Discretize the polar coordinate space into a plurality of bins, each bin represented by a value indicating the number of times the curve passes through the bin;

[0569] When determining whether the bin value exceeds a predetermined threshold, transform the position of the bin in the polar coordinate space into the Cartesian coordinate space;

[0570] Form lines in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin;

[0571] Use a clustering algorithm to group substantially parallel lines and form a rectangular frame for the integer grid space from the grouped lines;

[0572] Calculate a homography matrix configured to move the points from the Cartesian coordinate space to the integer grid; and

[0573] Use the calculated homography matrix to map each point to the integer grid.

[0574] 89. The computer-implemented method according to clause 88, wherein normalizing the position in the image includes constructing a transformation matrix to calculate the mean of the position and setting the variance of the position to one.

[0575] 90. The computer-implemented method according to clause 88 or 89, wherein normalizing the position in the image further includes rotating the Cartesian coordinate space by a certain angle to compensate for the rotation caused by the approach angle of the aerial vehicle towards the landing surface.

[0576] 91. The computer-implemented method according to any one of clauses 88 to 90, wherein the bin value is incremented by one for each instance of the curve of the position passing through the bin.

[0577] 92. The computer-implemented method according to any one of clauses 88 to 91, wherein the first association algorithm further includes optimizing one or more lines in the Cartesian coordinate space by rejecting one or more lines based on a fit to the detected positions of the light sources in the image.

[0578] 93. The computer-implemented method according to clause 92, wherein the first association algorithm further comprises iteratively optimizing the one or more lines.

[0579] 94. The computer-implemented method according to any one of clauses 88 to 93, further comprising using reference characters to label each position on the integer grid.

[0580] 95. The computer-implemented method according to clause 94, wherein the labeling is based on a predefined sequence.

[0581] 96. The computer-implemented method according to any one of clauses 88 to 95, wherein mapping each point to the integer grid indicates an offset distance, which is the distance between a reference position on the integer grid and the corresponding mapped point.

[0582] 97. The computer-implemented method according to clause 96, further comprising rejecting error detections from the association based on the offset distance.

[0583] 98. The computer-implemented method according to clause 97, wherein rejecting the error detections comprises comparing the offset distance with a threshold offset distance.

[0584] 99. The computer-implemented method according to clause 98, further comprising:

[0585] identifying the mapped points with an offset distance greater than the threshold offset distance as error detections; and

[0586] rejecting the association when it is determined that the number of the error detections exceeds an allowable threshold.

[0587] 100. The computer-implemented method according to any one of clauses 88 to 99, further comprising using a second association algorithm to associate the positions representing the detected light sources in the image with the corresponding positions of the light sources on the landing surface.

[0588] 101. The computer-implemented method according to clause 100, wherein the first association algorithm comprises a grid association algorithm, and the second association algorithm comprises an Iterative Closest Point (ICP) algorithm, a Thin Plate Spline Robust Point Matching (TPS-RPM) algorithm, a point tracking algorithm, or a combination thereof.

[0589] 102. A system, comprising:

[0590] a first plurality of light sources arranged in a predefined pattern on a landing surface for an aerial vehicle; and

[0591] A second plurality of light sources disposed along a flight path of the aerial vehicle to guide the vehicle, wherein characteristics of light emitted from each of the first plurality of light sources and the second plurality of light sources are configured to be modulated relative to time, and wherein the modulated light emitted from each of the first plurality of light sources and the second plurality of light sources is configured to be detectable by a camera mounted on the aerial vehicle.

[0592] 103. The system according to clause 102, wherein the second plurality of light sources are disposed on a roof top of a building, pole, tower, natural structure, or structure located along the flight path.

[0593] 104. A system comprising:

[0594] A landing surface for an aerial vehicle, the landing surface comprising:

[0595] A line light source of a pre - determined pattern on the landing surface for the aerial vehicle, wherein characteristics of light emitted from the plurality of line light sources are configured to be modulated relative to time, and wherein the modulation is configured to encode a signal representative of an identification of the landing surface.

[0596] 105. The system according to clause 104, further comprising a plurality of point light sources arranged in a pre - determined pattern on the landing surface for the aerial vehicle, wherein characteristics of light emitted from the plurality of point light sources are configured to be modulated relative to time.

[0597] 106. The system according to clause 104 or 105, wherein a shape of the line light source of the pre - determined pattern is rectangular.

[0598] 107. The system according to clause 106, wherein edges of the rectangular pre - determined pattern of the line light source comprise two or more collinear line segments, and wherein each line segment is configured to represent a single bit of information based on an activation state of the line light source in the line segment.

[0599] 108. The system according to any one of clauses 104 to 107, further comprising an aerial vehicle comprising a camera mounted on the aerial vehicle, the camera being configured to receive the encoded signal.

[0600] 109. The system according to clause 108, further comprising a processor communicatively associated with the camera, the processor being configured to decode the encoded signal received by the camera and generate an output based on the decoding.

[0601] 110. The system according to any one of clauses 106 to 109, wherein the shape of the line light source of the predetermined pattern is triangular, circular, elliptical, polygonal, or a combination thereof.

[0602] 111. A method for identifying a landing surface for an aerial vehicle, the method comprising:

[0603] Receiving, by a camera mounted on the aerial vehicle, an encoded signal from a plurality of line light sources arranged in a first predetermined pattern on the landing surface for the aerial vehicle, wherein the encoded signal represents an identification of the landing surface, and the encoded signal is encoded by modulating a characteristic of light emitted from the line light sources; and

[0604] Using a processor associated with the camera to decode the received encoded signal to generate an output including information associated with the identification of the landing surface,

[0605] wherein the line light sources of the first predetermined pattern include collinear line segments, each line segment being configured to represent a single bit of information based on an activation state of the line light sources in the line segment.

[0606] 112. The method according to clause 111, wherein in a first activation state, the line segment represents a bit value of one, and wherein in a second activation state, the line segment represents a bit value of zero.

[0607] 113. The method according to clause 112, wherein the first activation state is an on state of the line light sources in the line segment, and the second activation state is an off state.

[0608] 114. The method according to any one of clauses 111 to 113, further comprising receiving, by the camera mounted on the aerial vehicle, light emitted from point light sources of a second predetermined pattern on the landing surface for the aerial vehicle, wherein a characteristic of the light emitted from the point light sources is modulated with respect to time.

[0609] 115. A method for estimating the pose of an aerial vehicle, the method comprising:

[0610] Providing a distribution combination of light sources on a portable landing surface at a landing location for the aerial vehicle;

[0611] Using ultra-wideband signals between the light sources to calibrate the relative positions of the light sources to determine the configuration of the distribution combination;

[0612] Transmitting information associated with the determined configuration of the distribution combination of light sources to an aerial vehicle approaching the landing location.

[0613] 116. The method according to clause 115, further comprising estimating the pose of the aerial vehicle based on the distribution combination configuration and an image of the landing surface captured by a camera mounted on and associated with the aerial vehicle.

[0614] 117. The method according to clause 115 or 116, wherein the portable landing surface comprises a rapidly deployable landing surface, a redeployable landing surface, a roll-up mat, fabric, tarp, mesh or a mesh structure.

[0615] 118. The method according to any one of clauses 115 to 117, wherein the portable landing surface is configured to conform to the profile of the landing site.

[0616] 119. The method according to any one of clauses 115 to 118, wherein the distribution combination of the light sources comprises point light sources and line light sources.

[0617] 120. The method according to any one of clauses 115 to 119, wherein the distribution combination of the light sources comprises battery-powered light sources configured to be remotely operated.

[0618] 121. A method for estimating the pose of an aerial vehicle, the method comprising:

[0619] Providing a distribution combination of light sources on a portable landing surface in a predetermined pattern;

[0620] Setting the portable landing surface on a landing site having a profile, the portable landing surface being configured to conform to the profiled landing site;

[0621] Estimating the configuration of the distribution combination of light sources in the set portable landing surface; and

[0622] Estimating the pose of the aerial vehicle based on the estimated distribution combination configuration and an image of the landing surface captured by a camera mounted on and associated with the aerial vehicle.

[0623] 122. The method according to clause 121, wherein the portable landing surface comprises a rapidly deployable landing surface, a redeployable landing surface, a roll-up mat, fabric, tarp, mesh or a mesh structure.

[0624] 123. The method according to clause 121 or 122, wherein the distribution combination of the light sources comprises a combination of point light sources and line light sources.

[0625] 124. The method according to any one of clauses 120 to 123, wherein the distribution combination of the light sources includes a point light source and a line light source.

[0626] 125. The method according to any one of clauses 121 to 124, wherein the distribution combination of the light sources includes a battery-powered light source configured to be operated remotely.

[0627] 126. An aerial vehicle, comprising:

[0628] A camera configured to generate an image based on information received from a plurality of light sources located on a landing surface for the aerial vehicle;

[0629] A processor associated with the camera and configured to receive the image and perform the following operations:

[0630] Use a detection algorithm to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light detectable by the camera;

[0631] Associate the positions in the image representing the detected light sources with the corresponding positions of the light sources on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode;

[0632] Execute one or more association algorithms in the first operation mode and generate a confidence score for the association;

[0633] Execute one or more tracking algorithms in the second operation mode based on the confidence score obtained from the first operation mode; and

[0634] Determine one of the position or orientation of the aerial vehicle based on the performed association.

[0635] 127. The aerial vehicle according to clause 126, wherein the processor is further configured to automatically switch between the first operation mode and the second operation mode based on a pre-determined threshold confidence score.

[0636] 128. The aerial vehicle according to clause 127, wherein the processor is further configured to request user input to switch between the first operation mode and the second operation mode based on a pre-determined threshold confidence score.

[0637] 129. The aerial vehicle according to any one of clauses 126 to 128, wherein the processor is further configured to sequentially execute the first operation mode and the second operation mode.

[0638] 130. The aerial vehicle according to any one of clauses 126 to 129, wherein the processor is further configured to:

[0639] Switch from the first operating mode to the second operating mode, and

[0640] After switching from the first operating mode to the second operating mode, execute the first operating mode and the second operating mode in parallel.

[0641] 131. The aerial vehicle according to any one of clauses 126 to 130, wherein the one or more association algorithms include a grid association algorithm, a thin plate spline robust point matching (TPS-RPM) association algorithm, or an iterative closest point (ICP) algorithm.

[0642] 132. The aerial vehicle according to any one of clauses 126 to 131, wherein the one or more tracking algorithms include local associated point tracking or pose-based point tracking.

[0643] 133. The aerial vehicle according to any one of clauses 126 to 132, wherein the detection algorithm is configured to detect the modulation of the characteristics of the plurality of light sources over time.

[0644] 134. The aerial vehicle according to clause 133, wherein the plurality of light sources includes a combination of line light sources and point light sources.

[0645] 135. The aerial vehicle according to clause 134, wherein the line light source and the point light source are arranged on the landing surface in a predetermined pattern.

[0646] 136. The aerial vehicle according to clause 135, wherein the landing surface includes a portable landing surface, a rollable landing surface, a redeployable landing surface, a tarpaulin, a net, a net structure, or a combination thereof.

[0647] 137. The aerial vehicle according to any one of clauses 126 to 136, wherein performing the one or more association algorithms includes the following steps:

[0648] Normalize the positions in the image representing the detected light sources in a Cartesian coordinate space;

[0649] Transform the normalized positions into curves in a polar coordinate space, wherein the collinear normalized positions in the Cartesian coordinate space form curves intersecting at a common point in the polar coordinate space;

[0650] Discretize the polar coordinate space into a plurality of bins, each bin being represented by a value indicating the number of times the curve passes through the bin;

[0651] When determining whether a bin value exceeds a predetermined threshold, transform the position of the bin in the polar coordinate space to the Cartesian coordinate space;

[0652] Form lines in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin;

[0653] Use a clustering algorithm to group substantially parallel lines and form a rectangular frame for the integer grid space from the grouped lines;

[0654] Calculate a homography matrix configured to move the points from the Cartesian coordinate space to the integer grid; and

[0655] Use the calculated homography matrix to map each point to the integer grid.

[0656] 138. The aerial vehicle according to clause 137, wherein the processor is further configured to normalize the positions in the image by constructing a transformation matrix to calculate the mean of the positions and setting the variance of the positions to one.

[0657] 139. The aerial vehicle according to clause 137, wherein the processor is further configured to normalize the positions in the image by rotating the Cartesian coordinate space by a certain angle to compensate for the rotation caused by the approach angle of the aerial vehicle towards the landing surface.

[0658] 140. The aerial vehicle according to any one of clauses 137 to 139, wherein the processor is further configured to increment the bin value by one for each instance of a curve passing through the position of the bin.

[0659] 141. The aerial vehicle according to any one of clauses 137 to 140, wherein the processor is further configured to optimize one or more lines in the Cartesian coordinate space by eliminating one or more lines based on a fit to the detected position of a light source in the image.

[0660] 142. The aerial vehicle according to any one of clauses 137 to 141, wherein the processor is further configured to label each position on the integer grid with a reference character, and wherein the labels are based on a predefined sequence.

[0661] 143. The aerial vehicle according to any one of clauses 137 to 142, wherein the processor is further configured to map each point to an offset distance on the integer grid, the offset distance being the distance between a reference position on the integer grid and the corresponding mapped point.

[0662] 144. The aerial vehicle according to any one of clauses 137 to 143, wherein the processor is further configured to eliminate error detections from the association based on the offset distance, the elimination of the error detections including comparing the offset distance with a threshold offset distance.

[0663] 145. The aerial vehicle according to any one of clauses 137 to 144, wherein the processor is further configured to identify the mapped points with an offset distance greater than the threshold offset distance as error detections, and is configured to eliminate the association when determining that the number of error detections exceeds an allowable threshold.

[0664] 146. The aerial vehicle according to any one of clauses 126 to 145, wherein determining the position or the orientation of the aerial vehicle is further based on information from one or more of a global positioning system (GPS) or an inertial navigation system (INS).

[0665] 147. The aerial vehicle according to any one of clauses 126 to 146, wherein determining one of the position or the orientation of the aerial vehicle based on the performed association includes determining both the position and the orientation of the aerial vehicle.

[0666] 148. The aerial vehicle according to any one of clauses 126 to 147, further comprising:

[0667] A lift propeller; and

[0668] A controller configured to actuate the lift propeller based on the determined position or orientation of the aerial vehicle.

[0669] 149. The aerial vehicle according to any one of clauses 126 to 148, further comprising:

[0670] A tilt propeller; and

[0671] A controller configured to actuate the tilt propeller based on the determined position or orientation of the aerial vehicle.

[0672] 150. The aerial vehicle according to any one of clauses 126 to 149, further comprising:

[0673] A tilt actuator; and

[0674] A controller configured to actuate the tilt actuator based on the determined position or orientation of the aerial vehicle.

[0675] 151. The aerial vehicle according to any one of clauses 126 to 150, further comprising:

[0676] A control surface; and

[0677] A controller configured to actuate the control surface based on the determined position or orientation of the aerial vehicle.

[0678] 152. The aerial vehicle according to clause 151, wherein the control surface comprises one of a flaperon or an elevon.

[0679] 153. A navigation system for an aerial vehicle, comprising:

[0680] A camera configured to generate an image based on information received from a plurality of light sources arranged in a predetermined pattern on a landing surface for the aerial vehicle;

[0681] A processor associated with the camera and configured to receive the image and perform the following operations:

[0682] Activate the camera mounted on the aerial vehicle using the processor to enable reception of an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface for the aerial vehicle, the light having a characteristic that is modulated with respect to time;

[0683] Enable the camera to generate at least two images of the light source based on the received input signal;

[0684] Detect the light source in the at least two images using a detection algorithm;

[0685] Associate the positions in the image representing the detected light sources with the corresponding positions of the light sources on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode;

[0686] Execute one or more association algorithms in the first operation mode;

[0687] Execute one or more tracking algorithms in the second operation mode based on the results obtained from the first operation mode; and

[0688] Determine one of the position or orientation of the aerial vehicle based on the performed association.

[0689] 154. The navigation system according to clause 153, further comprising: the plurality of light sources are arranged in the predetermined pattern on the landing surface.

[0690] 155. The navigation system according to clause 154, wherein the characteristics of the light emitted from each of the light sources are modulated with respect to time.

[0691] 156. The navigation system according to any one of clauses 153 to 155, further comprising a controller configured to actuate the lift propeller based on the determined position or orientation of the aerial vehicle.

[0692] 157. The navigation system according to any one of clauses 153 to 156, further comprising a controller configured to actuate the tilt propeller based on the determined position or orientation of the aerial vehicle.

[0693] 158. The navigation system according to any one of clauses 153 to 157, further comprising a controller configured to actuate the tilt actuator based on the determined position or orientation of the aerial vehicle.

[0694] 159. The navigation system according to any one of clauses 153 to 158, further comprising a controller configured to actuate the control surface based on the determined position or orientation of the aerial vehicle.

[0695] 160. The navigation system according to clause 159, wherein the control surface includes one of a flaperon or a elevon.

[0696] 161. A system comprising:

[0697] A landing surface for an aerial vehicle; and

[0698] A plurality of light sources arranged in a predetermined pattern, the characteristics of the light emitted from each of the light sources being configured to be modulated with respect to time,

[0699] wherein the plurality of light sources include line light sources and point light sources, and

[0700] wherein the landing surface includes a portable landing surface.

[0701] 162. The system according to clause 161, further comprising:

[0702] A first processor configured to modulate the characteristics of the light emitted from the light sources with respect to time; and

[0703] A second processor configured to:

[0704] Activate a camera mounted on an aerial vehicle to receive an input signal associated with light emitted from a light source;

[0705] Enable the camera to generate an image of the light source based on the received input signal;

[0706] Determine one of the position or orientation of the aerial vehicle based on the generated image, wherein determining the position or the orientation includes:

[0707] Detect at least one light source among the light sources in the image;

[0708] Determine which one of the at least one light source among the light sources arranged in a predetermined pattern is the detected light source; and

[0709] Determine the position or the orientation of the aerial vehicle based on the determination of which one of the at least one light source among the light sources arranged in the predetermined pattern is the detected light source.

[0710] 163. The system according to clause 162, further comprising a controller configured to actuate a lift propeller based on the determined position or orientation of the aerial vehicle.

[0711] 164. The system according to clause 162 or 163, further comprising a controller configured to actuate a tilt propeller based on the determined position or orientation of the aerial vehicle.

[0712] 165. The system according to any one of clauses 162 to 164, further comprising a controller configured to actuate a tilt actuator based on the determined position or orientation of the aerial vehicle.

[0713] 166. The system according to any one of clauses 162 to 165, further comprising a controller configured to actuate a control surface based on the determined position or orientation of the aerial vehicle.

[0714] 167. The system according to any one of clauses 162 to 166, wherein the control surface includes one of a flaperon or an elevon.

[0715] 168. The system according to any one of clauses 161 to 167, wherein the portable landing surface is configured to conform to the contour of a landing site.

[0716] 169. The system according to any one of clauses 161 to 168, wherein the plurality of light sources includes battery-powered light sources configured to be remotely operated.

[0717] 170. The system according to any one of clauses 161 to 169, wherein the portable landing surface comprises a deployable landing surface, a roll-up mat, a fabric, a tarp, a net, or a net structure.

[0718] 171. The system according to any one of clauses 161 to 169, wherein the second processor is further configured to use an ultra-wideband signal between the light sources to calibrate the relative positions of the light sources.

[0719] 172. The system according to any one of clauses 161 to 171, wherein the plurality of light sources comprises a plurality of infrared light sources.

[0720] 173. A computer-readable medium storing instructions that, when executed by at least one processor of a device, cause the device to perform the method according to any one of clauses 57 to 69, 71 to 75, 79 to 101, or 111 to 125.

[0721] The foregoing description has been presented for purposes of illustration. It is not exhaustive and does not limit the invention to the precise forms or embodiments disclosed. Modifications and adaptations of the invention will be apparent to those skilled in the art by considering the specification and practice of the disclosed embodiments of the invention herein.

Claims

1. An aerial vehicle, comprising: a camera configured to generate an image based on information received from a plurality of light sources, the plurality of light sources being configured to emit light detectable by the camera and arranged on a landing surface for the aerial vehicle; a processor associated with the camera and configured to receive the image and perform the following operations: using a detection algorithm to detect light sources in the image; associating positions in the image representing detected light sources with corresponding positions of the light sources on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode; performing one or more association algorithms and generating a confidence score for the association in the first operation mode; performing one or more tracking algorithms in the second operation mode based on the confidence score obtained from the first operation mode; and determining one of the position or orientation of the aerial vehicle based on the performed association.

2. The aerial vehicle according to claim 1, wherein the processor is configured to automatically switch between the first operation mode and the second operation mode based on a pre-determined threshold confidence score.

3. The aerial vehicle according to claim 2, wherein the processor is configured to request user input to switch between the first operation mode and the second operation mode based on a pre-determined threshold confidence score.

4. The aerial vehicle according to any one of claims 1 to 3, wherein the processor is configured to sequentially perform the first operation mode and the second operation mode.

5. The aerial vehicle according to any one of claims 1 to 4, wherein the processor is configured to: switch from the first operation mode to the second operation mode, and after switching from the first operation mode to the second operation mode, perform the first operation mode and the second operation mode in parallel.

6. The aerial vehicle according to any one of claims 1 to 5, wherein the one or more association algorithms include a grid association algorithm, a thin plate spline robust point matching (TPS-RPM) association algorithm, or an iterative closest point (ICP) algorithm.

7. The aerial vehicle according to any one of claims 1 to 6, wherein the one or more tracking algorithms include local associated point tracking or pose-based point tracking.

8. The aerial vehicle according to any one of claims 1 to 7, wherein the detection algorithm is configured to detect modulation of characteristics of the plurality of light sources over time.

9. The aerial vehicle according to claim 8, wherein the plurality of light sources includes a combination of line light sources and point light sources.

10. The aerial vehicle according to any one of claims 1 to 8, wherein performing the one or more association algorithms includes the following steps: normalizing the positions in the image representing the detected light sources in a Cartesian coordinate space; Transform the normalized positions into curves in a polar coordinate space, where collinearly normalized positions in the Cartesian coordinate space form curves that intersect at a common point in the polar coordinate space; Discretize the polar coordinate space into a plurality of bins, each bin being represented by a value indicating the number of times the curve passes through the bin; When determining whether a bin value exceeds a predetermined threshold, transform the position of the bin in the polar coordinate space into the Cartesian coordinate space; Form lines in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin; Use a clustering algorithm to group substantially parallel lines and form a rectangular frame for an integer grid space from the grouped lines; Calculate a homography matrix configured to move the points from the Cartesian coordinate space to the integer grid; And Use the calculated homography matrix to map each point to the integer grid.

11. The aerial vehicle according to claim 10, wherein the processor is configured to normalize the positions in the image by constructing a transformation matrix to calculate the mean of the positions and setting the variance of the positions to one.

12. The aerial vehicle according to claim 10 or 11, wherein the processor is configured to normalize the positions in the image by rotating the Cartesian coordinate space by an angle to compensate for the rotation caused by the approach angle of the aerial vehicle towards the landing surface.

13. The aerial vehicle according to any one of claims 10 to 12, wherein the processor is configured to increment the bin value by one for each instance of the curve of the positions passing through the bin.

14. The aerial vehicle according to any one of claims 10 to 13, wherein the processor is configured to optimize one or more lines in the Cartesian coordinate space by removing one or more lines based on a fit to the detected positions of the light sources in the image.

15. The aerial vehicle according to any one of claims 10 to 14, wherein the processor is configured to label each position on the integer grid with a reference character, and wherein the labels are based on a predefined sequence.

16. The aerial vehicle according to any one of claims 10 to 15, wherein the processor is configured to map each point to the integer grid indicating an offset distance, the offset distance being the distance between a reference position on the integer grid and the corresponding mapped point.

17. The aerial vehicle according to any one of claims 10 to 16, wherein the processor is configured to remove false detections from the association based on the offset distance, the removal of the false detections including comparing the offset distance with a threshold offset distance.

18. The aerial vehicle according to any one of claims 10 to 17, wherein the processor is configured to identify mapped points with an offset distance greater than the threshold offset distance as false detections, and is configured to remove the association when determining that the number of false detections exceeds an allowable threshold.

19. The aerial vehicle according to any one of claims 1 to 18, wherein determining the position or the orientation of the aerial vehicle is further based on information from one or more of a global positioning system (GPS) or an inertial navigation system (INS).

20. The aerial vehicle according to any one of claims 1 to 19, further comprising: a controller configured to actuate components of the aerial vehicle based on the determined position or orientation of the aerial vehicle; wherein the components include one of a lift propeller, a tilt propeller, a tilt actuator, or a control surface.

21. A method of operating an aerial vehicle, the method comprising: generating an image using a camera based on information received from a plurality of light sources located on a landing surface for the aerial vehicle; using a detection algorithm to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light detectable by the camera; associating positions in the image representing the detected light sources with corresponding positions of the light sources on the landing surface, wherein performing the association includes a first operation mode and a second operation mode, wherein the first operation mode includes performing one or more association algorithms and generating a confidence score for the association; the second operation mode includes performing one or more tracking algorithms based on the confidence score obtained from the first operation mode; and determining one of the position or the orientation of the aerial vehicle based on the performed association.

22. A navigation system for an aerial vehicle, comprising: a camera configured to generate an image based on information received from a plurality of light sources arranged in a predetermined pattern on a landing surface for the aerial vehicle; a processor associated with the camera and configured to receive the image and perform the following operations: activating, using the processor, a camera mounted on the aerial vehicle to enable receipt of an input signal associated with light emitted from light sources arranged in a predetermined pattern on the landing surface for the aerial vehicle, the light having a characteristic that is modulated with respect to time; enabling the camera to generate at least two images of the light sources based on the received input signal; using a detection algorithm to detect the light sources in the at least two images; associating positions in the image representing the detected light sources with corresponding positions of the light sources on the landing surface, wherein the processor is configured to perform the association in a first operation mode and a second operation mode; performing one or more association algorithms in the first operation mode; performing one or more tracking algorithms in the second operation mode based on results obtained from the first operation mode; and determining one of the position or the orientation of the aerial vehicle based on the performed association.

23. The navigation system according to claim 22, further comprising: the plurality of light sources arranged in the predetermined pattern on the landing surface.

24. The navigation system according to claim 23, wherein the landing surface includes the landing surface of a vertical takeoff and landing airport.

25. The navigation system according to claim 23, wherein the landing surface includes a portable landing surface, and the portable landing surface includes one of a deployable landing surface, a roll-up mat, a fabric, a tarp, a net, or a net structure.

26. The navigation system according to any one of claims 23 to 25, wherein the characteristics of the light emitted from each of the light sources are modulated with respect to time.

27. The navigation system according to any one of claims 22 to 26, further comprising: a controller configured to actuate components of the aerial vehicle based on the determined position or orientation of the aerial vehicle, wherein the components include one of a lift propeller, a tilt propeller, a tilt actuator, or a control surface.

28. A system comprising: a portable landing surface for an aerial vehicle, the portable landing surface comprising: a plurality of light sources arranged in a predetermined pattern, and the characteristics of the light emitted from each of the light sources are configured to be modulated with respect to time, wherein the plurality of light sources include line light sources and point light sources.

29. The system according to claim 28, comprising a processor configured to use ultra-wideband signals between the light sources to calibrate the relative positions of the light sources.

30. The system according to claim 28 or 29, wherein the plurality of light sources include a plurality of infrared light sources.

Images