System and method for active light based precision positioning of an aircraft in a GPS-denied environment

Abstract

Systems and methods are disclosed that provide guidance to assist eVTOL aerial vehicles in landing and takeoff operations at a landing location in a GPS-denied environment. An exemplary system includes an aerial vehicle that includes a camera configured to generate images based on information transmitted by a plurality of light sources positioned near a landing surface for the aerial vehicle, and a controller circuit configured to receive the generated images and determine a position and orientation of the aerial vehicle based on the received images, wherein the light sources are arranged in a predetermined pattern on the landing surface, and wherein a characteristic of light emitted from each of the light sources is modulated with respect to time.

Description

[0001] Cross-references to related applications

[0002] This disclosure claims priority to U.S. Patent Application No. 18 / 451,055 (Attorney's File No. 16163.0047-00000), filed August 16, 2023, entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHT BASED PRECISION LOCALIZATION OF AIRCRAFTS IN GPS-DENIED ENVIRONMENTS," which in turn claims priority to U.S. Provisional Patent Application No. 63 / 420,616 (Attorney's File No. 16163.6001-00000), filed October 30, 2022, entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHT BASED PRECISION LOCALIZATION OF AIRCRAFTS IN GPS-DENIED ENVIRONMENTS," and U.S. Provisional Patent Application No. 63 / 420,616 (Attorney's File No. 16163.6001-00000), filed October 31, 2022, entitled "SYSTEMS AND METHODS FOR ACTIVE-LIGHT BASED PRECISION." The priority of U.S. Provisional Patent Application No. 63 / 381,571 (Attorney’s File No.: 16163.6001-01000) “LOCALIZATION OF AIRCRAFTS IN GPS-DENIED ENVIRONMENTS” is incorporated herein by reference in its entirety for all purposes.

[0003] The invention and its various embodiments described in this patent application were made, at least in part, with the support of the Department of Defense (Contract No.: FA8649-22-P-0797). To correct a clerk's error, the term "FA8649-21-P-0038" in the two previously filed provisional patent applications (Serial Nos. 63 / 420,616 and 63 / 381,571) shall be replaced, and hereby replaced, with "FA8649-22-P-0797". The U.S. federal government may reserve certain licensing rights to this invention. Technical Field

[0004] This disclosure relates generally to the field of powered air vehicles. More specifically and without limitation, this disclosure relates to electric vertical takeoff and landing (eVTOL) air vehicles and methods for providing them with highly accurate, reliable, active light-based landing and takeoff positioning guidance. Certain aspects of this disclosure generally relate to precision landing and takeoff systems that can be used in other types of aircraft but provide specific advantages in air vehicles. Summary of the Invention

[0005] The embodiments of this disclosure generally relate to the field of electrically powered vertical takeoff and landing (eVTOL) airborne vehicles. Moreover, and without limitation, this disclosure relates to systems and methods for providing guidance to assist eVTOL airborne vehicles in landing and takeoff operations at landing locations in GPS-denied environments or in areas where GPS is weakened and has limited accuracy. This disclosure further relates to methods for providing landing and takeoff guidance and estimating the attitude of the airborne vehicle relative to a landing surface. This method may include a dynamic combination of infrared or visible spectral reference light sources distributed at known fixed locations around a designated landing site. These light sources are viewed by an onboard camera as the aircraft approaches the landing site. The pattern of light sources projected onto the camera's image plane can be used to reliably calculate the camera's attitude (position and orientation) to achieve the appropriate level of accuracy required for precise eVTOL landing.

[0006] One aspect of this disclosure relates to a system comprising a landing surface for an airborne vehicle. The landing surface may include a plurality of light sources arranged in a predetermined pattern, wherein the characteristics of the light emitted from each of the light sources are configured to be modulated relative to time.

[0007] Another aspect of this disclosure relates to an aerial vehicle comprising a camera configured to generate images based on information transmitted from a plurality of light sources located near a landing surface used to control the vehicle; and controller circuitry configured to receive the generated images and determine the position and orientation of the aerial vehicle based on the received images. The light sources are arranged on the landing surface in a predetermined pattern, and the characteristics of the light emitted from each of these light sources are modulated relative to time.

[0008] Another aspect of this disclosure relates to a system comprising a plurality of light sources arranged on a landing surface for an aerial vehicle, the arrangement of which defines a set of intersecting virtual lines, wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0009] Another aspect of this disclosure relates to a method for estimating the attitude of an airborne vehicle. The method may include: providing a landing surface comprising light sources arranged in a predetermined pattern; modulating the characteristics of light emitted from the light sources relative to time; receiving an input signal associated with the light emitted from the light sources using a camera mounted on the airborne vehicle; generating an image of the light sources based on the received input signal; and determining the position and orientation of the airborne vehicle based on the image. Determining the position and orientation of the airborne vehicle includes detecting at least one light source in the image, determining which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern, and determining the position and orientation of the airborne vehicle based on the determination of which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern.

[0010] Another aspect of this disclosure relates to a computer-implemented system for estimating the attitude of an airborne vehicle. The system may include a landing surface and at least one processor, the landing surface including light sources arranged in a predetermined pattern. The processor may be configured to: modulate the characteristics of light emitted from the light sources relative to time; activate a camera mounted on the airborne vehicle to receive an input signal associated with the light emitted from the light sources; enable the camera to generate an image of the light sources based on the received input signals; and determine the position and orientation of the airborne vehicle based on the generated image. Determining the position and orientation includes: detecting at least one light source in the image; determining which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern; and determining the position and orientation of the airborne vehicle based on the determination of which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern.

[0011] Another aspect of this disclosure relates to a computer-implemented method for estimating the attitude of an airborne vehicle, the method comprising the following operations performed by at least one processor: modulating, relative to time, the characteristics of light emitted from light sources arranged in a predetermined pattern on a landing surface for the airborne vehicle; activating a camera mounted on the airborne vehicle to receive an input signal associated with the light emitted from the light sources; enabling the camera to generate an image of the light sources based on the received input signal; and determining the position and orientation of the airborne vehicle based on the image. Determining the position and orientation includes: detecting at least one light source in the image; determining which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern; and determining the position and orientation of the airborne vehicle based on the determination of which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern.

[0012] Another aspect of this disclosure relates to a non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a device to cause the device to perform a method. The method may include: modulating, relative to time, the characteristics of light emitted from light sources arranged in a predetermined pattern on a landing surface for an airborne vehicle; activating a camera mounted on the airborne vehicle to receive an input signal associated with the light emitted from the light sources; enabling the camera to generate an image of the light sources based on the received input signals; and determining the position and orientation of the airborne vehicle based on the image. Determining the position and orientation includes: detecting at least one light source in the image; determining which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern; and determining the position and orientation of the airborne vehicle based on the determination of which of the detected light sources is one of the at least one light sources arranged in a predetermined pattern.

[0013] Another aspect of this disclosure relates to an aerial vehicle. The aerial vehicle may include: a camera configured to generate an image based on information received from a plurality of light sources located on a landing surface for the aerial vehicle; and a processor associated with the camera. The processor may be configured to receive the image and to perform the following operations: using a detection algorithm to detect light sources in the image, these light sources being arranged on the landing surface and configured to emit light detectable by the camera; associating the location of the detected light source in the image with a corresponding location of the light source on the landing surface, wherein the processor is configured to perform the association in a first operating mode and a second operating mode; executing one or more association algorithms in the first operating mode and generating a confidence score for the association; executing one or more tracking algorithms in the second operating mode based on the confidence score obtained from the first operating mode; and determining either the position or orientation of the aerial vehicle based on the association performed.

[0014] Another aspect of this disclosure relates to a method for operating an airborne vehicle. The method may include: generating an image using a camera based on information received from a plurality of light sources located on a landing surface for the airborne vehicle; detecting light sources in the image using a detection algorithm, the light sources being arranged on the landing surface and configured to emit light detectable by the camera; associating the location of the detected light source in the image with a corresponding location of the light source on the landing surface, wherein the association includes a first operating mode and a second operating mode, wherein the first operating mode includes executing one or more association algorithms and generating a confidence score for the association; the second operating mode includes executing one or more tracking algorithms based on the confidence score obtained from the first operating mode; and determining either the position or orientation of the airborne vehicle based on the association performed.

[0015] Another aspect of this disclosure relates to a navigation system for an airborne vehicle. The navigation system may include: a camera configured to generate an image based on information received from a plurality of light sources arranged in a predetermined pattern on a landing surface for the airborne vehicle; a processor associated with the camera and configured to receive the image and to perform the following operations: activating the camera mounted on the airborne vehicle using the processor to receive an input signal associated with light emitted from light sources arranged in a predetermined pattern on the landing surface for the airborne vehicle, the light having characteristics modulated relative to time; enabling the camera to generate at least two images of the light source based on the received input signal; detecting the light source in the at least two images using a detection algorithm; associating the position of the detected light source in the image with a corresponding position of the light source on the landing surface, wherein the processor is configured to perform the association in a first operating mode and a second operating mode; executing one or more association algorithms in the first operating mode; executing one or more tracking algorithms in the second operating mode based on the result obtained from the first operating mode; and determining either the position or orientation of the airborne vehicle based on the association performed.

[0016] Another aspect of this disclosure relates to a system. The system may include: a landing surface for an airborne vehicle; and a plurality of light sources arranged in a predetermined pattern, the characteristics of light emitted from each of these light sources being configured to be modulated relative to time, wherein the plurality of light sources include line light sources and point light sources, and wherein the landing surface includes a portable landing surface. Attached Figure Description

[0017] Figure 1A A schematic diagram illustrating a conventional Instrument Landing System (ILS) for guiding aircraft horizontally and vertically along a runway is shown.

[0018] Figure 1B and Figure 1C Examples are given of the localizer and glide slope signal transmissions used by the pilot to assist in horizontal and vertical guidance for landing.

[0019] Figure 2 An exemplary landing / takeoff approach using optical navigation in a GPS-denied environment is illustrated, consistent with the disclosed embodiments.

[0020] Figure 3 An exemplary precision landing and takeoff system consistent with the disclosed embodiments is illustrated, as well as a data communication path between the aircraft, the vertical takeoff and landing airport, and the control unit.

[0021] Figure 4A and Figure 4BExemplary landing surfaces or vertical take-off and landing airports and exemplary light sources are illustrated, consistent with some of the disclosed embodiments.

[0022] Figure 5 An exemplary focal plane array (FPA) camera image of a light source for an exemplary vertical takeoff and landing airport, consistent with some of the disclosed embodiments, is illustrated.

[0023] Figure 6 An exemplary overview of the algorithms and data pipelines during the operation of a precision landing and takeoff system consistent with some of the disclosed embodiments is illustrated.

[0024] Figure 7 An exemplary detection algorithm for identifying the location of a light source in an image captured by a camera is illustrated, consistent with some of the disclosed embodiments.

[0025] Figure 8 A data plot illustrating a comparison of signals before and after signal processing using a bandpass filter is shown, consistent with some of the disclosed embodiments.

[0026] Figure 9 A data plot comparing the signal before and after a Discrete Fourier Transform (DFT) calculation, consistent with some of the disclosed embodiments, is illustrated.

[0027] Figure 10A and Figure 10B Examples are illustrated with simulated data plots, consistent with some disclosed embodiments, demonstrating the effect of the number of light sources on positioning accuracy in the horizontal and vertical directions, respectively.

[0028] Figure 11A and Figure 11B Examples of simulated data plots, consistent with some disclosed embodiments, illustrate the effect of the size of the distribution combination of light sources on positioning accuracy in the horizontal and vertical directions.

[0029] Figure 12A and Figure 12B Examples of simulated data plots, consistent with some disclosed embodiments, illustrate the effect of centroid error on the robustness of the algorithm in the horizontal and vertical directions, respectively.

[0030] Figure 12C and Figure 12D Examples are shown, consistent with some disclosed embodiments, of simulated data plots illustrating horizontal and vertical positioning errors of a landing trajectory during approach for an airborne vehicle.

[0031] Figure 13A , Figure 13B , Figure 13C and Figure 13DAn exemplary matching and association process using a thin-plate spline algorithm, consistent with some of the disclosed embodiments, is illustrated.

[0032] Figure 14A An exemplary camera image with detected points is shown, consistent with some of the disclosed embodiments.

[0033] Figure 14B A schematic diagram illustrating detected points normalized within a predefined space, consistent with some disclosed embodiments, is shown.

[0034] Figure 14C Examples consistent with some disclosed embodiments include crossing... Figure 4B An exemplary Hough transformation space for mappings of lines to a given point in the space.

[0035] Figure 14D Examples consistent with some disclosed embodiments are illustrated. Figure 14C A schematic diagram of the lines formed by the discretization of points in Hough space.

[0036] Figure 14E An exemplary Hough transform space with line optimization is illustrated, consistent with some of the disclosed embodiments.

[0037] Figure 14F Examples consistent with some of the disclosed embodiments are illustrated. Figure 14E An exemplary k-means clustering representation of the optimized line in the diagram.

[0038] Figure 14G An exemplary representation of a mapped point in a two-dimensional space, consistent with some of the disclosed embodiments, is illustrated.

[0039] Figure 14H Examples consistent with some of the disclosed embodiments are illustrated. Figure 14G An example projection of a point onto an integer grid space.

[0040] Figure 14J Examples are illustrated based on sources consistent with some disclosed embodiments. Figure 14H An example of detection with associated annotations for information about the light source.

[0041] Figure 15A An exemplary image illustrating the shift between a frame point and the corresponding newly predicted position using homography, consistent with some of the disclosed embodiments, is shown.

[0042] Figure 15B An exemplary image is illustrated, consistent with some of the disclosed embodiments, generated by using nearest neighbor-based search to identify corresponding detections of known associations being tracked.

[0043] Figure 16AAn exemplary combination of linear patterns of light source distribution consistent with some of the disclosed embodiments is illustrated.

[0044] Figure 16B An exemplary combination of star-shaped patterns of light source distribution consistent with some of the disclosed embodiments is illustrated.

[0045] Figure 17A and Figure 17B An exemplary distribution pattern of light sources consistent with some of the disclosed embodiments is illustrated.

[0046] Figure 18A and Figure 18B An exemplary set of lines using the Random Sample Consistency (RANSAC) sampling method, consistent with some of the disclosed embodiments, is illustrated.

[0047] Figure 18C A visualization illustrating a process for calculating interior points after finding a suitable set of candidate lines, consistent with some disclosed embodiments, is provided.

[0048] Figure 18D An example is illustrated using an angular cross ratio to determine the identification of lines in a distribution combination, consistent with some of the disclosed embodiments.

[0049] Figure 19 An exemplary voting scheme for determining point identification using a linear cross ratio, consistent with some of the disclosed embodiments, is illustrated.

[0050] Figure 20A This is a flowchart illustrating an exemplary method for pose estimation using a data association algorithm, consistent with some of the disclosed embodiments.

[0051] Figure 20B An exemplary drawing illustrating a simulated circular trajectory of flight around a distribution combination is shown, consistent with some of the disclosed embodiments.

[0052] Figure 21 Examples 2100A and 2100B are illustrated, consistent with some of the disclosed embodiments, respectively indicating the altitude estimation and corresponding error of the flight trajectory shown in FIG20.

[0053] Figure 22 Examples 2200A and 2200B are illustrated, consistent with some of the disclosed embodiments, respectively indicating the north estimate and the baseline true value and the corresponding error for the flight trajectory shown in Figure 20.

[0054] Figure 23 Examples 2300A and 2300B are illustrated, consistent with some of the disclosed embodiments, respectively indicating the eastward estimation and baseline true value of the flight trajectory shown in Figure 20, as well as the corresponding error data plots.

[0055] Figure 24A A schematic diagram of an exemplary random dot marker consistent with some of the disclosed embodiments is shown.

[0056] Figure 24B and Figure 24C An exemplary point identification method using the Local Likelihood Permutation Hash (LLAH) algorithm is illustrated, consistent with some of the disclosed embodiments.

[0057] Figure 24D An exemplary discretization method for creating a discretized cross ratio sequence is illustrated, consistent with some of the disclosed embodiments.

[0058] Figure 24E An exemplary hash table for keypoint registration, consistent with some of the disclosed embodiments, is illustrated.

[0059] Figure 25 An exemplary area cross ratio calculation for a subgroup of coplanar light is illustrated, consistent with some of the disclosed embodiments.

[0060] Figure 26A , Figure 26B and Figure 26C Exemplary waveforms representing intensity modulation and camera shutter speed operations for encoding / decoding information algorithms, consistent with some of the disclosed embodiments, are illustrated.

[0061] Figure 27A , Figure 27B and Figure 27C Exemplary modulation schemes for data transmission using a light source, consistent with some of the disclosed embodiments, are illustrated.

[0062] Figure 28 This is a flowchart illustrating an example method for data association and synthesis consistent with some of the disclosed embodiments.

[0063] Figure 29 This is a flowchart illustrating an example method for data association and synthesis consistent with some of the disclosed embodiments.

[0064] Figure 30 It is a schematic illustration of an exemplary arrangement of linear light in a combination of light source distributions consistent with some of the disclosed embodiments.

[0065] Figure 31A This is a schematic illustration of an exemplary data encoding scheme consistent with some of the disclosed embodiments.

[0066] Figure 31B This is a schematic illustration of an exemplary encoding scheme for data transmission using a combination of linearly shaped light and point sources, consistent with some of the disclosed embodiments.

[0067] Figure 31C A flowchart illustrating an exemplary method for pose estimation using a line light source, consistent with some of the disclosed embodiments, is shown.

[0068] Figure 32A and Figure 32B This is a schematic illustration of an exemplary GPS multipath error consistent with some of the disclosed embodiments.

[0069] Figure 33 This is a schematic illustration of an exemplary pipeline for data augmentation configured to use GPS to enhance a Precision Landing and Takeoff (PLaTO) system, consistent with some of the disclosed embodiments.

[0070] Figure 34 This is a schematic illustration of an exemplary pipeline for data augmentation configured to use the PLATO system to enhance GPS, consistent with some of the disclosed embodiments.

[0071] Figure 35 This is a schematic illustration of an exemplary pipeline for data enhancement configured to use the PLATO system to enhance INS, consistent with some of the disclosed embodiments.

[0072] Figure 36 This is a schematic illustration of an exemplary pipeline for data enhancement, consistent with some disclosed embodiments, configured to use an optical positioning system utilizing an extended Kalman filter (EKF) to enhance INS.

[0073] Figure 37 Exemplary systems, consistent with some of the disclosed embodiments, are illustrated to demonstrate the integration of the PLATO system with an aircraft to support manned or unmanned flight.

[0074] Figure 38 An example is illustrated, consistent with some disclosed embodiments, of data plotting the aircraft's altitude as a function of horizontal distance when the system is used to enhance INS.

[0075] Figure 39 An example is illustrated, consistent with some disclosed embodiments, of data plotting above ground altitude versus time and corresponding error when the system is used to enhance INS.

[0076] Figure 40 An example is illustrated, consistent with some of the disclosed embodiments, of how the system is used to enhance INS estimation and time-related data plotting, as well as corresponding errors.

[0077] Figure 41 An example is illustrated, consistent with some of the disclosed embodiments, of how the system is used to enhance INS when estimating time and plotting data for corresponding errors.

[0078] Figure 42 This is a flowchart illustrating an exemplary method for determining the position and orientation of an airborne vehicle, consistent with some of the disclosed embodiments.

[0079] Figure 43A and Figure 43B Exemplary, rapidly deployable combinations of light sources consistent with some of the disclosed embodiments are illustrated.

[0080] Figure 44 An exemplary VTOL aircraft consistent with the disclosed embodiments is shown.

[0081] Figure 45 An exemplary VTOL aircraft consistent with the disclosed embodiments is shown.

[0082] Figure 46 An exemplary top plan view of a VTOL aircraft consistent with the disclosed embodiments is shown.

[0083] Figure 47 An exemplary propeller rotation of a VTOL aircraft consistent with the disclosed embodiments is shown.

[0084] Figure 48 An exemplary power connection in a VTOL aircraft consistent with the disclosed embodiments is shown.

[0085] Figure 49 An exemplary architecture of an electric propulsion unit consistent with the disclosed embodiments is shown.

[0086] Figure 50 An exemplary top plan view of a VTOL aircraft consistent with the disclosed embodiments is shown.

[0087] Figure 51 A plot of positional errors in the X, Y, and Z directions as a function of the distance to the distribution combination of the light source, obtained from a field dataset, is shown, consistent with the disclosed embodiments. Detailed Implementation

[0088] This disclosure relates to components primarily intended for use in electric vertical takeoff and landing (eVTOL) aircraft used in unconventional aircraft. For example, the eVTOL aircraft of this disclosure may be designed for frequent (e.g., more than 50 flights per workday), short-duration flights (e.g., less than 100 miles per flight) over, into, and out of densely populated areas. The aircraft may be designed to carry 4 to 6 passengers or commuters with a desired low-noise and low-vibration experience. Therefore, their components may need to be configured and designed to withstand frequent use without wear, to generate less heat and vibration, and the aircraft may need to include mechanisms for effectively controlling and managing the heat or vibration generated by the components. Additionally, it may be anticipated that several of these aircraft will operate close to each other over congested metropolitan areas. Therefore, their components may need to be configured and designed to generate low levels of noise both inside and outside the aircraft, and to be configured and designed with various safety and backup mechanisms. For example, for safety reasons, the aircraft may need to be propelled by a distributed propulsion system to avoid the risk of single points of failure, and they may need to be capable of routine takeoff and landing on runways. Moreover, compared to traditional airport runways, aircraft may need to be able to take off and land safely vertically from and into relatively confined spaces (e.g., vertical takeoff and landing airports, parking areas, or runways) while carrying approximately four to six passengers or commuters with attached baggage. These usage requirements may impose design constraints on aircraft size, weight, and operational efficiency (e.g., drag, energy use), which may affect the design and configuration of aircraft components.

[0089] The disclosed embodiments provide new and improved configurations for aircraft components not observed in conventional aircraft, and / or identified design criteria for components that differ from those in conventional aircraft. These alternative configurations and design criteria, combined with solutions to the shortcomings and challenges of conventional components, result in the embodiments disclosed herein for a variety of configurations and designs for eVTOL aircraft components.

[0090] In some embodiments, the eVTOL aircraft of this disclosure may be designed to perform both vertical and conventional takeoff and landing, wherein the distributed electric propulsion system enables vertical flight, forward flight, and transitions. Thrust can be generated by supplying high-voltage electrical power to the electric engines of the distributed electric propulsion system, each of which can convert the high-voltage electrical power into mechanical shaft power to rotate the propeller. The embodiments disclosed herein may relate to optimizing the energy density of the electric propulsion system. Embodiments may include electric engines connected to an onboard electric power source, which may include devices capable of storing energy, such as batteries or capacitors, or may include one or more systems for utilizing or generating electricity, such as fuel-powered generators or solar panel arrays. Some of the disclosed embodiments provide weight and space reductions in aircraft components, thereby improving aircraft efficiency and performance. Given concerns about safety in passenger transport, the disclosed embodiments implement new and improved safety protocols and system redundancies in the event of failure to minimize any individual point of failure in the aircraft propulsion system. Some of the disclosed embodiments also provide new and improved methods to meet aviation and transport laws and regulations. For example, the U.S. Federal Aviation Administration enforces federal laws and regulations that require safety components such as fire barriers adjacent to engines that use excessive amounts of oil or other flammable materials.

[0091] In a preferred embodiment, the distributed electric propulsion system may include twelve electric engines mounted on booms at the front and rear of the aircraft's main wing. The front electric engines are tiltable during flight between a horizontal orientation position (e.g., to generate forward thrust) and a vertical orientation position (e.g., to generate vertical lift). The front electric engines may be clockwise or counterclockwise in terms of propeller rotation direction. The rear electric engines may be fixed in a vertical orientation position (e.g., to generate vertical lift). These rear electric engines may also be clockwise or counterclockwise in terms of propeller rotation direction. In some embodiments, the aircraft may have various combinations of front and rear electric engines. For example, the aircraft may have six front and six rear electric engines, four front and four rear electric engines, or any other combination of front and rear engines, including embodiments where the number of front and rear electric engines is not equal. In some embodiments, the aircraft may have four front and tail propellers, wherein at least four of these propellers include tiltable propellers.

[0092] In a preferred embodiment, for Vertical Takeoff and Landing (VTOL) missions, both the front and rear electric motors provide vertical thrust during takeoff and landing. During the forward flight phase, the front electric motor provides horizontal thrust, while the propeller of the rear electric motor retracts to a fixed position to minimize drag. The rear electric motor can be actively retracted using position monitoring. The transition from vertical to horizontal flight, and vice versa, can be achieved via a tilting propeller subsystem. The tilting propeller subsystem redirects thrust between the primary vertical direction during vertical flight mode and the primary horizontal direction during forward flight mode. A variable pitch mechanism alters the collective angle of the propeller hub assembly blades of the front electric motor for operation during hovering, transition, and cruise phases.

[0093] In some embodiments, during conventional takeoff and landing (CTOL) missions, the forward electric engine provides horizontal thrust for wing-borne takeoff, cruise, and landing. In some embodiments, the tail electric engine may not be used to generate thrust during CTOL missions, and the tail propeller may be retracted in place.

[0094] This document describes exemplary embodiments with reference to the accompanying drawings. The drawings are not necessarily drawn to scale. While examples and features of the disclosed principles are described herein, modifications, rewrites, and other implementations are possible without departing from the spirit and scope of the disclosed embodiments. Furthermore, the words “comprising,” “having,” “containing,” and “including,” and other similar forms, are intended to be semantically equivalent and are open-ended, as one or more items following any of these words do not imply an exhaustive list of such items or a limitation to only the listed items. It should also be noted that, as used herein and in the appended claims, the singular forms “a / an” and “the” include plural references unless the context clearly indicates otherwise.

[0095] Throughout this disclosure, the term "disclosed embodiment" refers to an example of the inventive ideas, concepts, and / or manifestations described herein. Numerous related and unrelated embodiments are described throughout this disclosure. The fact that some "disclosed embodiments" are described as exhibiting features or characteristics does not imply that other disclosed embodiments must share this feature or characteristic.

[0096] The embodiments described herein include computer-readable media (e.g., non-transitory computer-readable media) containing instructions that, when executed by at least one processor, cause the at least one processor to perform a method or a set of operations. A non-transitory computer-readable medium can be any medium capable of storing data in any memory in a manner readable by any computing device having a processor to implement a method or any other instructions stored in memory. A non-transitory computer-readable medium can be implemented to include any combination of software, firmware, and hardware. Software can preferably be implemented as an application tangibly embodied in a program storage unit or computer-readable medium consisting of parts or a combination of certain means and / or devices. The application can be uploaded to and executed by a machine including any suitable architecture. Preferably, the machine can be implemented on a computer platform having hardware such as one or more central processing units (“CPU”), memory, and input / output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described in this disclosure can be part of the microinstruction code or part of an application program or any combination thereof, which can be executed by the CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units can be connected to the computer platform, such as additional data storage units and printing units. Furthermore, a non-transitory computer-readable medium can be any computer-readable medium other than a transiently propagated signal.

[0097] Memory may include any mechanism for storing electronic data or instructions, including random access memory (RAM), read-only memory (ROM), hard disk, optical disk, magnetic media, flash memory, and other permanent, fixed, volatile, or non-volatile memory. Memory may include one or more separate storage devices communal or distributed for storing data structures, instructions, or any other data. Memory may further include a memory portion containing instructions for execution by a processor. Memory may also be used as a working memory device for the processor or as a temporary storage device.

[0098] Some embodiments may involve at least one processor. "At least one processor" can constitute any physical device or group of devices having circuitry that performs logical operations on one or more inputs. For example, at least one processor may include one or more integrated circuits (ICs), including application-specific integrated circuits (ASICs), microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a server, a virtual server, or other circuitry suitable for executing instructions or performing logical operations. Instructions executed by at least one processor may, for example, be preloaded into memory integrated with or embedded in the controller, or may be stored in a separate memory.

[0099] In some embodiments, at least one processor may include more than one processor. Each processor may have a similar configuration, or the processors may have different configurations that are electrically connected or disconnected from each other. For example, the processors may be separate circuits or integrated into a single circuit. When more than one processor is used, the processors may be configured to operate independently or collaboratively. The processors may be electrically coupled, magnetically coupled, optically coupled, acoustically coupled, mechanically coupled, or coupled in other ways that allow them to interact.

[0100] As used herein, unless otherwise expressly stated, the term "or" covers all possible combinations unless impractical. For example, if a statement indicates that a component may include A or B, then unless otherwise expressly stated or impractical, the component may include A, or B, or A and B. As a second example, if a statement indicates that a component may include A, B, or C, then unless otherwise expressly stated or impractical, the component may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0101] In the following description, various working examples are provided for illustrative purposes. However, it should be understood that this disclosure may be practiced without one or more of these details. Reference will now be made in detail to non-limiting examples of this disclosure, examples of which are illustrated in the accompanying drawings. Examples are described below with reference to the accompanying drawings, wherein similar reference numerals refer to similar elements. When similar reference numerals are shown, corresponding descriptions are not repeated, and interested readers may refer to the previously discussed drawings for a description of similar elements.

[0102] This document describes various embodiments with reference to systems, methods, apparatus, or computer-readable media. The purpose is that disclosure of one is disclosure of all. For example, it should be understood that the disclosure of a computer-readable medium described herein also constitutes a disclosure of methods implemented by a computer-readable medium, as well as systems and apparatus for implementing those methods via, for example, at least one processor. It should be understood that this form of disclosure is for ease of discussion only, and within the contemplated scope of this disclosure, one or more aspects of one embodiment herein may be combined with one or more aspects of other embodiments herein.

[0103] Consistent with this disclosure, some implementations may involve networks. A network can constitute any combination or type of physical and / or wireless computer networking arrangement for exchanging data. For example, a network can be the Internet, a private data network, a virtual private network using a public network, a Wi-Fi network, a mesh network, a local area network (LAN), a wide area network (WAN), and / or other suitable connections and combinations that enable information exchange among various components of the system. In some implementations, a network may include one or more physical links for exchanging data, such as Ethernet, coaxial cable, twisted-pair cable, fiber optic cable, or any other suitable physical medium for exchanging data. A network may also include a public wired network and / or a wireless cellular network. A network can be secure or insecure. In other embodiments, one or more components of the system may communicate directly via a dedicated communication network. Direct communication may use any suitable technology, including, for example, BLUETOOTH. TM BLUETOOTHLE TM (BLE), Wi-Fi, Near Field Communication (NFC), or other suitable communication methods that provide a medium for exchanging data and / or information between individual entities.

[0104] Reference will now be made to exemplary embodiments, which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, wherein the same numerals in different drawings denote the same or similar elements unless otherwise indicated. The implementations set forth in the following description of the exemplary embodiments do not represent all implementations consistent with this disclosure. Instead, they are merely examples of apparatuses and methods consistent with aspects relevant to the subject matter recited in the appended claims. Some embodiments may be described in the context of providing systems and methods in electric vertical takeoff and landing (eVTOL) aircraft or airborne vehicles without limiting the scope of this disclosure. However, this disclosure is not limited thereto. Other types of airborne vehicles (such as, but not limited to, unmanned airborne vehicles (UAVs), manned airborne vehicles, conventional vertical takeoff and landing (VTOL) aircraft, hybrid VTOL, and other airborne vehicles) may utilize the systems and methods disclosed herein.

[0105] Advanced Air Mobility (AAM) is an emerging field of aeronautics that involves the use of small aircraft for daily transportation and other services, and many AAM aircraft are expected to take off and land on new infrastructure known as vertical takeoff and landing (VTOL) airports. As described herein, a VTOL airport refers to a landing site or landing surface on which an airborne aircraft, such as an eVTOL, lands or takes off. In some embodiments, a VTOL airport may also be referred to as a vertiplex or vertistop. The location of a VTOL airport can be determined based on a number of factors, including but not limited to physical obstacles, federal and state or local regulatory restrictions, surrounding use, and so on. Physical obstacles can be fixed, anticipated, mobile, or temporary. An example of anticipated physical obstacles might be an adjacent property with development rights for a 40-story building, but which is currently vacant land. Some examples of physical obstacles might include nearby tall buildings, antennas, towers (communication towers and water towers), trees, power lines, utility poles, billboards, land use planning for the VTOL site, land title ownership rights, and so on.

[0106] In some cases, regulatory restrictions may include both the current land use designation of the VTOL airport site and the rights of the property owner. For example, in an air rights transaction or transfer of development rights, an owner may sell the right to build in the space above their property to a buyer who wants to construct a larger building than they were originally permitted to build. For instance, if a hangar operator sells the air rights above its hangar, a proposed VTOL terminal that would extend into that space may not be built without the approval of the air rights owner. Height zones are geographical areas where maximum building height is restricted and should also be considered when site selection for VTOL airports. Physical considerations, including physical barriers, can be weighed and balanced in light of anticipated future development patterns and the vision of the jurisdiction, as it seeks to accommodate population migration, increases or decreases in density, and current trends such as development toward mixed-use communities where residential and commercial buildings are close together.

[0107] Mobile or temporary physical obstacles include structures of altered or temporary nature. Mobile or temporary physical obstacles can include considerations that are both planned and anticipated. Planned considerations are those involving processes in which the VTOL operator may have the opportunity to provide input, while anticipated considerations are those that are not notified in advance or are likely to occur during the VTOL's lifecycle. Some examples of temporary structures might include temporary VTOLs, construction cranes, flying debris, construction phase divisions, noise, lightning protection equipment, non-acoustic interference factors, electrostatic discharge, urban wind shadow areas, or future local land use. While these considerations reflect temporary and potentially insignificant events during the VTOL's operational lifecycle, they are still worth considering to support safe and efficient operation. Furthermore, VTOL site selection decisions may also be influenced by the anticipated frequency of certain temporary considerations. For example, locating a VTOL near tall trees increases the likelihood of debris periodically entering the VTOL's mobile area, including the possibility that at some point in the future, as the trees grow, they may penetrate the VTOL's airspace and pose a hazard to general aviation.

[0108] When selecting a location for a vertical takeoff and landing (VTOL) airport and designing its operation, considering the surrounding area can be crucial. Surrounding uses encompass considerations that arise outside the VTOL property but within the local vicinity. These considerations can influence the VTOL during site selection, design, or operation, and may change throughout the VTOL's lifecycle. A VTOL can also influence the surrounding area and modify these considerations. Some examples of surrounding uses that may influence VTOL site selection include critical infrastructure, local fire stations, subway or bus stations, local land use, distance to maintenance or repair facilities, and the downwind direction of wind farms. In some cases, surrounding uses may be influenced by the VTOL. Examples of this scenario include nearby schools, properties below approach and departure paths, noise-sensitive areas, visual disturbances (e.g., solar panel reflectivity), zoos, protected wildlife habitats, and the privacy of VTOL neighbors. The proximity of the VTOL to existing infrastructure can be a primary site selection factor. Infrastructure considerations include current local land use (e.g., schools, hospitals, parks, or other noise-sensitive areas), emergency response (e.g., fire stations), and direct connectivity with other transport options (i.e., multimodal transport). For early VTOL airport site selection, proximity to these types of existing infrastructure can enable timely development and operation by reducing development lead time for these subordinate criteria (e.g., land use designated for transport). On the other hand, if a VTOL airport is located too close to other types of infrastructure, flight operations may be hampered. For example, proximity to wind turbine farms may restrict approach and departure paths and interfere with airflow, thus hindering safe flight operations. Several other factors may be considered when designing or configuring VTOL airports, including but not limited to aircraft performance in the VTOL airport environment, passenger comfort, economic considerations (such as development costs, maintenance costs, and revenue generation), environmental considerations, airspace considerations, demand considerations, emergency considerations, communications and data management, safety considerations, safety and usability, automation, etc.

[0109] For use in urban air traffic, landing and takeoff of eVTOL aircraft in urban environments may require highly accurate and complete positioning capable of operating in challenging GNSS environments. As used herein, a GPS-denied or GPS-challenged environment refers to an environment lacking reliable access to Global Positioning System (GPS) or Global Navigation Satellite System (GNSS) signals. In a GPS-denied environment, GPS signals may be weakened, interrupted, denied, interfered with, hacked, or simply disabled due to multipath effects or obstruction of satellite signals. Satellite signals may be denied in challenging environments due to the lack of a clear line-of-sight path between the satellite and the user antenna. Signals can be interrupted or weakened due to adverse weather conditions, low or poor visibility, high density of tall buildings in urban environments, unfavorable or uncooperative landing conditions, etc.

[0110] Aircraft (e.g., Figure 3The air vehicle (310) may include an electrically powered air vehicle or an eVTOL aircraft. eVTOL aircraft, which can be used for on-demand urban air transport services, can provide an alternative mode of transport in urban environments with low direct operating costs, low noise, and zero emissions. Furthermore, eVTOL aircraft offer an alternative form of air transport that is versatile (capable of vertical takeoff and landing from a basic landing zone), economical (reduced procurement and operating costs), easy to use (enabling operators with little or no flight training or experience to operate), and safe (designed to better withstand failure modes). In this regard, the development of distributed electric propulsion (DEP) enables inexpensive, quiet, and reliable short-range VTOL aircraft. The use of DEP offers significant flexibility, potentially allowing for new aircraft configurations, architectures, and control methods. Additionally, electric propulsion is scale-free in terms of achieving motor power at highly similar levels of weight and efficiency on a large scale. For example, redundant DEP can be used to improve fault tolerance and flight safety. Using electric motors can also improve ground safety through noise reduction, heat dissipation and potential toxic fumes, as well as the ability to shut off rotating propellers or rotors before passengers enter or leave. Additionally, while the propellers or ducted fans of a DEP still generate noise (the noise level and frequency will depend on tip speed, disk load, and other design parameters), by combining a DEP configuration (with multiple smaller rotors directly driven by electric motors), tip speed limiting, removal or minimization of engine / turbine / gear noise sources, and the potential use of fixed wings for efficient forward flight, eVTOLs are expected to have a modified or reduced noise profile compared to conventional helicopters of similar size, with a target noise reduction of 15 dB or more. Battery-powered eVTOL aircraft can also have a reduced environmental impact with zero operating emissions. Furthermore, using DEPs instead of complex shafts, cross couplings, and gear assemblies is expected to reduce both procurement, maintenance, and operating costs. When extended range is required, the aircraft can be designed with a hybrid electric propulsion system that takes advantage of operating smaller engines at peak efficiency. Operating the hybrid unit engine at idle or off during takeoff and landing can further reduce the aircraft's noise signature at lower altitudes.

[0111] Figure 1A This illustration shows a conventional Instrument Landing System (ILS) that provides horizontal and vertical guidance for aircraft along a runway. An ILS is a standard precision landing aid that provides accurate azimuth and descent guidance signals to guide an aircraft to land on the runway in normal or adverse weather conditions. An ILS may include three subsystems: a localizer, a glide slope, and a beacon. Figure 1B As shown, the localizer provides horizontal guidance for approaching aircraft, such as... Figure 1CAs shown, the glide slope provides vertical guidance for approaching aircraft, and the marker beacon provides distance information during the approach. In some cases, the marker beacon may be replaced by a distance measurement device (DME). In addition, the ILS may further include high-intensity lighting at the runway ends to help pilots orient the runway and transition from approach to visual landing.

[0112] In the ILS system, such as Figure 1B and Figure 1C As shown, two or more radio frequencies (RF) are broadcast, one of which is spatially offset from the other. The spatially offset RF signal repeats in both the horizontal and vertical directions. In some cases, signal sensors associated with the aircraft can measure the strength of the two signals. If one signal is stronger than the other, the aircraft deviates from the configured landing trajectory. Based on this information, the pilot can correct the course to continuously align with the centerline, thus navigating along the glide slope to the runway. While ILS and associated systems are well-established for conventional passenger aircraft, they may be unsuitable for eVTOL air vehicles in urban environments due to non-standard flight approach trajectories, interference with nearby ILS systems or all buildings, or the large footprint of the ILS.

[0113] Now for reference Figure 2 It illustrates an exemplary landing / takeoff approach using optical navigation in a GPS-denied environment, consistent with some embodiments of this disclosure. Figure 2 The diagram illustrates an example view 200 of an eVTOL aircraft approaching a landing site or landing location with an active marker in its camera's field of view in a GPS-denied environment, such as an urban environment with high density and tall structures. An exemplary airborne vehicle 210 (such as an eVTOL) can approach a vertical takeoff and landing (VTOL) airport 220 using one or more approach paths 260. One of the advantages of using an eVTOL is that, unlike conventional route-based landing approaches in ILS systems, the airborne vehicle can approach the VTOL airport from any direction. View 200 illustrates another approach direction (approach 2) by way of example. In some embodiments, the approach angle can vary between 7° and 9°, as shown. The airborne vehicle 210 may include a camera with a field of view 270. Figure 2 (Not illustrated here, but will be discussed in later chapters).

[0114] Now for reference Figure 3This document illustrates exemplary precision landing and takeoff systems and data communication systems consistent with some embodiments of this disclosure. As described herein, a precision landing and takeoff system refers to an optical navigation-based eVTOL positioning system operating in GPS-denied environments. As previously discussed, eVTOL air vehicles (such as air vehicle 310) can be used in urban air mobility (UAM) applications that foster commercial passenger services such as air taxis, or in public service applications such as firefighting, medical assistance delivery, emergency search and rescue operations, disaster relief operations, law enforcement, etc. In some embodiments, the air vehicle can be autonomous, i.e., unmanned.

[0115] A precision landing and takeoff system may include an airborne vehicle 310 comprising an onboard optical inspection device 315, a vertical takeoff and landing (VTOL) airport 320 comprising a marker 350, and a ground control unit 330. The optical inspection device 315 may include a camera 311 and a processor 312. In some embodiments, the optical inspection device 315, the ground control unit 330, and one or more VTOL airports 320 may communicate wirelessly with each other during landing or takeoff operations of the eVTOL airborne vehicle 310. Communication between the optical inspection device 315, the ground control unit 330, and one or more VTOL airports 320 may include receiving and transmitting data or information associated with providing landing or takeoff guidance to the airborne vehicle 310.

[0116] A precision landing and takeoff system may include one or more vertical takeoff and landing (VTOL) airfields 320 (also illustrated as VTOL airfield 420 in Figure 4). In some embodiments, VTOL airfield 320 or VTOL airfield 420 may include a landing surface or landing location for an eVTOL aircraft, such as an airborne vehicle 310. VTOL airfield 320 may include a plurality of light sources arranged in a predetermined pattern, wherein the characteristics of the light emitted from each of these light sources are configured to be modulated relative to time. Each VTOL airfield may include markers 350 or a dynamically distributed combination of active light sources. As used herein, an active light source (ALS) refers to a light source whose characteristics can be modulated over time. For example, the intensity of light emitted from an active light source can be modulated over time. Other characteristics that can be modulated over time include, but are not limited to, the frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light. Examples of active light sources may include, but are not limited to, light-emitting diodes (LEDs).

[0117] In some embodiments, the landing and takeoff area of ​​the vertical takeoff and landing (VTOL) airport 320 may be rectangular, circular, triangular, substantially rectangular, substantially circular, or substantially triangular, or a combination thereof, or other suitable shapes. As used herein, the landing and takeoff area refers to the area on which an airborne vehicle (e.g., airborne vehicle 310) can land or take off / take off from the VTOL airport. In some embodiments, active light sources arranged in a predetermined pattern may resemble the shape of the landing and takeoff area, such that the active light sources define the boundaries of the landing and takeoff area. In some embodiments, the predetermined pattern of the active light sources defining the boundaries of the landing and takeoff area may include low-intensity light sources located in the camera's field of view when the airborne vehicle approaches a landing target to assist in landing or takeoff.

[0118] In some embodiments, the active light sources may be arranged in substantially axisymmetric shapes (such as circles, squares, or rectangles). In some embodiments, the active light sources may be arranged in elliptical, triangular, trapezoidal, or other shapes. In some embodiments, the active light sources may be spaced equally or unequally in an axisymmetric shape. In an unequally spaced arrangement in an axisymmetric shape, the distance between adjacent active light sources may be non-uniform. In some embodiments, the active light sources may be arranged based on a grid pattern, wherein the light sources are uniformly spaced on the landing platform. Other arrangements (such as suitable arrangements) are also possible.

[0119] In some embodiments, the active light source may be arranged in an asymmetrical shape to maximize the detectability, uniqueness, or deception resistance and interference immunity of the active light source and its associated vertical take-off and landing airport. Figure 5 An example of an asymmetrical shape for arranging active light sources is shown (discussed later).

[0120] In some embodiments, the marker 350 may include a combination of infrared (IR) or visible spectral reference light sources (e.g., active light sources) distributed at known locations within the vertical takeoff and landing (VTOL) airfield 320. In this respect, the precision landing and takeoff system may be referred to as an Active Reference Light Pattern Positioning (AFLPL) system. Some of the advantages of using active light sources as markers at VTOL airfields for optical navigation of eVTOL air vehicles include:

[0121] i. Authentication and Security — Modulate one or more properties of the emitted light to achieve authentication and improve security.

[0122] ii. It can be easily distinguished from the surrounding environment, thus allowing for the reduction or elimination of visual clutter in urban environments.

[0123] iii. Enhanced range, detectability, and data transmission—One or more characteristics can be modulated to enhance range, detectability, and data transmission, including delivering messages to aircraft or enabling low-bandwidth communication.

[0124] iv. Day / Night Capability — The aircraft can operate during both day and night.

[0125] v. Interference and Spoofing Resistance—Optical AFLPL methods are inherently more resistant to interference and spoofing than radio frequency (RF) signals. The modulation capabilities of AFLPL can further enable the implementation of authentication codes.

[0126] vi. No major regulatory hurdles—The AFLPL approach may provide a simpler path to FAA certification, thereby enhancing the acceptability and feasibility of active light sources in public service applications. Furthermore, unlike RF-based ranging and communication methods, AFLPL will not require FCC allocation of the limited availability of RF spectrum.

[0127] Furthermore, in military applications, AFLPL narrowband near-IR illumination can provide a lower level of detection than RF emission.

[0128] vii. High Positioning and Guidance Accuracy—Positioning accuracy compared to GPS or GNSS may be equal to or higher in the vicinity of a VTOL airport, specifically in environments where GPS is challenged or denied. Position accuracy error may be less than 1m within 100m of the VTOL airport and decreases as the aircraft approaches the airport. In some cases, position accuracy error may be less than 10cm within 10m of the VTOL airport.

[0129] viii. High redundancy and high reliability—active light sources may be a relatively low-cost option and have no weight in the air.

[0130] This allows for the use of multiple light sources at vertical takeoff and landing airports to provide uninterrupted guidance.

[0131] ix. Enhanced visibility—The infrared (IR) wavelength of the emitted light can enhance visibility and detectability in adverse weather conditions such as fog, rain, storms, and lightning.

[0132] x. Low implementation complexity—Vertical takeoff and landing (VTOL) airports may require minimal infrastructure. Visible and infrared light sources can be integrated into the VTOL airport structure and recessed into the landing field with relatively low technical complexity and cost.

[0133] In some embodiments, the positions of active light sources in a distribution array at a vertical takeoff and landing (VTOL) airport can be designed to provide optimized positioning across the entire landing trajectory. In some embodiments, the distribution array may include a first set of light sources arranged in a first predetermined pattern, wherein each light source in the first set is configured to be within the field of view of a camera associated with the aircraft when the aircraft is at a first distance from the landing surface. The first set of light sources may include light sources with higher intensity and located at a greater distance from the landing target, improving performance when the aircraft is at a greater distance from the VTOL airport. The first set of light sources may be within the camera's field of view at the greater distance and outside the camera's field of view when the aircraft approaches the VTOL airport or landing target. The distribution array may further include a second set of light sources arranged in a second predetermined pattern, wherein each light source in the second set is configured to be within the camera's field of view when the aircraft is at a second distance from the landing surface. The second set of light sources may include light sources with lower intensity and located at a smaller distance, such that the second set of light sources remains within the camera's field of view when the aircraft is at its final approach or within a predetermined approach distance. The intensity of the second set of light sources may differ from that of the first set of light sources to avoid interfering with the detection of the first set of light sources from a greater distance. In some embodiments, the intensity of the first set of light sources may be higher than that of the second set of light sources. In some embodiments, the area covered by the first set of light sources may be larger than the area covered by the second set of light sources. In other words, compared to the second set of light sources, the first set of light sources may be distributed over a larger area so that when an airborne vehicle is within a predetermined approach distance, only the second set of light sources can be detected.

[0134] In some embodiments, the light sources in the distribution array may be arranged to maximize the detectability of each location by maximizing the spacing between each light source. In some embodiments, the light sources may be arranged to maximize the ability to identify a light source from multiple light sources, such as by minimizing the symmetry of the arrangement pattern. In some other embodiments, a predetermined pattern of light sources may be associated with a landing surface. For example, the distribution array may include a uniquely identifiable pattern for each VTOL airport, such that the VTOL airport can be identified based on the arrangement pattern of the light sources in the VTOL airport.

[0135] In some embodiments, additionally or alternatively, the active light source (e.g., a reference point) may be located on the ground, on the top of a building, or on other objects along a common flight path. Some possible locations include, but are not limited to, roads, tops of light poles, tops of buildings, tops of antennas, or other tall structures. This will allow for accurate location information throughout the flight, not just when the landing surface or landing site is visible. Furthermore, the same algorithm can be used to calculate the location, but because the light source pattern is spread over a much larger area, the system can operate for the entire or most of the flight duration, not just during the final approach to the landing site.

[0136] In some embodiments, the size and dimensions of one or more vertical takeoff and landing (VTOL) airports may vary. The size of the VTOL airport can determine the level of performance when the aircraft is far from the airport. While the reference pattern within the distribution array may not need to be identical for each VTOL airport, prior knowledge of the layout or arrangement of the reference light markers may be required. However, in some embodiments, the light sources may be configured to transmit information related to their own location, in which case prior knowledge of the light source's location or arrangement may not be necessary. In some embodiments, the distribution array pattern or arrangement may be stored in a system database or server. Information stored in the database may be accessible and updatable in real time or based on user input.

[0137] In some embodiments, one or more characteristics of the emitted light can be modulated to transmit information. Modulation can be performed by one or more methods, including but not limited to switching on or off, frequency modulation, amplitude modulation, duty cycle modulation, synchronization options, etc. The transmitted information may include the identification of the VTOL airport, the location of the light source, the identification of the light source, and the operational status of the VTOL airport. In some embodiments, the transmitted information may include encoded signals authenticating the landing surface or VTOL airport. It should be understood that combinations of modulation methods and information transmitted by modulation can be appropriately applied. For example, the frequency of light emitted from the light source can be modulated to transmit information associated with the identification of the light source, and the duty cycle can be modulated to transmit information associated with the identification of the VTOL airport. In another example, the frequency of light emitted from the light source can be modulated to transmit information associated with the identification of the VTOL airport. In some embodiments, the frequency can be modulated to indicate the operational status of the VTOL airport, such as normal operation, abnormal operation, maintenance, etc.

[0138] In some embodiments, the wavelength of light emitted from one or more light sources may be determined based on several factors, including but not limited to improving detectability by maximizing the difference between the emitted light and the background light, minimizing the absorption of the emitted light by atmospheric or weather conditions, maximizing the sensitivity of the light detected by the camera, or reducing visible light pollution around the landing target.

[0139] In some embodiments, the wavelength of light emitted from one or more light sources is in the range of 800 nm to 1550 nm. In some embodiments, the wavelength of light emitted from one or more light sources is in the range of 800 nm to 850 nm. In a preferred embodiment, the wavelength of light emitted from one or more light sources is 810 nm. In some embodiments, the wavelength of light emitted from one or more light sources is 1310 nm. In some embodiments, the wavelength of light emitted from one or more light sources is 1550 nm.

[0140] In some embodiments, as shown in the figure Figure 4A As shown, one or more light sources 450 may be recessed relative to the landing surface of the vertical takeoff and landing airport 420. Figure 4B An exemplary light source 450 is illustrated. In some embodiments, one or more light sources may be recessed, convex, or coplanar with the landing surface. In some embodiments, the light source 450 may include a protective cover or encapsulation cover to prevent the ingress of moisture, dust, or other particles that may affect the performance of the light source. In some embodiments, the protective cover may be configured to transmit substantially all light emitted from the light source, such that absorption by the protective cover is negligible or nonexistent.

[0141] In some embodiments, each light source may further include an optical sensor configured to detect a portion of light emitted from at least one other light source. This may require synchronizing the camera capture rate with the modulation of the light source to reduce errors that may occur when capturing images during transitions (discussed in later sections). In some embodiments, the landing surface or vertical takeoff and landing (VTOL) airport may further include controller circuitry configured to operate the light source. In this context, operating the light source may include activating, deactivating, or modulating the characteristics of the light source by, for example, adjusting an electrical signal applied to the corresponding light source. The controller circuitry may further include time management circuitry, power management circuitry, sequencing circuitry, etc. In some embodiments, one or more processors may be configured to remotely control the operation of the light source.

[0142] In some embodiments, the wavelength of the emitted light can be modulated to increase the detectability or range of the light source, or to transmit data from a corresponding reference point. The inventors recognize that while the wavelength can be adjusted, this can negatively impact the simplicity and feasibility of the system. For example, a hyperspectral camera might be required to detect wavelength changes. Although such cameras exist and are commercially available, they can be complex, unreliable, and computationally intensive. Furthermore, since the wavelength of an LED is primarily determined by its material composition and transmission medium, such light sources can be experimental, unreliable, and expensive.

[0143] In some embodiments, the VTOL airport 320 (or VTOL airport 420) may be a portable landing surface. Portable VTOL airports (not illustrated herein) may include retractable landing mats, fabric, or tarpaulins. This can be particularly useful in situations requiring landing at non-cooperative locations with restricted or temporary landing infrastructure, such as military operations, firefighting operations, disaster relief operations, medical aid distribution operations, etc. In some embodiments, the portable VTOL airport may include battery-powered active light sources incorporated therein, such that they can be remotely activated, deactivated, or modulated.

[0144] In some embodiments, a vertical takeoff and landing (VTOL) airport may include a plurality of landing surfaces, wherein each landing surface includes a plurality of light sources arranged in a predetermined pattern, and wherein the characteristics of the light emitted from each of these light sources are configured to be modulated relative to time.

[0145] In some embodiments, one or more landing surfaces of a vertical takeoff and landing (VTOL) airport may be horizontally displaced relative to each other, for example, within a VTOL airport hub, a VTOL complex, or a large area comprising multiple VTOL airports. The horizontally displaced VTOL airports may be coplanar or substantially coplanar. As used herein, the terms "coplanar" or "substantially coplanar" refer to the landing surfaces of a VTOL airport being on the ground, analogous to cars parked in a parking lot. In some embodiments, one or more VTOL airports may be vertically displaced relative to each other, such that they are non-coplanar, for example, in a VTOL airport hangar comprising multiple VTOL airports. In some embodiments, one or more VTOL airports may be horizontally and vertically displaced relative to each other, such that they are offset relative to each other along horizontal and vertical axes, thereby allowing for a higher density of VTOL airports.

[0146] Return to reference Figure 3 The airborne vehicle 310 may include an optical detection device 315, which further includes a camera 311 and a processor 312. The camera 311 may be configured to generate an image based on information transmitted from a light source (e.g., an active light source 350) located near the landing surface for the airborne vehicle. For example, the camera 311 may be configured to capture an image of light emitted by the light source. The processor 312 may include controller circuitry configured to receive the generated image and determine the position and orientation of the airborne vehicle based on the received image.

[0147] In some embodiments, camera 311 may include a color, monochrome, hyperspectral camera. Camera 311 may be mounted on airborne vehicle 310 such that camera 311 provides a planar view of the light source on the landing surface. A planar view of the light source may be desired when the airborne vehicle is within the landing distance (where vertical descent is from 50 ft above ground level (AGL) to 0 AGL) or when the airborne vehicle is taking off. In some embodiments, camera 311 may be mounted on airborne vehicle 310 such that camera 311 provides a forward-looking view of the light source on the landing surface. This may be desirable for maximizing visibility during approach. In some embodiments, one or more cameras may be mounted on the airborne vehicle to capture multiple frames or views from different angles during landing or takeoff.

[0148] In some embodiments, camera 311 may always be on but is activated when a reference point or active light source on the landing surface is detected to capture images and / or report measurement results. Alternatively, camera 311 may be turned on or activated to conserve power when the aircraft is within detection range. In alternative embodiments, camera 311 may be turned on or activated for a predetermined duration, at a predetermined time, or by an activation signal from an external processor (e.g., a flight control computer or ground control unit 330), or by the operator of the aircraft. In some embodiments, camera 311 may be configured to be activated after the aircraft is within a predetermined distance from the landing surface. The predetermined distance may be based on several factors, including but not limited to weather conditions, landing surface conditions, etc. In a preferred embodiment, the predetermined distance may be 500 m or less.

[0149] In some embodiments, camera 311 may include an optical filter configured to allow a range of wavelengths of light emitted from each of the light sources. In other words, the optical filter of camera 311 may be configured to reject wavelengths that are significantly different from a reference transmission wavelength. As used herein, a reference transmission wavelength refers to the wavelength or range of light emitted by one or more reference markers (e.g., an active light source on a landing surface). For example, if an active light source is configured to emit light at a wavelength of 810 nm, the optical filter may allow a wavelength range of 808 nm to 812 nm and reject wavelengths outside the permissible transmission range. In some embodiments, the sensitivity of the optical detection of camera 311 may be adjusted to filter incoming wavelengths.

[0150] In some embodiments, the permissible wavelength range is in the range of 800 nm to 850 nm. In a preferred embodiment, the permissible wavelength range is approximately 810 nm. In some embodiments, the permissible wavelength range is approximately 1310 nm. In some embodiments, the permissible wavelength range is approximately 1550 nm. In some embodiments, the optical filter may be configured to permissive wavelengths corresponding to the emitted light. As used herein, the term "approximately" refers to an approximation that makes the permissible wavelength range within ±2 nm or less. The optical filter may be a low-pass filter, a high-pass filter, or a band-pass filter. Based on the detected light source, camera 311 may generate an FPA image, such as... Figure 5 As shown.

[0151] Now for reference Figure 6 This illustrates an exemplary overview of an algorithm and data pipeline 600 during the operation of a precision landing and takeoff system, consistent with some embodiments of this disclosure. As shown, a camera (e.g., camera 311) may be configured to receive optical signals from light sources on a vertical takeoff and landing (VTOL) airport, as well as background information such as scene information. The camera, including optical filters, may be configured to output a series of still images, or a video stream, or an FPA image. Based on the generated output signals, a processor may be configured to execute one or more algorithms to detect, correlate, estimate the attitude of the aircraft, and decode information from encoded signals from active light sources. Detection algorithms allow the identification of the location of active light sources in image frames captured by the camera. Correlation algorithms allow the association or mapping of identified active light sources in the image to corresponding active light sources on the landing surface. Attitude estimation algorithms allow the determination of the aircraft's attitude based on the correlated active light sources.

[0152] Detection

[0153] Identifying the location of an active light source in an image frame may involve distinguishing the received signal from background noise. This can be done using background subtraction and thresholding operations. If the active light source is modulated such that it is completely on in one frame and completely off in another, the frame where the light source is completely off can be used as the background image to remove the background from the fully on image by subtraction. An exemplary subtraction algorithm is provided here. It should be understood that other suitable subtraction and thresholding techniques can be used to identify the location of active light sources in an image.

[0154] For example, if I_n is an MxN matrix of pixel values ​​corresponding to the nth image, then I_diff = abs(I_n - I_(n-1)) is the difference in pixel values ​​between consecutive images. The resulting difference frame (I_diff) can be thresholded to generate a mask that identifies which pixels in the original image correspond to the active light source. Therefore, I_mask = where (I_diff > threshold). If an active light source is identified in the image, its position can be calculated using centroid mathematics, which utilizes sub-pixel accuracy to determine the light source's location.

[0155] Center of mass = ∑(Pi*Xi) / ∑Pi, where P i X is the value of the i-th pixel indicated by the mask, and X is the position of that pixel (xy pair). Additionally or alternatively, image filtering techniques such as temporal filtering and spatial filtering can be used to detect and locate active light sources in an image.

[0156] In some embodiments, improving the detection or localization of the location of an active light source in an image frame may include performing a registration method. This may be desirable for images in which the background is moving rapidly due to the motion of a camera (such as a camera mounted on a moving aerial vehicle). The active light source may change position in the image between open and closed frames. In this case, it may be necessary to shift the images to align them between frames, thereby aligning the background for subtraction. To achieve this, one of several techniques, including feature matching, translation matching, or current state estimation, may be used. In feature matching, features common to each frame can be identified and their positions in each frame can be determined. Images may be translated and / or distorted to align the features. In translation matching, for small changes, two images may be shifted one pixel at a time to determine the position where the background most closely matches. In current state estimation, if the camera's position and rotation speed are known or can be estimated from previous images, the image distortion required to align each frame can be estimated. Additionally, registration may be performed globally or locally over one or more regions of interest around the active light source. It should be understood that other techniques for improving detection may be employed as alternatives or in combination with the techniques described herein.

[0157] In some embodiments, identifying the location of an active light source in an image frame or improving detection may include tracking the location of the identified active light source to reduce computational load in subsequent calculations. To adequately predict the location of an active light source, certain measurements of velocity or time-varying changes may be useful. This can be achieved by using aircraft position and velocity from an external system (such as GPS or an inertial navigation system (INS)), using internal estimates of aircraft position and velocity derived from changes in position over time, or using changes in pixel position over time (such as tracking changes in active light sources between frames and time extrapolation). Tracking can be used to reduce computational load by calculating the region of interest for detection rather than the entire image, improve accuracy by providing estimates for registration, or calculate additional information (such as velocity or acceleration) that can be reported to other devices on the aircraft.

[0158] Figure 7 An exemplary detection algorithm 700 for identifying the location of a light source in an image captured by a camera, consistent with some embodiments of this disclosure, is illustrated. As shown, the camera may receive a noise signal and generate an output typically in the form of an image or image stream. The image may be processed using applied bandpass filters, Discrete Fourier Transform (DFT) calculations on the filtered image, low-pass filters, and thresholding. Figure 8 and Figure 9 Examples of data plots illustrating comparisons of signals before and after signal processing using bandpass filters and DFT calculations, consistent with some embodiments of this disclosure, are provided. In some embodiments, one or more characteristics of an active light source may be modulated relative to time to improve the detection of active light sources in a captured image.

[0159] Data association

[0160] As used in the context of this disclosure, data association refers to the process of matching detected points in a camera image with known points in a database of known locations of active light sources, optical markers, or reference points on a landing surface. Some prior art techniques for data association may include modulating one or more light sources to transmit a unique identifier of the light source and determining location based on light modulation. However, such methods can be challenging, such as, but not limited to, inaccurate identification due to cross-signaling, poor signal-to-noise ratio (SNR), high background noise, etc. As previously mentioned and disclosed in some embodiments of this disclosure, one or more data association algorithms may be performed to map detected points in a camera image to known locations of active light sources. The selection of a data association algorithm may depend on several factors, including, but not limited to, the existence of an acceptable data association, the reliability, accuracy, robustness, etc., of the association result obtained. As an example, after finding an acceptable and correct association, points may be tracked between images, for example, by a point tracking method (discussed in detail later). As another example, while a mesh association algorithm may not rely on an initial association, the algorithm may not always produce a solution. Therefore, it may be necessary to execute two or more data association algorithms in parallel or sequentially to establish acceptable data associations and associate each identified reference point in the camera image with the three-dimensional (3D) location of the identified reference point. Some aspects of this disclosure relate to methods and systems for data association and their advantages.

[0161] a. Iterative Closest Point Algorithm

[0162] In some embodiments, the Iterative Closest Point (ICP) algorithm may be used. This algorithm may include the following steps.

[0163] 1. Given a location estimate, estimate the location of the ALS in the image.

[0164] 2. For each detected point, find the point with the nearest (geometric distance) estimate.

[0165] 3. Sum the distances between each pair of points (measured and estimated).

[0166] 4. Determine small perturbations (xy translation and rotation) to the estimated points, which reduces the error distance calculated in step 3.

[0167] 5. Update the estimated location of the point based on the perturbation calculated in step 4.

[0168] 6. Return to step 2 until the error converges.

[0169] 7. After the solution has converged, obtain the association obtained from step 2 (nearest neighbor).

[0170] While simple to implement, ICP can be sensitive to inaccuracies in the initial pose estimation. In some cases where the step size might be too small, it can take a long time to converge and the computational cost can be substantial.

[0171] b. Robust point matching algorithm for thin plate splines

[0172] Splines are numerical functions defined by polynomial functions. Spline functions exhibit high smoothness at the points where polynomial segments connect (called nodes). Feature-based methods for non-rigid registration may face challenges associated with the correspondence between points between two or more sets of features. In this context, the correspondence between two sets of features refers to the association between each identified reference point in a 2D image and the 3D location of a reference point (e.g., an active light source on the ground). The framework of non-rigid point matching, or robust point matching (RPM) algorithms can be extended to include spline-based deformations, specifically plate splines. Some methods that solve for both correspondences and transformations include ICP (discussed previously). The ICP algorithm assigns binary correspondences at each step using nearest neighbor relationships. This estimate of the correspondence can be used to optimize the transformation, and vice versa. While the ICP algorithm is simple and fast, and guarantees convergence to a local minimum, this can be insufficient, especially with large deformations. Furthermore, the correspondence deteriorates rapidly with outliers, rendering the ICP algorithm inadequate. In spline theory, generating a spatial mapping that conforms to two sets of landmark points through smooth interpolation is a pervasive problem. This is because, once non-rigidity is allowed, there are countless ways to map one set of points to another. Smoothness constraints are desirable because they prevent mappings that are too arbitrary or outliers. In other words, the behavior of the mapping can be controlled by choosing a specific smoothness factor based on prior knowledge.

[0173] Now for reference Figures 13A to 13D It illustrates the correspondence and matching process using a robust point matching algorithm for thin-plate splines, consistent with some of the disclosed embodiments.

[0174] Figure 13A The coordinate space is shown, comprising a normalized grid ordered according to the order of ground-based active light sources. For example... Figure 13A As shown, the exemplary normalized mesh contains 25 points in a 5x5 array format. Each point in the normalized mesh is numbered from 0 to 24 according to the order of light on the ground. In some embodiments, the normalized mesh may be rotated based on the heading of the aircraft approaching the landing surface. The coordinate space further illustrates the normalized detected points received from the detection algorithm. The normalized detected points are indicated using a "+" sign. The normalized detected points may overlap with the normalized mesh points. Circular boundaries indicate the radius of potential matches to be evaluated using robust point matching.

[0175] The Thin Plate Spline Robust Point Matching (TPS-RPM) algorithm can include performing distance-based point matching and association over a large search area. Figure 13B The diagram shows the coordinate space including normalized grid points and the locations of detected points after deformation of the thin plate spline (e.g., as shown in the diagram). Figure 13D (As shown). After the thin plate spline is deformed, a line can be drawn to connect the normalized grid points with the corresponding normalized detected points (if any).

[0176] The TPS-RPM algorithm may further include determining TPS distortion that can be used to reduce errors for associated points. The algorithm may further include reducing the size of the search region and iteratively performing TPS distortion until the normalized detected points and normalized grid points converge. Figure 13C The association between normalized grid points and the detected normalized points is shown, indicated by the line connecting the two.

[0177] In some embodiments, given the iterative nature of the TPS-RPM algorithm, matching the known pattern with detected points using a normalized initial guess can be extremely time-consuming. Therefore, to improve the overall performance of the algorithm, specifically when the aircraft approaches a vertical takeoff and landing (VTOL) airport where there are significant changes between frames, the TPS-RPM algorithm can operate in two modes. The first mode uses a normalized initial guess to estimate the system's attitude. Based on the confidence metric in the estimated attitude, other association algorithms (such as ICP, cross-ratio, mesh association, etc.) can be used to supplement or assist TPS-RPM by seeding it with the aircraft's current attitude, and using that current attitude to project the known pattern into the camera frame, using this projection as our initial guess before normalization. In practice, this approach significantly reduces the number of iterations required to converge to a solution, thus allowing TPS-RPM to be used at full rate.

[0178] c. Grid-based association algorithm

[0179] Now for reference Figure 14AThis represents an exemplary camera image 1400 with detected points (indicated by a square grid) consistent with the disclosed embodiments. Camera image 1400 may be an image captured by a camera mounted on an airborne aircraft that approaches at a non-zero angle relative to a plane perpendicular to the landing surface. In some embodiments, image 1400 may include any number of detected points in an arrayed or non-arrayed arrangement. It should be understood that although image 1400 illustrates a 5x5 grid of detected points, other grid patterns or random patterns including any number of detected points may also be used. In some embodiments, the detected reference points or points may be numbered in a known, identifiable order. For example, the detected points in image 1400 may be numbered from 0 to 24, starting from the top row and moving from the top left to the top right, with the numbering repeated from left to right for each row. It should be understood that the numbering order should never be construed as limiting. The numbering format is exemplary and non-limiting.

[0180] Figure 14B This represents a normalized image space 1410, which includes the normalized detected points of image 1400. In some embodiments, the detected points may be normalized such that they fit within a predefined image space, for example, any unit (au) from -100 to +100. Figure 14B As shown. In some embodiments, normalization may include constructing a transformation matrix using the mean and variance of the points, such that the mean of the detected points is centered and the variance of the detected points is equal to one (value 1). Some advantages of normalizing the detected points within the normalized image space are that all points are within a unit distance and that these points are not offset. Additionally, any processing (e.g., data manipulation) performed on one or more points may affect the remaining points equally.

[0181] After normalizing the detected points, such as Figure 14B As shown in the normalized image space 1410, lines passing through multiple detected points in the normalized image space 1410 can be identified by performing the Hough transform technique. As mentioned herein, the Hough transform technique can be used to isolate features of specific shapes, such as lines, circles, ellipses, etc., within an image comprising multiple points in parameterized form. In line detection using the Hough transform, each input measurement (e.g., coordinate point) indicates its contribution to a globally consistent solution (e.g., the physical line that produces that image point). As an example, when fitting a set of line segments to a set of discrete image points (e.g., pixel locations), a lack of knowledge about the desired number of line segments may not constrain the possible solutions for fitting line segments through image points.

[0182] In the context of image analysis, the coordinates (i.e., x, y) of points on edge segments in the image are known and therefore act as constants in the parametric equation xcosθ + ysinθ = r, while r and θ are unknown variables. Plotting the possible (r, θ) values ​​defined by each (x, y) point in the Cartesian image space onto a curve (i.e., a sine curve) in the polar Hough parameter space, as shown by... Figure 14C The Hough transform image 1420 is illustrated in the figure. This point-to-curve transformation is the Hough transform for straight lines. When viewed in the Hough parameter space, collinear points in the Cartesian image space produce a curve that intersects at a common point (r, θ).

[0183] The Hough transform can be used to identify one or more parameters of a curve fitted to a given set of points. In some cases, the Hough transform can also help determine what the features are (i.e., to detect features with parameterized descriptions) and how many features are present in an image. Curves generated from collinear points in a gradient image intersect at peaks (r, θ) in the Hough transform space. These intersection points characterize straight line segments from the original image. Extractor mechanisms can be employed to extract local maxima (e.g., intersection points) from an array of accumulators. For example, one approach may include applying thresholding and thinning to isolated clusters of local maxima in an image of an accumulator array or a Hough transform image 1420. In the context of the Hough transform, thresholding refers to setting a predefined limit on the maximum number in the accumulator array, where values ​​may be equal to or greater than the predefined maximum.

[0184] In some embodiments, identifying lines passing through multiple detected points in the normalized image space 1410 may include, among other steps, using a Hough transform to map all lines passing through a given point in a single sine curve in the Hough-transformed image 1420, discretizing the Hough space into multiple bins 1424, and incrementing a bin 1424 by one if the sine curve passes through it. Each detected point in the normalized image space 1410 is transformed into the Hough space, and if a bin 1424 in the Hough space has a value higher than a predetermined threshold, that point in the Hough space will be mapped to a line passing through at least that many points in the image during inverse Hough transformation. As an example, in a 5x5 grid array of detected points shown in the normalized image space 1410, the bin threshold may be set to 4. The point in the Hough space represented by a bin 1424 with a bin threshold of 4 will be mapped to a line passing through at least four points in the normalized image space 1410.

[0185] In some embodiments, the data association algorithm may further include optimizing the line by, for example, eliminating or removing lines with poor fits to the detected points. Discretization in Hough space may produce lines (such as...) Figure 14D(As shown in image space 14130), these lines may deviate slightly from the best-fit line through a set of detected points. Alternatively or additionally, discretization may also produce multiple lines that are close to each other. Data association algorithms may include optimizing a set of best-fit lines by eliminating lines with poor fits. The steps for optimization may include selecting a line from the set and plotting it in image space, identifying all points within a predefined distance of the line, performing simple linear regression to define the line that best fits these points, and removing other lines within a certain distance in the Hough transform space. These steps may be repeated any number of times as needed.

[0186] Figure 14E Indicated in the above reference Figure 14D The discussed line optimization step is followed by the Hough transform image 1440. In the Hough transform image 1440, cluster 1444 corresponds to the lines present after the optimization step. One or more loops of line optimization can be performed iteratively as needed.

[0187] In some embodiments, associating each identified reference point in the image with a point in the 2D image of the detected point may include inverse Hough transformation, i.e., mapping the points from a polar coordinate image space (e.g., Hough transform space) to a regular grid (e.g., Cartesian coordinate image space). Doing so allows for the identification of missed points as well as points not included on the grid. Inverse Hough transformation may include the step of identifying groups of parallel lines by identifying groups of all Hough points with similar theta(θ) values, and the step of using a clustering algorithm (such as k-means clustering) to identify groups of lines with similar theta(θ) values. Figure 14F Examples consistent with the disclosed embodiments are illustrated. Figure 14E The example k-means clustering representation 1450 for optimized lines is shown below. It should be understood that other clustering algorithms may also be used. In representation 1450, clusters of lines with similar slope (or theta) values ​​can be formed, and the distribution of points in representation 1450 indicates the range of slopes of the lines within the clusters. Thus, representation 1450 illustrates four groups of substantially parallel lines with one-dimensional (θ) clustering. For example, the substantially parallel lines of the first group could be represented as having a slope in the range of 0 to 1 radian, the substantially parallel lines of the second group could be represented as having a slope in the range of 1 to 2 radians, the substantially parallel lines of the third group could be represented as having a slope in the range of 2 to 2.5 radians, and the substantially parallel lines of the fourth group could be represented as having a slope in the range of 2.5 to 3 radians.

[0188] From representation 1450, two points from the group with the most points and two lines with different theta values ​​can be selected. The intersection of four lines in image space can be used to generate four points forming a rectangle, such as... Figure 14GThe representation is shown in 1460. The vertices of the rectangle can be labeled 0, 1, 2, and 3 (counterclockwise from the top right vertex to the bottom right vertex). In some embodiments, a homography matrix can be computed that shifts points in image space to an integer grid, preferably a square grid. In the context of this disclosure, homography is a transformation that occurs between two planes. In other words, homography is a mapping between two planar projections of an image using a transformation matrix that allows shifting one view to another by multiplying the homography matrix by points in the view to find their corresponding positions in another view of the same scene. When it comes to this disclosure, at least four points may be required to compute the homography, as the homography matrix has eight free variables (each point may contain x and y, for a total of eight equations).

[0189] After calculating homography, the calculated homography can be used to map all points to an integer grid. In some embodiments, the mapped points can be rescaled such that the minimum and maximum values ​​are located at... Figure 14H At the edge of the integer grid 1470 shown.

[0190] One of several ways to determine a successful mapping using an association algorithm is to determine whether each point in image 1460 is mapped to a separate, discrete location on a square integer grid 1470. Each reference location on the square integer grid (e.g., integer grid 1470) can be labeled or numbered using reference characters (e.g., numbers, alphanumeric characters, letters, or other suitable characters) based on a predefined sequence. The mapping from image 1460 to points on integer grid 1470 can indicate the distance or "offset" between the reference location on integer grid 1470 and the mapped point. The offset can be represented in arbitrary units or can indicate the actual offset distance between the reference location and the mapped point.

[0191] In some embodiments, the association between the identified reference point in the image and the 3D position of the reference point may further include excluding false detections and out-of-range detections. False detection may include detecting light sources that do not conform to or are not recognized as valid light sources, such as, but not limited to, reflections from objects, temporary light sources with similar characteristics, etc. excluding false detections may include discarding points that are more than a predetermined threshold offset distance from the reference position. In some embodiments, the predetermined threshold offset distance may be an absolute integer value, a fraction, a percentage, or other number representing the distance between two adjacent reference positions.

[0192] Additionally, in some embodiments, data associations that generate mapped integer grids (e.g., with mapped integer grid 1470) can be discarded based on the number of identified false detections. For example, if the number of false detections exceeds a predetermined threshold, data associations can be discarded, resulting in no associations at all, thus making the data association algorithm a reliable data association algorithm. Figure 14J The detected image 1480 includes annotations of the detected light sources based on data association.

[0193] d. Point tracking algorithm

[0194] Now for reference Figure 15A The illustration shows an exemplary image consistent with the disclosed embodiments, demonstrating the shift between a frame point and its corresponding newly predicted location using homography for point tracking. Point tracking can be used to generate associations for a new set of detected points. One of several ways to perform point tracking may include propagating existing associations from one image frame to subsequent image frames based on a previous set of associated points and corresponding pose estimates. In some embodiments, point tracking may include, but is not limited to, extracting corresponding features between two sets of frames, using the corresponding features to compute homography, applying the homography to previously known associations, and using nearest neighbor type search to map associations to new detections.

[0195] In some embodiments, point tracking may include extracting features or unique features from the previous frame group and identifying corresponding features in the current frame group. In this context, the previous frame group and the current frame group may refer to the (n-1)th frame group and the (n)th frame group, where n is an integer. When the aircraft is at position (p-1) at a given time (t-1), the previous frame group (the (n-1)th frame group) may include multiple pixels of an image captured by a camera (e.g., a camera mounted on the aircraft), and when the aircraft is at position (p0) at a given time (t0), the current frame group (n0) may include multiple pixels of an image captured by a camera. As used herein, the current frame group refers to the frame group immediately following the previous frame group, such that there is no frame group between them.

[0196] In some embodiments, two sets of features extracted from two frame groups can be used to determine a homography matrix, which can be configured to transform any point in one image to a corresponding point in the other image. The determined homography can be configured to shift points from the previous frame group to predicted positions in subsequent frame groups.

[0197] In some embodiments, point tracking can be implemented as locally associated tracking by applying determined homography to previously associated detection points in a previous frame group. Alternatively or additionally, point tracking can be determined as pose-based tracking. Exemplary pose-based tracking techniques may include using previously known position estimates, projecting known reference points into image frames using pose estimates, and applying homography to the projected points as a prediction of where these points will be located in the current frame.

[0198] In some embodiments, such as Figure 15B As shown, the nearest neighbor search step can be used to find the correspondence between the detected points from the detection step and the predicted points from the tracking step, so that an associated identification can be found for each detection. Image 1520 shows an exemplary image generated consistent with the disclosed embodiments by using nearest neighbor-based search to identify corresponding detections of known associations to the tracking.

[0199] In the context of this disclosure and in optical signal processing, temporal filtering refers to isolating the frequency components of a time series of an image into a specific frequency band or range. Filters used for temporal filtering can have any canonical form, such as finite impulse response or infinite impulse response. Spatial filtering refers to the process of altering the properties of an optical image by selectively removing specific spatial frequencies that constitute an object. In spatial filtering techniques, the Fourier transform of the input function can be manipulated by a filter. Spatial filters can be convolutional filters (where the kernel moves across the image) or some other form of spatially oriented filter. As an example, Gaussian blur can be applied to an image to remove high spatial frequencies.

[0200] As previously described, the Active Reference Light Pattern Positioning (AFLPL) method (also referred to herein as Precision Landing and Takeoff (PLaTO)) offers significant advantages for positioning eVTOL aircraft during the approach and landing phases when GPS is challenged, impaired, or completely unavailable. However, to best utilize AFLPL, it is necessary to address and develop an understanding of its potential limitations and methods for mitigating these limitations. This paper provides a brief description of the limitations and mitigation strategies.

[0201] i. Geometric constraints —The primary geometric limitation of AFLPL is the decrease in accuracy as the range to the illumination distribution combination increases. For a fixed-focal-length camera, the maximum useful horizontal range is approximately 100 times the size of the illumination distribution combination. A second geometric limitation is the constraint imposed by the camera's field of view (FOV) on the approach trajectory to the landing site. This limitation can be mitigated by using multiple cameras with different viewing orientations and FOVs. Given the small size, light weight, and low power consumption of the cameras, two or three cameras can be used with minimal impact on aircraft cost and performance.

[0202] ii. Detection and SNR Limitations —For visible and near-infrared systems, the intensity of background light from the sun and other artificial sources can be significantly higher than that of the proposed distributed illumination combination. The AFLPL light source needs sufficient brightness, modulation capability, and a combination of different wavelengths to enable detection from cluttered and backlit, but primarily static, backgrounds. A high proportion of GPS denied landing scenarios can be achieved by employing a combination of visible, near-infrared, and long-wavelength infrared sources and detectors.

[0203] iii. Data association The perspective n-point algorithm requires that the positions of a distributed combination of light sources be known, and that point source detection in the camera's focal plane be associated with each individual light source in the distribution. This data association problem becomes further complicated as point sources leave the field of view due to camera movement or appearing / closing. This problem is well-studied and fundamental to many computer vision applications, and many standard algorithms, such as RANSAC, have been developed to solve the association problem. It is important to note that association with a reference point consisting of active point sources is significantly less complex than association with visual features from unstructured, passive imagery. In the AFLPL method, we are dealing with a small number of distinct point sources with high detection probabilities under favorable placement. Additionally, association can be facilitated by temporally modulating each reference source using a unique code, or by spectral matching of light emitted by each source at different wavelengths.

[0204] The feasibility of the AFLPL method has been verified through simulation studies and related analyses exploring the accuracy of localization solutions generated by various perspective n-point (PnP) algorithms under various operating conditions, including the number of distributed light sources, the physical size of the distributed combination, and the imaging quality of the camera system. Two methods were used to model the reference point-based localization of eVTOL aircraft. Equations for modeling a pinhole camera with a 4K imager (3840x2160 resolution) and a 90-degree field-of-view lens were implemented. Equations were also implemented with… Figure 2 The landing profile shown corresponds to the aircraft trajectory and the lighting distribution configuration. These models are used to examine the sensitivity of positioning accuracy to variations in the number of light sources in the distribution, the size of the distribution, and camera imaging errors, as shown in the reference... Figure 10A , Figure 10B , Figure 11A , Figure 11B , Figure 12A and Figure 12B The subject of discussion.

[0205] Now for reference Figure 10A and Figure 10BThis illustrates simulated data plots, consistent with some embodiments of this disclosure, showing the effects of the size of the distribution combination on positioning accuracy in the horizontal and vertical directions, respectively. To investigate the effect of the number of distribution combination points on positioning accuracy, the eVTOL camera is positioned along... Figure 2 The landing profile is defined at discrete locations. At each location, an embodiment of the proposed system is used to calculate the camera's position and altitude. To generate... Figure 10A and Figure 10B The plotting process involves varying the number of light sources. For each number of light sources within each range, the light sources are uniformly distributed within a rectangular volume of 40m x 40m x 20m. 1000 randomly positioned light configurations are used to calculate the mean positioning error. The camera position is compared to the actual camera position, and the horizontal and vertical components of the positioning error are calculated and averaged over 1000 runs to generate the plot. The positions of light spots in the image frame are rounded to the nearest pixel. As shown, the positioning error decreases with the number of light sources used for imaging. If at least 15 sources are used, the positioning error is less than 10m within a 1200m range and decreases to less than 5m within a 500m range. As the aircraft approaches the landing target, the positioning error decreases to the centimeter horizontal range.

[0206] Figure 11A and Figure 11B Simulated data plots illustrating the effect of the size of a light source distribution combination on positioning accuracy in the horizontal and vertical directions are shown, consistent with some embodiments of this disclosure. The simulated data plots are generated by varying the horizontal size of the light source distribution combination while keeping the number of light sources and the vertical size of the distribution combination constant. The vertical dimension of the distribution combination is 20m, and 20 light sources are used. For each distribution combination size at each camera position, the 20 light sources are uniformly distributed within the distribution combination volume. One thousand randomly positioned light configurations are used to calculate the mean positioning error. The camera positions are compared to actual camera positions, and the horizontal and vertical components of the positioning error are calculated and averaged over 1000 runs to generate the plot. As shown, the positioning error decreases with the size of the distribution combination. If a horizontal baseline of at least 30m is used, the positioning error is less than 10m in the range of 1200m and decreases to less than 1m in the range of 500m. As the aircraft approaches the landing target, the positioning error decreases to the centimeter horizontal range.

[0207] Figure 12A and Figure 12B Simulated data plots illustrating the effect of centroid error on the robustness of the algorithm in the horizontal and vertical directions are shown, consistent with some embodiments of this disclosure. Small errors can be introduced by the optics and image sensor when the light spot is imaged by a camera. Figure 12A and Figure 12BThe plots shown depict the impact of these errors on positioning accuracy. These plots were generated by rounding the floating-point pixel positions of the light sources (their true centroid positions on the image plane) to the nearest integer (average error of 0.3 pixels) and then adding different amounts of normally distributed error with a mean of 0, 1, 2, or 3 pixels. This was done while keeping the number of light sources (20) and the size of the distribution combination (40x40x20m) constant in each range. 1000 randomly positioned light configurations with added imaging errors were used to calculate the mean positioning error. The camera position was compared to the true camera position, and the horizontal and vertical components of the positioning error were calculated and averaged over 1000 runs to generate the plots. As shown, the positioning error increases with the error in the centroid position. If a horizontal baseline of at least 30m is used, the positioning error is less than 10m in the 1200m range and decreases to less than 1m in the 500m range. As the aircraft approaches the landing target, the positioning error decreases to the centimeter horizontal range. Errors of 0.3 pixels and 1.3 pixels satisfy the horizontal error requirement.

[0208] Now for reference Figure 12C and Figure 12D This illustration shows simulated data plots, consistent with some embodiments of this disclosure, depicting horizontal and vertical positioning errors during the approach of a simulated aircraft. The algorithm was tested using an eVTOL simulation environment. The positioning error increases during the transition from the forward-facing camera to the nadir camera. We anticipate that this transition error can be addressed by running the nadir positioning algorithm before the transition and fusing the results with those from the forward-facing camera, rather than by making an abrupt switch between the two.

[0209] Light distribution combination design

[0210] In some embodiments, the accuracy of data association may be affected by distortion of the light source pattern due to viewing angle. For example, the pattern of a light source viewed from directly above the landing surface may have a different shape than when viewed from a shallow viewing angle. This difference in the shape of the light source pattern based on viewing angle is called "viewpoint transformation." Under viewpoint transformation, many identifying features may be lost. Therefore, it may be necessary to design a reference pattern for active light sources and develop corresponding data association algorithms to identify reference pattern points in images captured by cameras mounted on airborne aircraft. Further, a software pipeline is needed to generate highly accurate attitude estimation using multiple batches of aerial images with a reference pattern of active light sources laid out on the ground. In some situations (such as dense urban environments or severe weather conditions), obtaining an estimate of the aircraft's position or orientation can be challenging. In such situations, performing data association and subsequent attitude estimation without prior knowledge or estimation of the aircraft's position and / or orientation can further exacerbate the problem. Therefore, there is a further need to design benchmark patterns and data association algorithms that can perform data association and subsequent pose estimation on isolated batches of benchmark patterns based on light sources captured from arbitrary viewpoints, without the need for additional sensors or measurement mechanisms. The proposed benchmark pattern design and data association algorithms are designed to address one or more of the challenges identified herein. In some embodiments of this disclosure, the proposed benchmark pattern design and data association algorithms can also allow the pose estimation pipeline to continue operating despite obstacles or malfunctions that impede observation of some distributed combinations of light sources in the benchmark pattern or when additional ambient light sources are visible in the camera image.

[0211] Many geometric properties existing in three-dimensional space are inconsistent when mapped to two-dimensional space under projection transformations. For example, length, area, centroid, and parallelism in a camera image all depend on the camera's position and orientation relative to the image subject. However, the cross ratio is constant regardless of the viewpoint from which a camera image is taken and serves as a primary principle in benchmark distribution combination design. In some embodiments, benchmark distribution combination design utilizes the projection-invariant property of the cross ratio. The cross ratio is a view-invariant property that can be used for accurate data association. As used herein, "cross ratio" refers to the ratio of four values, each calculated based on a unique subgroup of features, where the product of two of these values ​​is divided by the product of the other two. For example, the linear cross ratio is the ratio of the length ratios between collinear points, and the angular cross ratio is the ratio of the angle ratios between intersecting lines.

[0212] Now for reference Figure 16A This example illustrates a line with four points A, B, C, and D located at different distances from each other. The linear cross ratio can be calculated as follows:

[0213]

[0214] Cross ratios are constant values ​​for a line segment, independent of the viewing angle. The choice of line segment used in this calculation may produce different cross ratios, but they remain the same regardless of the viewing angle. Cross ratios are view-invariant. For example, for a given set of four points on a line, a total of six cross ratios can be calculated. These six cross ratio values ​​can be used to calculate a single invariant that is unique to the spatial distribution of the four points on the line.

[0215] Now for reference Figure 16B This illustrates an exemplary star-shaped pattern of light source distribution consistent with the disclosed embodiments. The light sources can be positioned along line segments AA, BB, CC, and DD arranged in a star configuration, such that different angles can be formed through the intersecting lines. In such a configuration, the angle intersection ratio can be calculated as follows:

[0216]

[0217] Or it can be calculated as:

[0218]

[0219] In some embodiments, the landing surface may include a distribution of light sources. Multiple light sources may be arranged on the landing surface for an airborne vehicle, the arrangement defining a set of intersecting virtual lines, with the light sources positioned along each virtual line, wherein the distance between adjacent light sources on each virtual line is non-uniform, such as... Figure 17A As shown, this figure illustrates a viewing angle-invariant active light source distribution combination design 1700 consistent with embodiments of this disclosure. Figure 17A As shown, the geometric features of the distribution combination design 1700 include four lines (AOA, OB, OC, and OD) that intersect at a single point O and are labeled lines 1702, 1704, 1706, and 1708. Each of lines 1704, 1706, and 1708 can be a virtual line connecting five light sources B1-B5, C1-C5, and D1-D5, respectively. Line 1702 can connect ten light sources A1-A10. Different lines in the distribution combination, and the light sources belonging to these lines, can be detected by Random Sample Consensus (RANSAC) or other methods. Given this set of lines, there can be two unique angular cross ratios, one calculated using the sequential lines ABCD, and the other calculated using the sequential lines BCDA. Because line AOA can be distinguished from other lines by the number of light sources (i.e., ten light sources compared to the five light sources on OB, OC, and OD), the correct identification of each of the other lines can be determined by calculating the cross ratio obtained using three lines in clockwise or counterclockwise order from line OA.

[0220] To correlate light detected in a camera image with a specific light in a baseline distribution combination (e.g., distribution combination design 1700), a data association algorithm (such as mesh association, ICP, plate spline, or point tracking, or a combination thereof) can be configured to fit a line to a cloud of detected points in the camera image, identify specific lines in the distribution combination, and identify specific points within each identified line. The data association algorithm may include one or more of the following steps.

[0221] In some embodiments, the data association algorithm may include the step of using RANSAC to determine a set of four lines that are the best fit among the point clouds detected in the camera image, having a single intersection point and the correct angular cross ratio. Figure 18A and Figure 18B Examples of potential RANSAC sampling methods, such as random pair sampling and k-nearest neighbor (k-NN) sampling, are given respectively.

[0222] In some embodiments, using the RANSAC sampling method may include the following steps: (a) sampling four pairs of points from the point cloud of detected light and drawing a line through each pair to form four lines. The sampling of point pairs may be random, or pairs may be drawn from a set of k nearest neighbors to increase the probability of finding point pairs on the same lines in a distribution combination. The type of sampling used for RANSAC (random pairing or k-NN sampling) may depend on how efficiently a set of k nearest neighbors can be computed by comparing randomly sampled point pairs using a K-dimensional (KD) tree. (b) computing the intersection points represented by the least-squares solution to the system of equations describing the set of four lines sampled in step (a). If the error associated with the solution is higher than a predetermined threshold, i.e., the point that best fits the set of four lines is far from each of these lines, then the set of lines does not have sufficiently close intersection points, and the algorithm may return to step (a). Step (c) involves calculating the angular cross ratio of the set of lines when determining that they intersect at a single point, by starting with any line and including the other three lines in a clockwise or counterclockwise order. As previously described, only two possible angular cross ratios may exist within the distribution combination design 1700. Therefore, if the calculated cross ratio falls outside a predetermined error threshold for the expected cross ratio, the algorithm may return to step (a). In step (d), when determining the set of four lines intersecting at a single point and matching the expected angular cross ratio of the distribution combination, the number of points as interior points to the set of lines is determined, such as... Figure 18C As shown. If the number of interior points is greater than the current best interior point count, then the current set of lines can become the best set of lines and the current interior point count can become the new best interior point count. Step (d) may further include assigning each interior point to the line in the image that is closest to it.

[0223] In some embodiments, the data association algorithm may further include the step of determining the identifier of each line in the set of lines obtained during the determination of the best fit and correct angle cross ratio of the four lines using the known angle cross ratio of the distribution combination design. The step of determining the identifier of each of these lines in the set of lines using the known angle cross ratio of the distribution combination design may include line fitting for each of the four lines in the best line set using the interior points assigned to each line in step (d) above. Based on the line fitting, the line with the largest number of interior points can be identified as the baseline (AOA) of the distribution combination design 1700, and the angle cross ratio obtained using lines ordered clockwise or counterclockwise from the distribution combination baseline is calculated. This cross ratio will be matched against one of two possible known distribution combination angle cross ratios, and therefore, the group of lines used to calculate this cross ratio should be identified as lines with a pre-calculated cross ratio for matching, such as... Figure 18D As shown.

[0224] In some embodiments, the data association algorithm may further include the steps of voting for each point in line 1900 using a known linear cross ratio designed for the distribution combination of each line in the distribution combination and determining the identifier of each point, such as... Figure 19 As shown. Line 1900 can connect at least four light sources. In other words, line 1900 can contain at least four points. In some embodiments, line 1900 can contain five or more points, six or more points, seven or more points, eight or more points, nine or more points, ten or more points, or any suitable number of points. Figure 19 As shown, line 1900 connects six points (points 1-6). For each line, the points can be sorted in order of distance from the distribution combination vertex or intercept (e.g., point 1 in line 1900). For each subgroup of four points on line 1900, the linear cross ratio can be calculated and compared with a pre-calculated known linear cross ratio of the distribution combination line. Figure 19 Table 1910 illustrates known cross ratios, and Table 1920 illustrates calculated cross ratios for subgroups of four points. Table 1910, with known cross ratios, may include the cross ratios for subgroups of four points on each line 1702, 1704, 1706, or 1708. Table 1920, with estimated or calculated cross ratios, may include the cross ratios for subgroups of four points on line 1900.

[0225] When comparing the known cross ratio with the calculated cross ratio, a voting table 1930 can be generated. The voting table is a grid of rows and columns that includes vote counts. If the calculated linear cross ratio (e.g., the cross ratio of line 1900) is within a predetermined threshold of any of the known cross ratios of lines 1702, 1704, 1706, or 1708, then each point used to calculate the cross ratio receives a vote for its corresponding point in the group used for the known cross ratio.

[0226] In some embodiments, voting strength can be calculated for each benchmark point. As used herein, “voting strength” is the ratio of the vote count for the candidate point with the highest number of votes to the candidate point with the second-highest number of votes. If the voting strength for a benchmark point is greater than a predetermined threshold for voting strength, the benchmark point can be assigned the identifier of the candidate point that received the highest number of votes. As an example, the known cross ratio of line B1B2B3B4 is 1.35, and the calculated cross ratio of line 1900, which is closest in value to connecting points 1, 3, 4, and 5 (labeled as line 1345 in Table 1920), is 1.36. If the difference between the known cross ratio and the calculated cross ratio (1.35 - 1.36 = -0.1) is within a predetermined threshold difference, each point used to calculate the cross ratio (e.g., points 1, 3, 4, and 5) can receive votes for its corresponding point in the group used for the known cross ratio. A corresponding point refers to the position of a point on a line relative to a vertex. Point B1 on line 1704 corresponds to point 1 on line 1900, point B2 on line 1704 corresponds to point 2 on line 1900, point B3 on line 1704 corresponds to point 3 on line 1900, point B4 on line 1704 corresponds to point 4 on line 1900, and point B5 on line 1704 corresponds to point 5 on line 1900.

[0227] Return to reference Figure 17B This illustrates an exemplary distribution and combination design 1750 of light sources, where an equal number of light sources (e.g., light sources 1751, 1753, and 1755) can be arranged on each virtual line 1752, 1754, 1756, 1758, and 1760, which intersect at point 1780. Multiple light sources can be arranged at a landing surface for an aerial vehicle, the arrangement defining a set of intersecting virtual lines, with light sources arranged on each virtual line, wherein the distance between adjacent light sources on each virtual line is non-uniform, such as... Figure 17B As shown, this diagram illustrates a viewing angle-invariant active light source distribution combination design 1750 consistent with embodiments of this disclosure. It should be understood that although only three light sources are marked on line 1752, more light sources may exist on each line (indicated by small circles). Figure 17B(Unmarked in the text). The linear cross ratio for each virtual line is independent of the viewing angle. It should be understood that although each virtual line is illustrated as containing an equal number of light sources, other combinations and arrangements are possible. For example, five light sources can be arranged on one virtual line, and six light sources can be arranged on adjacent virtual lines.

[0228] Each of lines 1752, 1754, 1756, 1758, and 1760 can be a virtual line connecting light sources. For example, virtual line 1752 can connect at least light sources 1751, 1753, and 1755. All virtual lines can intersect at a virtual intersection point 1780. In this case, the cross ratio of the angles formed by the intersecting lines remains unchanged in the projection. Different lines in the distribution combination and the light sources belonging to these lines can be detected by RANSAC or other methods. Intersection points can be determined by the intersection of multiple lines. The cross ratios for (A, B, C, D) and (B, C, D, E) can be used to determine the order of the angular distribution of the lines. The ordering of points along individual lines can also be calculated using linear cross ratios, as previously referenced. Figure 17A As described.

[0229] In some embodiments, each active light source (e.g., Figure 3 The light source 350 can be configured to perform self-identification via a modulation scheme. For example, the light source can be modulated to transmit data. In some embodiments, the data may include an encoded authentication or identification signal, which, when received by an optical detector or camera 311, can be used to identify the location of the active light source, or to identify the landing surface, etc.

[0230] Rapidly deployable distribution combinations of light

[0231] In some embodiments, the landing surface may include a portable landing surface. For example... Figure 43A and Figure 43B As shown, a portable vertical takeoff and landing (VTOL) airport may include a retractable landing mat, fabric, or tarpaulin, or a rapidly deployable landing surface 4310 including multiple light sources 4325. In some embodiments, the rapidly deployable landing surface 4310 may include a distributed combination of IR light sources. In some embodiments, the rapidly deployable landing surface 4310 may include a combination of point light sources and line light sources (see reference). Figure 30 (as described in Figure 31). This can be particularly useful in situations requiring landing at non-cooperative locations with restricted or temporary landing infrastructure, such as military operations, firefighting operations, disaster relief operations, medical aid distribution operations, etc. In some embodiments, the portable vertical takeoff and landing (VTOL) airport may include battery-powered active light sources incorporated therein, such that they can be remotely activated, deactivated, or modulated.

[0232] In some embodiments, for example, the rapidly deployable landing surface 4310 may include a distribution of light sources placed on a rollable mat that can be carried in a backpack. The rapidly deployable landing surface 4310 can be deployed at a specially designated landing site in emergency situations, adverse conditions, rescue operations, etc. Figure 43B In this context, the rapidly deployable landing surface 4340 can be a flexible mesh or mesh structure that includes a light source 4325 woven or clamped therein.

[0233] In some embodiments, estimating the aircraft's attitude based on images captured from cameras mounted on the aircraft may include transmitting a distributed configuration of light sources 4325 arranged on or within a rapidly deployable landing surface 4310 or 4340 to an onboard processor associated with the aircraft. The distributed configuration can be determined by automatically ranging between the light sources using ultra-wideband (UWB) signals to calibrate the relative positions of the light sources. As used herein, ultra-wideband signals can be used to transmit information across a wide bandwidth (>500 MHz). This allows for the transmission of large amounts of signal energy without interfering with conventional narrowband and carrier transmissions in the same frequency band.

[0234] In some embodiments, the light source 4325 may be woven into a rapidly deployable landing surface 4340 (e.g., a mesh or flexible mesh structure) that can be deployed at a specially arranged landing site. The landing site may be uneven, contoured, or non-planar, and the rapidly deployable landing surface 4310 or 4340 may be conformable to the landing site. The arrangement of the light source 4325 may be located at a known location on the rapidly deployable landing surface 4340, so the approximate configuration or location of the light distribution combination may be known. In some embodiments, multiple top-view images, such as those obtained from the aircraft, are used to calibrate the light position. Alternatively or additionally, automatic ranging between the light sources using UWB signals may be performed to calibrate the relative positions of the light sources, thereby learning the distribution combination configuration.

[0235] Automatic generation of light distribution combination patterns

[0236] In some embodiments, a pattern of combined light distributions can be automatically generated to maximize one or more properties of the distribution combination. For example, using predefined metrics, the distribution combination can be designed to maximize the ability to distinguish between lights used for detection and data association purposes, thereby effectively maximizing the correctness and robustness of the data association process. One possible metric is the variance of the linear or angular cross ratios between multiple lines.

[0237] Incorporating uncertainty into light position estimation

[0238] As previously described, the data association step involves associating the position of a light source in the image plane with the physical light position on the ground. In some cases, the association of light sources between the image and the ground can be challenging because there may be uncertainties in the physical light position due to measurement errors, or uncertainties in the light image position (on the image plane) due to imaging errors. In some embodiments, incorporating uncertainty information from these position measurements into the data association algorithm makes it possible to calculate the probability of a correct match, thereby allowing the use of confidence metrics in the data association algorithm and facilitating the decision-making process.

[0239] Posture recovery / posture estimation

[0240] In some embodiments, determining the position and orientation of an aerial vehicle may include detecting at least one light source in an image, determining which of the detected light sources is one of at least one light source arranged in a predetermined pattern, and determining the position and orientation of the aerial vehicle based on the determination of which of the detected light sources is one of at least one light source arranged in a predetermined pattern. In a preferred embodiment, a localization algorithm (e.g., PnP) may need to detect at least four light sources in the image plane (and determine their pixel positions), and those light source pixel positions may be correctly associated with their corresponding physical light sources arranged on the ground in a predetermined pattern (and known positions) to produce an estimate of the attitude of the camera and / or the aerial vehicle (e.g., camera 311, aerial vehicle 310).

[0241] In another preferred embodiment, the localization algorithm may need to detect at least five light sources in the image plane, and the pixel locations of those light sources are correctly associated with their corresponding physical light sources arranged in a predetermined pattern. Generally, the accuracy of camera and / or aircraft pose estimation increases with the number of light sources detected in the image plane and correctly associated with their corresponding physical light source locations in a predetermined pattern, as previously referenced. Figure 10A and Figure 10B The subject of discussion.

[0242] In some embodiments, determining the position and orientation of an airborne vehicle may include detecting three light sources in an image plane and correctly associating these three light sources with their corresponding physical light source positions in a pre-determined pattern. Standard triangulation methods can be used to identify two potential locations using three points. In the case of landing surfaces and vertical takeoff and landing (VTOL) airports, one of the two potential solutions may be eliminated because it would be below the landing surface, potentially narrowing the solution down to a single location.

[0243] In some embodiments, if orientation information such as yaw, pitch, and roll, or a 3D gravity vector is known, determining the position of an airborne vehicle may include detecting two light sources in an image plane and correctly associating those two light sources with their corresponding physical light source positions in a pre-determined pattern. In some embodiments, if the altitude information of the aircraft is known, determining the position and / or orientation of the airborne vehicle may include detecting a light source in an image plane and correctly associating that light source with its corresponding physical light source position in a pre-determined pattern.

[0244] In some embodiments, one or more light sources in a distributed composite design (e.g., design 1700 or 175) may be unobservable and inaccurately identifiable due to adverse conditions such as, but not limited to, severe weather, dense urban environments, poor light transmittance, additional ambient light, etc. In such scenarios, it may be necessary to still be able to perform data association and attitude estimation for airborne vehicles approaching or taking off from the landing surface. The proposed attitude estimation algorithm and system address some of the challenges mentioned above.

[0245] Now for reference Figure 20A This is a process flowchart illustrating an exemplary method 2000 for estimating the attitude of an airborne vehicle, consistent with embodiments of this disclosure. Method 200 can be used in precision landing and takeoff systems as well as data communication systems (e.g., Figure 3 The pose estimation algorithm is used in the PLATO system 300. For example, the processor (e.g., Figure 3 The processor 312 can be configured to execute the pose estimation algorithm and can be programmed to implement the pose estimation algorithm steps. It should be understood that the steps performed in method 2000 can be reordered, added, removed, or edited as needed. Computer implementation of the pose estimation algorithm enables system 300 (including cameras, light detectors, microprocessors, memory, or storage devices mounted on an aircraft) to perform the following steps of pose estimation method 2000.

[0246] In step 2010, a camera (e.g., camera 311 of system 300) is configured to capture a continuous stream of images. The camera application programming interface (API) may be configured to receive the continuous camera image stream at a frame rate of 100 frames per second (fps). The camera API may be further configured to transmit three consecutive frames of the camera images to a photodetector. In some embodiments, the distributed combination design of the reference light may flicker at a certain frequency, such that each distributed combination light is turned on in at least one of the three frames and turned off in at least one frame.

[0247] In step 2020, the light detector is configured to generate an output including the pixel locations of the detected light source based on the frames received from the camera in step 2010. Generating the output includes constructing maximum and minimum images consisting of the corresponding maximum and minimum grayscale intensities at each pixel across the three images, and subtracting the minimum image from the maximum image to remove any ambient background light. Due to the flickering of the light on and off, the distribution combination forms a high contrast with the background and is easily detected in the minimum-maximum image. Subtracting the minimum image from the maximum image includes subtracting the intensity of each pixel in the minimum image from the intensity of each corresponding pixel in the maximum image. In some embodiments, flickering may include adjusting the light intensity between the "minimum" intensity and the "maximum" intensity. The minimum intensity may include zero (light off) or any intensity below the maximum intensity of the light source, such that the difference between the maximum and minimum values ​​can be distinguished by the detector.

[0248] In step 2030, based on the pixel positions of the detected light in the minimum-maximum image received from the photodetector, a data association algorithm is used to identify which distribution combination of light corresponds to the detected light source. The data association algorithm performed in this step can be a previously referenced algorithm. Figures 18A to 18D and Figure 19 The algorithm described, or other suitable data association algorithm. In some embodiments, the determination of a set of four best-fitting lines with a single intersection point and the correct angular cross ratio in a point cloud detected in a camera image using the RANSAC sampling method can be performed in parallel to improve efficiency and reduce the time required for RANSAC to determine a set of lines with high confidence.

[0249] In step 2040, the pose estimator API is configured to receive the pixel positions of the identified distributed composite points in the camera image and perform an iterative perspective n-point (PnP) algorithm to generate a pose estimate of the camera in the distributed composite coordinate system.

[0250] The pose estimation pipeline is configured to run end-to-end in less than 30 milliseconds (i.e., at a frequency >33 Hz), allowing for the generation of real-time pose measurements while acquiring camera images at 100 fps. During real-time operation, the camera API can be configured to run separately from the data association and pose estimation APIs, enabling the acquisition of the next batch of images while performing light detection, data association, and pose estimation algorithms on the current set of images, thus making the pipeline more efficient.

[0251] Example—Simulation and hardware testing of a pipeline for benchmark distribution combination, data association, and pose estimation.

[0252] The aforementioned baseline distribution combination, data association algorithm, and attitude estimation pipeline have been tested in simulation and on hardware. Hardware results were obtained by running the attitude estimation pipeline on an Intel NUC mounted on a hexacopter that uses a camera equipped with an infrared (IR) filter to image IR light sources arranged on the ground in the shape of the baseline distribution combination. Several types of trajectories were flown around the distribution combination at distances up to 200 meters to test the robustness of the data association algorithm and the attitude estimation pipeline to the camera's viewpoint. Utilizing data from... Figure 20B The results of one of the test trajectories shown will be compared with the position estimate calculated by the attitude pipeline and the baseline true value measurement provided by the real-time motion technology (RTK) GPS sensor mounted on the hexacoach. (As shown in...) Figure 21 , Figure 22 and Figure 23 As can be seen, the accuracy of the pose pipeline is within a few percent of the distance between the camera and the distribution combination, which is also typical for trajectories in other tests.

[0253] Figure 21 Examples are given for Figure 20B The flight trajectory shown is an altitude estimate 2100A plotted using RTK GPS ground truth data, along with the corresponding error 2100B. Figure 22 Examples are given for Figure 20B The flight trajectory is shown as a function of the baseline truth value in plot 2200A and the corresponding error plot 2200B. Figure 23 Examples are given for Figure 20B The flight trajectory is shown as an estimated plot 2300A as a function of the baseline truth value, and the corresponding error plot 2300B.

[0254] Now for reference Figure 24A This illustrates an exemplary random distribution pattern of active light sources on the ground, consistent with some of the disclosed embodiments. In some embodiments, the distribution pattern 2400 may include light sources randomly arranged on the ground. The randomly placed light sources may be IR light sources. In scenarios of automated landing (such as the landing of eVTOL airborne vehicles), dense urban environments or impaired visibility due to inclement weather and low light conditions can pose significant challenges to safe landing and takeoff operations.

[0255] like Figure 24AAs shown, the random point distribution combination design 2400 can be based on a reference marker called a Random Point Marker (RDM), which implements the Local Likelihood Permutation Hash (LLAH) algorithm to identify randomly placed points. Due to the clustering nature of the method used to compute the descriptors, the LLAH algorithm is a naturally robust algorithm against occlusion. In the context of eVTOL landing and takeoff operations in dense environments, occlusion can occur due to failed light, covered light, or some light temporarily disappearing from the line of sight as the aircraft approaches the landing surface. Furthermore, the distribution combination placement on the landing field may be feasible because the points are randomly placed and are not limited to square tags, rectangular tags, or any specific shape.

[0256] In some embodiments, the LLAH random point identification method may include steps of distribution combination design, keypoint registration, and keypoint retrieval. Designing the distribution combination may include creating a random point marker by generating random x and y coordinates for N points to fit within a selected marker size. In some embodiments, newly generated points that overlap with existing points may be discarded or excluded from consideration. Due to the sensitivity of the cross-ratio equation to point location, the random distribution of points results in naturally unique cross-ratios. While other reference markers may be limited to a certain shape, random point markers can take any shape, as long as the points are arranged on a planar surface.

[0257] In some embodiments, keypoint registration in the LLAH algorithm can benefit from cross-ratios to ensure invariance under viewpoint changes. While affine invariants can be used because they require fewer feature points to compute descriptors, using cross-ratios offers several advantages over affine invariants. For example, in this scenario, low angles near the landing site can cause severe viewpoint distortion of points, and it may be necessary not to assume affine transformations for local point clusters. Furthermore, because the number of reference points (e.g., IR light sources on the ground) may be relatively small, affine invariance may be redundant, or even inefficient in some cases.

[0258] In some embodiments, in order to uniquely identify points using the LLAH algorithm, a descriptor can be computed for each keypoint in the distribution combination. As used herein, the descriptor is a sequence of discretized cross ratios. Figure 24B An exemplary distribution combination including a keypoint p and n nearest neighbors is illustrated. The computational descriptor may include determining the n nearest neighbors for each keypoint p in the distribution combination, and selecting a combination of m points from the n nearest neighbors, such as... Figure 24B As shown. Among the selected m points, a combination of five points can be used to calculate the cross ratio. As implemented in this paper, the keypoint p can be one of the five points used to calculate the cross ratio.

[0259] Figure 24C An example is illustrated using a combination of five points from m = 7 points to calculate the cross ratio to create a discretized sequence of cross ratios. In some embodiments, the cross ratios can be discretized, and the sequence of discretized cross ratios from each combination can be used as a descriptor for a keypoint p. In this method, the bounds for discretization are selected by calculating all cross ratio combinations for the distribution combination, sorting them, and splitting them between a selected number of buckets. The upper and lower bounds of the buckets can be defined by the cross ratios with the lowest and highest values ​​in that bucket. This method of discretizing these cross ratios is used because the values ​​of the cross ratios for a single distribution combination are not represented by a uniform distribution. Typically, there are many cross ratios with low values ​​(less than 20) and very few with high values ​​(>100). Therefore, a finer range may be needed when discretizing the lower values.

[0260] Now for reference Figure 24D This illustrates the discretization of crossover ratios consistent with some disclosed embodiments to create a discretized sequence of crossover ratios. In some embodiments, the sequence of discretized crossover ratios is used as a descriptor instead of crossover ratio values. Figure 24D As shown, two sequences of cross-ratios can be generated (e.g., group 1 and group 2). After discretizing the cross-ratios, groups 1 and 2 have exactly the same cross-ratio values. However, the sequence in which the values ​​appear may be unique. For example, although the discretized cross-ratio (0,2,0,3) for group 1 and the discretized cross-ratio (2,3,0,0) for group 2 are equal, the sequences of cross-ratios in these groups are different. To make the sequences easy to replicate during retrieval, the nearest neighbors of each keypoint are sorted in clockwise order before computing any combinations or descriptors. Each descriptor can have a dimension of mC5, and each point will have nCm descriptors. Figure 24E As shown, a hash index can be calculated from each descriptor, and the keypoint ID and tagger ID can be stored together with the descriptor at the hash index.

[0261] In some embodiments, the LLAH algorithm may further include keypoint retrieval. To perform tagger retrieval using matching, descriptors can be computed for each detected point in the image using the same method as previously described. However, because the orientation of the distribution combination in the camera view may differ from the orientation during the registration step, simply sorting these points clockwise before computing the sequence may not yield accurate results. The first point among the nearest neighbors in the sorted distribution combination may differ from the first point in the actual distribution combination. Therefore, all n clockwise orders will be computed and used for voting. For example, if the original clockwise order of the points is a, b, c, d, e, f, then the actual retrieval must also compute descriptors for b, c, d, e, f, a, etc.

[0262] Descriptors can be used to compute the hash index required to reference the table. Voting is performed on each keypoint ID and tagger ID candidate found in the table. For each tagger candidate, RANSAC is then used to compute homography to confirm a match. In this case, there is only one tagger candidate, so homography can be computed once the votes for each and / or multiple identified keypoints exceed a predetermined threshold. At this point, we have identified our keypoints.

[0263] In some embodiments, all points (e.g., active light sources) may be non-coplanar or non-collinear. In such cases, the area cross ratio (ACR) can be used. For example, using any five arbitrary points, the ACR can be calculated based on the ratio of the areas of the triangles defined by three of those five points, as follows:

[0264]

[0265] Where Z0, Z1, Z2, Z3, and Z4 are any points that form a triangle, such as... Figure 25 As shown.

[0266] As shown above, this equation can be used to determine an example cross ratio value. Similar to the linear cross ratio, six unique area cross ratios can be determined for a given combination of five non-collinear and non-coplanar points. A single invariant value can be calculated from the six (6) area cross ratios.

[0267] In some embodiments, the area cross ratio algorithm can be used for data association. The algorithm includes the following steps:

[0268] 1. Given a set of points to be associated, select 5 unassociated or arbitrary points.

[0269] 2. Calculate the single invariant value mentioned above.

[0270] 3. Find the invariant in the pre-calculated table that best matches the calculated value.

[0271] 4. Follow the associated entries to reach the second pre-calculated table that is unique for that single invariant value.

[0272] 5. Using the cross ratio values ​​calculated in step 2, find a set of values ​​that are the closest matches to determine the order of these points.

[0273] 6. Based on the sorting of the points used in steps 1 and 5, the identifiers of all 5 points can be determined.

[0274] 7. Remove these 5 points from the consideration and repeat step 1.

[0275] 8. If there are fewer than 5 points, the previously identified points can be used to complete a set of 5 points.

[0276] In some embodiments, in table matching techniques for data association, a pre-computed table can be formed by enumerating all possible groups of 5 points in a 3D distribution with known geometry. The j-invariant area cross ratio generated when each of the 5 points in each group is selected as the center point can be calculated. These values ​​form a table where rows correspond to the IDs of the center points and columns correspond to a group of 4 non-center points in the group of 5 points. It should be understood that some cells in the table are empty because a single point cannot be both a center point and a non-center point in the group of 5 points. One method for completing the data association process may include searching the table and matching the j-invariant values ​​calculated based on the observed 2D image points against those pre-computed values ​​in the table. By utilizing the points already matched in the table traversal algorithm, the number of possible j-invariant values ​​in the table for potential matches at a given time step can be reduced. This allows us to efficiently solve for the optimal match in a region-growing approach, where we first find and identify a single group of 5 points and then incrementally add the group we are considering, one point at a time. Furthermore, when determining the threshold for matching the j-invariant value, uncertainty propagation techniques can be used to account for the calibration uncertainty of pixel and distribution combinations.

[0277] In some embodiments, the Hungarian association matrix or the Moncler algorithm can be used to perform data association. In the Hungarian association matrix technique, an exemplary 5x5 matrix of fractions can be used, as shown in Table 1 below.

[0278]

[0279] Table 1. 5x5 matrix of fractions

[0280] A large number, such as 1.0, can be subtracted from all the numbers to convert it into cost, and the result is shown in Table 2 below.

[0281]

[0282] Table 2.

[0283] In the next step, the minimum cost can be subtracted from all rows, as shown in Table 3 below.

[0284]

[0285] Table 3.

[0286] In the next step, the minimum cost can be subtracted from all columns, as shown in Table 4 below.

[0287]

[0288] Table 4.

[0289] In the next step, draw as few rows or columns as possible to connect all the zero values ​​in Table 4. Find the minimum value from the non-zero elements of the matrix shown in Table 5 below.

[0290]

[0291] Table 5.

[0292] In the next step, the minimum value (e.g., 15 in Table 5) is subtracted from all the unconnected elements, leaving the remaining elements. The resulting table is shown below as Table 6.

[0293]

[0294] Table 6.

[0295] In the next step, identify the points in the table. If any points are missing, repeat the steps in Table 5.

[0296]

[0297] Table 7.

[0298] In pose recovery or pose estimation algorithms, the physical pose of an aircraft can be estimated based on identified points and points associated in the data association step described above. In some embodiments, the Perspective n-Point (PnP) algorithm can be used to calculate the relative pose (attitude and position) of the camera with respect to the landing surface. Various implementations of the PnP algorithm are available (e.g., such as in the OpenCV library). The accuracy of PnP calculations can be improved when the distribution combination points are not coplanar. In some cases, the distribution combination of VTOL airports may be coplanar, so the robustness of various PnP methods to coplanarity can be explored. PnP calculations are relatively fast compared to the detection and association steps and are unlikely to limit the speed of the pose recovery pipeline (detection, association, pose recovery). PnP will produce camera pose estimates at the camera's frame rate. Because the camera's pose relative to the aircraft is known, the aircraft's pose relative to the landing point can be calculated based on the camera pose information.

[0299] PnP works by minimizing the reprojection error of 2D points observed in an image relative to 3D points reprojected into the image, while optimizing the camera's 3D pose (position and orientation). Various algorithms exist to solve this optimization, and it can be solved using nonlinear least squares techniques.

[0300] In addition to geometry-based methods, it may be necessary to use methods such as the Extended Kalman Filter (EKF) to fuse PnP-type pose solutions with IMU information. This fusion approach offers several advantages, including enabling pose solutions to be computed at a higher update rate than that of the airborne camera, and providing increased redundancy and robustness to the pose recovery process, thus allowing for the mathematically rigorous removal of outlier PnP solutions.

[0301] In some embodiments, a tightly coupled rolling temporal estimation (MHE) formula can be used, where raw camera visualization and inertial measurement unit (IMU) information are processed simultaneously. The MHE formula method is less sensitive to nonlinearity and is therefore more accurate than the loosely coupled EKF method, while possessing the same advantages.

[0302] In some embodiments, other image-based navigation aids are used, such as visual odometry, optical flow, or intermediate homography information derived from images generated by a camera, to enhance the system's accuracy. Measurements of the rate of change of aircraft speed and attitude can be fused with attitude estimation (i.e., utilizing Kalman filters or other sensor fusion techniques) to improve the overall accuracy of the system.

[0303] In some embodiments, one or more properties of the light emitted from an active light source can be used to improve detection via background subtraction. Now refer to Figures 26A to 26CThe example illustrates exemplary waveforms consistent with some embodiments of this disclosure, representing intensity modulation and camera shutter speed operations used for encoding / decoding information algorithms.

[0304] For background subtraction, it might be necessary to have one frame with light on at 100% intensity and another frame with light completely off or at 0% intensity. The maximum difference in light intensity emitted from the light source allows subtracting every pixel between the two frames, thus removing all constant light sources. One of several ways to achieve this is to switch the lights on and off using half the camera's shutter speed, such as... Figure 26A As shown. In Figure 26A In this context, Tm indicates the modulation period, and Ts indicates the shutter period. However, such techniques can be limited by timing alignment issues. For example, when the camera sampling rate (also referred to as the camera capture rate in this paper) is aligned with the transition of the active light source, such as... Figure 26B As shown. In such cases, because camera exposure is not instantaneous, the camera might capture a 1 / 2 on and 1 / 2 off configuration, resulting in a 50% pixel value. This not only reduces the overall signal strength but can also affect the background subtraction algorithm, as each frame will be at the same 50% value, potentially making it impossible to detect the reference point.

[0305] To mitigate timing alignment issues related to the transition between the camera capture rate and the active light source, the camera capture speed can be synchronized with the modulation frequency of the light source. However, several problems may arise associated with the synchronization process, such as all light must be synchronized together, thus requiring camera synchronization as well, and synchronization to pulses can be complex because a reference point must be identified before synchronization can occur. While achievable, this could significantly increase the complexity of the algorithm.

[0306] In some embodiments, timing alignment between the camera capture rate and the change of the active light source can be overcome by flashing at a rate different from the shutter speed, such as... Figure 26C As shown. While this overcomes the problem of applying 50% of the pixel value to each frame, it doesn't completely remove parts of the frame and may introduce two sequential frames with the same exposure (in...). Figure 26C(Shown as two consecutive dark bars). One way to mitigate the problem of identical exposure might be to increase the sample size from 2 images to 3 images. In such a case, for background subtraction, instead of subtracting frame 2 from frame 1, we take the maximum and minimum pixel values ​​for a given pixel location across all three images and subtract them. However, for a given set of 3 images, one image might be at 100% intensity, while another might be at 0% intensity, but the third frame might be partially exposed, or two subsequent frames might have the same exposure. By setting the flash frequency to a non-multiple of the shutter speed, we can mathematically prove that each batch of three images will have at least one image open and one image closed.

[0307] In some embodiments, the camera capture rate is at least 100 frames per second. In some embodiments, the adjustment of the camera capture rate is based on the modulation of the light source. The adjustment of the capture rate may include synchronizing the camera capture rate with the modulation rate of the light source. In some embodiments, the flicker rate of the light source is approximately 30 Hz. In some embodiments, the flicker rate of the light source may be adjusted based on the camera capture rate.

[0308] In some embodiments, the controller is configured to adjust the camera's capture rate based on the modulation of the light source. The controller may be further configured to adjust the light source's flicker rate based on the camera's capture rate. The controller may be further configured to adjust the camera's bit transmission rate, wherein the camera's bit transmission rate is 10 Hz or higher.

[0309] In some embodiments, the controller is further configured to generate an output signal that includes information associated with the position and orientation of the airborne vehicle, based on the output signal from the camera. The information associated with the position of the airborne vehicle may include the GPS coordinates of the airborne vehicle. The controller may be further configured to transmit the information associated with the position and orientation of the airborne vehicle to an external processor.

[0310] In some embodiments, one or more properties of light emitted from an active light source can be used to transmit information associated with the light source or a landing surface associated with the light source. Conventional techniques for data transmission may have several drawbacks, including, but not limited to, the need for a universal clock signal to trigger sampling, the possibility that binary values ​​of on=1 and off=0 may be insufficient, and the need for synchronous sampling in clockless transmission modes.

[0311] In some embodiments, the duty cycle of the active light source can be adjusted. An example is a bit-by-bit transmission mode where 1 indicates a 70% duty cycle and 0 indicates a 30% duty cycle. However, for a reference point, the duty cycle alternates at a rate lower than the flicker rate of the light source. For example, the binary value 9 can be represented as four bits 1001, such as... Figure 27AAs shown. Once the reference point is identified by the camera, the light intensity can be averaged across multiple samples to calculate the average duty cycle for that period and the bit value assigned to that period, such as... Figure 27B As shown. The transmission and averaging can be varied to fit requirements for transmission rate and robustness to noise. Alternatively, the same averaging technique can be used to calculate the average duty cycle, and 1 or 0 can be represented by the rising or falling edge, as shown. Figure 27C As shown.

[0312] In some embodiments, the camera capture rate and the flicker frequency of the active light source can be synchronized to reduce errors that will occur when capturing images during transition periods, such as... Figure 26B As shown. In some embodiments, each active light source can be synchronized via a synchronization pulse transmitted through a connected wired network. In some embodiments, the synchronization pulse can be transmitted wirelessly via RF transmission. The frequency of the synchronization pulse can be appropriately selected. In some embodiments, synchronization can be performed via an optical sensor associated with each active light source. In such cases, each active light source needs to be able to detect light transmission from at least one other light source. Since each light source is synchronized with its neighboring light sources, all active light sources can be synchronized.

[0313] In some embodiments, if an active light source is synchronized, the camera may be synchronized with the active light source by a synchronization pulse transmitted wirelessly via RF transmission. In some embodiments, an onboard processor (e.g., processor 312) may adjust the camera's frame rate based on the quality of a benchmark detection.

[0314] Association Synthesis

[0315] As previously described, more than one algorithm can be used for data association to associate each identified reference point in a 2D image with its 3D location on the ground. Some of these algorithms include ICP, TPS-RPM, point tracking, linear cross-ratio, angular cross-ratio, and grid association. Each data association algorithm may offer several advantages when used individually, but it can also present challenges. Due to their different strengths and applicability, it may be necessary to combine two or more of these data association algorithms to produce more reliable and robust associations for use in precise landing and takeoff operations of eVTOL in GPS-denied environments.

[0316] Now for reference Figure 28 It is a flowchart illustrating an example method 2800 for data association and synthesis consistent with the disclosed embodiments. Figure 28The exemplary method shown includes: using a camera mounted on the aircraft to capture images of light sources (reference points) of a vertical takeoff and landing airport laid out on the ground; determining the pixel position of each reference point in the image (detection step); associating each identified reference point in the two-dimensional image with the three-dimensional position of the reference point on the ground (data association step); determining the orientation and position of the aircraft based on the association (attitude estimation step) and verifying the attitude; performing one or more sampling algorithms (e.g., RANSAC) to remove outliers; and generating the attitude of the aircraft by performing a perspective n-point transformation and applying an attitude filter.

[0317] In some embodiments, the data association synthesis pipeline may include executing one or more data association algorithms to generate associations between identified reference points in a 2D image and the 3D locations of those reference points on the ground. In a preferred embodiment, two or more data association algorithms may be executed to generate associations. The generated associations may be compiled, for example, in a data storage server or memory, to form an aggregated list of data or a larger set of information associated with the generated associations.

[0318] In some embodiments, for each data association algorithm performed, a PnP algorithm may also be executed to accurately determine the attitude (attitude and orientation) generated based on the association. Additionally or alternatively, for each data association algorithm performed, the determined attitude may be verified, for example, by confirming that the aircraft is above the ground, confirming that the aircraft is within a certain distance of the assigned vertical takeoff and landing airport or landing surface, confirming that the aircraft is pointing in the correct overall direction, etc. Once the determined attitude is verified, the associated information can be added to the aggregated data list.

[0319] The data association synthesis pipeline may further include identifying all unique points from the aggregated data list to form a second aggregated data list and performing a sampling method (e.g., RANSAC sampling) or similar algorithm to remove outliers. A third aggregated data list, including association information after outlier removal, forms a final aggregated data list, which can be used to perform a full PnP transform to generate an attitude estimate of the aircraft. In some embodiments, one or more attitude filters (such as, but not limited to, Kalman filters, extended Kalman filters, or other suitable attitude filters) may be used to generate the attitude estimate of the aircraft.

[0320] Now for reference Figure 29 This is a flowchart illustrating an example method 2900 for data association and synthesis consistent with the disclosed embodiments. Figure 28The exemplary method shown includes: using a camera mounted on the aircraft to capture images of light sources (reference points) of a vertical takeoff and landing airport laid out on the ground; determining the pixel position of each reference point in the image (detection step); associating each identified reference point in the two-dimensional image with the three-dimensional position of the reference point on the ground (data association step); determining the orientation and position of the aircraft based on the association (attitude estimation step) and verifying the attitude; performing one or more sampling algorithms (e.g., RANSAC) to remove outliers; and generating the attitude of the aircraft by performing a perspective n-point transformation and applying an attitude filter.

[0321] Compared to method 2800, in some embodiments, each data association algorithm performed can be considered as an independent sensor, such as Figure 29 As shown. For each data association algorithm executed, a complete PnP transform and RANSAC sampling can be performed before combining all measurements using a pose filter (such as a Kalman filter).

[0322] Data association algorithms (such as grid association and TPS-RPM) are inherently computationally intensive and iterative, respectively. To overcome these and other problems, an association pipeline with dual-mode operation can be implemented. The two modes in the dual-mode operation can be a "lost space" mode and a "tracking" mode.

[0323] The "Lost Space" mode can be useful when no prior information is available, such as during the maiden flight of an airborne vehicle, or when a new light source is installed, a connection is lost, or attitude information is lost due to transmission challenges. The Lost Space mode can run at a lower rate and can run mesh association and TPS-RPM algorithms to associate detected light with known patterns, thus providing a set of associated points to a Pose Estimation (PnP) algorithm configured to generate pose estimates. Based on the confidence level of the associations provided by the Lost Space mode, the processor can be configured to switch to a tracking mode, which runs at a higher rate than the Lost Space mode.

[0324] Tracking modes can be useful when prior information is available or obtained from lost space patterns. In tracking modes, tracking algorithms (such as, but not limited to, locally correlated point tracking, pose-based point tracking, and ICP) can be seeded using previously computed poses and correlations. These algorithms can generate correlations, which can be provided individually or fed as a synthetic superset into the pose estimate. Such pose estimates can then be output by the system at a higher rate.

[0325] To further improve correlation throughput, a lost space correlation algorithm can be executed in parallel after switching to tracking mode to provide correction. Because the correlation algorithm is independent of previous states and does not accumulate errors, overall pose estimation can be improved. A threshold can be determined based on the system's confidence metric. If the confidence metric exceeds the predetermined threshold, the system can switch back to lost space mode until better pose measurements from the sensors are obtained. In some embodiments, the switching between lost space mode and tracking mode can be performed automatically or autonomously by the system. However, in some embodiments, the switching can be performed manually through user intervention or user input. Therefore, providing a correlation pipeline with dual-mode operation, allowing switching between modes based on available information or correlation throughput, may be beneficial.

[0326] Line light source

[0327] As previously discussed, active light sources or reference points positioned on the ground in a vertical takeoff and landing (VTOL) airport can be point sources, such as LEDs. Point sources of light (such as LEDs) may be easy to install and modulate; however, in some cases, it may be necessary to enhance the overall signal-to-noise ratio (SNR) of the light signal originating from a ground-based source. While the amount of light emitted from the source can be increased by installing more LEDs, point sources cannot emit light that is dispersed over a large area, making them insufficient for high SNR applications. Therefore, systems and methods may be needed to enhance the signal strength and SNR of the light from the source and received by the photodetector.

[0328] See Figure 30 This is a schematic illustration of an exemplary arrangement of linear light in a distribution combination of light sources consistent with the disclosed embodiments. Landing surface 3000 may include a plurality of point sources 3030 and a plurality of line sources 3020. In some embodiments, landing surface 3000 may include line sources as an alternative to or supplement to point sources 3030. In some embodiments, landing surface 3000 may be a rectangular, square, triangular, circular, or elliptical landing area, or any other suitable shape. In some embodiments, line sources 3020 may be arranged along all sides of landing surface 3000 (e.g., a rectangular or square landing surface). Each side of landing surface 3000 may include a plurality of collinear line segments, each line segment including a line source. In some embodiments, adjacent line sources of the plurality of line segments may be separated by point sources. In some embodiments, no light source may be provided between adjacent line sources of the plurality of line segments, such that discontinuous rows of line sources can be formed along one side of landing surface 3000.

[0329] Using a line source offers several advantages, including but not limited to a higher signal-to-noise ratio due to the larger dispersion of light generated from a line source compared to a point source, more robust line detection algorithms, higher data throughput from a line source compared to a point source, compatibility with a range of algorithms, and a simple and reliable coding scheme.

[0330] One of the benefits of using line sources in the distribution of ground-based light sources is compatibility with detection algorithms, data association algorithms, PnP transform, pose recovery or pose estimation algorithms, and data encoding. For example, background subtraction techniques, point detection algorithms (if using a combination of point and line sources), and line detection after background subtraction can remain unchanged and can be directly transferred from point source detection algorithms.

[0331] Figure 31A This is a schematic illustration of an exemplary data encoding scheme 3100 using a line light source, as shown on a landing surface 3000, consistent with the disclosed embodiments. Each of these edges can be segmented into multiple collinear segments without affecting the detection or association algorithms. In some embodiments, each segment can be used to represent a single data bit. As an example, the landing surface 3000 (each landing surface comprising four edges of four segments) can be configured to transmit 16 bits of data per transmission cycle. It should be understood that the number of segments can vary as needed.

[0332] Figure 31A The example illustrates an exemplary encoding scheme such as an on / off scheme. In the on / off encoding scheme, since line detection and association only apply to a single segment lit on each line, encoding can be performed by stopping the blinking of the line segments. Additionally, at least one line segment may need to blink for pose recovery during the transmission of any binary representation. Alternatively, multiple fixed segments that always blink may be available for pose recovery, but can be excluded from the transmission. As an example, in Figure 31A In the four-segment encoding shown, the segment closest to the corner of each line can always be switched without affecting the integrity of attitude recovery, making the inner segments available for data transmission. The encoded binary bit representation of value 13 is shown in exemplary scheme 3100. In some embodiments, the blinking of the line segments can be programmed to represent predefined values ​​associated with the markings on the landing surface.

[0333] Now for reference Figure 31BThis illustrates an exemplary encoding scheme 3150 for data transmission using a combination of linearly shaped light and point sources, consistent with some disclosed embodiments. An exemplary combined landing surface, such as landing surface 3110, may include a plurality of line light sources 3120 forming the edges of a pattern (e.g., rectangles, squares, triangles, etc.), and a plurality of point light sources 3125 distributed within an area defined by the line light sources 3120. In some embodiments, the point light sources 3125 may have a predetermined pattern distribution with known locations, or a randomly generated pattern distribution with unknown locations of the point light sources.

[0334] In some embodiments, data transmission between a landing surface and an airborne aircraft configured to land on or take off from the landing surface may include providing an encoding scheme. Encoding scheme 3150 may include marking the intersections of line light sources 3120. The marking scheme may include using heading information from INS or one or more previous iterations to mark these points in a predetermined, known order. As an example, in Figure 31B In the rectangular pattern shown, northeast can be marked as "0", southeast as "1", southwest as "2", and northwest as "3". As another example, intersections can be marked based on the number of nearby point light sources. The corner with the fewest nearby point light sources can be marked as "0", and the corner with the most nearby point light sources can be marked as "3". It should be understood that although only two marking schemes have been discussed, other suitable marking schemes can be applied.

[0335] Data transmission may further include projecting the light source onto a normalized grid 3130. One of several ways of projecting the light source includes calculating and applying a homography matrix to remove distortion associated with the projection. The normalized grid 3130 may be divided into multiple predetermined subspaces 3140. Although the normalized grid 3130 is shown as being divided into nine subspaces 3140, the normalized grid 3130 may be divided into any number of subspaces based on the number of point light sources, the density of point light sources, the area of ​​point light sources, or as needed. For each subspace, if a point light source is detected (i.e., activated or switched on), it may be marked as "1", and if no point light source is detected, it may be marked as "0". The labels for each subspace may be combined in a predetermined order to generate a binary value that can be configured to represent or identify the landing space. As an example, as shown in the normalized grid 3130, the labels for the nine subspaces 3140 may be combined to form the binary value 100010011, which represents the number 275. In some embodiments, the number 275 may represent an identification associated with the landing surface, or the spatial orientation of the landing surface, or an authentication code that can be used to verify user identity or aircraft identification, etc.

[0336] Figure 31C A flowchart illustrating an exemplary method 3160 for pose estimation using a line light source, consistent with some of the disclosed embodiments, is provided. Method 3160 may be performed in conjunction with or in lieu of a method for pose estimation using a point light source (as previously described).

[0337] Method 3160 includes, but is not limited to, the following steps: receiving at least two images from a camera mounted on an airborne vehicle; performing background subtraction from the received images; performing line detection using line detection algorithms such as Hough transform, line detection filters, etc.; determining the locations of line intersections; labeling each intersection point using a labeling scheme; performing a PnP transform algorithm using the labeled points; and selecting a valid pose from the results. The steps represented by shaded blocks (such as background subtraction, connected components, centroids, associations, and PnP) are processes present in pose estimation methods using point sources. One or more steps for pose estimation using point sources may be additionally or alternatively used for pose estimation using line sources.

[0338] In some embodiments, the marking scheme may include using heading information from INS or one or more previous iterations to mark the points in a predetermined, known order. As an example, in Figure 31B In the rectangular pattern shown, northeast can be marked as "0", southeast as "1", southwest as "2", and northwest as "3". As another example, intersections can be marked based on the number of nearby point light sources. The corner with the fewest nearby point light sources can be marked as "0", and the corner with the most nearby point light sources can be marked as "3".

[0339] Data Augmentation

[0340] In urban environments, GPS signals can be delayed, blocked, distorted, or completely undetectable due to reflections or obstructions from densely packed structures such as tall buildings and towers. GPS signals can be reflected by buildings, walls, vehicles, and in some cases even the ground. Glass, metal, and damp surfaces are known to be strong reflectors of light. These reflected signals can interfere with the reception of signals directly from satellites and may be received via multiple paths, such as reflections from other surfaces and structures in the vicinity of the aircraft. This phenomenon is called multipath interference or multipath effect. Figure 32B As shown. However, in some cases, direct signals from GPS satellites (e.g., GPS satellite 3220) may be blocked or obstructed by, for example, tall buildings 3210, and only reflected signals are received. This phenomenon is called non-line-of-sight (NLOS) reception, such as... Figure 32A As shown.

[0341] In the context of this disclosure, multipath effects can pose several challenges and may be more cumbersome than NLOS reception because measurements from GPS signals may not only be distorted but may also be undetectable. In some cases, a single signal may be received twice, or the signal may be significantly delayed, which can directly affect the time of arrival (TOA) calculations required to generate a position. These effects can be exacerbated during landing and takeoff as the aircraft approaches the landing surface and is at an altitude where signals may be blocked, reflected, or distorted by surrounding structures. The proposed Precise Landing and Takeoff (PLaTO) systems and methods can be used to eliminate one or more of these effects to improve the overall accuracy of position measurements, as disclosed in some embodiments herein.

[0342] a. Using GPS to enhance PLATO

[0343] As previously mentioned, the algorithms used in PLATO can be iterative in nature, and therefore, convergence time and signal latency can be reduced by providing an initial guess. In some existing systems and methods, executing one or more steps of the algorithm can take milliseconds or even seconds if an initial guess is unavailable, making the algorithm inefficient and negatively impacting data throughput. However, in some embodiments, GPS signals can be used to reduce commuting time in “lost space” mode by seeding the algorithm with an initial guess of the aircraft’s current position. Even if the guess is inaccurate, it can speed up the algorithm by reducing the number of iterations required to achieve convergence.

[0344] Now for reference Figure 33 This illustrates a data augmentation pipeline 3300, consistent with the disclosed embodiments, configured to enhance the accuracy or speed of an aircraft's horizontal positioning by combining GPS and inertial navigation system (INS) measurements. In practice, the aircraft may be configured to use some form of GPS positioning, and information from the INS may also be available.

[0345] The data augmentation pipeline 3300 may include receiving information associated with the aircraft's position based on GPS signals from one or more GPS satellites. In some embodiments, the position information may include the aircraft's position coordinates. The pipeline 3300 may further include receiving information associated with the position of a landing surface or vertical takeoff and landing (VTOL) airport, which may already exist in a database. In some embodiments, the position of the landing surface may initially be determined based on GPS signals and stored in an accessible database for later use. The pipeline 3300 may further include receiving information associated with the aircraft's attitude (pose, position, and orientation) based on INS measurements. Using information associated with the aircraft's position in 3D space, the landing surface's position in 3D space, and the orientation of a camera configured to determine the aircraft's attitude allows for the determination of a region of interest 3350. In this context, a "region of interest" refers to an area assigned to the landing space or VTOL airport for an eVTOL aircraft. In some embodiments, region 3355 (indicated by a pixelated area surrounding region 3350) may be excluded from consideration to reduce processing time. In some embodiments, as previously described, the GPS information associated with the aircraft’s location may not have to be accurate, and even initial guesses can help reduce processing time.

[0346] b. Using PLATO to enhance GPS measurements

[0347] Now for reference Figure 34 This illustrates a data augmentation pipeline 3400, consistent with the disclosed embodiments, configured to use information from the PLATO system to enhance GPS measurements.

[0348] As previously described, multipath effects in dense urban environments present several challenges associated with receiving GPS signals, and thus negatively impact the accuracy of location measurements. In some embodiments, the PLATO system can be used to help mitigate some of these problems to improve the overall accuracy of GPS-based location measurements.

[0349] In some embodiments, pipeline 3400 may include using position measurements from a PLATO system to define the position estimate provided by GPS signals. In some cases, GPS signals may be affected by multipath effects, NLOS reception, or both, and therefore, GPS signals may only provide a position estimate but not an accurate measurement. For reliable GPS signals, at least five satellites may be required. In some scenarios where more than five satellites are present, the use of one or more satellites may be excluded if the position reference exceeds the limits of the PLATO estimate. Figure 34As shown, if the number of detected satellites (N) is less than the minimum number of satellites required to generate a reliable GPS signal, pipeline 3400 can verify one or more satellites. If the number of satellites exceeds the minimum number, one or more satellites can be discarded, and the GPS signal can be calculated.

[0350] In some embodiments, if GPS satellites cannot be verified, they are considered incapable of providing position estimates or are poorly performing satellites and can be removed from consideration. In such cases, the remaining satellites can be used to calculate the position. In some embodiments, verifying existing satellites may include estimating GPS signals from available GPS satellites, determining for each satellite whether the error between the PLATO signal and the GPS signal is less than a predefined threshold error limit, and calculating the position based on the signals received from the satellites after determination. If the error is greater than the predefined threshold limit, the signals from the satellites and the satellites themselves can be removed from consideration.

[0351] In some embodiments, the positions of individual satellites can be determined based on a given position measurement from PLATO. Information associated with the positions of the individual satellites can be used to eliminate multipath effects or synthesize satellite signals. Multipath effect elimination can be achieved by performing one or more of the following steps.

[0352] i. Provides various satellite measurements, such as Time Difference of Arrival (TDOA) and position measurements from the PLATO system.

[0353] ii. Calculate the expected TDOA signal for each satellite, given the position reported by the PLATO system.

[0354] iii. Compare the measured TDOA signal with the expected TDOA signal and remove outliers that fall outside the given range.

[0355] iv. Use one or more expected TDOAs to calculate the location, allowing GPS to use fewer satellites than is typically required to calculate the location.

[0356] In some embodiments, based on position measurements from PLATO and fewer than a minimum number of GPS satellites (e.g., two satellites), the position probability distributions of the two measurements can be combined with a Kalman filter to reduce the measurement error to below the measurement error that either system could produce alone.

[0357] c. Using PLATO to enhance INS measurements

[0358] In the context of this disclosure, using the last known position and extrapolating it based on INS measurements is referred to as dead reckoning navigation. This method can be useful for short distances, but for longer distances, INS sensor drift will lead to a large accumulation of errors over extended periods. This is likely due to the fact that INS directly measures acceleration and generates velocity and position estimates by integrating acceleration over time. Therefore, small shifts in acceleration can accumulate into large position errors over time. In some embodiments, measurements from PLATO can be used to enhance INS measurements, such as... Figure 35 As shown.

[0359] In some embodiments, the absolute time-varying position correction of the aircraft can be obtained from the PLATO system. This can be achieved by fusing position and velocity estimates from the PLATO system with INS measurements to generate high-fidelity measurements. One or more camera images from the PLATO system can be used to perform visual odometry to enhance INS measurements. In the context of this disclosure, "visual odometry" is similar to point tracking described in the data association algorithms above, but the end result is a change in position between frames rather than a change in location. One of the advantages of visual odometry is that it does not require a reference point to exist in the frame, meaning it can be used even when the landing surface or VTOL airport is not in the vicinity during normal flight of the aircraft. Visual odometry may include the following steps:

[0360] 1. Provide information at time t. a and t b Images captured at the location (a) and (b);

[0361] 2. Identify the distinctive features in images (a) and (b);

[0362] 3. Use nearest neighbor or a suitable algorithm to associate features in image (a) with features in image (b); and

[0363] 4. Estimate the change in camera position between image (a) and image (b).

[0364] In some embodiments, the results of visual odometry can be used to correct for errors in t during image capture. a With t b The drift that occurs within the time range between these points. In some embodiments, t a With t b The difference between them is one, such that at t a and t bThe captured images are continuous. Changes in position can be similarly integrated over time to generate a relative position from the starting point. Another advantage of visual odometry is that it is less prone to drift effects because it depends on the camera's field of view and the environment, rather than on acceleration measurements.

[0365] d. INS assisted by optical positioning system

[0366] Now for reference Figure 36 This is an exemplary pipeline 3600 for data augmentation methods consistent with some of the disclosed embodiments. Pipeline 3600 represents a method for using a fixed hysteresis smoother algorithm to assist INS measurements performed by an optical positioning system. In some embodiments, the optical positioning system using the fixed hysteresis smoother algorithm can be used in conjunction with INS on an aircraft to provide a more accurate position estimate than when used alone. Standard filtering implementations would utilize an extended Kalman filter (EKF) to fuse the optical positioning attitude solution with inertial navigation system data. However, as an alternative to the EKF, attitude map optimization methods (such as, for example, a fixed hysteresis smoother) can be used to allow the use of sensor data across time windows (not just instantaneous) to compute the optimal position estimate. In some embodiments, using a fixed hysteresis smoother algorithm can improve the accuracy of position estimates at higher frequencies (e.g., 500 Hz).

[0367] e. Integrating PLATO with aircraft

[0368] Now for reference Figure 37 This illustrates an exemplary system 3700, consistent with some disclosed embodiments, demonstrating the integration of a PLATO system with an aircraft to support manned or unmanned flight. The electric propulsion system for eVTOL may include an electric engine that provides mechanical shaft power to a propeller assembly to generate thrust. In some embodiments, the electric engine of the electric propulsion system may include a high-voltage power system that supplies high-voltage power to the electric engine and / or a low-voltage system that supplies low-voltage DC power to the electric engine. Some embodiments may include an electric engine that digitally communicates with a flight control system (“FCS”) including a flight control computer (“FCC”) 3750, which may send and receive signals to and from the electric engine, including command and response data or status. Some embodiments may include an electric engine capable of receiving and transmitting operating parameters from and to the FCC, including speed, voltage, current, torque, temperature, vibration, propeller position, and any other values ​​of the operating parameters.

[0369] In some embodiments, the flight control system may include a system capable of communicating with the electric engine to send and receive analog / discrete signals to the electric engine and controlling devices that can redirect the thrust of the tilt propeller between the primary vertical direction during vertical flight mode and the primary horizontal direction during forward flight mode. In some embodiments, the system may be referred to as a tilt propeller system (“TPS”) and may be capable of transmitting and orienting additional features of the electric propulsion system.

[0370] In some embodiments, system 3700 may transmit measured attitude (position and orientation) to FCC 3750. In some embodiments, FCC 3750 may fuse estimated attitudes from other sources (such as GPS 3710, INS 3720, altimeter 3710, PLATO 3740, etc.) to generate an optimal estimate of the aircraft's attitude. This can be performed using a variety of sensor fusion techniques (such as Kalman filters, extended Kalman filters, fixed hysteresis smoothers, or other methods for performing sensor fusion).

[0371] In manned aircraft, the final position estimate can be used to provide visual feedback to the pilot. In unmanned aircraft, the final position estimate can be used to calculate flight control commands such as motor commands, flight surface controls, or other control signals used to maneuver the aircraft in flight.

[0372] Example—EKF utilizes location data from PLATO

[0373] Figures 38 to 41 The results of a real-time flight test using an extended Kalman filter based on position data from the PLATO system are illustrated. Figure 38 An example is shown comparing height estimates plotted as a function of horizontal distance measured by RTK-GPS ground truth data and EKF. Figure 38 As shown, the EKF line (dashed line) is mostly closely attached to the RTK line (solid line). Figure 39 An example is shown comparing the altitude of an aircraft above ground measured using EKF and RTK-GPS reference data, along with the corresponding errors. For example... Figure 39 As shown, the EKF line (dashed line) is mostly closely attached to the RTK line (solid line). Figure 40 An example is given of the East estimate as a function of the baseline truth value and the corresponding error plot. For example... Figure 40 As shown, the EKF line (dashed line) is mostly closely attached to the RTK line (solid line). Figure 41 Examples include plots of the north estimate as a function of the baseline truth value and corresponding error plots. For instance... Figure 41 As shown, the EKF line (dashed line) is mostly closely attached to the RTK line (solid line).

[0374] Now for reference Figure 42 This is a flowchart illustrating an example method 4200 for estimating the attitude of an airborne vehicle, consistent with some embodiments of this disclosure. Method 4200 may be implemented using computing devices and systems, such as those disclosed herein. In some embodiments, method 4200 may be performed by at least one processor of a computer-implemented system. The corresponding steps and operations of these components for method 4200 are described below. It should be understood that components and operations may be combined, modified, and / or rearranged depending on the application and system embodiment.

[0375] like Figure 42 As shown, at step 4210, a landing surface comprising light sources arranged in a predetermined pattern can be provided. The landing surface may be a vertical takeoff and landing airport for an eVTOL airborne vehicle. Each of these light sources may be an active light source configured to emit light, and one or more characteristics of the emitted light may be modulated over time. The light sources may be arranged in a predetermined pattern, wherein the position of each light source is known. The distribution combination design of the arranged light sources may include an arrangement of light sources defining a set of intersecting virtual lines, wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0376] At step 4220, one or more characteristics of a light source on the landing surface can be modulated over time. The characteristics of the light source may include the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light. Modulating one or more characteristics of the emitted light can be configured to identify the landing surface, identify the light source, identify the position of the light source, identify the operational state of the landing surface, or encode signals for authenticating the landing surface. The characteristics of the light source can be modulated by a controller on the landing surface.

[0377] At step 4230, a camera mounted on the airborne vehicle receives an input signal associated with light emitted from a light source. The camera is mounted on the airborne vehicle at a known location and with a known orientation. The camera may use optical filters or lenses to allow a range of wavelengths.

[0378] At step 4240, based on the received input signal, the camera can generate an output in the form of a still image, a series of still images, or a streaming video, which is information captured from the landing surface and the surrounding environment.

[0379] At step 4250, the processor may determine the position and orientation of the airborne vehicle based on information from images captured by the camera. Determining the position and orientation of the airborne vehicle may include detecting at least one light source in the image, determining which of the detected light sources is arranged in a predetermined pattern, and determining the position and orientation of the airborne vehicle based on the determination of which of the detected light sources is arranged in a predetermined pattern. The processor may be configured to perform one or more algorithms to estimate the attitude of the airborne vehicle based on information received from the camera.

[0380] Figure 44 This is an illustration of a perspective view of an exemplary VTOL aircraft consistent with the disclosed embodiments. Figure 45 This is another illustration of a perspective view of an exemplary VTOL aircraft in an alternative configuration consistent with embodiments of this disclosure. Figure 44 and Figure 45 Examples of VTOL aircraft 4400 and 4500, consistent with embodiments of this disclosure, are shown in cruise configuration and vertical takeoff, landing, and hovering configurations (also referred to herein as "takeoff and landing" configurations). Figure 44 and Figure 45 Corresponding elements may have similar designations and refer to similar components of aircraft 4400 and 4500. Aircraft 4400 and 4500 may include fuselages 4402 and 4502, wings 4404 and 4504 mounted to fuselages 4402 and 4502, and one or more rear stabilizers 4406 and 4506 mounted to the rear of fuselages 4402 and 4502. Multiple elevator propellers 4412 and 4512 may be mounted to wings 4404 and 4504 and may be configured to provide lift for vertical takeoff, landing, and hovering. Multiple tilt propellers 4414 and 4514 may be mounted to wings 4404 and 4504 and may tilt between an elevator configuration and a cruise configuration, in which the tilt propellers provide a portion of the lift required for vertical takeoff, landing, and hovering, such as... Figure 45 As shown, in this cruise configuration, the multiple tilting propellers provide forward thrust to the aircraft 4400 for level flight, such as... Figure 44 As shown. As used herein, a tilt propeller lift configuration refers to any tilt propeller orientation in which tilt propeller thrust primarily provides lift to the aircraft, while a tilt propeller cruise configuration refers to any tilt propeller orientation in which tilt propeller thrust primarily provides forward thrust to the aircraft.

[0381] In some embodiments, the lift propellers 4412, 4512 may be configured to provide lift force only, wherein all horizontal thrust is provided by the tilt propellers. Therefore, the lift propellers 4412, 4512 may be configured to have a fixed position and generate thrust only during the takeoff, landing, and hovering phases of flight. Simultaneously, the tilt propellers 4414, 4514 may tilt upwards into a lift configuration in which the thrust from the propellers 4414, 4514 is directed downwards to provide additional lift force.

[0382] For forward flight, tilting propellers 4414 and 4514 can tilt from their lift configuration to their cruise configuration. In other words, the orientation of tilting propellers 4414 and 4514 can change from an orientation in which the tilting propeller thrust is oriented downwards (to provide lift during vertical takeoff, landing, and hovering) to an orientation in which the tilting propeller thrust is oriented backwards (to provide forward thrust to aircraft 4400 and 4500). The tilting propeller assembly for a specific electric engine can tilt about an axis of rotation defined by the mounting point connecting the boom and the electric engine. When aircraft 4400 and 4500 are in fully forward flight, lift can be provided entirely by wings 4404 and 4504. Meanwhile, in cruise configuration, lift propellers 4412 and 4512 can be shut off. The blades 4420 and 4520 of lift propellers 4412 and 4512 can be maintained in a low-drag position for aircraft cruise. In some embodiments, the lift propellers 4412 and 4512 may each have two blades 4420 and 4520, which can be locked for cruising in a minimum drag position, where one blade is directly in front of the other, as shown in FIG474. In some embodiments, the lift propellers 4412 and 4512 have more than two blades. In some embodiments, the tilt propellers 4414 and 4514 may include more blades 4416 and 4516 than the lift propellers 4412 and 4512. For example, as... Figure 44 and Figure 45 As shown, the elevator propellers 4412 and 4512 may each include, for example, two blades, while the tilt propellers 4414 and 4514 may each include more blades, such as the five blades shown. In some embodiments, each of the tilt propellers 4414 and 4514 may have two to five blades, and possibly more, depending on the aircraft's design considerations and requirements.

[0383] In some embodiments, the aircraft may include a single wing 4404, 4504 on each side of the fuselage 4402, 4502 (or a single wing extending across the entire aircraft). At least a portion of elevators 4412, 4512 may be located behind the wings 4404, 4504, and at least a portion of jib propellers 4414, 4514 may be located in front of the wings 4404, 4504. In some embodiments, all of the elevators 4412, 4512 may be located behind the wings 4404, 4504, and all of the jib propellers 4414, 4514 may be located in front of the wings 4404, 4504. According to some embodiments, all elevators 4412, 4512 and jib propellers 4414, 4514 may be mounted to the wings—that is, no elevators or jib propellers may be mounted to the fuselage. In some embodiments, the elevator propellers 4412 and 4512 may all be located behind the wings 4404 and 4504, and the tilt propellers 4414 and 4514 may all be located in front of the wings 4404 and 4504. According to some embodiments, all elevator propellers 4412 and 4512 and tilt propellers 4414 and 4514 may be positioned inside the ends of the wings 4404 and 4504.

[0384] In some embodiments, the elevator propellers 4412, 4512 and the tilt propellers 4414, 4514 can be mounted to the wings 4404, 4504 via booms 4422, 4522. Booms 4422, 4522 can be mounted below the wings 4404, 4504, on the top of the wings, and / or integrated into the wing profile. In some embodiments, the elevator propellers 4412, 4512 and the tilt propellers 4414, 4514 can be directly mounted to the wings 4404, 4504. In some embodiments, one elevator propeller 4412, 4512 and one tilt propeller 4414, 4514 can be mounted to each boom 4422, 4522. The elevator propellers 4412, 4512 can be mounted at the rear end of the booms 4422, 4522, and the tilt propellers 4414, 4514 can be mounted at the front end of the booms 4422, 4522. In some embodiments, the lift propellers 4412 and 4512 may be mounted in a fixed position on the booms 4422 and 4522. In some embodiments, the tilt propellers 4414 and 4514 may be hinged to the front end of the booms 4422 and 4522. The tilt propellers 4414 and 4514 may be mounted to the booms 4422 and 4522 such that, when in their cruise configuration, the tilt propellers 4414 and 4514 are aligned with the body of the booms 4422 and 4522, forming a continuous extension at the front end of the booms 4422 and 4522, which is used to minimize drag during forward flight.

[0385] In some embodiments, aircraft 4400, 4500 may include, for example, a wing on each side of fuselage 4402, 4502 or a single wing extending across the aircraft. According to some embodiments, at least one wing 4404, 4504 is a high wing mounted to the upper side of fuselage 4402, 4502. According to some embodiments, the wing includes control surfaces such as flaps and / or ailerons. According to some embodiments, wings 4404, 4504 may have been designed with profiles to reduce drag during forward flight. In some embodiments, the wingtip profile may be curved and / or tapered to minimize drag.

[0386] In some embodiments, the rear stabilizers 4406, 4506 include control surfaces such as one or more rudders, one or more elevators, and / or one or more combinations of rudder-elevator. The wing may have any suitable design. In some embodiments, the wing has a tapered leading edge.

[0387] In some embodiments, the elevator propellers 4412, 4512 or the yaw propellers 4414, 4514 may be deflected relative to at least one other elevator propeller 4412, 4512 or yaw propeller 4414, 4514. As used herein, deflection refers to the relative orientation of the axis of rotation of the elevator / yaw propeller about a line parallel to the forward-reverse direction, analogous to the roll degree of freedom of an aircraft. Deflection of the elevator propellers and / or yaw propellers can help minimize damage from a propeller burst by orienting the plane of rotation of the elevator / yaw propeller disk (blades plus the hub on which the blades are mounted) so as not to intersect with critical parts of the aircraft (such areas of the fuselage where personnel may be located, critical flight control systems, batteries, adjacent propellers, etc.) or other propeller disks, and can provide enhanced yaw control during flight.

[0388] Figure 46 This is an illustration of a top plan view of an exemplary VTOL aircraft consistent with embodiments of this disclosure. The aircraft 4600 shown in the figure may be, respectively, in... Figure 44 and Figure 45The diagram shows a top plan view of aircraft 4400 and 4500. As discussed herein, aircraft 4600 may include twelve electric propulsion systems distributed across aircraft 4600. In some embodiments, the distribution of electric propulsion systems may include six forward electric propulsion systems 4614 and six tail electric propulsion systems 4612 mounted on booms at the forward and aft ends of the main wing 4604 of aircraft 4600. In some embodiments, the length of the rear end of boom 4624 from wing 4604 to elevator propeller 4612 may include similar rear end lengths of boom 4624 across multiple rear ends of boom. In some embodiments, the length of the rear end of boom may vary across the six exemplary rear ends of boom. For example, each rear end of boom 4624 may include different lengths from wing 4604 to elevator propeller 4612, or subgroups of the rear ends of boom may be similar in length. In some embodiments, the front end of boom 4622 may include various lengths across the front end of boom from wing 4604 to tilt propeller 4614. For example, as... Figure 46 As shown, the length of the boom 4622 from the tiltrotor 4614 closest to the fuselage to the tip of the boom 4622 from the wing 4604 may be greater than the length from the wing 4604 to the tip of the boom 4622 furthest from the fuselage from the tiltrotor 4614. Some embodiments may include boom tips of similar length at six exemplary tips, or any other length distribution from the wing 4604 to the tip of the boom from the tiltrotor 4614. Some embodiments may include an aircraft 4600 having eight electric propulsion systems, wherein the eight electric propulsion systems have four forward electric propulsion systems 4614 and four tail electric propulsion systems 4612, or any other distribution of forward and tail electric propulsion systems, including embodiments in which the number of forward electric propulsion systems 4614 is less than or greater than the number of tail electric propulsion systems 4612. Additionally, Figure 46 An exemplary embodiment of a VTOL aircraft 4600 is depicted, having a front propeller 4614 in a horizontal orientation for horizontal flight and a tail propeller blade 4620 in a retracted position for forward flight.

[0389] As disclosed herein, the front electric propulsion system and the rear electric propulsion system may be of clockwise (CW) or counterclockwise (CCW) type. Some embodiments may include various front electric propulsion systems having a mixture of both CW and CCW types. In some embodiments, the rear electric propulsion system may have a mixture of CW and CCW type systems within the rear electric propulsion system.

[0390] Figure 47This is a schematic diagram illustrating exemplary propeller rotation of a VTOL aircraft consistent with the disclosed embodiments. The aircraft 4700 shown in the figure may be, respectively, shown in Figure 1, Figure 2 and Figure 3 The diagram shows a top plan view of aircraft 4400, 4500, and 4600. Aircraft 4700 may include six forward electric propulsion systems, three of which have a CW type 4724, and the remaining three have a CCW type. In some embodiments, three tail electric propulsion systems may have a CCW type 4728, while the remaining three tail electric propulsion systems may have a CW type 4730. Some embodiments may include an aircraft 4700 having four forward electric propulsion systems and four tail electric propulsion systems, each having two CW types and two CCW types. In some embodiments, the propellers may rotate in opposite directions relative to adjacent propellers to eliminate torque steer experienced by the aircraft's fuselage or wings due to propeller rotation. In some embodiments, the difference in rotation direction may be achieved using the direction of engine rotation. In other embodiments, all engines may rotate in the same direction, and different propeller rotation directions may be achieved using gear mechanisms.

[0391] Some embodiments may include an aircraft 4700 having a front electric propulsion system and a rear electric propulsion system, wherein the number of CW type 4724 and CCW type 4726 in the front electric propulsion system, in the rear electric propulsion system, or in both the front and rear electric propulsion systems is not equal.

[0392] Figure 48This is a schematic diagram illustrating exemplary power connections in a VTOL aircraft consistent with the disclosed embodiments. VTOL aircraft may have various power systems connected to diagonally opposite electric propulsion systems. In some embodiments, the power systems may include high-voltage power systems. Some embodiments may include high-voltage power systems connected to the electric engines via high-voltage channels. In some embodiments, aircraft 4800 may include six power systems comprising batteries 4826, 4828, 4830, 4832, 4834, and 4836 stored within the wings 4870 of aircraft 4800. In some embodiments, aircraft 4800 may include six forward electric propulsion systems having six electric engines 4802, 4804, 4806, 4808, 4810, and 4812, and six tail electric propulsion systems having six electric engines 4814, 4816, 4818, 4820, 4822, and 4824. In some embodiments, batteries may be connected to diagonally opposite electric engines. In this configuration, a first power system 4826 may supply power to an electric motor 4802 via a power connection channel 4838 and to an electric motor 4824 via a power connection channel 4840. In some embodiments, the first power system 4826 may also be paired with a fourth power system 4832 via a power connection channel 4842 having a fuse to prevent excessive current from flowing through power systems 4826 and 4832. Further, with respect to this embodiment, the VTOL aircraft 4800 may include a second power system 4828 paired with a fifth power system 4834 via a power connection channel 4848 having a fuse, and may supply power to electric motors 4810 and 4816 via power connection channels 4844 and 4846, respectively. In some embodiments, a third power system 4830 may be paired with a sixth power system 4836 via a power connection channel 4854 having a fuse, and may supply power to electric motors 4806 and 4820 via power connection channels 4850 and 4852, respectively. The fourth power system 4832 can also supply power to electric motors 4808 and 4818 via power connection channels 4856 and 4858, respectively. The fifth power system 4834 can also supply power to electric motors 4804 and 4822 via power connection channels 4860 and 4862, respectively. The sixth power system 4836 can also supply power to electric motors 4812 and 4814 via power connection channels 4864 and 4866, respectively.

[0393] As disclosed herein, an electric propulsion system may include an electric engine connected via a high-voltage channel or power connection channel to a high-voltage power system (such as a battery) located within the aircraft. Some embodiments may include various batteries stored within the aircraft wing, which has a high-voltage channel running through the aircraft (including the wing and boom) to the electric propulsion system. In some embodiments, multiple high-voltage power systems may be used to create an electric propulsion system with multiple high-voltage power sources to avoid the risk of single point of failure. In some embodiments, the aircraft may include multiple electric propulsion systems that may be patterned and wired to various batteries or power sources stored throughout the aircraft. It should be recognized that such a configuration may be beneficial in avoiding the risk of single point of failure, where a failure of one battery or power source could cause a portion of the aircraft to lose the thrust required to continue flight or perform a controlled landing. For example, if a VTOL has two forward electric propulsion systems and two aft propulsion systems, the forward and aft electric propulsion systems on opposite sides of the VTOL aircraft may be connected to the same high-voltage power system. In such configurations, if one high-voltage power system fails, the forward and aft electric propulsion systems on opposite sides of the VTOL aircraft will remain operational, providing a more balanced flight or landing compared to a forward and aft electric propulsion system failing on the same side of the VTOL aircraft. Some embodiments may include four forward electric propulsion systems and four aft electric propulsion systems, with diagonally opposite electric engines connected to a common battery or power source. Some embodiments may include various configurations of electric engines electrically connected to the high-voltage power system, such that the risk of a single point of failure is avoided in the event of a power source failure, and that the flight phase during which the failure occurs can continue, or the aircraft can proceed to an alternative flight phase in response to the failure.

[0394] As discussed above, an electric propulsion system may include an electric engine that provides mechanical shaft power to a propeller assembly to generate thrust. In some embodiments, the electric engine of the electric propulsion system may include a high-voltage power system that supplies high-voltage power to the electric engine and / or a low-voltage system that supplies low-voltage DC power to the electric engine. Some embodiments may include an electric engine that digitally communicates with a flight control system (“FCS”) including a flight control computer (“FCC”), which may send signals to and receive signals from the electric engine, including command and response data or status. Some embodiments may include an electric engine capable of receiving and transmitting operating parameters from and to the FCC, including speed, voltage, current, torque, temperature, vibration, propeller position, and any other values ​​of the operating parameters.

[0395] In some embodiments, the flight control system may include a system capable of communicating with the electric engine to send and receive analog / discrete signals to the electric engine and controlling devices that can redirect the thrust of the tilt propeller between the primary vertical direction during vertical flight mode and the primary horizontal direction during forward flight mode. In some embodiments, the system may be referred to as a tilt propeller system (“TPS”) and may be capable of transmitting and orienting additional features of the electric propulsion system.

[0396] Figure 49 A block diagram illustrating an exemplary architecture and design of an electric propulsion unit 4900 consistent with the disclosed embodiments is shown. In some embodiments, the electric propulsion system 4902 may include an electric engine subsystem 4904 that can supply torque via an axial propeller subsystem 4906 to generate thrust for the electric propulsion system 4902. Some embodiments may include an electric engine subsystem 4904 that receives low-voltage DC (LV DC) power from a low-voltage system (LVS) 4908. Some embodiments may include an electric engine subsystem 4904 that receives high-voltage (HV) power from a high-voltage power system (HVPS) 4910, which includes at least one battery or another device capable of storing energy. In some embodiments, the high-voltage power system may include more than one battery or another device capable of storing energy that supplies high-voltage power to the electric engine subsystem 4904. It should be appreciated that such a configuration can be advantageous because it avoids the risk of a single point of failure, where a single battery failure would cause the electric propulsion system 4902 to fail.

[0397] Some embodiments may include an electric propulsion system 4902, which includes an electric engine subsystem 4904 that receives signals from and transmits signals to a flight control system 4912. In some embodiments, the flight control system 4912 may include a flight control computer capable of sending commands to and receiving status and data from the electric engine subsystem 4904 using a controller area network (“CAN”) data bus signal. It should be understood that while a CAN data bus signal is used between the flight control computer and the electric engine, some embodiments may include any form of communication capable of sending and receiving data from the flight control computer to and from the electric engine. In some embodiments, the flight control system 4912 may also include a tilting propeller system (“TPS”) 4914 capable of sending analog discrete data to and receiving analog discrete data from the tilting propeller’s electric engine subsystem 4904. The tilt propeller system 4914 may include devices capable of transmitting operating parameters to the electric engine subsystem 4904 and articulating the orientation of the propeller subsystem 4906 to redirect the thrust of the tilt propeller by using mechanical components such as gearbox assemblies, linear actuators and any other configured components to change the orientation of the propeller subsystem 4906 during various phases of flight.

[0398] As discussed throughout, exemplary VTOL aircraft may have various types of electric propulsion systems, including tilt propellers and lift propellers, which include a front electric engine capable of tilting during various phases of flight and a tail electric engine that remains in one orientation and can operate only during certain phases of flight (i.e., takeoff, landing, and hovering).

[0399] In some embodiments, the flight control system may include a system capable of controlling control surfaces and their associated actuators in an exemplary VTOL aircraft. Figure 50 This is an illustration of a top plan view of an exemplary VTOL aircraft 5000 consistent with embodiments of this disclosure. The aircraft 5000 shown in the figure may be, respectively, in... Figure 44 and Figure 45 The diagram shows a top plan view of aircraft 4400 and 4500. In some embodiments, aircraft 5000 may be similar to... Figure 46The aircraft 4600. In the aircraft 5000, in addition to the propeller blades discussed previously, the control surface may also include ailerons 5072 and rudders 5074. Ailerons 5072 may combine the functions of one or more flaps, one or more ailerons, and / or one or more spoilers. Rudders 5074 may combine the functions of one or more rudders and / or one or more elevators. In some embodiments, the control surface may include, for example, flaps, ailerons, spoilers, rudders, or elevators. In the aircraft 5000, in addition to the electric propulsion system discussed previously, actuators may also include control surface actuators (CSAs) associated with, for example, ailerons 5072 and rudders 5074.

[0400] Working Example—Landing using IR random dot markers

[0401] The random point marker has proven to be robust against occlusion and reliable. The algorithm used in this example does not rely solely on frame-by-frame point tracking to identify points at shallow viewing angles; instead, it improves the nearest neighbor and descriptor computation to ensure that the marker can be redetected even if tracking fails.

[0402] Light detection The light source flickers at 1 / 3 of the camera's frame rate, and frames are processed in batches of three. The maximum and minimum grayscale intensities at each pixel between the three images are determined to construct the maximum and minimum images. Next, the minimum image is subtracted from the maximum image to remove any ambient background IR light. In the minimum-maximum image, the distribution combination will contrast sharply with the background, and keypoint registration and retrieval of both can then be performed using the pixel locations of the light.

[0403] Keypoint registration:Before live pose estimation can be performed, the distribution ensemble must be registered using known light positions and IDs. To identify points using LLAH, multiple "descriptors" are computed for each keypoint. To compute the descriptors, LLAH finds n nearest neighbors for each keypoint. During this step, the distribution ensemble is rescaled to have a 1:1 length-to-width ratio. Otherwise, the nearest neighbors might change under drastic viewpoint changes. Since cross-ratios are order-dependent, the nearest neighbors are sorted according to their clockwise position relative to the keypoints. Then, a combination of m points is selected from the sorted nearest neighbors. Among these m points, a cross-ratio is computed using a combination of four points and the keypoint. The cross-ratio for each of the four-point combinations is computed, discretized, and stored in the order they were computed. Finally, the descriptors are stored starting with the lowest discretized cross-ratio value in the sequence, while maintaining the original order. Overall, each descriptor will have a dimension of mC4, and each point will have nCm descriptors. A hash index is calculated for each descriptor, and the keypoint ID is stored at that index along with the associated descriptor.

[0404] Key point retrieval During live processing, taggers can be retrieved using either matching or tracking. If all keypoints were identified in the previous frame, the algorithm defaults to basic point tracking between the previous and current frames. Otherwise, the algorithm attempts both matching and tracking, using the result from the method with more identified points. To use matching for tagger retrieval, a descriptor is computed for each detected point in the image using the same method as for registration. The descriptor is then used to compute a hash index. At this index, voting is taken for each keypoint ID candidate with a matching descriptor. The keypoint is identified by the ID with the most votes exceeding a certain threshold, provided that ID has not yet been used to identify another point. After all keypoints have been processed during the matching process, homography between the live frame and points from known distribution combinations is then computed to confirm matches and identify points whose IDs were not assigned during matching.

[0405] Posture estimation If at least 20 of the 25 points are successfully identified in a frame, OpenCV's Iterative Perspective n-Point (PNP) pose estimation is used to determine the aircraft's pose relative to the distribution combination. The pose estimation is accepted if the change in roll between the previous and current frames does not exceed 2 degrees. This constraint ensures that impossible PNP poses are eliminated, as the aircraft should not be able to rotate 2 degrees within a single frame.

[0406] resultThe algorithm uses a helicopter to approach a randomly placed distribution combination during the day to obtain live data. Detecting IR LEDs can be more difficult during the day than at night due to interference from external reflections. The algorithm is able to identify the light at a distance of approximately 200m from the distribution combination. The current Python implementation of this data association algorithm takes 923ms to retrieve frames using matching and 0.371ms using tracking. For this dataset, 21% of these frames were retrieved purely by tracking, while the remainder required a matching process. The mean positional errors in each direction are -1.46m, -0.41m, and -0.39m, with standard deviations of 0.35m, 0.34m, and 0.22m, respectively. Figure 51 As shown.

[0407] Embodiments of this disclosure may be further described with respect to the following terms:

[0408] 1. A system comprising:

[0409] A landing surface for an airborne vehicle, the landing surface comprising:

[0410] Multiple light sources are arranged in a predetermined pattern, wherein the characteristics of the light emitted from each of the light sources are configured to be modulated relative to time.

[0411] 2. The system according to Clause 1, wherein the light source comprises a first set of light sources arranged in a first predetermined pattern, and wherein each of the first set of light sources is configured to be within the field of view of a camera associated with the air vehicle when the air vehicle is at a first distance from the landing surface.

[0412] 3. The system according to Clause 2, wherein the light source comprises a second set of light sources arranged in a second predetermined pattern, and wherein each of the second set of light sources is configured to be within the field of view of the camera when the aircraft is at a second distance from the landing surface.

[0413] 4. The system according to Clause 3, wherein each of the first group of light sources is configured to be outside the field of view of the camera when the aircraft is at the second distance from the landing surface.

[0414] 5. The system according to clause 3 or 4, wherein the area covered by the first group of light sources is larger than the area covered by the second group of light sources.

[0415] 6. The system according to any one of clauses 3 to 5, wherein the intensity of the first group of light sources is configured to be higher than the intensity of the second group of light sources.

[0416] 7. The system according to any one of clauses 1 to 6, wherein the predetermined pattern of the light source is associated with the landing surface.

[0417] 8. The system according to any one of clauses 1 to 7, wherein the light sources are arranged such that one of the light sources is uniquely identifiable in the predetermined pattern.

[0418] 9. The system according to any one of clauses 1 to 8, wherein modulation of the properties of the emitted light is configured to identify the landing surface.

[0419] 10. The system according to any one of clauses 1 to 8, wherein the modulation of said characteristics of the emitted light is configured to identify one of the light sources.

[0420] 11. The system according to any one of clauses 1 to 8, wherein modulation of the said characteristics of the emitted light is configured to identify the position of one of the light sources.

[0421] 12. The system according to any one of clauses 1 to 8, wherein modulation of the properties of the emitted light is configured to identify the state of the landing surface.

[0422] 13. The system according to claim 1 of any one of claims 1 to 9, wherein the modulation of said characteristics of the emitted light is configured to encode a signal authenticating the landing surface.

[0423] 14. The system according to any one of clauses 1 to 13, wherein modulation of the characteristics of the emitted light includes modulation of the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light.

[0424] 15. The system according to any one of clauses 1 to 14, wherein the wavelength of the emitted light from the light source is in the range of 800 nm to 850 nm.

[0425] 16. The system according to Clause 15, wherein the wavelength of the emitted light is approximately 810 nm.

[0426] 17. The system according to Clause 15, wherein the wavelength of the emitted light is approximately 1310 nm.

[0427] 18. The system according to Clause 15, wherein the wavelength of the emitted light is approximately 1550 nm.

[0428] 19. The system according to any one of Clauses 1 to 18, wherein the landing surface is a portable landing surface comprising a retractable landing mat, fabric, or tarpaulin.

[0429] 20. The system according to any one of clauses 1 to 19, further comprising controller circuitry configured to operate the light source.

[0430] 21. The system according to any one of clauses 1 to 20, wherein each of the light sources is recessed relative to the landing surface.

[0431] 22. The system according to any one of clauses 1 to 21, wherein each of the light sources includes an optical sensor configured to detect a portion of light emitted from at least one other light source among the light sources.

[0432] 23. The system according to any one of clauses 1 to 22, further comprising a plurality of landing surfaces, wherein each landing surface includes a plurality of light sources arranged in a predetermined pattern, and wherein the characteristics of the light emitted from each of the light sources are configured to be modulated relative to time.

[0433] 24. The system according to Clause 23, wherein the landing surfaces are displaced horizontally relative to each other.

[0434] 25. The system according to Clause 23, wherein the landing surfaces are vertically displaced relative to each other.

[0435] 26. The system according to Clause 23, wherein the landing surfaces are displaced horizontally and vertically relative to each other.

[0436] 27. A system comprising:

[0437] Aircraft, including:

[0438] A camera, configured to generate images based on information transmitted from multiple light sources located near the landing surface of the aircraft; and

[0439] The controller circuit is configured as follows:

[0440] Receive the generated image; and

[0441] The position and orientation of the airborne vehicle are determined based on the received images.

[0442] The light sources are arranged in a predetermined pattern on the landing surface, and the characteristics of the light emitted from each of the light sources are modulated relative to time.

[0443] 28. The system according to Clause 27, wherein the camera is configured to provide a planar view of the light source on the landing surface.

[0444] 29. The system according to Clause 27, wherein the camera is configured to provide a forward-looking view of the light source on the landing surface.

[0445] 30. The system according to any one of Clauses 27 to 29, wherein the camera includes an optical filter configured to allow wavelength ranges of light emitted from each of the light sources.

[0446] 31. The system described in Clause 30, wherein the permitted wavelength range is between 800 nm and 850 nm.

[0447] 32. The system as described in Clause 30, wherein the permitted wavelength range is approximately 810 nm.

[0448] 33. The system as described in Clause 30, wherein the permitted wavelength range is approximately 1310 nm.

[0449] 34. The system described in Clause 30, wherein the permitted wavelength range is approximately 1550 nm.

[0450] 35. The system according to any one of clauses 30 to 34, wherein the optical filter includes a bandpass filter configured to allow the wavelength range of light emitted from each of the light sources.

[0451] 36. The system according to any one of clauses 27 to 35, wherein the controller is further configured to adjust the capture rate of the camera based on the modulation rate of the light source.

[0452] 37. The system according to Clause 36, wherein the adjustment of the capture rate includes the synchronization of the capture rate of the camera with the modulation rate of the light source.

[0453] 38. The system according to clause 36 or 37, wherein the capture rate of the camera is at least 100 frames per second (Hz).

[0454] 39. The system according to any one of clauses 36 to 38, wherein the controller is further configured to adjust the flicker rate of the light source based on the capture rate.

[0455] 40. The system according to Clause 39, wherein the flicker rate of the light source is 30 Hz.

[0456] 41. The system according to any one of clauses 36 to 40, wherein the controller is further configured to adjust the bit transmission rate of the camera.

[0457] 42. The system according to Clause 41, wherein the bit transmission rate of the camera is 10 Hz or higher.

[0458] 43. The system according to any one of clauses 36 to 42, wherein the controller is further configured to transmit a synchronization pulse to synchronize the capture rate of the camera with the modulation rate of the light source.

[0459] 44. The system according to any one of Clauses 27 to 43, wherein modulation of the properties of light emitted from each of the light sources includes modulation of the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the emitted light.

[0460] 45. The system according to any one of clauses 27 to 44, wherein the camera is configured to be activated based on an activation signal from an external processor associated with the aircraft, an operator of the aircraft, or the controller.

[0461] 46. ​​The system according to any one of clauses 27 to 45, wherein the controller is further configured to generate an output signal including information associated with the position and orientation of the airborne vehicle.

[0462] 47. The system according to Clause 46, wherein the information associated with the location of the airborne vehicle includes the global positioning system (GPS) coordinates of the airborne vehicle.

[0463] 48. The system according to clause 46 or 47, wherein the information associated with the orientation of the air vehicle includes the orientation of the air vehicle relative to the landing surface.

[0464] 49. The system according to any one of clauses 46 to 48, wherein the controller is further configured to transmit the information associated with the position and orientation of the airborne vehicle to the external processor.

[0465] 50. The system according to any one of Clauses 27 to 49, wherein the camera is a color, monochrome, or hyperspectral camera.

[0466] 51. The system according to any one of clauses 27 to 50, wherein the camera is configured to be activated after the air vehicle is within a predetermined distance from the landing surface.

[0467] 52. The system according to Clause 51, wherein the predetermined distance is 500m or less.

[0468] 53. A system comprising:

[0469] Multiple light sources are arranged on the landing surface of an aerial vehicle, the arrangement of which defines a set of intersecting virtual lines, the light sources being arranged on each virtual line, wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0470] 54. The system according to Clause 53, wherein an equal number of light sources are arranged on each virtual line.

[0471] 55. The system described in Clause 53 or 54, wherein the linear cross ratio for each virtual line is independent of the viewing angle.

[0472] 56. The system according to any one of clauses 53 to 55, wherein the intersecting virtual lines define a plurality of regions, and wherein the area cross ratio for each region is independent of the viewing angle.

[0473] 57. A method for estimating the attitude of an airborne vehicle, comprising:

[0474] Provides a landing surface including light sources arranged in a predetermined pattern;

[0475] The characteristics of light emitted from the light source relative to time modulation;

[0476] A camera mounted on the aircraft is used to receive input signals associated with the light emitted from the light source;

[0477] Generate an image of the light source based on the received input signal;

[0478] Determining the position and orientation of the airborne vehicle based on the image, wherein determining the position and orientation of the airborne vehicle includes:

[0479] Detect at least one of the light sources in the image;

[0480] Determine which of at least one of the detected light sources is arranged in a predetermined pattern; and

[0481] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least one of the light sources arranged in the predetermined pattern is the detected light source.

[0482] 58. The method according to Clause 57, further comprising encoding information associated with the at least one of the light sources in the characteristics of the modulated light emitted from the light source.

[0483] 59. The method according to clause 57 or 58, wherein determining which of the at least one of the light sources arranged in the predetermined pattern is the detected light source includes using a processor to decode the encoded information associated with the light source.

[0484] 60. The method according to any one of clauses 57 to 59, wherein detecting the at least one light source among the light sources in the image further comprises background subtraction and thresholding.

[0485] 61. The method according to any one of clauses 57 to 60, wherein detecting the at least one light source among the light sources in the image further comprises image filtering techniques.

[0486] 62. The method according to Clause 61, wherein the image filtering includes temporal filtering, spatial filtering, or a combination thereof.

[0487] 63. The method according to any one of clauses 57 to 62, further comprising storing information associated with the predetermined pattern in a database.

[0488] 64. The method according to Clause 63, wherein determining which of the at least one of the light sources arranged in the predetermined pattern is the detected light source comprises matching points in the image with points in the database.

[0489] 65. The method according to any one of clauses 57 to 64, wherein determining which of the at least one of the light sources arranged in the predetermined pattern is the detected light source further includes calculating the cross ratio of the positions of the corresponding light sources.

[0490] 66. The method according to any one of clauses 57 to 65, wherein the predetermined pattern includes an arrangement of light sources defining a set of intersecting virtual lines, the light sources being arranged on each virtual line, and wherein the distance between adjacent light sources on each virtual line is non-uniform.

[0491] 67. The method described in Clause 66, wherein the linear cross ratio for each virtual line is independent of the viewing angle.

[0492] 68. The method according to clause 66 or 67, wherein the intersecting virtual lines define multiple regions, and wherein the area cross ratio for each region is independent of the viewing angle.

[0493] 69. The method according to any one of clauses 57 to 68, wherein the characteristics of the modulated emitted light include the intensity, frequency, amplitude, wavelength, phase, bandwidth, or duty cycle of the modulated emitted light.

[0494] 70. The method according to any one of clauses 57 to 69, wherein determining the position and orientation of the airborne vehicle comprises:

[0495] Detect at least four light sources in the image;

[0496] Determine which of the at least four light sources are arranged in the predetermined pattern; and

[0497] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least four light sources are arranged in the predetermined pattern.

[0498] 71. The method according to any one of clauses 57 to 69, wherein determining the position and orientation of the airborne vehicle comprises:

[0499] Detect at least five light sources in the image;

[0500] Determine which of the at least five light sources are arranged in the predetermined pattern; and

[0501] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least five light sources are arranged in the predetermined pattern.

[0502] 72. A computer-implemented system for estimating the attitude of an airborne vehicle, the system comprising:

[0503] A landing surface, comprising light sources arranged in a predetermined pattern; and

[0504] At least one processor is configured as follows:

[0505] The characteristics of light emitted from the light source relative to time modulation;

[0506] Activate the camera mounted on the airborne vehicle to receive the input signal associated with the light emitted from the light source;

[0507] This enables the camera to generate an image of the light source based on the received input signal;

[0508] Determining the position and orientation of the airborne vehicle based on the generated image, wherein determining the position and the orientation includes:

[0509] Detect at least one of the light sources in the image;

[0510] Determine which of at least one of the detected light sources is arranged in a predetermined pattern; and

[0511] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least one of the light sources arranged in the predetermined pattern is the detected light source.

[0512] 73. A computer-implemented method for estimating the attitude of an airborne vehicle, the method comprising the following operations performed by at least one processor:

[0513] The characteristics of light emitted from a light source arranged in a predetermined pattern on the landing surface of the aircraft relative to time modulation;

[0514] Activate the camera mounted on the aircraft to enable it to receive input signals associated with light emitted from the light source;

[0515] This enables the camera to generate an image of the light source based on the received input signal;

[0516] The position and orientation of the aerial vehicle are determined based on the image, wherein the position and orientation are determined.

[0517] To include:

[0518] Detect at least one of the light sources in the image;

[0519] Determine which of at least one of the detected light sources is arranged in a predetermined pattern; and

[0520] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least one of the light sources arranged in the predetermined pattern is the detected light source.

[0521] 74. The computer-implemented method according to Clause 73, wherein determining the position and orientation of the airborne vehicle comprises:

[0522] Detect at least four light sources in the image;

[0523] Determine which of the at least four light sources are arranged in the predetermined pattern; and

[0524] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least four light sources are arranged in the predetermined pattern.

[0525] 75. The computer-implemented method according to clause 73 or 74, wherein determining the position and orientation of the airborne vehicle comprises:

[0526] Detect at least five light sources in the image;

[0527] Determine which of the at least five light sources are arranged in the predetermined pattern; and

[0528] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least five light sources are arranged in the predetermined pattern.

[0529] 76. A non-transitory computer-readable medium storing a set of instructions executable by at least one processor of a device to cause the device to perform a method, the method comprising:

[0530] The characteristics of light emitted from a light source arranged in a predetermined pattern on a landing surface for an aircraft, relative to time modulation;

[0531] Activate the camera mounted on the aircraft to enable it to receive input signals associated with light emitted from the light source;

[0532] This enables the camera to generate an image of the light source based on the received input signal;

[0533] The position and orientation of the aerial vehicle are determined based on the image, wherein the position and orientation are determined.

[0534] To include:

[0535] Detect at least one of the light sources in the image;

[0536] Determine which of at least one of the detected light sources is arranged in a predetermined pattern; and

[0537] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least one of the light sources arranged in the predetermined pattern is the detected light source.

[0538] 77. The non-transitory computer-readable medium according to Clause 76, wherein the set of instructions executable by the at least one processor of the device causes the device to determine the position and orientation of the airborne vehicle, and wherein determining the position and orientation of the airborne vehicle includes:

[0539] Detect at least four light sources in the image;

[0540] Determine which of the at least four light sources are arranged in the predetermined pattern; and

[0541] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least four light sources are arranged in the predetermined pattern.

[0542] 78. The non-transitory computer-readable medium according to Clause 76, wherein the set of instructions executable by the at least one processor of the device causes the device to determine the position and orientation of the airborne vehicle, and wherein determining the position and orientation of the airborne vehicle includes:

[0543] Detect at least five light sources in the image;

[0544] Determine which of the at least five light sources are arranged in the predetermined pattern; and

[0545] The position and orientation of the airborne vehicle are determined based on the determination of which of the at least five light sources are arranged in the predetermined pattern.

[0546] 79. A computer-implemented method for locating a light source for a landing surface of an airborne vehicle, the method comprising the following operations performed by at least one processor:

[0547] Activate the camera mounted on the air vehicle to receive an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface for the air vehicle, the light having characteristics modulated relative to time.

[0548] This enables the camera to generate at least two images of the light source based on the received input signal;

[0549] This enables the detector to use a detection algorithm to detect at least one of the light sources in the at least two images.

[0550] A light source, wherein the detection algorithm includes the following steps:

[0551] The difference in pixel intensity values ​​between the at least two images is determined using a subtraction algorithm.

[0552] A pre-determined threshold pixel intensity is applied to the difference frame to generate a mask;

[0553] The mask is used to identify the location of pixels in the image representing the light source; and

[0554] Based on the position of the pixel in the image, the centroid algorithm is used to calculate the position of the light source on the landing surface.

[0555] 80. The computer-implemented method according to Clause 79, wherein the detection algorithm further comprises generating the difference frame based on a determined difference in pixel intensity values ​​of the at least two images.

[0556] 81. The computer-implemented method according to clause 79 or 80, wherein the properties of light are modulated to fully activate the light source in a first image of the at least two images and to fully deactivate the light source in a second image of the at least two images.

[0557] 82. The computer-implemented method according to Clause 81, wherein the first image comprises a signal image and the second image comprises a background image.

[0558] 83. The computer-implemented method according to clause 81 or 82, wherein the first image and the second image comprise consecutive images.

[0559] 84. The computer-implemented method according to any one of clauses 81 to 83, wherein the detection algorithm further comprises performing image registration to achieve background subtraction by aligning the first image and the second image.

[0560] 85. The computer-implemented method according to Clause 84, wherein the image registration is performed using techniques including feature matching, translation matching, current state estimation, or combinations thereof.

[0561] 86. The computer-implemented method according to any one of clauses 79 to 85, wherein the detection algorithm further comprises tracking the location of the identification of the pixel representing the light source by extrapolating a combination of information based on the speed of the airborne vehicle and the time elapsed between capturing the at least two images.

[0562] 87. The computer-implemented method according to any one of clauses 79 to 86, wherein the air vehicle includes an electric vertical takeoff and landing aircraft.

[0563] 88. A computer-implemented method for mapping a position in an image to a position on a landing surface for an airborne vehicle, the method comprising the following operations performed by at least one processor:

[0564] A detection algorithm is used to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light that can be detected by a camera mounted on the airborne aircraft.

[0565] A first association algorithm is used to associate the location of a detected light source in the image with the corresponding location of the light source on the landing surface, wherein the association algorithm includes the following steps:

[0566] Normalize the position of the detected light source in the image in Cartesian coordinate space;

[0567] The normalized positions are transformed into curves in polar coordinate space, wherein the collinear normalized positions in the Cartesian coordinate space form curves that intersect at common points in the polar coordinate space.

[0568] The polar coordinate space is discretized into multiple bins, each bin being represented by the number of times an indicator curve passes through that bin;

[0569] When it is determined whether the bin value exceeds a predetermined threshold, the position of the bin in the polar coordinate space is transformed to the Cartesian coordinate space;

[0570] Lines are formed in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin;

[0571] Clustering algorithms are used to group substantially parallel lines and form rectangular frames for integer grid space from the grouped lines.

[0572] The calculation is configured to move the point from the Cartesian coordinate space to an integer grid, and the homography matrix is ​​calculated.

[0573] The calculated homography matrix is ​​used to map each point to the integer grid.

[0574] 89. The computer-implemented method according to Clause 88, wherein normalizing the position in the image includes constructing a transformation matrix to calculate the mean of the position and setting the variance of the position to one.

[0575] 90. The computer-implemented method according to clause 88 or 89, wherein normalizing the position in the image further comprises rotating the Cartesian coordinate space by an angle to compensate for the rotation caused by the approach angle of the airborne vehicle toward the landing surface.

[0576] 91. The computer-implemented method according to any one of clauses 88 to 90, wherein the bin value is incremented by one for each instance of a curve passing through the location of the bin.

[0577] 92. A computer-implemented method according to any one of claims 88 to 91, wherein the first association algorithm further comprises optimizing one or more lines by eliminating one or more lines in the Cartesian coordinate space based on fitting a detected position of the light source in the image.

[0578] 93. The computer-implemented method according to Clause 92, wherein the first association algorithm further comprises iteratively optimizing the one or more lines.

[0579] 94. The computer-implemented method according to any one of clauses 88 to 93, further comprising using reference characters to mark each position on the integer grid.

[0580] 95. The computer-implemented method according to Clause 94, wherein the markers are based on a predefined sequence.

[0581] 96. A computer-implemented method according to any one of clauses 88 to 95, wherein each point is mapped to an integer grid indicating an offset distance, the offset distance being the distance between a reference position on the integer grid and the corresponding mapped point.

[0582] 97. The computer-implemented method according to Clause 96, further comprising excluding erroneous detections from the association based on the offset distance.

[0583] 98. The computer-implemented method according to Clause 97, wherein eliminating the error detection includes comparing the offset distance with a threshold offset distance.

[0584] 99. The computer-implemented method according to Clause 98, further comprising:

[0585] Mapped points whose offset distance is greater than the threshold offset distance are identified as error detections; and

[0586] The association is removed when the number of false detections exceeds an allowable threshold.

[0587] 100. The computer-implemented method according to any one of clauses 88 to 99, further comprising using a second association algorithm to associate the location of a detected light source in the image with a corresponding location of the light source on the landing surface.

[0588] 101. The computer-implemented method according to Clause 100, wherein the first association algorithm comprises a mesh association algorithm, and the second association algorithm comprises an Iterative Closest Point (ICP) algorithm, a Thin Plate Spline Robust Point Matching (TPS-RPM) algorithm, a point tracking algorithm, or a combination thereof.

[0589] 102. A system comprising:

[0590] The first plurality of light sources are arranged in a predetermined pattern on the landing surface of an aircraft; and

[0591] A second plurality of light sources are arranged along the flight path of the airborne vehicle to guide the vehicle, wherein the characteristics of the light emitted from each of the first plurality of light sources and the second plurality of light sources are configured to be modulated relative to time, and wherein the modulated light emitted from each of the first plurality of light sources and the second plurality of light sources is configured to be detectable by a camera mounted on the airborne vehicle.

[0592] 103. The system according to Clause 102, wherein the second plurality of light sources are disposed on the rooftop of a building, pole, tower, natural structure or structure positioned along the flight path.

[0593] 104. A system comprising:

[0594] A landing surface for an airborne vehicle, the landing surface comprising:

[0595] Line light sources with a predetermined pattern, on the landing surface of the aircraft, wherein the characteristics of the light emitted from the plurality of line light sources are configured to be modulated relative to time, and wherein the modulation is configured to encode a signal representing an identifier of the landing surface.

[0596] 105. The system according to Clause 104, further comprising a plurality of point light sources arranged in a predetermined pattern on the landing surface for the airborne vehicle, wherein the characteristics of the light emitted from the plurality of point light sources are configured to be modulated relative to time.

[0597] 106. The system according to clause 104 or 105, wherein the shape of the line light source of the predetermined pattern is rectangular.

[0598] 107. The system according to Clause 106, wherein the edges of the rectangular predetermined pattern of the line light source comprise two or more collinear segments, wherein each segment is configured to represent a single bit of information based on the activation state of the line light source in the segment.

[0599] 108. The system according to any one of clauses 104 to 107, further comprising an airborne vehicle including a camera mounted on the airborne vehicle, the camera being configured to receive coded signals.

[0600] 109. The system according to Clause 108, further comprising a processor communicatively associated with the camera, the processor being configured to decode the encoded signal received by the camera and generate an output based on the decoding.

[0601] 110. The system according to any one of clauses 106 to 109, wherein the shape of the line light source of the predetermined pattern is a triangle, a circle, an ellipse, a polygon, or a combination thereof.

[0602] 111. A method for identifying a landing surface for an airborne vehicle, the method comprising:

[0603] A camera mounted on the aircraft receives encoded signals from a plurality of line light sources arranged in a first predetermined pattern on the landing surface for the aircraft, wherein the encoded signals represent the markings of the landing surface, and the encoded signals are encoded by modulating the characteristics of the light emitted from the line light sources; and

[0604] The received encoded signal is decoded using a processor associated with the camera to generate an output that includes information associated with the markings on the landing surface.

[0605] The first predetermined pattern of line light sources includes collinear segments, each segment being configured to represent a single bit of information based on the activation state of the line light source within the segment.

[0606] 112. The method according to Clause 111, wherein in a first active state, the line segment represents a bit value of one, and wherein in a second active state, the line segment represents a bit value of zero.

[0607] 113. The method according to Clause 112, wherein the first activation state is the on state of the line light source in the line segment, and the second activation state is the off state.

[0608] 114. The method according to any one of clauses 111 to 113, further comprising receiving light emitted by a point light source in a second predetermined pattern on the landing surface of the airborne vehicle by a camera mounted on the airborne vehicle, wherein the characteristics of the light emitted from the point light source are modulated relative to time.

[0609] 115. A method for estimating the attitude of an airborne vehicle, the method comprising:

[0610] A distribution combination of light sources is provided on a portable landing surface at the landing site of the said airborne vehicle;

[0611] The relative positions of the light sources are calibrated using ultra-wideband signals between the light sources to determine the configuration of the distribution combination;

[0612] Information associated with a defined combination of light source distributions is transmitted to an airborne aircraft approaching the landing site.

[0613] 116. The method according to Clause 115, further comprising estimating the attitude of the air vehicle based on the distributed combination configuration and images of the landing surface captured by a camera mounted on and associated with the air vehicle.

[0614] 117. The method according to clause 115 or 116, wherein the portable landing surface includes a rapidly deployable landing surface, a re-deployable landing surface, a rollable pad, fabric, tarpaulin, mesh, or mesh structure.

[0615] 118. The method according to any one of clauses 115 to 117, wherein the portable landing surface is configured to conform to the contour of the landing site.

[0616] 119. The method according to any one of clauses 115 to 118, wherein the distribution combination of the light sources includes point light sources and line light sources.

[0617] 120. The method according to any one of clauses 115 to 119, wherein the distribution combination of the light sources includes battery-powered light sources configured to operate remotely.

[0618] 121. A method for estimating the attitude of an airborne vehicle, the method comprising:

[0619] A distribution combination of light sources is provided on the portable landing surface in a predetermined pattern;

[0620] The portable landing surface is placed on a contoured landing site, and the portable landing surface is configured to conform to the contoured landing site;

[0621] The configuration of the distribution combination of light sources in the estimated portable landing surface; and

[0622] The attitude of the aircraft is estimated based on the estimated distribution combination configuration and images of the landing surface captured by cameras mounted on and associated with the aircraft.

[0623] 122. The method according to Clause 121, wherein the portable landing surface includes a rapidly deployable landing surface, a re-deployable landing surface, a rollable pad, fabric, tarpaulin, mesh, or mesh structure.

[0624] 123. The method according to clause 121 or 122, wherein the distribution combination of the light sources includes a combination of point light sources and line light sources.

[0625] 124. The method according to any one of clauses 120 to 123, wherein the distribution combination of the light sources includes point light sources and line light sources.

[0626] 125. The method according to any one of clauses 121 to 124, wherein the distribution combination of the light sources includes battery-powered light sources configured to operate remotely.

[0627] 126. An aerial vehicle comprising:

[0628] A camera is configured to generate images based on information received from multiple light sources located on the landing surface of the aircraft.

[0629] A processor, associated with the camera and configured to receive the image and perform the following operations:

[0630] A detection algorithm is used to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light that can be detected by the camera;

[0631] The processor is configured to associate the position of the detected light source in the image with the corresponding position of the light source on the landing surface, wherein the processor is configured to perform the association in a first operating mode and a second operating mode.

[0632] In the first operating mode, one or more association algorithms are executed and confidence scores of the associations are generated;

[0633] One or more tracking algorithms are executed in the second operating mode based on the confidence score obtained from the first operating mode; and

[0634] The location or orientation of the airborne vehicle is determined based on the correlation performed.

[0635] 127. The airborne vehicle as described in Clause 126, wherein the processor is further configured to automatically switch between the first operating mode and the second operating mode based on a predetermined threshold confidence score.

[0636] 128. The airborne vehicle as described in Clause 127, wherein the processor is further configured to request user input to switch between the first operating mode and the second operating mode based on a predetermined threshold confidence score.

[0637] 129. An airborne aircraft according to any one of clauses 126 to 128, wherein the processor is further configured to sequentially execute the first operating mode and the second operating mode.

[0638] 130. The airborne aircraft according to any one of clauses 126 to 129, wherein said processor is further configured to:

[0639] Switching from the first operating mode to the second operating mode, and

[0640] After switching from the first operating mode to the second operating mode, the first operating mode and the second operating mode are executed in parallel.

[0641] 131. An airborne vehicle according to any one of clauses 126 to 130, wherein the one or more association algorithms include a mesh association algorithm, a thin plate spline robust point matching (TPS-RPM) association algorithm, or an iterative nearest point (ICP) algorithm.

[0642] 132. An airborne vehicle according to any one of clauses 126 to 131, wherein the one or more tracking algorithms include local associated point tracking or attitude-based point tracking.

[0643] 133. An airborne aircraft according to any one of clauses 126 to 132, wherein the detection algorithm is configured to detect the modulation of the characteristics of the plurality of light sources relative to time.

[0644] 134. The aerial vehicle as described in Clause 133, wherein the plurality of light sources comprises a combination of line light sources and point light sources.

[0645] 135. The airborne vehicle as described in Clause 134, wherein the line light source and the point light source are arranged in a predetermined pattern on the landing surface.

[0646] 136. The airborne vehicle as described in Clause 135, wherein the landing surface includes a portable landing surface, a rollable landing surface, a retractable landing surface, a tarpaulin, a mesh, a mesh structure, or a combination thereof.

[0647] 137. An airborne aircraft according to any one of clauses 126 to 136, wherein performing the one or more association algorithms includes the following steps:

[0648] Normalize the position of the detected light source in the image in Cartesian coordinate space;

[0649] The normalized positions are transformed into curves in polar coordinate space, wherein the collinear normalized positions in the Cartesian coordinate space form curves that intersect at common points in the polar coordinate space.

[0650] The polar coordinate space is discretized into multiple bins, each bin being represented by the number of times an indicator curve passes through that bin;

[0651] When it is determined whether the bin value exceeds a predetermined threshold, the position of the bin in the polar coordinate space is transformed to the Cartesian coordinate space;

[0652] Lines are formed in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin;

[0653] Clustering algorithms are used to group substantially parallel lines and form rectangular frames for integer grid space from the grouped lines.

[0654] The calculation is configured to move the point from the Cartesian coordinate space to an integer grid, and the homography matrix is ​​calculated.

[0655] The calculated homography matrix is ​​used to map each point to the integer grid.

[0656] 138. The airborne vehicle according to Clause 137, wherein the processor is further configured to normalize the position in the image by constructing a transformation matrix to calculate the mean of the position and setting the variance of the position to one.

[0657] 139. The airborne vehicle as described in Clause 137, wherein the processor is further configured to normalize the position in the image by rotating the Cartesian coordinate space by an angle to compensate for rotation caused by the approach angle of the airborne vehicle toward the landing surface.

[0658] 140. An airborne aircraft according to any one of clauses 137 to 139, wherein the processor is further configured to increment the bin value by one for each instance of a curve passing through the location of the bin.

[0659] 141. An airborne aircraft according to any one of clauses 137 to 140, wherein the processor is further configured to optimize one or more lines by eliminating one or more lines in the Cartesian coordinate space based on fitting the detected position of the light source in the image.

[0660] 142. An airborne aircraft according to any one of clauses 137 to 141, wherein the processor is further configured to use reference characters to label each position on the integer grid, and wherein the label is based on a predefined sequence.

[0661] 143. An airborne aircraft according to any one of clauses 137 to 142, wherein the processor is further configured to map each point to an integer grid indicating an offset distance, the offset distance being the distance between a reference position on the integer grid and the corresponding mapped point.

[0662] 144. An airborne aircraft according to any one of clauses 137 to 143, wherein the processor is further configured to remove erroneous detections from the association based on the offset distance, the removal of erroneous detections comprising comparing the offset distance with a threshold offset distance.

[0663] 145. An airborne aircraft according to any one of clauses 137 to 144, wherein the processor is further configured to identify mapped points whose offset distance is greater than the threshold offset distance as false detections, and is configured to discard the associations when it is determined that the number of false detections exceeds an allowable threshold.

[0664] 146. An airborne aircraft according to any one of clauses 126 to 145, wherein determining the position or orientation of the airborne aircraft is further based on information from one or more of a Global Positioning System (GPS) or an Inertial Navigation System (INS).

[0665] 147. An airborne aircraft according to any one of clauses 126 to 146, wherein determining one of the airborne aircraft's position or orientation based on a performed association includes determining both the airborne aircraft's position and its orientation.

[0666] 148. The airborne aircraft according to any one of clauses 126 to 147, further comprising:

[0667] Elevating propeller; and

[0668] A controller is configured to actuate the lift propellers based on a determined position or orientation of the airborne vehicle.

[0669] 149. The airborne aircraft according to any one of clauses 126 to 148, further comprising:

[0670] Inclined propeller; and

[0671] A controller is configured to actuate the tilting propeller based on the determined position or orientation of the airborne vehicle.

[0672] 150. The airborne aircraft according to any one of clauses 126 to 149, further comprising:

[0673] Tilting actuator; and

[0674] A controller is configured to actuate the tilt actuator based on the determined position or orientation of the airborne vehicle.

[0675] 151. The airborne aircraft according to any one of clauses 126 to 150, further comprising:

[0676] Control surfaces; and

[0677] A controller configured to actuate the control surface based on the determined position or orientation of the airborne vehicle.

[0678] 152. The airborne aircraft as described in Clause 151, wherein the control surface includes either ailerons or directional elevators.

[0679] 153. A navigation system for an airborne vehicle, comprising:

[0680] The camera is configured to generate images based on information received from multiple light sources arranged in a predetermined pattern on the landing surface for an aerial vehicle.

[0681] A processor, associated with the camera and configured to receive the image and perform the following operations:

[0682] The processor is used to activate a camera mounted on the airborne vehicle, enabling the reception of an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface of the airborne vehicle, the light having characteristics modulated relative to time.

[0683] This enables the camera to generate at least two images of the light source based on the received input signal;

[0684] Use a detection algorithm to detect light sources in at least two images;

[0685] The processor is configured to associate the position of the detected light source in the image with the corresponding position of the light source on the landing surface, wherein the processor is configured to perform the association in a first operating mode and a second operating mode.

[0686] Execute one or more association algorithms in the first operation mode;

[0687] Based on the results obtained from the first operating mode, execute one or more tracking algorithms in the second operating mode; and

[0688] The location or orientation of the airborne vehicle is determined based on the correlation performed.

[0689] 154. The navigation system according to Clause 153, further comprising: the plurality of light sources arranged on the landing surface in the predetermined pattern.

[0690] 155. The navigation system according to Clause 154, wherein the characteristics of the light emitted from each of the light sources are modulated relative to time.

[0691] 156. The navigation system according to any one of clauses 153 to 155, further comprising a controller configured to actuate a lift propeller based on the determined position or orientation of the airborne vehicle.

[0692] 157. The navigation system according to any one of clauses 153 to 156, further comprising a controller configured to actuate a tilting propeller based on the determined position or orientation of the airborne vehicle.

[0693] 158. The navigation system according to any one of clauses 153 to 157, further comprising a controller configured to actuate a tilt actuator based on the determined position or orientation of the airborne vehicle.

[0694] 159. The navigation system according to any one of clauses 153 to 158, further comprising a controller configured to actuate a control surface based on the determined position or orientation of the airborne vehicle.

[0695] 160. The navigation system according to Clause 159, wherein the control surface includes either ailerons or directional elevators.

[0696] 161. A system comprising:

[0697] Landing surfaces for aircraft; and

[0698] Multiple light sources are arranged in a predetermined pattern, and the characteristics of the light emitted from each light source are configured to be modulated relative to time.

[0699] The plurality of light sources includes line light sources and point light sources, and

[0700] The landing surface mentioned above includes a portable landing surface.

[0701] 162. The system according to Clause 161, further comprising:

[0702] A first processor, configured to modulate the characteristics of light emitted from the light source relative to time; and

[0703] The second processor is configured as follows:

[0704] Activate the camera mounted on the aircraft to receive the input signal associated with the light emitted from the light source;

[0705] This enables the camera to generate an image of the light source based on the received input signal;

[0706] Determining either the position or the orientation of the airborne vehicle based on the generated image, wherein determining the position or the orientation includes:

[0707] Detect at least one of the light sources in the image;

[0708] Determine which of at least one of the detected light sources is arranged in a predetermined pattern; and

[0709] The position or orientation of the airborne vehicle is determined based on the determination of which of the at least one of the light sources arranged in the predetermined pattern.

[0710] 163. The system according to Clause 162, further comprising a controller configured to actuate the lift propellers based on a determined position or orientation of the airborne vehicle.

[0711] 164. The system according to clause 162 or 163 further includes a controller configured to actuate a tilting propeller based on the determined position or orientation of the airborne vehicle.

[0712] 165. The system according to any one of clauses 162 to 164, further comprising a controller configured to actuate a tilt actuator based on the determined position or orientation of the airborne vehicle.

[0713] 166. The system according to any one of clauses 162 to 165, further comprising a controller configured to actuate a control surface based on the determined position or orientation of the airborne vehicle.

[0714] 167. The system according to any one of clauses 162 to 166, wherein the control surface includes one of ailerons or directional elevators.

[0715] 168. The system according to any one of clauses 161 to 167, wherein the portable landing surface is configured to conform to the contour of the landing site.

[0716] 169. The system according to any one of clauses 161 to 168, wherein the plurality of light sources includes battery-powered light sources configured to operate remotely.

[0717] 170. The system according to any one of clauses 161 to 169, wherein the portable landing surface comprises a retractable landing surface, a rollable pad, fabric, tarpaulin, mesh, or mesh structure.

[0718] 171. The system according to any one of clauses 161 to 169, wherein the second processor is further configured to use an ultra-wideband signal between the light sources to calibrate the relative positions of the light sources.

[0719] 172. The system according to any one of clauses 161 to 171, wherein the plurality of light sources comprises a plurality of infrared light sources.

[0720] 173. A computer-readable medium storing instructions that, when executed by at least one processor of a device, cause the device to perform the method according to any one of clauses 57 to 69, 71 to 75, 79 to 101, or 111 to 125.

[0721] The foregoing description has been presented for illustrative purposes. It is not exhaustive and does not limit the invention to the precise forms or embodiments disclosed herein. Modifications and adaptations of the invention will be apparent to those skilled in the art upon consideration of the description and practice of the disclosed embodiments of the invention.

Claims

1. An aerial vehicle comprising: A camera is configured to generate images based on information received from multiple light sources arranged in a predetermined pattern on the landing surface of the aircraft. A processor, associated with the camera and configured to receive the image and perform the following operations: A detection algorithm is used to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light that can be detected by the camera; The locations of detected light sources in the images are associated with corresponding locations of the light sources on the landing surface. The association includes a first operating mode and a second operating mode. The first operating mode includes executing one or more association algorithms to map the detected light sources to a known pattern and generate a confidence score for the association. The second operating mode includes executing one or more tracking algorithms based on the confidence score obtained from the first operating mode to track the locations of the plurality of light sources between the images. as well as The location or orientation of the airborne vehicle is determined based on the correlation performed.

2. The airborne vehicle of claim 1, wherein the processor is further configured to automatically switch between the first operating mode and the second operating mode based on a predetermined threshold confidence score.

3. The airborne vehicle of claim 2, wherein the processor is further configured to request user input to switch between the first operating mode and the second operating mode based on a predetermined threshold confidence score.

4. The airborne vehicle according to claim 1, wherein the processor is further configured to sequentially execute the first operating mode and the second operating mode.

5. The airborne vehicle according to claim 1, wherein the processor is further configured to: Switching from the first operating mode to the second operating mode, and After switching from the first operating mode to the second operating mode, the first operating mode and the second operating mode are executed in parallel.

6. The airborne vehicle according to claim 1, wherein the one or more association algorithms include a mesh association algorithm, a thin plate spline robust point matching (TPS-RPM) association algorithm, or an iterative nearest point (ICP) algorithm.

7. The aerial vehicle of claim 1, wherein the one or more tracking algorithms include local associated point tracking or attitude-based point tracking.

8. The airborne vehicle of claim 1, wherein the detection algorithm is configured to detect the modulation of the characteristics of the plurality of light sources relative to time.

9. The aerial vehicle according to claim 8, wherein the plurality of light sources comprises a combination of line light sources and point light sources.

10. The airborne vehicle of claim 1, wherein performing the one or more association algorithms includes the following steps: Normalize the position of the detected light source in the image in Cartesian coordinate space; The normalized positions are transformed into curves in polar coordinate space, wherein the collinear normalized positions in the Cartesian coordinate space form curves that intersect at common points in the polar coordinate space. The polar coordinate space is discretized into multiple bins, each bin being represented by the number of times an indicator curve passes through that bin; When it is determined that the bin value exceeds a predetermined threshold, the position of the bin in the polar coordinate space is transformed to the Cartesian coordinate space; Lines are formed in the Cartesian coordinate space, each line connecting at least a plurality of points equal to the value of the corresponding bin; Clustering algorithms are used to group substantially parallel lines and form rectangular frames for integer grid space from the grouped lines. The homography matrix configured to move the point from the Cartesian coordinate space to an integer grid is calculated; as well as The calculated homography matrix is ​​used to map each point to the integer grid.

11. The aerial vehicle of claim 10, wherein the processor is further configured to normalize the position in the image by constructing a transformation matrix to calculate the mean of the position and setting the variance of the position to one.

12. The airborne vehicle of claim 10, wherein the processor is further configured to normalize the position in the image by rotating the Cartesian coordinate space by an angle to compensate for rotation caused by the approach angle of the airborne vehicle toward the landing surface.

13. The airborne vehicle of claim 10, wherein the processor is further configured to increment the bin value by one for each instance of a curve passing through the location of the bin.

14. The aerial vehicle of claim 10, wherein the processor is further configured to remove one or more lines in the Cartesian coordinate space based on fitting the detected position of the light source in the image.

15. The airborne vehicle of claim 10, wherein the processor is further configured to use reference characters to label each position on the integer grid based on a predefined sequence.

16. The airborne vehicle of claim 10, wherein the processor is further configured to map each point to an integer grid indicating an offset distance, the offset distance being the distance between a reference position on the integer grid and the corresponding mapped point.

17. The airborne vehicle of claim 16, wherein the processor is further configured to remove erroneous detections from the association based on the offset distance, the removal of erroneous detections comprising comparing the offset distance with a threshold offset distance.

18. The airborne vehicle of claim 17, wherein the processor is further configured to identify mapped points whose offset distance is greater than the threshold offset distance as erroneous detections, and is configured to discard the associations when it is determined that the number of erroneous detections exceeds an allowable threshold.

19. The airborne vehicle of claim 1, wherein determining the position or orientation of the airborne vehicle is further based on information from one or more of the Global Positioning System (GPS) or Inertial Navigation System (INS).

20. The airborne vehicle according to claim 1, further comprising: A controller configured to actuate components of the aircraft based on a determined position or orientation of the aircraft; The component said therein includes one of a lifting propeller, a tilting propeller, a tilting actuator, or a control surface.

21. A method for operating an aerial vehicle, the method comprising: Images are generated using a camera based on information received from multiple light sources arranged in a predetermined pattern on the landing surface of the aircraft. A detection algorithm is used to detect light sources in the image, the light sources being arranged on the landing surface and configured to emit light that can be detected by the camera; The location of the detected light source in the image is associated with the corresponding location of the light source on the landing surface, wherein the association includes a first operating mode and a second operating mode. The first operating mode includes executing one or more association algorithms to map the detected light source to a known pattern and generate a confidence score for the association; The second operating mode includes performing one or more tracking algorithms based on the confidence score obtained from the first operating mode to track the positions of the plurality of light sources between the images; as well as The location or orientation of the airborne vehicle is determined based on the correlation performed.

22. A navigation system for an airborne vehicle, comprising: The camera is configured to generate images based on information received from multiple light sources arranged in a predetermined pattern on the landing surface for an aerial vehicle. A processor, associated with the camera and configured to receive the image and perform the following operations: The processor is used to activate a camera mounted on the airborne vehicle, enabling the reception of an input signal associated with light emitted from a light source arranged in a predetermined pattern on the landing surface of the airborne vehicle, the light having characteristics modulated relative to time. This enables the camera to generate at least two images of the light source based on the received input signal; A detection algorithm is used to detect the light source in the at least two images; Associating the position of the detected light source in the image with the corresponding position of the light source on the landing surface, wherein the association includes a first operating mode and a second operating mode; The first operating mode includes executing one or more association algorithms to map the detected light source to a known pattern and generate a confidence score for the association; The second operating mode includes performing one or more tracking algorithms based on a confidence score obtained from the first operating mode to track the positions of the plurality of light sources between the images; as well as The location or orientation of the airborne vehicle is determined based on the correlation performed.

23. The navigation system according to claim 22, further comprising: The plurality of light sources are arranged on the landing surface in the predetermined pattern.

24. The navigation system of claim 23, wherein the landing surface comprises the landing surface of a vertical takeoff and landing airport.

25. The navigation system of claim 23, wherein the landing surface comprises a portable landing surface.

26. The navigation system of claim 25, wherein the portable landing surface comprises a re-deployable landing surface.

27. The navigation system of claim 25, wherein the portable landing surface comprises a rollable pad.

28. The navigation system of claim 25, wherein the portable landing surface comprises fabric.

29. The navigation system of claim 25, wherein the portable landing surface comprises a waterproof tarpaulin.

30. The navigation system of claim 25, wherein the portable landing surface comprises a mesh.

31. The navigation system of claim 25, wherein the portable landing surface comprises a mesh structure.

32. The navigation system of claim 23, wherein the characteristics of the light emitted from each of the light sources are modulated relative to time.

33. The navigation system according to claim 22, further comprising: A controller configured to actuate components of the aircraft based on a determined position or orientation of the aircraft. The component said therein includes one of a lifting propeller, a tilting propeller, a tilting actuator, or a control surface.

Images