Motion-Based Calibration of Unmanned Aerial Vehicles

Abstract

A calibration of the unmanned aerial vehicle is performed without the use of a magnetometer. The unmanned aerial vehicle generates a first acceleration vector in a navigational reference frame and a second acceleration vector in a GPS reference frame. The unmanned aerial vehicle estimates a heading of the unmanned aerial vehicle based on the first acceleration vector and the second acceleration vector. The unmanned aerial vehicle performs a calibration based on the estimated heading of the unmanned aerial vehicle.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Patent Application No. 17 / 875,850, filed July 28, 2022, and U.S. Provisional Patent Application No. 63 / 321,217, filed March 18, 2022, the disclosures of each of which are incorporated by reference in their entireties herein.

[0002] The present disclosure relates to aircraft calibration. [Background technology]

[0003] Unmanned Aerial Vehicles (UAVs) typically require periodic calibration of one or more sensors. One of these sensors is the UAV's magnetometer. The magnetometer calibration procedure is time consuming and often the resulting magnetometer calibration is unreliable.

[0004] The present disclosure is best understood from the following detailed description when read in conjunction with the accompanying drawings, in which: It is emphasized that, according to common practice, the various features of the drawings are not to scale. To the contrary, dimensions of the various features have been arbitrarily expanded or reduced for clarity. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 illustrates an example of a UAV system. [Figure 2A] FIG. 1 is a diagram showing an example of a UAV seen from above. [Figure 2B] FIG. 1 is a diagram showing an example of a UAV seen from below. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of a UAV. [Figure 4] FIG. 1 is a block diagram illustrating exemplary software functions of a UAV system. [Diagram 5] 1 is a flowchart illustrating an example of a motion-based calibration method for a UAV. [Figure 6]1 is a flowchart illustrating an example of a method for estimating a heading of a UAV. [Figure 7A] FIG. 13 illustrates an exemplary graphical user interface (GUI) display for entering a motion-based calibration mode. [Figure 7B] FIG. 13 illustrates an exemplary graphical user interface (GUI) display for entering a motion-based calibration mode. [Figure 7C] FIG. 13 illustrates an exemplary graphical user interface (GUI) display for entering a motion-based calibration mode. [Figure 8A] FIG. 13 illustrates an exemplary GUI display with user actions for performing motion-based calibration. [Figure 8B] FIG. 13 illustrates an exemplary GUI display with user actions for performing motion-based calibration. [Figure 8C] FIG. 13 illustrates an exemplary GUI display with user actions for performing motion-based calibration. [Figure 9] FIG. 13 illustrates an exemplary GUI display for a calibration mode option. [Figure 10] 13 is an example of a plot comparison of heading estimates during motion-based calibration, UAV yaw angle, and heading estimate uncertainty. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] The autonomous navigation capabilities of a UAV traditionally rely on various on-board sensors that generate data based on the UAV and / or the environment in which the UAV is operating. The data is typically processed on the UAV to determine one or more aspects of the UAV's functionality, including, for example, where and how the UAV will fly, whether to capture images and what to focus on in those images, whether to follow a subject or follow a defined flight path, etc. This processing typically takes into account various environmental and UAV constraints, such as the location of obstacles (e.g., objects) in the environment in which the UAV is operating, indications of whether those obstacles are stationary or moving, the speed capabilities of the UAV, and other external factors acting on the UAV in flight.

[0007] One or more navigation sensors on a UAV must be calibrated to generate an accurate heading for the UAV. One common sensor used for UAV navigation is the magnetometer on the UAV. Traditional UAV magnetometers require periodic calibration. However, magnetometer calibration can be time consuming. Furthermore, it can be difficult to know if the magnetometer calibration is valid at a given time. Because magnetometer calibration relies on the Earth's magnetic field, magnetometers are susceptible to magnetic interference, for example from nearby metal objects, the UAV itself, metal in the environment, or any combination thereof. Furthermore, magnetometers are difficult to use during flight because they can be affected by rotating motors. Thus, traditional UAV magnetometers suffer from the drawback of being unreliable.

[0008] The embodiments of the present disclosure address problems such as these with a magnetometer-independent calibration method. The UAV disclosed herein is configured for motion-based calibration that can be performed without the use of a magnetometer. The UAV is configured to calculate an Inertial Measurement Unit (IMU) vector related to the UAV direction and a Global Positioning System (GPS) vector related to the GPS orientation, and calibrate the UAV based on the correlation between the UAV direction and the GPS orientation. Some embodiments are described herein as being performed via the UAV or as being implemented in a flight control subsystem onboard the UAV. However, alternative embodiments may be performed on an aircraft that is not a UAV and / or may be implemented remotely on a user device or server that communicates with the UAV.

[0009] Some embodiments disclosed herein include various engines, each of which is configured, programmed, configured, or otherwise adapted to perform a function or set of functions. As used herein, the term engine refers to a tangible device, component, or arrangement of components implemented using hardware, such as an application specific integrated circuit (ASIC) or field programmable gate array (FPGA), or as a combination of hardware and software, such as a processor-based computing platform and a set of program instructions that transform the computing platform into a dedicated device for implementing a particular function. An engine may also be implemented as a combination of the two, where certain functions are facilitated solely by hardware and other functions are facilitated by a combination of hardware and software.

[0010] As an example, the software may exist on a tangible, machine-readable storage medium in executable or non-executable form. Software existing in non-executable form may be compiled, translated, or otherwise converted into executable form before or during execution. As an example, the software, when executed by the engine's underlying hardware, causes the hardware to perform specified operations. Thus, the engine is physically configured, specifically configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a specified manner or to perform some or all of any operations described herein in connection with that engine.

[0011] Considering the example where the engines are temporarily configured, each engine may be instantiated at different times. For example, if the engines comprise general purpose hardware processor cores configured using software, the general purpose hardware processor cores may be configured as different engines at different times. The software may accordingly configure the hardware processor cores to, for example, configure a particular engine at one time and a different engine at a different time.

[0012] In particular implementations, at least a portion, and possibly all, of the engines execute on the processor(s) of one or more computers that execute the operating system, system programs, and application programs, while the engines may be implemented using multitasking, multithreading, distributed (cluster, peer-to-peer, cloud, etc.) processing, or other such techniques where appropriate. Thus, each engine may be realized in a variety of suitable configurations and generally should not be limited to the particular implementations illustrated herein unless such limitations are expressly invoked.

[0013] In addition, the engine itself may be composed of one or more sub-engines, each of which may be considered an engine in its own right. Further, in the embodiments described herein, each of the various engines corresponds to a defined function. However, it should be understood that in other contemplated embodiments, each function may be distributed among one or more engines. Similarly, in other contemplated embodiments, multiple defined functions may be implemented by a single engine that performs those functions, possibly together with other functions, or may be distributed among a set of engines in a manner different than that specifically shown in the examples herein.

[0014] To describe some implementations in more detail, reference will first be made to an example of a hardware and software structure that may be used to implement motion-based calibration of a UAV. Figure 1 illustrates an example of a UAV system 100. The system 100 includes a UAV 102, a controller 104, a dock 106, and a server 108.

[0015] The UAV 102 is a device that may be autonomously controlled by one or more on-board processing aspects or may be remotely controlled by an operator, for example, using the controller 104. The UAV 102 may be implemented as one of many types of unmanned aerial vehicles configured for aerial operation. For example, the UAV 102 may be a device commonly referred to as a drone, but may otherwise be a device configured to fly with a human operator present therein. In particular, the UAV 102 may be a multi-rotor vehicle. For example, the UAV 102 may be lifted and propelled by four fixed-pitch rotors, and may achieve in-flight positioning by varying the angular velocity of each of these rotors.

[0016] The controller 104 is a device configured to control at least some operations associated with the UAV 102. The controller 104 may communicate with the UAV 102 via a wireless communication link (e.g., via a Wi-Fi network, a Bluetooth link, a ZigBee link, or another network or link), receive video or images, and / or issue commands (e.g., commands related to taking off, landing, following, manual control, and / or performing autonomous or semi-autonomous navigation of the UAV 102). The controller 104 may be or include a dedicated device. Alternatively, the controller 104 may be or include a mobile device, e.g., a smartphone, tablet, laptop, or other device, capable of executing software configured to communicate with the UAV 102 and at least partially control the UAV.

[0017] The dock 106 is a structure that can be used for takeoff and / or landing operations of the UAV 102. In particular, the dock 106 can include one or more fiducials that can be used by the UAV 102 for autonomous takeoff and landing operations. For example, the fiducials may generally include markings that can be detected using one or more sensors of the UAV 102 to guide the UAV 102 from or to a particular location on or in the dock 106. In some implementations, the dock 106 may further include components for charging a battery of the UAV 102 while the UAV 102 is on or in the dock 106. The dock 106 may be a protective enclosure from which the UAV 102 is launched. The location of the dock 106 may correspond to a launch point of the UAV 102.

[0018] The server 108 is a remote computing device that can receive information usable for the operation of the UAV 102 and / or transmit information acquired by the UAV 102. For example, the server 108 may be used to train learning models usable by one or more aspects of the UAV 102 to implement the functionality of the UAV 102. In another example, the server 108 may receive a signal from the server 108 that includes information usable to update aspects of the UAV 102. The server 108 may communicate with the UAV 102 over a network, such as the Internet, a local area network, a wide area network, or another public or private network.

[0019] In some implementations, system 100 may include one or more additional components not shown in Figure 1. In some implementations, one or more components shown in Figure 1, such as server 108, may be omitted from system 100.

[0020] An exemplary diagram of UAV 102, which may be, for example, UAV 200 shown in FIG. 1, is shown in FIG. 2A and FIG. 2B. FIG. 2A illustrates an example of UAV 200 as viewed from above. UAV 200 includes a propulsion mechanism 202 including several propellers (e.g., four) and a motor configured to rotate the propellers. For example, UAV 200 may be a quadcopter drone. UAV 200 includes an image sensor including a high-resolution image sensor 204. This image sensor 204 may be, for example, gimbal mounted to support stable, blur-free image capture and object tracking. UAV 200 also includes image sensors 206, 208, 210 spaced around the top periphery of UAV 200 and covered with respective fisheye lenses to provide a wide field of view and support stereoscopic computer vision. Image sensors 206, 208, and 210 generally have a lower resolution than that of image sensor 204. UAV 200 also includes other internal hardware, such as a processing unit (not shown). In some implementations, the processing unit is configured to automatically fold the propellers when entering a dock (e.g., dock 106 shown in FIG. 1), allowing the dock to have a smaller footprint than the area swept by the propellers of propulsion mechanism 202.

[0021] FIG. 2B illustrates an example of the UAV 200 as viewed from below. From this perspective, three more image sensors 212, 214, 216 are visible, located at the bottom of the UAV 200. These image sensors 212, 214, and 216 may also be covered with respective fisheye lenses to provide a generally wide field of view and support stereoscopic computer vision. The various image sensors of the UAV 200 may enable visual inertial odometry (VIO) for high-resolution localization and obstacle detection and avoidance. For example, the image sensors may be used to capture images including infrared data that may be processed for day or night mode navigation of the UAV 200. The UAV 200 also includes a battery in a battery pack 220 attached to the bottom of the UAV 200 and has conductive contacts 218 that enable battery charging. The bottom surface of the battery pack 220 may be the bottom surface of the UAV 200.

[0022] Figure 3 is a block diagram illustrating an example of a hardware configuration of a UAV 300, which may be, for example, the UAV 102 shown in Figure 1. The UAV 300 includes a processing unit 302, a data storage unit 304, a sensor interface 306, a communication interface 308, a propulsion control interface 310, a user interface 312, and an interconnect 314 by which the processing unit 302 may access other components.

[0023] The processing unit 302 is operable to execute instructions stored in the data storage unit 304 or elsewhere. The processing unit 302 is a processor with a random access memory (RAM) for temporarily storing instructions read from the data storage unit 304 or elsewhere while the instructions are being executed. The processing unit 302 may include a single processor or multiple processors, each having single or multiple processing cores. Alternatively, the processing unit 302 may include another type of device or multiple devices capable of manipulating or processing data. The processing unit 302 may be located in a processing unit such as a central processing unit (CPU) or a graphics processing unit (GPU).

[0024] Data storage device 304 is a non-volatile information storage device, such as, for example, a solid state drive, a read only memory device (ROM), an optical disk, a magnetic disk, or another suitable type of storage device, such as a non-transitory computer readable memory. Storage device 304 may include another type of device or devices that may store data for retrieval or processing by processing device 302. Processing device 302 may access and manipulate the data stored in data storage device 304 via an interconnect 314, which may be, for example, a bus or a wired or wireless network (e.g., a vehicle area network).

[0025] The sensor interface 306 is configured to control and / or receive data from one or more sensors of the UAV 300. The data may refer to, for example, one or more of temperature measurements, pressure measurements, Global Positioning System (GPS) data, acceleration measurements, angular velocity measurements, magnetic flux measurements, visible spectrum images, infrared images, images including infrared and visible spectrum data, and / or other sensor outputs. For example, the one or more sensors from which the data is generated may include one or more of an image sensor 316, an accelerometer 318, a gyroscope 320, a geolocation sensor 322, a barometer 324, and / or another sensor. In some implementations, the accelerometer 318 and the gyroscope 320 may be combined as an inertial measurement unit (IMU). In some implementations, the sensor interface 306 may implement a serial port protocol (e.g., I2C or SPI) for communicating with one or more sensor devices over conductors. In some implementations, the sensor interface 306 may include a wireless interface for communicating with one or more sensor groups via low-power, short-range communication technologies (eg, using vehicle area network protocols).

[0026] The communication interface 308 facilitates communication with one or more other devices, such as a paired dock (e.g., dock 106), a controller (e.g., controller 104), or another apparatus, such as a user computing device (e.g., a smartphone, tablet, or other device). The communication interface 308 may include a wireless interface and / or a wired interface. For example, the wireless interface may facilitate communication over a Wi-Fi network, a Bluetooth link, a ZigBee link, or another network or link. In another example, the wired interface may facilitate communication over a serial port (e.g., RS-232 or USB). The communication interface 308 further facilitates communication over a network, which may be, for example, the Internet, a local area network, a wide area network, or another public or private network.

[0027] The propulsion control interface 310 is used by the processing unit to control the propulsion system of the UAV 300 (e.g., including one or more propellers driven by electric motors). For example, the propulsion control interface 310 may include circuitry for converting digital control signals from the processing unit 302 into analog control signals for the actuators (e.g., the electric motors that drive the respective propellers). In some implementations, the propulsion control interface 310 may implement a serial port protocol (e.g., I2C or SPI) for communicating with the processing unit 302. In some implementations, the propulsion control interface 310 may include a wireless interface for communicating with the one or more motors via low-power short-range communications (e.g., vehicle area network protocols).

[0028] User interface 312 allows for the input and output of information from / to a user. In some implementations, user interface 312 may include a display, which may be a liquid crystal display (LCD), a light emitting diode (LED) display (e.g., an OLED display), or another suitable display. In some such implementations, user interface 312 may be or include a touch screen. In some implementations, user interface 312 may include one or more buttons. In some implementations, user interface 312 may include a position input device, such as a touch pad, a touch screen, or another suitable human or machine interface device.

[0029] In some implementations, UAV 300 may include one or more additional components not shown in Figure 3. In some implementations, one or more components shown in Figure 3, such as user interface 312, may be omitted from UAV 300.

[0030] Figure 4 is a block diagram illustrating example software functionality of a UAV system, which may be, for example, the system 100 shown in Figure 1. In particular, the software functionality is represented as on-board software 400 operating on a UAV, for example, the UAV 102 shown in Figure 1. The on-board software 400 includes an acceleration vector generation tool 402, an autonomous navigation tool 404, and a global heading update tool 406.

[0031] The acceleration vector generation tool 402 configures the UAV for motion-based calibration. The acceleration vector generation tool 402 configures the UAV to obtain acceleration signals from one or more accelerometers and angular rate signals from one or more gyroscopes. The acceleration vector generation tool 402 configures the UAV to fuse the one or more accelerometers and the one or more gyroscopes in a complementary filter. The acceleration vector generation tool 402 configures the complementary filter to obtain and combine the acceleration and angular rate signals into a combined signal for output.

[0032] The acceleration vector generation tool 402 configures the UAV to estimate an orientation of the UAV body. The estimate of the orientation of the UAV body may be based on a composite signal output from the complementary filter. The acceleration vector generation tool 402 configures the UAV to calculate a first acceleration vector in a navigational reference frame, for example, based on the estimated orientation of the UAV body. The acceleration vector generation tool 402 configures the UAV to determine a global velocity from a GPS signal. The acceleration vector generation tool 408 configures the UAV to calculate a second acceleration vector in a GPS reference frame.

[0033] Autonomous navigation tools 404 include functionality for enabling autonomous flight of the UAV. The autonomous flight functionality of the UAV generally includes switching between using a camera for vision-based navigation and using the UAV's onboard GPS and IMU for position-based navigation. In particular, the autonomous flight of the UAV may use position-based navigation if objects in the environment in which the UAV is operating are determined to be at least some distance away from the UAV, and the autonomous flight of the UAV may instead use vision-based navigation if those objects are determined to be no further away than that distance from the UAV.

[0034] In position-based navigation, the UAV may receive a series of position signals through a GPS receiver. The received GPS signals may indicate the position of the UAV in a world reference frame. The UAV may use the position signals from the GPS receiver to determine the position and velocity of the UAV. The UAV may determine acceleration and orientation signals in the navigation reference frame based on acceleration signals from one or more accelerometers and angular rate signals from one or more gyroscopes, which may be associated with an IMU onboard the UAV, for example.

[0035] In vision-based navigation, one or more on-board cameras of the UAV may continuously or periodically collect data that can be used to generate images. The images may be processed in real-time or substantially real-time to identify objects in the environment in which the UAV is operating and determine the relative position of the UAV with respect to those objects. Depth estimation may be performed to determine the relative position of the UAV with respect to the objects. Performing depth estimation includes modeling depth values ​​for various pixels of an image generated based on data collected using the on-board cameras. The depth values ​​may be modeled according to RGB inputs collected for the target pixels, for example. Based on these depth values ​​and the output from the on-board IMU, the UAV may avoid object collisions by evaluating a trajectory of the UAV towards the detected object.

[0036] The global heading update tool 406 includes functionality for estimating and updating the UAV heading in a GPS reference frame (e.g., based on the latitude, longitude, and attitude of the UAV). The global heading update tool 406 configures the UAV to estimate the UAV heading by running a histogram filter. The histogram filter can be run recursively by aligning the first acceleration vector and the second acceleration vector. The global heading update tool 406 can configure the histogram filter to provide an uncertainty value to determine whether the motion-based calibration procedure is complete.

[0037] The global heading update tool 406 configures the UAV to obtain a time window of acceleration from the first acceleration vector and the second acceleration vector. The global heading update tool 406 configures the UAV to perform batch optimization, for example, to improve an estimate of the UAV heading. The global heading update tool 406 can configure the UAV to improve the estimate of the UAV heading by accounting for a time delay between the IMU and GPS signals. The global heading update tool 406 configures the UAV to update an estimate of the UAV heading and to calibrate the UAV based on the estimate of the UAV heading.

[0038] FIG. 5 is a flow chart illustrating an example of a motion-based calibration method 500 for a UAV. At 502, the motion-based calibration method includes acquiring sensor data. The sensor data is acquired when a user performs a manual movement or motion with the UAV. For example, the manual movement or motion may be a lateral hand-waving motion (e.g., swinging back and forth horizontally) while holding the UAV. The acquired sensor data may include gyroscope sensor data, accelerometer sensor data, and GPS sensor data. In some examples, additional sensor data may be acquired, such as magnetometer sensor data, barometer sensor data, or both.

[0039] At 504, the motion-based calibration method 500 includes determining a global heading uncertainty value. The global heading uncertainty value is a value of the confidence (or lack thereof) in the estimated UAV heading. The global heading uncertainty value may be based on the uncertainty of the measured azimuth angle. The uncertainty of the measured azimuth angle may be based on a calculation of the mean and standard deviation of the measured azimuth angles. The measured azimuth angle may range from -180 degrees to +180 degrees. The measured azimuth angle is calculated based on the sensor data obtained in operation 502. The global heading uncertainty value is determined continuously. In some implementations, the global heading uncertainty value may be determined continuously or periodically.

[0040] At 506, the motion-based calibration method 500 includes determining whether the global heading uncertainty is less than a threshold. The global heading uncertainty is a statistical measure of the reliability of the estimated global heading (e.g., the UAV heading in a GPS reference frame based on the latitude, longitude, and attitude of the UAV). For example, the sensor data may be continuously acquired (at 502) and the global heading uncertainty may be continuously determined (at 504) until the global heading uncertainty is less than the threshold. For example, the threshold may be a measured azimuth angle of 15 degrees or less.

[0041] At 508, the motion-based calibration method 500 includes initializing a filter, such as an extended Kalman filter. The filter is initialized to perform state estimation of the UAV while it is navigating. The filter may perform the state estimation based on accelerometer sensor data, gyroscope sensor data, GPS sensor data, barometer sensor data, or any combination thereof. The state estimation is based on an estimated global heading and includes UAV position, UAV velocity, and UAV orientation, allowing the UAV to be controlled in dark (e.g., low light) environments, such as at night, where visual sensors are ineffective.

[0042] At 510, the motion-based calibration method 500 includes generating a notification. The notification may be a visual notification, an audible notification, a tactile notification, or any combination or variation thereof, providing an indication to the user that the calibration can be performed and that manual movement or exercise with the UAV can be stopped. The notification may be generated in response to determining that the uncertainty value is below (or is below) a threshold. In one example, the notification may be generated and transmitted for display on a screen of a remote device, such as a controller. For example, the notification may include text such as "calibration finished," "stop shaking," or "drop drone for takeoff." In another example, the notification may be a visual notification, such as illuminating an LED on the UAV a particular color. For example, the notification may include illuminating an LED on the UAV green or another color. In another example, the notification may be an audible notification. The audible notification may be a sound emitted from the UAV, the controller, or both. In yet another example, the notification may be a haptic notification, for example, the haptic notification vibrates the UAV and / or the controller. It should be appreciated that the notification may include combinations or variations of the above examples.

[0043] At 512, the motion-based calibration method 500 includes preparing for takeoff. Preparing for takeoff includes calibrating the UAV based on the global heading value determined to have an uncertainty value less than a threshold. Additionally, preparing for takeoff may include determining that the UAV is on a level and stable surface (e.g., solid ground for ground launch or a steady hand for hand-held launch) based on the state estimation, and performing various pre-flight checks. The determination that the UAV is on a stable surface may be based on sensor data, such as image sensor data, GPS data, accelerometer data, gyroscope data, or any combination thereof.

[0044] FIG. 6 is a flow chart illustrating an example of a method 600 for estimating heading of a UAV. At 602, the method 600 includes fusing an accelerometer and a gyroscope sensor in a complementary filter. The complementary filter is a sensor fusion technique that includes a low-pass filter and a high-pass filter. In an inertial sensor-based attitude estimation, the dynamic motion characteristics of the gyroscope sensor are complementary to the dynamic motion characteristics of the accelerometer sensor. The complementary filter is configured to obtain acceleration signals from one or more accelerometers during a lateral hand-waving motion and combine them with angular rate signals obtained from one or more gyroscopes. The complementary filter is configured to output the acceleration signal and the angular rate signal as a composite signal. The composite signal may be an average of the acceleration signal and the angular rate signal. The composite signal may be adjusted by assigning weights to the acceleration signal and the angular rate signal. In some examples, the composite signal may be adjusted based on other factors, such as a barometer value.

[0045] At 604, the method 600 includes estimating an orientation of the UAV body. The estimation of the orientation of the UAV body may be based on the composite signal output from the complementary filter. The estimation of the orientation of the UAV body may be performed by measuring a gravity vector and ignoring other vectors, such as an external acceleration vector based on an acceleration signal and an angular rate signal.

[0046] At 606, the method 600 includes calculating a first acceleration vector in a navigational reference frame. The navigational reference frame is a local reference frame when the UAV is powered on and is based on acceleration data from the accelerometer and gyro sensors. In the navigational reference frame, the UAV is aligned with a gravity vector and GPS data can be ignored. The first acceleration vector can be calculated based on an estimated orientation of the UAV body. The first acceleration vector is associated with the IMU and can be referred to as an IMU vector.

[0047] At 608, the method 600 includes determining a velocity from the GPS signals in the GPS reference frame. Determining the velocity may include calculating a velocity of the UAV based on the GPS signals obtained during the lateral hand-waving motion.

[0048] At 610, the method 600 includes applying a low pass filter to the velocity. At 612, the method 600 includes calculating a second acceleration vector in a navigation reference frame. The second acceleration vector may be calculated based on an output of the low pass filter. The low pass filter is configured to smooth the data to remove noise. The second acceleration vector may be associated with a GPS and may be referred to as a GPS vector.

[0049] At 614, the method 600 includes estimating the UAV heading. The UAV heading can be estimated by recursively running a histogram filter by aligning the first acceleration vector with the second acceleration vector. In one example, the horizontal portion of the first acceleration vector may be aligned with the horizontal portion of the second acceleration vector. In this example, the first and second acceleration vectors are three-dimensional and have X, Y, and Z values. Two dimensions may be used (e.g., X and Y values) to calculate the UAV heading. In this example, the UAV heading may be calculated as follows: Vector 1 (x1, y1, z1) and Vector 2 (x2, y2, z2) -> Yaw 1 = atan2 (y1, x1), Yaw 2 = atan2 (y2, x2) -> Heading = Yaw 1 - Yaw 2. In this example, the Z value is essentially ignored when calculating the UAV heading.

[0050] The histogram filter is configured to converge the estimated UAV heading between the navigational reference frame and the GPS reference frame. The histogram filter may be configured to provide an uncertainty value of the estimated UAV heading angle to determine whether the procedure is complete. For example, the procedure may be determined to be complete when the uncertainty value falls below a threshold value, for example 15 degrees. The estimation of the UAV heading may be performed continuously.

[0051] At 616, the method 600 includes obtaining a time window of acceleration from the IMU and GPS vectors calculated in operations 606 and 612, respectively. The time window may include the last 5 or 10 seconds of the acquired data. At 618, the motion-based calibration method 600 includes performing a batch optimization. The batch optimization operation may be performed on the time window of acquired data to refine the UAV heading given the initial UAV heading calculated in operation 614. The batch optimization operation may improve the estimate of the UAV heading by taking into account the time delay between the IMU and GPS signals. The batch optimization operation may include using a set of acceleration data collected during the motion-based calibration and processing the set of acceleration data using a least mean squares algorithm, such as a Levenberg-Marquardt algorithm, to optimize the estimate of heading given an initial estimate from a histogram filter.

[0052] At 620, the method 600 includes updating the UAV heading. Updating the UAV heading may include resetting an estimated heading between the navigational reference frame and the GPS reference frame to zero. The UAV heading may be updated based on an output of the batch optimization operation. Updating the UAV heading may include storing the updated heading, for example, in a memory of the UAV, a memory of the controller, or both. The updated UAV heading may be used to calibrate the UAV for flight. In some examples, the batch operations to refine the UAV heading may continue during flight operations of the UAV. Updating the UAV heading may include initializing the UAV for flight.

[0053] At 622, the method 600 may include continuing to estimate the UAV heading. The UAV heading may be estimated by recursively running a histogram filter by aligning the first acceleration vector and the second acceleration vector. The histogram filter may be configured to provide an uncertainty value of the estimated UAV heading angle to determine whether the procedure is complete. For example, the procedure may be determined to be complete when the uncertainty value falls below a threshold value, e.g., 15 degrees. The estimation of the UAV heading may be performed continuously, for example, during flight.

[0054] 7A-7C are diagrams illustrating exemplary graphical user interface (GUI) displays for entering a motion-based calibration mode. As shown in FIG. 7A, GUI display 702 is an example of a main screen that includes a settings icon 704. When the settings icon 704 is activated, such as by pressing a touch display, a GUI display 706 is presented to the user. The GUI display 706 includes a number of settings that the user can enable, disable, or change. In this example, the hand gesture setting 708 is set to "off." To activate motion-based calibration, the user selects the hand gesture setting 708, which causes a GUI display 710 to be displayed on the controller.

[0055] GUI display 710 includes a toggle switch 712. A user may activate hand calibration by sliding toggle switch 712 to the "on" position, as shown in FIG. 7A. After sliding toggle switch 712 to the "on" position, a user may select back button 714 to return to the previous GUI display, as shown as GUI display 716. As shown in GUI display 716, motion-based calibration is activated, as represented by hand gesture motion setting 718 being set to "on."

[0056] Once motion-based calibration is active, the user may return to the main screen, as shown in GUI display 720 of FIG. 7B. At this point, the UAV is ready to fly. When the user selects start flight button 722, the autonomous engine may be started, as shown in GUI 724. Once the autonomous engine is loaded, the system may determine that motion-based calibration is required, as shown in GUI display 726. GUI display 726 includes instructions for the user to execute, such as "quickly shake the drone side to side at arm's length," as shown in FIG. 7B. The system is configured to collect data, such as accelerometer data, gyroscope data, GPS data, or any combination thereof, while the user is shaking the UAV. Once the system determines that sufficient data has been collected, the system may display instructions, such as "stop shaking" and / or "download drone for takeoff," as shown in GUI display 728 of FIG. 7C. Once the UAV determines that the UAV is set up, for example using data from one or more sensors, the system may start the autonomous engine in preparation for takeoff, as shown in GUI display 730.

[0057] 8A-8C are diagrams illustrating example GUI displays with user actions for performing motion-based calibration. As shown in FIG. 8A, GUI display 802 indicates that UAV 804 has been placed on the ground by user 806 and is ready to fly. When user 806 selects start flight button 808, the autonomous engine may be started, as shown in GUI 810. Once the autonomous engine is loaded, the system may determine that motion-based calibration is required, as shown in GUI display 812. GUI display 812 includes instructions for user 806 to execute, such as "quickly swing the drone left and right at arm's length," as shown in FIG. 8A. As shown in FIG. 8B, user 806 may lift UAV 804 off the ground and swing the UAV left and right as instructed in GUI display 814. As shown in image 816, the user swings UAV 804 to the right, then to the left, as shown in image 818. The user 806 continues this waving motion, and the system is configured to collect data, such as accelerometer data, gyroscope data, GPS data, or any combination thereof, while the user 806 is waving the UAV 804. Once the system determines that sufficient data has been collected, the system may display instructions such as "Stop shaking" and / or "Put drone down for takeoff," as shown in GUI display 820 of FIG. 8B. As shown in image 822, the user 806 stops shaking the UAV 804 as instructed. The system displays GUI display 820 until the user 806 places the UAV 804 on the ground, as shown in image 824. Once the UAV 804 determines that it is placed, as shown in image 826, the system may start the autonomous engine in preparation for takeoff, as shown in GUI display 828 shown in FIG. 8C. Once the autonomous engine is loaded, the UAV 804 may view the environment, as shown in GUI display 830. The UAV 804 may check the environment, for example for obstacles, to determine if it is safe to take off. Once the UAV 804 determines that it is safe to take off, the system displays a launch button 832 on the GUI display 834.In one example, the UAV 804 can take off when the user 806 presses the launch button 832 .

[0058] 9 is an example GUI display 900 for calibration mode options. As shown in FIG. 9, the GUI display 900 may display a hand-wave calibration option 902 and a magnetometer calibration option 904. In some implementations, hand-wave calibration may be available as the default GPS night flight calibration option. Options to switch between the hand-wave calibration option 902 and the magnetometer calibration option 904 may be nested in the GPS night flight menu. Changes to these options may persist across flight and power cycles.

[0059] 10 is an example of a plot comparison 1000 of heading estimate 1002, UAV yaw angle 1004, and heading estimate uncertainty 1006 during motion-based calibration. The motion-based calibration starts with an initial heading estimate of zero (0) degrees, as shown at 1008, an initial UAV yaw angle of +180 degrees, as shown at 1010, and an initial uncertainty of 250, as shown at 1012.

[0060] The motion of the UAV during the motion-based calibration is shown as 1014. As shown at 1014, the UAV yaw angle may vary between +180 degrees and -180 degrees during the motion-based calibration. As the motion is performed, the heading estimate improves as the relative heading between the navigational and GPS reference frames converge at 1016 and the uncertainty decreases at 1018. If, for example, the uncertainty falls below a threshold at 1020, a notification is generated at 1022 instructing the user to land the UAV (e.g., on the ground). Upon determining that the UAV has landed, a filter, such as an extended Kalman filter, is initialized at 1024 and the relative heading between the navigational and GPS reference frames is reset to zero (0) at 1026.

[0061] The embodiments of the present disclosure can be described in terms of functional block components and various processing operations. Such functional block components can be realized by a number of hardware or software components that perform the specified functions. For example, the disclosed embodiments can employ various integrated circuit components (e.g., memory elements, processing elements, logic elements, look-up tables, etc.) that can perform various functions under the control of one or more microprocessors or other controllers.

[0062] Similarly, where elements of the disclosed embodiments are implemented using software programming or elements, the systems and techniques may be implemented in programming or scripting languages ​​such as C, C++, Java, JavaScript, Assembler, etc., and various algorithms are implemented with combinations of data structures, objects, processes, routines, or other programming elements.

[0063] Functional aspects may be implemented in algorithms executed on one or more processors. Additionally, implementations of the systems and techniques disclosed herein may employ numerous conventional techniques for electronics configuration, signal processing or control, data processing, and the like. The terms "mechanism" and "component" are used broadly and are not limited to mechanical or physical implementations, but may include software routines in combination with processors and the like. Similarly, the terms "system" or "tool" as used herein and in the drawings may be understood to correspond to functional units implemented using software, hardware (e.g., integrated circuits such as ASICs), or a combination of software and hardware, depending on their context, in any case. In certain contexts, such systems or mechanisms may be understood to be processor-implemented software systems or mechanisms that are part of or callable by executable programs, and may themselves be composed in whole or in part of such linked systems or mechanisms.

[0064] The above disclosed embodiments or portions of the embodiments may take the form of a computer program product, for example accessible from a computer usable or computer readable medium. The computer usable or computer readable medium may be, for example, any device that can tangibly store, store, communicate, or transmit a program or data structure for use by or in association with a processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device.

[0065] Other suitable media may be utilized. Such computer usable or computer readable media may be referred to as non-transitory memory and may include volatile or non-volatile memory that changes over time. The memory of the devices described herein need not be physically stored by the device, unless otherwise specified, but may be remotely accessible by the device and need not be contiguous with other memory that the device may physically store.

[0066] Although the present disclosure has been described in connection with particular embodiments, it is to be understood that the disclosure is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent arrangements as permitted under law.

Claims

1. An unmanned aerial vehicle (UAV), an accelerometer configured to generate an acceleration signal; a gyroscope configured to generate an angular rate signal; a global positioning system (GPS) sensor configured to detect GPS signals; continuously determining a global heading based on sensor data including an uncertainty, the global heading being the heading of the UAV in a GPS reference frame based on the latitude, longitude, and attitude of the UAV, the uncertainty being based on a standard deviation of the measured azimuth angles; determining that the uncertainty value is less than a threshold value; in response to determining that the uncertainty value is less than the threshold value; initializing an extended Kalman filter to perform state estimation of the UAV; and, if the uncertainty value is less than the threshold, calibrating at least one of the gyroscope, the accelerometer, or the GPS sensor based on the global heading; a processor configured to: An unmanned aerial vehicle (UAV) comprising:

2. The UAV of claim 1 , wherein the measured azimuth angle ranges from −180 degrees to +180 degrees.

3. The UAV of claim 1 or 2, wherein the processor is configured to acquire acceleration signals from the accelerometer, angular velocity signals from the gyroscope, and GPS signals from the GPS sensor until the uncertainty value is less than a threshold value.

4. The UAV of claim 1 or 2, wherein the threshold is a confidence value of the measured azimuth angle.

5. The processor: Generate a notification if the uncertainty value is less than the threshold value 3. The UAV of claim 1 or 2, further configured as follows:

6. The processor: transmitting the notification to the remote device via the remote device to trigger a visual, audio, or tactile notification; The UAV of claim 5 further configured as follows:

7. The UAV of claim 5 , wherein the notification includes one or more of a visual notification, an audible notification, and a tactile notification.

8. The UAV of claim 7 , wherein the visual notification comprises a text display or light-emitting diode (LED) illumination.

9. The non-transitory computer-readable medium storing instructions, when executed by an on-board computer of an unmanned aerial vehicle (UAV), causes the on-board computer to: Fusing the accelerometer and gyroscope sensors in a complementary filter; Estimating the orientation of the UAV; determining a first acceleration vector in a navigational reference frame; Determine speed from Global Positioning System (GPS) signals, determining a second acceleration vector in a GPS reference frame; and Estimating a heading of the UAV based on the first acceleration vector and the second acceleration vector; Non-transitory computer-readable medium.

10. Execution of instructions by an on-board computer of the UAV causes the on-board computer to: Obtaining a time window of acceleration data; and performing batch optimization to refine the estimated heading; The non-transitory computer-readable medium of claim 9.

11. The non-transitory computer-readable medium of claim 9 , wherein the navigational frame of reference is based on data from the accelerometer and gyroscope sensors.

12. At least one of the GPS reference frames is based on the latitude, longitude, and attitude of the UAV; determining the second acceleration vector includes applying a low pass filter; estimating the heading of the UAV includes recursively executing a histogram filter; the histogram filter providing uncertainty values; The non-transitory computer-readable medium of claim 9 or 10, wherein estimating the heading of the UAV includes aligning the first acceleration vector and the second acceleration vector.

13. 1. A method for use in an unmanned aerial vehicle (UAV), comprising: fusing accelerometer and gyroscope sensors in a complementary filter to estimate the orientation of the UAV; determining a first acceleration vector in a first frame of reference; determining speed from a Global Positioning System (GPS) signal; determining a second acceleration vector in a second frame of reference; Estimating a direction of travel of the UAV based on the first acceleration vector and the second acceleration vector; A method comprising:

14. the first reference frame is a navigation reference frame; and / or The method of claim 13 , wherein the second reference frame is a GPS reference frame.

15. The method of claim 13 or 14, wherein determining the second acceleration vector comprises applying a low pass filter.

Images