Systems and methods for modifying autonomous flight of unmanned aerial vehicles

Abstract

Systems and methods for modifying autonomous flight of one or more unmanned aerial vehicles (UAVs) are provided herein. In some instances, autonomous flight of a UAV can be implemented by first user input received at a first user interface. While autonomous flight is maintained, second user input received at a second user interface can modify autonomous flight of the UAV. Wherein the second user input modifies autonomous flight of the UAV during a duration of the second user input to assist a user and modify autonomous flight of the UAV during autonomous flight.

Description

TECHNICAL FIELD

[0001] Embodiments of the present invention relate to the field of unmanned aerial vehicle technology, and in particular to a system and method for modifying autonomous flight of an unmanned aerial vehicle. BACKGROUND

[0002] Aerial vehicles have a wide range of practical applications including surveillance, reconnaissance, exploration, logistics transportation, disaster relief, aerial photography, large-scale agricultural automation, real-time video broadcasting, and the like. In some applications, an aerial vehicle carrying a payload (e.g., a camera) can be controlled to fly around a target to acquire data or perform certain tasks. With advances in sensor and navigation technology, autonomous flight or control of aerial vehicles is possible. The utility of autonomously flying aerial vehicles can be improved. SUMMARY

[0003] Currently, aerial vehicles can fly along a pre-determined trajectory or an autonomously planned trajectory during autonomous flight. Examples of autonomous flight can include autonomous return to home of an aerial vehicle, autonomous navigation of an aerial vehicle along one or more waypoints, and / or autonomous flight to a point of interest. During autonomous flight, user intervention can be limited, or can interrupt autonomous flight of an aerial vehicle. However, in some situations, it can be desirable to enable a user to quickly intervene or supplement the ability of an aerial vehicle to autonomously fly. For example, during autonomous return to home of an aerial vehicle, input from a user can help avoid obstacles such as buildings (e.g., if the aerial vehicle does not have obstacle avoidance sensors). Furthermore, in some situations, a user can desire to slightly modify the flight of an aerial vehicle while still relying on the ability of the aerial vehicle to autonomously operate to complete a given task. For example, a user can wish to deviate from a selected target or destination.

[0004] Accordingly, there is a need for the ability to modify autonomous flight of an aerial vehicle. This ability can be provided through an intuitive and easy-to-use flight control system that allows a person to modify and / or influence autonomous flight of an aerial vehicle through interaction with a human-system interface. The burden of manually piloting an aerial vehicle by a user can be significantly reduced while the user can still exert some degree of control or modification when needed or beneficial.

[0005] Accordingly, in one aspect, a system for modifying autonomous flight of an unmanned aerial vehicle (UAV) is provided. The system includes a first user interface configured to receive a first user input, wherein the first user input provides one or more instructions to implement autonomous flight of the UAV, and a second user interface configured to receive a second user input, wherein the second user input provides one or more instructions to modify autonomous flight of the UAV.

[0006] In another aspect, a method of modifying autonomous flight of an unmanned aerial vehicle (UAV) is provided. The method includes receiving first user input at a first user interface, where the first user input provides one or more instructions to implement autonomous flight of the UAV, and receiving second user input at a second user interface, where the second user input provides one or more instructions to modify autonomous flight of the UAV.

[0007] In another aspect, a non-transitory computer-readable medium for modifying flight of an unmanned aerial vehicle (UAV) is provided. The non-transitory computer-readable medium includes code, logic, or instructions to: receive first user input at a first user interface, where the first user input provides one or more instructions to implement autonomous flight of the UAV, and receive second user input at a second user interface, where the second user input provides one or more instructions to modify autonomous flight of the UAV.

[0008] In another aspect, a system for modifying autonomous flight of an unmanned aerial vehicle (UAV) is provided. The system includes a flight controller configured to: (1) generate a first set of signals to implement autonomous flight of the UAV in response to first user input received at a first user interface, and (2) generate a second set of signals to modify autonomous flight of the UAV in response to second user input received at a second user interface.

[0009] In another aspect, a method of modifying autonomous flight of an unmanned aerial vehicle (UAV) is provided. The method includes generating, with a flight controller, a first set of signals to implement autonomous flight of the UAV in response to first user input received at a first user interface, and generating, with the flight controller, a second set of signals to modify autonomous flight of the UAV in response to second user input received at a second user interface.

[0010] In another aspect, a non-transitory computer-readable medium for modifying flight of an unmanned aerial vehicle (UAV) is provided. The non-transitory computer-readable medium includes code, logic, or instructions to: generate, with a flight controller, a first set of signals to implement autonomous flight of the UAV in response to first user input received at a first user interface, and generate, with the flight controller, a second set of signals to modify autonomous flight of the UAV in response to second user input received at a second user interface.

[0011] In another aspect, a system for modifying flight of an unmanned aerial vehicle (UAV) is provided. The system includes one or more processors individually or collectively configured to: implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while the autonomous flight is maintained.

[0012] In another aspect, a system for modifying flight of an unmanned aerial vehicle (UAV) is provided. The system includes one or more processors individually or collectively configured to: implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while the autonomous flight is maintained.

[0013] In another aspect, a system for modifying flight of an unmanned aerial vehicle (UAV) is provided. The system includes one or more processors individually or collectively configured to: implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while the autonomous flight is maintained.

[0014] In another aspect, a system for modifying flight of an unmanned aerial vehicle (UAV) is provided. The system includes one or more processors individually or collectively configured to: implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while the autonomous flight is maintained.

[0015] In another aspect, a system for modifying flight of an unmanned aerial vehicle (UAV) is provided. The system includes one or more processors individually or collectively configured to: implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while the autonomous flight is maintained.

[0016] It should be understood that different aspects of the application can be appreciated individually, collectively, separately, or in any combination. Various aspects of the application described herein can be applied to any of the particular applications set forth below or to any other type of movable object. Any illustration herein of an aircraft can be applied to any movable object, such as any vehicle. Also, systems, devices, and methods disclosed herein in the context of aerial motion (e.g., flight) can also be applied in the context of other types of motion, such as motion on the ground or on water, underwater motion, or space motion.

[0017] Other objects and features of the present application will become apparent from the descriptions, drawings, and claims.

[0018] By way of example, an example of a system for navigation according to an embodiment is shown.

[0019] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. BRIEF DESCRIPTION OF DRAWINGS

[0020] The novel features of the application are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present application will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the application are utilized, and the accompanying drawings of which:

[0021] Figure 1 An example of a system for navigation according to an embodiment is shown.

[0022] Figure 2 A user terminal according to an embodiment is shown.

[0023] Figure 3 A first user interface and a second user interface that work in cooperation according to an embodiment are shown.

[0024] Figure 4 A method for modifying autonomous flight of an unmanned aerial vehicle according to an embodiment is shown.

[0025] Figure 5 An autonomous flight path of an unmanned aerial vehicle modified by user input according to an embodiment is shown.

[0026] Figure 6 A side view of an autonomous flight path of a movable object modified by user input according to an embodiment is shown.

[0027] Figure 7 A force of user input proportionally modifies a force of autonomous flight of a movable object according to an embodiment is shown.

[0028] Figure 8 Behavior of the UAV upon reaching a threshold value is shown in accordance with an embodiment.

[0029] Figure 9 Behavior of the UAV after user input modifying autonomous flight of the movable object is released is shown in accordance with an embodiment.

[0030] Figure 10 New autonomous flight path of the UAV modified by user input is shown in accordance with an embodiment.

[0031] Figure 11 A UAV (UAV) is shown in accordance with an embodiment.

[0032] Figure 12 A schematic diagram of a system block diagram for controlling a movable object is shown in accordance with an embodiment.

[0033] Figure 13 A UAV flying in a curved trajectory in response to actuation of one or more dials is shown in accordance with an embodiment.

[0034] Figure 14 A top view of a UAV moving along an autonomous flight path at an increased or decreased speed is shown in accordance with an embodiment. DETAILED DESCRIPTION

[0035] The systems, methods, and devices provided herein can be used to give a human operator the ability to influence and / or modify the flight of an autonomously operating aerial vehicle. For example, intuitive and easy-to-use user input devices can be provided. The user input devices can be utilized to modify the flight of the aerial vehicle during autonomous operation of the aerial vehicle. In some cases, the autonomously operating vehicle can have a predetermined goal. For example, the autonomously operating vehicle can have a predetermined task to complete, or a target to head towards. Thus, when parameters of the autonomously operating vehicle (e.g., flight path and / or flight direction) are modified in accordance with user input, the autonomously operating vehicle can continue to complete its predetermined task, or continue to head towards its target. In some cases, to ensure that the aerial vehicle is able to complete its task or reach its destination under conditions of autonomous control rather than user input, a threshold of user-allowed modification can be provided.

[0036] In some cases, different user interfaces can be provided for receiving different types of user input. The different user interfaces can include hardware and / or software interfaces. For example, the different user interfaces can include physical buttons on the device or interactive buttons displayed on a screen. In some cases, the different user interfaces can include two different user interfaces. For example, a first user interface can be provided that can be used to improve ease of autonomous operation of the aerial vehicle. The first user interface can allow control of the aerial vehicle through interaction with a graphical human interface and significantly reduce the burden of manually piloting the aerial vehicle. The first user interface can be used to provide the aerial vehicle with autonomous tasks to be completed. In some cases, the autonomous tasks to be completed can be specified as simple commands (e.g., touching a target on a map). A second user interface can be provided that allows for simple and intuitive modification of the autonomous operation of the aerial vehicle (e.g., autonomous flight). For example, user input on the second user interface can slightly modify the trajectory of the aerial vehicle as the aerial vehicle autonomously navigates to its target. In some cases, user input provided on the second user interface can modify parameters of the autonomous flight while maintaining autonomous flight of the aerial vehicle.

[0037] The ability to influence and / or modify the flight of an autonomously operating aerial vehicle can improve the operability of the aerial vehicle under autonomous control. The burden of manually piloting the aerial vehicle can be significantly reduced, but adjustments (e.g., modifications) to the autonomous flight can also be allowed so that the user can influence or modify the autonomous flight when needed or beneficial. Separate functionality provided on different user interfaces can simplify control so that both skilled and unskilled users can take advantage of the benefits enabled by the inventions provided herein. Furthermore, separate functionality provided on different user interfaces can ensure that quick action to modify the autonomous flight of the aerial vehicle can be taken without confusion or error in the event of an emergency or unexpected situation.

[0038] The ability to modify autonomous flight can be particularly useful when the vehicle encounters an obstacle that was not detected (e.g., due to an error or when the vehicle lacks an obstacle sensor) but is noticed by the user. In these situations, the user can make a slight modification to the unmanned vehicle without interrupting autonomous operation of the vehicle to enable the given task to be completed after the modification. In some cases, the ability to modify autonomous flight can be particularly useful if, for example, the user needs to deviate from a given flight path to some extent (e.g., the user sees something of interest) while maintaining autonomous flight toward a target destination. As used herein, a flight path can refer to a path taken by a vehicle during flight. In some cases, a flight path can refer to a trajectory of a vehicle or a flight direction of a vehicle (e.g., in two or three dimensions). In some cases, a flight path can refer to a pre-set flight path (e.g., trajectory) that a vehicle is configured to follow. In some cases, a flight path can refer to an instantaneous flight direction of a vehicle.

[0039] By superimposing a directional component to the flight path, and / or superimposing an acceleration component or a velocity component to the vehicle, the user input can affect and / or modify the flight path of the vehicle. In some cases, the ability to modify autonomous flight can be particularly useful when the user needs the unmanned vehicle to fly in a particular flight mode, or to perform a maneuver that is not easily implemented (e.g., to make the unmanned vehicle fly in a helical ascent or descent). It should be noted that the ability to modify autonomous flight can be integrated into any type of vehicle and any vehicle capable of traversing air, water, land, and / or space.

[0040] It should be appreciated that different aspects of the application can be appreciated individually, collectively, or in any combination thereof. Various aspects of the application described herein can be applied to any of the particular applications set forth below or any other type of remotely controlled vehicle or movable object.

[0041] Figure 1 An example of a system for navigation according to an embodiment is shown. The navigation system can include a movable object 100 and a user terminal 106 that can communicate with the movable object. The movable object can be configured to carry a payload 104. The user terminal can be used to control one or more motion characteristics of the movable object and / or the payload. For example, the user terminal can be used to control the movable object to enable the movable object to navigate to a target area. The user terminal can be used to give the movable object instructions or commands that are transmitted to the movable object (e.g., a flight controller of the movable object) to enable autonomous flight of the movable object as described further herein. In some cases, the user terminal can be used to manually control the movable object and / or modify parameters of the movable object while the movable object is operating autonomously.

[0042] The movable object 100 can be any object that is capable of traversing an environment. The movable object can be capable of traversing air, water, land, and / or space. The environment can include objects that are not capable of motion (stationary objects) and objects that are capable of motion. Examples of stationary objects can include geographical features, plants, landmarks, buildings, overall structures, or any fixed structure. Examples of objects that are capable of motion include people, vehicles, animals, projectiles, etc.

[0043] In some instances, the environment can be an inertial frame of reference. An inertial frame of reference can be used to describe time and space in a uniform, isotropic, and time-independent manner. An inertial frame of reference can be established with respect to a movable object and can move with the movable object. Measurements in an inertial frame of reference can be converted to measurements in another frame of reference (e.g., a global frame of reference) by a transformation (e.g., Galilean transformation in Newtonian physics).

[0044] The movable object 100 can be a vehicle. The vehicle can be a self-propelled vehicle. The vehicle can traverse the environment with the aid of one or more propulsion units 107. The vehicle can be an aerial vehicle, a land-based vehicle, a water-based vehicle, or a space-based vehicle. The vehicle can be an unmanned vehicle. The vehicle can be capable of traversing the environment without an onboard human passenger. Alternatively, the vehicle can carry a human passenger. In some embodiments, the movable object can be an unmanned aerial vehicle (UAV).

[0045] The descriptions herein of unmanned aerial vehicles or any other type of movable object can generally apply to any other type of movable object or to various categories of movable objects, and vice versa. For example, the descriptions herein of unmanned aerial vehicles can apply to any unmanned land-, water-, or space-based vehicle. Further examples of movable objects are provided in greater detail elsewhere herein.

[0046] As mentioned above, the movable object can be capable of traversing an environment. The movable object can be capable of flying in three-dimensional space. The movable object can be capable of spatial translation along one, two, or three axes. The one, two, or three axes can be mutually orthogonal. The axes can be along a pitch axis, a yaw axis, and / or a roll axis. The movable object can be capable of rotation about one, two, or three axes. The one, two, or three axes can be mutually orthogonal. The axes can be a pitch axis, a yaw axis, and / or a roll axis. The movable object can be capable of movement in up to 6 degrees of freedom. The movable object can include one or more propulsion units that can assist the movable object in movement. For example, the movable object can be an unmanned aerial vehicle having one, two, or more propulsion units. The propulsion units can be configured to generate lift for the unmanned aerial vehicle. The propulsion units can include rotors. The movable object can be a multi-copter unmanned aerial vehicle.

[0047] The movable object can have any physical configuration. For example, the movable object can have a central body with one or more arms or branches extending from the central body. The arms can extend laterally or radially from the central body. The arms can be movable relative to the central body or can be stationary relative to the central body. The arms can support one or more propulsion units. For example, each arm can support one, two, or more propulsion units.

[0048] The movable object can have a housing. The housing can be formed from one, two, or more complete pieces. The housing can include a cavity in which one or more components are disposed. The components can be electronic components, such as a motion controller (e.g., a flight controller), one or more processors, one or more memory storage units, one or more sensors (e.g., one or more inertial sensors or any other type of sensor described elsewhere herein), one or more navigation units (e.g., a global positioning system (GPS) unit), one or more communication units, or any other type of component. The housing can have one cavity or multiple cavities. In some cases, a motion controller (e.g., a flight controller) can be in communication with and / or can control the operation of one or more propulsion units. The motion controller (or flight controller) can be in communication with and / or control the operation of one or more propulsion units via one or more electronic speed controller (ESC) modules. The motion controller (or flight controller) can be in communication with the ESC modules to control the operation of the propulsion units.

[0049] The movable object can support an on-board payload 104. The payload can have a fixed position relative to the movable object or can be movable relative to the movable object. The payload can be spatially translatable relative to the movable object. For example, the payload can be movable relative to the movable object along one, two, or three axes. The payload can be rotatable relative to the movable object. For example, the payload can be rotatable relative to the movable object about one, two, or three axes. The axes can be mutually orthogonal. The axes can be a pitch axis, a yaw axis, and / or a roll axis. Alternatively, the payload can be fixed to or integrated with the movable object.

[0050] The payload can be movable relative to the movable object via a carrier 102. The carrier can include one or more gimbal stages that can allow the carrier to move relative to the movable object. For example, the carrier can include a first gimbal stage that can allow the carrier to rotate relative to the movable object about a first axis, a second gimbal stage that can allow the carrier to rotate relative to the movable object about a second axis, and / or a third gimbal stage that can allow the carrier to rotate relative to the movable object about a third axis. Any of the descriptions and / or characteristics of the carrier described elsewhere herein can apply.

[0051] The payload can include a device that can sense the environment around the movable object, a device that can emit a signal to the environment, and / or a device that can interact with the environment.

[0052] One or more sensors can be provided as the payload, and can have the ability to sense the environment. The one or more sensors can include an imaging device. The imaging device can be a physical imaging device. The imaging device can be configured to detect electromagnetic radiation (e.g., visible light, infrared light, and / or ultraviolet light) and generate image data based on the detected electromagnetic radiation. The imaging device can include a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor that generates an electrical signal in response to the wavelength of light. The electrical signal thus generated can be processed to generate image data. The image data generated by the imaging device can include one or more images, which can be a still image (e.g., a photograph), a dynamic image (e.g., a video), or a suitable combination thereof. The image data can be multi-colored (e.g., RGB, CMYK, HSV) or monochromatic (e.g., grayscale, black and white, sepia). The imaging device can include a lens configured to direct light onto an image sensor.

[0053] The imaging device can be a camera. The camera can be a movie or video camera that captures dynamic image data (e.g., video). The camera can be a still camera that captures still images (e.g., photographs). The camera can capture both dynamic image data and still images. The camera can switch between capturing dynamic image data and still images. While certain embodiments provided herein are described in conjunction with cameras, it should be understood that the present disclosure can apply to any suitable imaging device, and any description herein relating to cameras can also apply to any suitable imaging device, and any description herein relating to cameras can also apply to other types of imaging devices. The camera can be used to generate a 2D image of a 3D scene (e.g., an environment, one or more objects, etc.). The image generated by the camera can represent a projection of the 3D scene onto a 2D image plane. Thus, each point in the 2D image corresponds to a 3D spatial coordinate in the scene. The camera can include optical elements (e.g., a lens, a mirror, a filter, etc.). The camera can capture color images, grayscale images, infrared images, etc. When the camera is configured to capture infrared images, it can be a thermal imaging device.

[0054] In some embodiments, the payload can include multiple imaging devices, or an imaging device with multiple lenses and / or image sensors. Other portions of the payload can capture multiple images substantially simultaneously. The multiple images can assist in creating a 3D scene, 3D virtual environment, 3D map, or 3D model. For example, right and left images can be captured and used for stereographic mapping. Depth maps can be computed from calibrated binocular images. Any number of images (e.g., 2 or more, 3 or more, 4 or more, 5 or more, 6 or more, 7 or more, 8 or more, 9 or more) can be captured simultaneously to assist in generating a 3D scene / virtual environment / model, and / or for depth mapping. The images can be directed substantially in the same direction, or in slightly different directions. In some cases, data from other sensors (e.g., ultrasonic data, LIDAR data, data from any other sensor as described elsewhere herein, or data from an external device) can assist in creating 2D or 3D images or maps.

[0055] The imaging device can capture images or image sequences at a particular image resolution. In some embodiments, the image resolution can be defined by the number of pixels in an image. In some embodiments, the image resolution can be greater than or equal to approximately 352x420 pixels, 480x320 pixels, 720x480 pixels, 1280x720 pixels, 1440x1080 pixels, 1920x1080 pixels, 2048x1080 pixels, 3840x2160 pixels, 4096x2160 pixels, 7680x4320 pixels, or 15360x8640 pixels. In some embodiments, the camera can be a 4K camera or a higher resolution camera.

[0056] The imaging device can capture image sequences at a particular capture rate. In some embodiments, image sequences can be captured at a standard video frame rate of, for example, approximately 24p, 25p, 30p, 48p, 50p, 60p, 72p, 90p, 100p, 120p, 300p, 50i, or 60i. In some embodiments, image sequences can be captured at a rate of less than or equal to approximately one image per 0.0001 seconds, 0.0002 seconds, 0.0005 seconds, 0.001 seconds, 0.002 seconds, 0.005 seconds, 0.01 seconds, 0.02 seconds, 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, or 10 seconds. In some embodiments, the capture rate can change according to user input and / or external conditions (e.g., rain, snow, wind, non-obvious environmental surface conditions).

[0057] The imaging device can have adjustable parameters. At different parameters, the imaging device can capture different images when subjected to the same external conditions (e.g., location, lighting). The adjustable parameters can include exposure (e.g., exposure time, shutter speed, aperture, film speed), gain, gamma, region of interest, binning / subsampling, pixel clock, offset, trigger, ISO, etc. Parameters related to exposure can control the amount of light that reaches an image sensor in the imaging device. For example, shutter speed can control the time light reaches the image sensor, and aperture can control the amount of light that reaches the image sensor in a given time. Parameters related to gain can control amplification of signals from the optical sensor. ISO can control the sensitivity of the camera to natural light.

[0058] In some alternative embodiments, the imaging device can extend beyond the physical imaging device. For example, the imaging device can include any technology capable of capturing and / or generating an image or video frame. In some embodiments, the imaging device can refer to an algorithm capable of processing an image acquired from another physical device.

[0059] The payload can include one or more types of sensors. Some examples of sensor types can include location sensors (e.g., global positioning system (GPS) sensors, mobile device transmitters that enable location triangulation), motion sensors, vision sensors (e.g., imaging devices capable of detecting visible light, infrared light, or ultraviolet light, such as cameras), proximity or distance sensors (e.g., ultrasonic sensors, lidar, time-of-flight or depth cameras), inertial sensors (e.g., accelerometers, gyroscopes, and / or gravity detection sensors that can form an inertial measurement unit (IMU)), altitude sensors, attitude sensors (e.g., compasses), pressure sensors (e.g., barometers), temperature sensors, humidity sensors, vibration sensors, audio sensors (e.g., microphones), and / or field sensors (e.g., magnetometers, electromagnetic sensors, radio sensors).

[0060] The sensing data provided by the sensors can be used to control the spatial layout, speed, and / or direction of the movable object (e.g., using a suitable processing unit and / or control module, as described below). Alternatively, the sensors can be used to provide data about the environment surrounding the movable object, such as weather conditions, proximity to potential obstacles, locations of geographical features, locations of man-made structures, etc.

[0061] The payload can include one or more devices capable of emitting a signal into the environment. For example, the payload can include an emitter along the electromagnetic spectrum (e.g., a visible light emitter, an ultraviolet emitter, an infrared emitter). The payload can include a laser or any other type of electromagnetic emitter. The payload can emit one or more vibrations, such as an ultrasonic signal. The payload can emit an audible sound (e.g., from a speaker). The payload can emit a wireless signal, such as a radio signal or other type of signal.

[0062] The payload is capable of interacting with the environment. For example, the payload can include a robotic arm. The payload can include an item for delivery, such as a liquid, a gas, and / or a solid component. For example, the payload can include a pesticide, water, fertilizer, fire retardant, food, packaging, or any other item.

[0063] Any example of a payload herein can apply to a device that can be carried by the movable object or can be part of the movable object. For example, one or more sensors 108 can be part of the movable object. One or more sensors can also be provided in addition to the payload. This can apply to any type of payload, such as the payloads described herein.

[0064] The movable object is capable of communicating with the user terminal 106. The user terminal can communicate with the movable object through a payload of the movable object and / or with a carrier of the movable object, where the carrier is used to support the payload. Any description herein of communicating with the movable object can also apply to communicating with a payload of the movable object, a carrier of the movable object, and / or one or more independent components of the movable object (e.g., a communication unit, a navigation unit, a propulsion unit, a power source, a processor, a memory storage unit, and / or an actuator).

[0065] Communication between the movable object and the user terminal can be conducted through wireless communication 116. For example, a communication system 110 can be provided on the movable object. A corresponding communication unit 114 can be provided on the user terminal and can be used to form a communication link (e.g., a wireless communication link) between the communication systems. Direct communication can be provided between the movable object and the user terminal. Direct communication can occur without any intermediary devices or networks. Indirect communication can be provided between the movable object and the user terminal. Indirect communication can occur with the aid of one or more intermediary devices or networks. For example, indirect communication can utilize a telecommunications network. Indirect communication can be performed with the aid of one or more routers, communication towers, satellites, or any other intermediary devices or networks. Examples of types of communication can include, but are not limited to, over the Internet, a local area network (LAN), a wide area network (WAN), Bluetooth, near field communication (NFC) technology, a network based on a mobile data network protocol (e.g., general packet radio service (GPRS), global system for mobile communications (GSM), enhanced data GSM environment (EDGE), 3G, 4G, or long term evolution (LTE) protocol, infrared (IR) communication technology, and / or Wi-Fi), and can be in the form of wireless, wired, or a combination thereof.

[0066] A user terminal can be any type of external device. A user terminal can individually or collectively refer to a device configured to receive user input. A user terminal can be configured with one or more user interfaces configured to receive user input. Examples of a user terminal can include, but are not limited to, a smartphone / cell phone, a tablet, a personal digital assistant (PDA), a notebook computer, a desktop computer, a media content player, a video game station / system, a virtual reality system, an augmented reality system, a wearable device (e.g., a watch, glasses, gloves, headwear (e.g., a hat, a headset, a virtual reality headset, an augmented reality headset, a head-mounted device (HMD), a hairband), a pendant, an armband, a legband, a shoe, a vest), a gesture recognition device, a microphone, any electronic device capable of providing or rendering image data, a remote control with a dial, or any other type of device. A user terminal can be a handheld object. A user terminal can be portable. A user terminal can be carried by a human user. In some cases, a human user can be remote from the user terminal, and the user can control the user terminal using wireless and / or wired communication. Various examples and / or characteristics of a user terminal are provided in more detail elsewhere herein.

[0067] The user terminal can include one or more processors capable of executing a non-transitory computer-readable medium providing instructions for one or more actions. The user terminal can include one or more memory storage devices including a non-transitory computer-readable medium including code, logic or instructions to perform one or more actions. The user terminal can include a software application that allows the user terminal to communicate with the movable object and receive image data from the movable object. The user terminal can include a communication unit 114 that can allow for communication with the movable object. In some instances, the communication unit can include one communication module, or multiple communication modules. In some instances, the user terminal can be capable of interacting with the movable object with a single communication link or multiple different types of communication links. The user terminal can be used to control the motion of the movable object. In some instances, the user terminal can be configured to enable autonomous operation of the movable device (e.g., autonomous flight) in response to user input, for example. In some instances, the user terminal can be configured to enable and / or modify autonomous operation of the movable device as described further below. In some instances, the user terminal can be selectively used to control any component of the movable object (e.g., operation of a payload, operation of a carrier, one or more sensors, communication, navigation, a landing stand, actuation of one or more components, power control, or any other function).

[0068] A user terminal can include one or more user interfaces provided on one or more devices. For example, a user terminal can include 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or more user interfaces. A user interface can refer to an interface where input is received by a user (e.g., a handler of a UAV). The input can be of any type. For example, a user can provide input by simply touching a portion of a user interface (e.g., a capacitive touch screen). For example, a user can actuate a mechanism on a user interface (e.g., a keyboard, a mouse, a button, a joystick, etc.) to provide user input. In some cases, a user can provide an audible signal (e.g., a voice command) to a user interface that is received by the user interface. In some cases, a user interface can be configured to sense, follow, or track a user's motion (e.g., eye movement, hand gestures, etc.) to receive user input. In some cases, a user interface can be configured to receive different degrees of user input. For example, a user can exert different amounts of force on a user interface or actuate a mechanism on a user interface with different degrees, which can be appropriately interpreted by the user interface (or one or more processors coupled with the user interface). For example, a user can provide input for different durations, which can be appropriately interpreted by the user interface (or one or more processors coupled with the user interface). Alternatively or additionally, a user input can be configured to receive and interpret user input as binary input. For example, a user touching a user interface can be interpreted as a command to implement autonomous flight of a movable object toward a target. Each user interface can be provided on a separate device. Alternatively, 2, 3, 4, 5, or more user interfaces can be provided on one device.

[0069] In some cases, different user interfaces can be configured to control different functions of a movable object and / or control different components of a movable object. For example, a first user terminal can be used to implement autonomous operation of a movable object, while a second user terminal can be used to modify (e.g., influence) the autonomous operation. In some cases, different devices can be configured to control different functions of a movable object and / or control different components of a movable object. Different user interfaces and / or devices can or can not be in communication with each other. For example, different devices can be in communication with each other through a wireless or wired communication link. Alternatively, each of the different devices can not be in communication with each other, but can be in communication with a UAV individually.

[0070] Figure 2A user terminal 200 according to an embodiment is shown. The user terminal can include one or more user interfaces. The user interfaces can be configured to receive input from a user (e.g., a pilot of the UAV). The user can be a pilot of the movable object. Each user interface can be provided on a separate device. Alternatively, different user interfaces can be provided on different devices, and the user terminal can include two or more devices. The user terminal can be configured to receive user input and generate and / or provide instructions (e.g., signals) sent to the movable object and / or a payload. In some cases, an input can be received at the user terminal that results in instructions to enable autonomous flight of the movable object. For example, a target can be specified on the user terminal, and instructions can be generated and sent to the movable object (e.g., a flight controller of the movable object) to cause the movable object to autonomously move toward the target. In some cases, user input can be received at the user terminal to influence or modify autonomous operation of the movable object. For example, the user terminal can be used to manually control movement of the movable object and / or to modify flight (e.g., autonomous flight) of the movable object.

[0071] In some cases, the user terminal can include a first user interface 202 and a second user interface 204. The first and second user interfaces can have different characteristics and can be used to accomplish different functions of the movable object. For example, the first user interface can be configured to receive user input to enable autonomous operation (e.g., autonomous flight) of the movable object. In some cases, the first user interface can be configured to receive user input indicating that the movable object is to autonomously accomplish a particular task or autonomously move toward a target. One input or a number of inputs that are not continuous can be sufficient to indicate autonomous operation (e.g., autonomous flight) of the movable object. It can not be necessary for the user to continuously monitor or monitor the movable object while using the first user interface.

[0072] The first user interface 202 can include a display 203. The display can be a screen. The display can be a touch screen, or can not be a touch screen. The display can be a light emitting diode (LED) screen, an organic light emitting diode (OLED) screen, a liquid crystal display (LCD) screen, a plasma screen, or any other type of screen. The display can be configured to display images. The images on the display can display views captured by a payload of a movable object. For example, images captured by an imaging device can be displayed on the display. In some cases, the images captured by the imaging device can be considered first person view (FPV) images. In some cases, one imaging device can be provided and one first person view image can be provided. Alternatively, multiple imaging devices with different fields of view can be provided. The views can be switched between multiple first person view images, or multiple first person view images can be displayed simultaneously. The multiple first person view images can correspond to (or be generated from) different imaging devices that can have different fields of view.

[0073] In another example, the images on the display can display a map that can be generated from information from a payload of a movable object. The map can be selectively generated from multiple imaging devices (right view cameras, left view cameras, or more cameras) that can utilize stereographic mapping techniques. In some cases, the map can be generated based on position information of the movable object relative to the environment, the imaging device relative to the environment, and / or the movable object relative to the imaging device. The position information can include attitude information, spatial position information, angular velocity, linear velocity, angular acceleration, and / or linear acceleration. The map can be selectively generated from one or more additional sensors as described in more detail elsewhere herein. The map can be a two-dimensional map or a three-dimensional map. In some cases, the two-dimensional map can display an overhead map. In some cases, the two-dimensional map can be a topographic map that displays various natural and man-made features. In some cases, the two-dimensional map can be a profile map (e.g., displaying elevation). In some cases, the view can be switched (e.g., between a topographic map and a profile map). In some cases, the view can be switched between two-dimensional and three-dimensional map views, or both two-dimensional and three-dimensional map views can be displayed simultaneously. The view can be switched between one or more first person views and one or more map views, or both one or more first person views and one or more map views can be displayed simultaneously.

[0074] In some embodiments, the image can be provided in a 3D virtual environment displayed on the first user interface (e.g., a virtual reality system or an augmented reality system). The 3D virtual environment can selectively correspond to a 3D map. The virtual environment can include a plurality of points or objects that can be manipulated by the user. The user can manipulate the points and objects by various actions in the virtual environment. Examples of the actions can include selecting one or more points or objects, dragging and dropping, panning, rotating, spinning, pushing, pulling, zooming in, zooming out, etc. Any type of motion action of the points or objects in the three-dimensional virtual space can be contemplated.

[0075] The first user interface can include a graphical user interface (GUI). The graphical user interface can display images that allow the user to control the actions of the movable object. The graphical user interface can display images that allow the user to instruct the movable object to operate autonomously or to complete a given task. For example, the user can be able to select a target for tracking, select an area for navigation (e.g., a predetermined area or a point of interest), select one or more waypoints for the navigation of the movable object to pass through, instruct the movable object to return to the user (e.g., the user terminal), etc. In some cases, the user can be able to simply instruct the movable object to complete a task by touching or clicking a point (e.g., a portion) on the first user interface. In some cases, the first user interface can include a capacitive touch screen. The first user interface can impart tap and go functionality to the movable object. By simply tapping the first user interface (e.g., a desired location of a map displayed on the first user interface), the movable object can be instructed to autonomously operate towards the tapped object and / or area. For example, for an image display on a display that displays a view captured by a payload carried by the movable object, a user tapping an object of interest can instruct the movable object to autonomously follow or track the object. For example, for an image display on a display that displays a map (e.g., a 2D or 3D map), a user tapping a location on the map can instruct the movable object to autonomously navigate to the tapped location.

[0076] In some cases, the user can select a target on the first user interface. The target can be selected in an image (e.g., of a display). The user can also select a portion (e.g., a point, an area, and / or an object) of the image to define the target and / or the direction. The user can select the target by directly touching a screen (e.g., a touch screen). The user can touch a portion of the screen. The user can touch a portion of the screen by touching a point on the screen. The user can select the target by selecting a portion of the image with a user interaction device (e.g., a mouse, a joystick, a keyboard, a trackball, a touchpad, a button, a verbal command, gesture recognition, a posture sensor, a heat sensor, a touch capacitive sensor, or any other device). The touch screen can be configured to detect the user's touch location, touch length, touch pressure, and / or touch action, whereby each of the above can indicate a specific input command of the user.

[0077] The movable object can be configured to move toward a target, navigate around a target, and / or visually track a target. The target can be a target destination. In some instances, the target destination can be a location selected in an image (e.g., a first-person perspective image) captured by an imaging device on the movable object. For example, a user can select a location (or multiple locations) in an image (e.g., a first-person perspective image) captured by the imaging device, such as by touching a point on the image. Clicking on the portion of the image can instruct the movable object to fly to the location with a flight path. In some instances, the target destination can be a location selected on a map. For example, locations 206, 208, and 210 can include targets that a user has selected on a map. The targets can be selected by touching a point on the map, for example, substantially as described above. Clicking on the portion of the map can instruct the movable object to fly to the target (e.g., autonomously) with a flight path. Clicking on a portion of a user interface to instruct a movable object to fly autonomously toward a target (e.g., a location or object) can be referred to herein as a tap to go function. In some instances, the target destination can be a predetermined or preset destination selected without the aid of an image or map (e.g., from a predetermined list, as a standalone feature, etc.). For example, the target destination can be a location of a user terminal, a location of a user, a specified waypoint, or a point of interest (e.g., a user-specified venue).

[0078] In some instances, the target can be a target object. The target object can be a stationary target or a moving target. In some instances, a user can specify whether the target is a stationary target or a moving target. Alternatively, the user can provide any other type of indication to indicate whether the target is a stationary target or a moving target. Alternatively, no indication can be provided and a decision can be made automatically by one or more processors, optionally without user input specifying whether the target is a stationary target or a moving target. A target object can be classified as a stationary target or a moving target based on its state of motion. In some instances, a target object can be moving or stationary at any given point in time. When a target object is moving, the target object can be classified as a moving target. Conversely, when the same target object is stationary, the target object can be classified as a stationary target.

[0079] A stationary target can remain substantially stable in an environment. Examples of stationary targets can include, but are not limited to, a landscape feature (e.g., a tree, a plant, a mountain, a hill, a river, a stream, a river bend, a canyon, a pebble, a rock, etc.) or a man-made feature (e.g., a structure, a building, a road, a bridge, a pole, a fence, a non-moving vehicle, a sign, a light, etc.). A stationary target can include a large target or a small target. A stationary target can be selected by a user. A stationary target can be identified. Alternatively, a stationary target can be mapped. A movable object can move and / or navigate around a stationary target, and / or track a stationary target. In some cases, a stationary target can correspond to a selected portion of a structure or object. For example, a stationary target can correspond to a particular region (e.g., a top floor) of a skyscraper.

[0080] A moving target is capable of moving in an environment. A moving target can be in motion at all times, or in motion for a period of time. A moving target can move in a fairly stable direction, or can change direction. A moving target can move in the air, on land, under ground, on water or under water, and / or in space. A moving target can be a living moving target (e.g., a person, an animal) or a non-living moving target (e.g., a moving vehicle, a moving machine, an object floating in the wind or carried by water, an object carried by a living target). A moving target can include one moving object or a group of moving objects. For example, a moving target can include a person or a group of moving people. A moving target can be a large target or a small target. A moving target can be selected by a user. A moving target can be identified. Alternatively, a moving target can be mapped. A movable object can move toward and / or navigate around a moving target, and / or track a moving target. A flight path can be planned for a movable object to navigate around a moving object. The path can be altered or updated as the moving object moves along the path. Alternatively, a movable object can move and / or navigate around a stationary object and / or visually track a moving object without a planned path.

[0081] A moving target can be any object configured to move in any suitable environment, such as in the air (e.g., a fixed-wing aircraft, a rotorcraft, or an aircraft with neither fixed wings nor rotors), in water (e.g., a boat or a submarine), on land (e.g., a motorized vehicle such as a car, a truck, a bus, a van, a motorcycle; a moving structure or frame such as a pole, a fishing rod; or a train), under ground (e.g., a subway), in space (e.g., a spaceplane, a satellite, or a probe), or any combination of these environments.

[0082] The mobile object can be free to move in the environment with respect to six degrees of freedom (e.g., three degrees of freedom of translation and three degrees of freedom of rotation). Alternatively, the movement of the mobile object can be constrained with respect to one or more degrees of freedom, e.g., by a predetermined path, trajectory, or direction. The movement can be actuated by any suitable actuation mechanism, e.g., an engine or motor. The actuation mechanism of the mobile object can be powered by any suitable energy source, e.g., electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy, or any suitable combination thereof. The mobile object can be self-propelled by a propulsion system, as described further below. The propulsion system can optionally be powered by an energy source, e.g., electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy, or any suitable combination thereof.

[0083] In some cases, the mobile object can be a vehicle, e.g., a remotely controlled vehicle. Suitable vehicles can include: a water-borne vehicle, an aircraft, a space vehicle, or a land vehicle. For example, the aircraft can be a fixed-wing aircraft (e.g., an airplane, a glider), a rotary-wing aircraft (e.g., a helicopter, a gyrocopter), an aircraft with both fixed and rotary wings, or an aircraft with neither (e.g., a dirigible, a hot air balloon). The vehicle can be self-propelled, e.g., by air, on or in water, in space, or on or under the ground. The self-propelled vehicle can utilize a propulsion system, e.g., a propulsion system including one or more engines, motors, wheels, axles, magnets, rotors, propellers, paddles, nozzles, or any suitable combination thereof. In some cases, the propulsion system can be used to take off from a surface, land on a surface, maintain its current position and / or orientation (e.g., hover), change orientation, and / or change position.

[0084] Selecting a target as described above can enable autonomous flight of the movable object. For example, selecting a target can generate instructions (e.g., via a communication system) that are sent to a flight controller of the movable object. The flight controller can receive the instructions and further generate signals to enable autonomous flight of the movable object. Autonomous flight can be flight toward the target. As described herein, the target can be a target destination (e.g., a location) and / or a target object. In some cases, multiple targets can be selected, and the movable object can fly along the targets. The movable object in an autonomous mode of operation (e.g., by input received at the first user interface) can include a predetermined flight speed at which it moves. The predetermined flight speed can be a default speed. In some cases, the predetermined flight speed can be user-configurable. In some cases, the predetermined flight speed can be equal to or less than about 2 m / s, 4 m / s, 6 m / s, 8 m / s, 10 m / s, 12 m / s, 15 m / s, 20 m / s, or 50 m / s.

[0085] The object in the autonomous operation (e.g., by input received at the first user interface) state can include a trajectory or a flight path. The autonomous flight can include an autonomous flight path of the movable object. In some instances, the flight path 205 of the autonomous flight can be displayed at the graphical user interface. Alternatively or additionally, a plurality of points 206, 208, 210 can be displayed at the graphical user interface, which indicate a target toward which the movable object autonomously flies. The target can indicate a target object and / or a target region toward which the movable object autonomously flies. In some instances, the flight path can include a preset direction, a preset trajectory, an autonomously planned trajectory, and / or a user-configured trajectory. In some instances, the flight path can be preset (e.g., take the shortest route at a particular height). In some instances, the user can select the flight path (e.g., from a number of different preset flight paths). In some instances, the user can generate a flight path of the movable object by drawing contours on the screen, e.g., using a user interaction device or a user accessory substantially as described above. In some instances, the flight path can be autonomously generated or semi-autonomously generated. In some instances, the flight path can be generated with respect to a target by considering a location, a direction, a pose, a size, a shape, and / or a geometry of the target.

[0086] In some instances, the flight path can be autonomously generated or semi-autonomously generated by considering other parameters (e.g., parameters of the movable object (e.g., size, weight, speed, etc.), jurisdictional parameters (e.g., laws and regulations), or environmental parameters (e.g., wind conditions, visibility, obstacles, etc.).

[0087] In some instances, the user can modify any portion of the flight path by adjusting (e.g., moving) different spatial points of the motion path on the screen (e.g., at the first user interface). Alternatively, the user can select a region from an existing set of regions on the screen, or draw a boundary of a region, a diameter of a region, or specify a portion of the screen in any other manner.

[0088] The movable object can move along the flight path when a withdrawal instruction is received or when a withdrawal condition is fulfilled. For example, the movable object can autonomously move along the motion path until a new path is input, when a portion of the motion path is changed, or a new target is input. The movable object can move along the flight path until a different flight path is selected. In some instances, the user can take manual control of the motion of the movable object at any time the movable object moves.

[0089] The user terminal can optionally include a second user interface 204. In some instances, the second user interface can be different from the first user interface. In some instances, the second user interface can be of a different type than the first user interface, or can be configured to receive different modes of user input. In some instances, the second user interface can include one or more mechanisms 212, 214 to receive user input. The one or more mechanisms can be actuatable. The one or more mechanisms can include any type of hardware mechanism, such as a dial, a physical button, or a scroll wheel. In some instances, the one or more mechanisms can include a software mechanism, such as an interactive button on a touch screen. While a dial is primarily described herein, it is to be understood that the use of other mechanisms (e.g., buttons, etc.) can be equally applicable.

[0090] The dial can also be referred to as a joystick (or handle). In some instances, the one or more dials can include a roll dial configured to affect rotation of the UAV about a roll axis; and / or a yaw dial configured to affect rotation of the UAV about a yaw axis. In some instances, the one or more dials can include a pitch dial. The pitch dial can be configured to affect a change in speed of the UAV. In some instances, the one or more dials can include a throttle dial. The throttle dial can be configured to affect a change in height (e.g., altitude) of the UAV. In some instances, the second user interface can be used to control movement of the movable object. The second user interface can be used to directly control movement of the movable object. Alternatively or additionally, the second user interface can be used to modify movement (e.g., flight) of the movable object in an autonomous control state. In some instances, the second user interface can be used to implement autonomous flight of the UAV.

[0091] While a dial can be assigned a particular name (e.g., pitch dial, yaw dial, etc.), it is to be understood that the assignment of the dial is arbitrary. For example, the user terminal (e.g., the second user interface) can operate in different modes. For example, the user terminal can operate in different modes using user-given commands (e.g., actuation of a switch). In different modes, the dial (e.g., dial 212 or 214) can be configured to affect operation of the UAV in different ways. In some instances, in one mode of operation, actuation of the mechanism can be configured to implement autonomous flight (e.g., flight in a predetermined direction or flight in a previously navigated direction), while in another mode of operation, actuation of the mechanism can be configured to affect flight of the UAV in an autonomous flight state.

[0092] In some cases, in the first mode, the dial lever 212 can be configured to affect the forward and backward movement of the unmanned aerial vehicle, while in the second mode, the dial lever 212 can be configured to affect the speed of the forward movement of the unmanned aerial vehicle. In a third mode of operation, the dial lever 212 can be configured to affect the height of the unmanned aerial vehicle and / or the rotation of the unmanned aerial vehicle about one or more axes. The user terminal can include 1, 2, 3, 4, 5, or more modes of operation. Furthermore, a given dial lever (e.g., dial lever 212 or 214) can include multiple functions, or can affect the flight (e.g., autonomous flight) of the unmanned aerial vehicle in multiple parameters. For example, forward and backward movement of the dial lever 212 can affect the change in height of the unmanned aerial vehicle, while left and right movement of the dial lever 212 can affect the rotation of the unmanned aerial vehicle about the roll axis.

[0093] In some cases, real-time control of the movable object can be implemented with the second user interface. In some cases, the first user interface and the second user interface can work in conjunction. Figure 3 A first user interface 302 and a second user interface 304 working in conjunction according to an embodiment are shown. The first user interface can be as previously described. For example, the first user interface can include a display configured to display one or more images. For example, the display can be configured to display an image of a map 306. The map can be a two-dimensional or three-dimensional map of the environment surrounding the movable object. Alternatively or additionally, the display can be configured to display a first-person perspective image 308 acquired by a payload connected to the movable object. For example, Figure 3 The first-person perspective image in the middle shows an obstacle 310 that the movable object is heading towards.

[0094] The second user interface can include one or more dials 314, 316. The dials can be utilized to affect (e.g., in real-time) parameters of the movable object. In some instances, the dials can affect and / or modify autonomous operation (e.g., autonomous flight) of the movable object. For example, a first user input can be received at the first user interface. The first user input can enable autonomous flight of the movable object. For example, a user tapping a target on the map 306 can generate instructions that are sent to a flight controller of the movable object that implements autonomous flight toward the target. Autonomous operation of the movable object can include a flight path 312. While autonomously navigating toward the target, the first user interface can display a first-person view image 308 in an image captured by a payload coupled to the movable object. Subsequent input received at the second user interface can affect or modify autonomous operation of the movable object. In some instances, input received at the second user interface can interrupt autonomous flight of the movable object. Interrupting autonomous flight via the second user interface can provide an efficient and simple method to quickly interrupt autonomous flight, for example, in an emergency or unexpected situation where it is not desirable to interact with the first user interface (e.g., a graphical user interface). In some instances, input received at the second user interface can modify autonomous flight of the movable object without interrupting autonomous operation. For example, due to input received at the second user input, the flight path of the movable object can be modified, but the movable object can continue to navigate to the designated target, for example, while and / or after the second user input is being received. For example, autonomous operation or flight of the movable object can be maintained despite user input on the second user interface so that the movable object continues to complete its mission (e.g., track a target, navigate to a desired location, etc.). In an example embodiment, the second user input can modify the flight path or trajectory of the movable object. For example, input at the second user interface can superimpose a directional component to the autonomous flight path of the movable object, or modify the autonomous flight path by superimposing an acceleration component or an acceleration component to the movable object.

[0095] This feature can be beneficial to provide user input in unexpected situations without interrupting autonomous flight of the movable object. In some cases, it can prove beneficial to distinguish between user interfaces (e.g., the first and second user interfaces) because the second user interface can provide an intuitive and simple control scheme to modify autonomous flight without interrupting the overall autonomous flight of the movable object. For example, this process is desirable in emergency situations or unexplainable situations or situations that are not detected by the movable object, when quick user input is necessary, but autonomous flight needs to continue afterwards. There can be a seamless transition between autonomous flight of the movable object and modification of autonomous flight by the second user input (e.g., in real-time). Additionally, there can be a seamless transition between the time the movable object considers the second user input and the time the movable object returns to autonomous flight (e.g., after the second user input).

[0096] For example, a movable object operating in an autonomous control state can encounter an obstacle 310. The movable object can fail to detect the obstacle (e.g., due to a malfunction or lack of obstacle sensors) and / or fail to complete an autonomous obstacle avoidance maneuver. In this case, a user can observe that there is an obstacle in the flight path of the autonomously operating movable object. By manipulating a dial on the second user interface, the user can easily avoid the obstacle. After avoiding the obstacle, the user can release the dial, and the movable object can continue autonomous operation, or complete its mission. For example, by attempting to provide input to the first user interface, the movable object can be unable to quickly avoid the obstacle. The second user interface can provide a convenient interface in which the user can achieve quick and intuitive control. After avoiding the obstacle, the movable object can continue autonomous operation, or complete its mission (e.g., track a target, navigate to a destination). In example embodiments, the second user input can modify a flight path or trajectory of the movable object. For example, input at the second user interface can superimpose a directional component to the autonomous flight path of the movable object, or can modify the autonomous flight path by superimposing an acceleration or acceleration component to the movable object.

[0097] For example, a movable object operating under autonomous control transmits a first-person view image to a first user interface. In some cases, a user can notice an object of interest 318 in the first-person view image. By manipulating a dial on a second user interface, the user can be able to steer (e.g., slightly modify the trajectory of the movable object) toward the object of interest without interrupting autonomous operation. After the user is satisfied, the user can release the dial, and the movable object can continue autonomous operation, or complete its mission. The movable object can continue autonomous operation without further input, and can seamlessly transition between autonomous operation and user-managed movement. In example embodiments, the second user input can modify a flight path or trajectory of the movable object. For example, input at the second user interface can superimpose a directional component onto the autonomous flight path of the movable object, or modify the autonomous flight path by superimposing an acceleration component or an acceleration component onto the movable object.

[0098] Figure 4 A method of modifying autonomous flight of an unmanned aerial vehicle is shown, in accordance with an embodiment. In step 401, autonomous flight of the unmanned aerial vehicle can be implemented. For example, one or more instructions can be provided to implement autonomous flight of the unmanned aerial vehicle. In some cases, a user can provide the one or more instructions (e.g., at a first user interface, as previously described). For example, the user can provide input at a handheld device or mobile device (e.g., a cell phone, a tablet, or a personal digital assistant). As previously described, the handheld device or mobile device can include a touchscreen, and can be configured to display images. In some cases, the images can include images received from a camera coupled to the unmanned aerial vehicle (e.g., a first-person view image) and / or a map image (e.g., a 2D or 3D map) showing a location of the unmanned aerial vehicle. By touching the handheld device or mobile device, the user can provide one or more instructions to implement autonomous flight of the unmanned aerial vehicle.

[0099] The one or more instructions can be transmitted to a flight controller of the unmanned aerial vehicle. In response to the transmitted one or more instructions, the flight controller can generate a first set of signals to implement autonomous flight of the unmanned aerial vehicle, e.g., with the aid of one or more processors. For example, the flight controller can generate a first set of signals instructing one or more propulsion units of the unmanned aerial vehicle to operate to implement autonomous flight of the unmanned aerial vehicle.

[0100] An autonomous flight can be any flight of a UAV that does not require continuous input (e.g., real-time input) from a user. In some instances, an autonomous flight can have a predetermined mission or objective. Examples of a predetermined mission or objective can include, but are not limited to, tracking or following a target object, flying to a target area or desired location, returning to the location of a user or user terminal. In some instances, an autonomous flight can have a predetermined objective that the UAV is moving towards. The objective can be a target object or a target destination. For example, an autonomous flight can be an autonomous flight towards a predetermined location specified by a user. In some instances, an autonomous flight can be a flight to a predetermined location, an autonomous return of a UAV, autonomous navigation along one or more waypoints, autonomous flight to a point of interest.

[0101] In some instances, an autonomous flight can include an autonomous flight trajectory or an autonomous flight path. In some instances, an autonomous flight can include an autonomous flight direction. A trajectory can be a flight trajectory in two or three dimensional coordinates. In some instances, an autonomous flight can have a preset trajectory. For example, a preset trajectory can take a shortest flight path, e.g., in flying towards a target (e.g., a target destination or a target obstacle) or in completing a mission. In some instances, an autonomous flight path can have an autonomously planned trajectory. For example, a flight controller of a UAV can calculate or autonomously plan a trajectory taking into account various parameters. The parameters can include, but are not limited to, environmental conditions, regulations and laws, known obstacles, known events, and objectives. Based on the various parameters, the flight controller can set an autonomously planned trajectory that is most suitable for the autonomous flight. In some instances, an autonomous flight can have a user configured trajectory. For example, a handler of a UAV can manually configure a trajectory or flight path that the autonomously operated UAV takes before the autonomous flight is implemented. In some instances, the trajectory or flight path of the autonomously flying UAV can be displayed on a user terminal, and / or a handheld device or a mobile device that receives user input to implement the autonomous flight of the UAV.

[0102] In step 403, the autonomous flight can be modified in response to user input. For example, during operation of the autonomous flight, a user can provide input at the user terminal. In some instances, the user input can be provided through the button or dial described supra. The input can provide one or more instructions to modify the autonomous flight of the UAV. The one or more instructions can be transmitted to the flight controller of the UAV, which can generate a second set of signals to modify the autonomous flight of the UAV. For example, the flight controller can generate a second set of signals that further instruct one or more propulsion units to operate to modify the autonomous flight of the UAV. In some instances, the modification of the autonomous flight can interrupt or stop the autonomous flight of the UAV, for example, until further input. For example, the user can manually control the UAV whose autonomous flight was interrupted. In some instances, the UAV whose autonomous flight was interrupted can hover at the location where the user input was provided until further instruction is given. Alternatively, the UAV whose autonomous flight was interrupted can return to the user, or the user terminal, or continue to land.

[0103] In some instances, the autonomous flight of the UAV can be modified without interrupting the autonomous flight. For example, when the autonomous flight is modified through user input, the UAV can continue to perform its mission (e.g., track a target object or move towards a target destination). In some instances, the UAV can continue to fly when the user input modifies the flight of the UAV. In some instances, the modification of the autonomous flight can affect the rotation of the UAV about one or more axes of the UAV (e.g., roll axis, yaw axis, pitch axis, etc.). In some instances, the modification of the autonomous flight can modify the autonomous flight path of the UAV while maintaining the autonomous flight. As described supra, the ability to modify the flight path of the UAV (e.g., affect the trajectory) while maintaining the autonomous flight can be advantageous because it enables small adjustments to handle unexpected situations (e.g., not discovered by the flight controller but noticed by the user) without interrupting the autonomous flight. Thus, a seamless transition between the autonomous flight of the UAV and user adjustments (e.g., in real-time) can be achieved.

[0104] In some instances, the method 400 can be implemented with one or more processors. For example, a system can be provided to modify the autonomous flight of a UAV. The system can include one or more processors individually or collectively configured to: implement an autonomous flight of a UAV, wherein the autonomous flight includes an autonomous flight path; and modify the autonomous flight path in response to user input, wherein the autonomous flight path is modified while maintaining the autonomous flight.

[0105] In some instances, the method 400 can be implemented with the aid of a non-transitory computer readable medium comprising code, logic, or instructions. For example, the non-transitory computer readable medium can comprise code, logic, or instructions to implement autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path; and respond to user input to modify the autonomous flight path, wherein the autonomous flight path is modified while maintaining the autonomous flight.

[0106] In some instances, a UAV can be used to implement the method 400. For example, the UAV can comprise a flight controller configured to generate: (1) a first set of signals for autonomous flight of the UAV, wherein the autonomous flight comprises an autonomous flight path, and (2) a second set of signals for modifying the autonomous flight path, wherein the autonomous flight path is modified while maintaining the autonomous flight; and one or more propulsion units configured to: (a) respond to the first set of signals to implement the autonomous flight of the UAV, and (b) respond to the second set of signals to modify the autonomous flight path of the UAV.

[0107] Figure 5 Autonomous flight of a UAV modified by user input is shown in accordance with an embodiment. In some instances, the UAV can operate autonomously. For example, the UAV 504 can autonomously fly toward a target 506, e.g., in accordance with user instructions. The autonomous flight can comprise an autonomous flight path 508. In some instances, a user can provide input (e.g., at a user terminal) to modify the autonomous flight of the UAV. In some instances, the user input can modify the autonomous flight path of the UAV. For example, the user can provide input at a remote controller 502 comprising one or more dials 510, 512. The dials can be configured to affect rotation of the UAV about one or more axes. For example, the one or more dials can comprise a roll dial configured to affect rotation of the UAV about a roll axis and / or a yaw dial configured to affect rotation of the UAV about a yaw axis. In some instances, the one or more dials can comprise a pitch dial. The pitch dial can be configured to affect a change in speed of the UAV. In some instances, the one or more dials can comprise a throttle dial. The throttle dial can be configured to affect a change in height (e.g., altitude) of the UAV.

[0108] By providing input, a user can actuate at least one of the one or more dials. User input received at the user terminal can provide one or more instructions to modify autonomous flight of the unmanned aerial vehicle. The one or more instructions can be transmitted to a flight controller of the unmanned aerial vehicle, which can generate a set of signals to modify autonomous flight of the unmanned aerial vehicle, for example, by affecting rotation of the unmanned aerial vehicle about one or more axes, by affecting a change in velocity of the unmanned aerial vehicle, or by affecting a change in altitude of the unmanned aerial vehicle. For example, the flight controller can generate a set of signals further instructing one or more propulsion units to operate to modify autonomous flight of the unmanned aerial vehicle, for example, by affecting rotation of the unmanned aerial vehicle about one or more axes. In some instances, actuation of a roll dial can affect rotation of the unmanned aerial vehicle about a roll axis, while actuation of a yaw dial can affect rotation of the unmanned aerial vehicle about a yaw axis, for example, while maintaining autonomous flight of the unmanned aerial vehicle. In some instances, actuation of a throttle dial can affect altitude of the unmanned aerial vehicle, while actuation of a pitch dial can affect velocity of the unmanned aerial vehicle.

[0109] In some instances, a user input (e.g., actuation of a dial lever) can superimpose a directional component to the autonomous flight path 508 of the UAV. For example, the user input can modify the distance 514 specific to the desired target 506 to cause the UAV to move toward a new target 516. In some instances, the UAV can move toward the new target without rotating about a yaw axis. For example, the UAV can move toward the new target without rotating the UAV about a roll axis. The superimposed directional component can or can not be perpendicular to the autonomous flight path 508 of the UAV. In some instances, the superimposed directional component can be along a reference plane. A reference plane as used herein can be any reference plane. The reference plane can be a relative reference plane depending on other factors. For example, the reference plane can be adjusted according to the state of the UAV (e.g., the position and / or orientation of the UAV). For example, the reference plane can be a lateral plane of the UAV, which can be adjusted according to the orientation of the UAV. In some instances, the reference plane can be a lateral plane of the UAV at a hover position. In some instances, the reference plane can be a lateral plane of the UAV at a vertical position. In some instances, the reference plane can be related to external factors or the environment. The reference plane can be related to a designated or predetermined reference plane (e.g., a lateral plane or a horizontal plane of the UAV). In some instances, the superimposed directional component can be both perpendicular to the autonomous flight path of the UAV and along the reference plane. The superimposed directional component can be a horizontal component, which can modify the autonomous flight path of the UAV in a horizontal direction. In some instances, the superimposed directional component can correspond to the degree of the user input, e.g., the duration of the user input or the force of the user input (e.g., the degree of actuation of one or more joysticks). For example, the superimposed directional component can increase gradually if the user input is maintained. For example, a greater directional component can be superimposed to a greater degree of actuation of a joystick than to a lesser degree of actuation of a joystick. The correspondence can be linear or non-linear. In some instances, the superimposed component can modify the flight path of the UAV according to any mathematical function (e.g., linear, exponential, etc.).

[0110] In some instances, the user input can superimpose a velocity component 518 to the unmanned vehicle to modify the autonomous flight path 520 of the unmanned vehicle. The superimposed velocity component can or can not be perpendicular to the autonomous flight path of the unmanned vehicle. In some instances, the superimposed velocity component can be along a reference plane. For example, the superimposed velocity component can be along a lateral plane and / or a horizontal plane of the unmanned vehicle. In some instances, the superimposed velocity component can be both perpendicular to the autonomous flight path of the unmanned vehicle and along a reference plane. The superimposed velocity component can be a horizontal component that can modify the autonomous flight path of the unmanned vehicle in a horizontal direction. In some instances, a velocity component related to rotating the unmanned vehicle about a roll axis can be superimposed. In some instances, a velocity component can be superimposed without affecting the rotation of the unmanned vehicle about a yaw axis. In some instances, the superimposed velocity component can continue to be applied while the user input is maintained, as Figure 5 shown in FIG. 6. In some instances, the superimposed velocity component can correspond to the degree of user input, e.g., the duration of the user input or the force of the user input (e.g., the degree of actuation of one or more joysticks). For example, the superimposed velocity component can gradually increase if the user input is maintained. For example, a greater velocity component can be superimposed to a greater degree of actuation of a joystick than to a lesser degree of actuation of a joystick. The correspondence can be linear or non-linear. In some instances, the superimposed component can modify the flight path of the unmanned vehicle according to any mathematical function (e.g., linear, exponential, etc.).

[0111] In some instances, the user input can superimpose an acceleration component 522 to the UAV to modify the autonomous flight path 524 of the UAV. The superimposed acceleration component can be perpendicular or not perpendicular to the autonomous flight path of the UAV. In some instances, the superimposed acceleration component can be along a reference plane. For example, the superimposed acceleration component can be along a lateral plane and / or a horizontal plane of the UAV. In some instances, the superimposed acceleration component can be both perpendicular to the autonomous flight path of the UAV and along a reference plane. The superimposed acceleration component can be a horizontal component that can modify the autonomous flight path of the UAV in a horizontal direction. In some instances, an acceleration component related to rotating the UAV about a roll axis can be superimposed. In some instances, an acceleration component can be superimposed without affecting the rotation of the UAV about a yaw axis. In some instances, the superimposed acceleration component can continue to be applied while the user input is maintained. In some instances, the superimposed acceleration component can correspond to the degree of the user input, e.g., the duration of the user input or the force of the user input (e.g., the degree of actuation of one or more joysticks). For example, the superimposed acceleration component can gradually increase if the user input is maintained. For example, a greater acceleration component can be superimposed to a greater degree of actuation of a joystick than to a lesser degree of actuation of a joystick. The correspondence can be linear or non-linear. In some instances, the superimposed acceleration component can modify the flight path of the UAV according to any mathematical function (e.g., linear, exponential, etc.).

[0112] In some instances, actuation of a roll lever can superimpose a directional component, a velocity component, and / or an acceleration component as previously described. For example, dial lever 510 can be an example of a roll lever. Alternatively, dial lever 512 can be an example of a roll lever. Actuation of a roll lever can superimpose a horizontal velocity component to the UAV. In some instances, rotation of the UAV about the roll axis can cause the superimposed velocity component. In some instances, the superimposed velocity component can correspond to the degree of actuation of the roll lever. For example, a roll lever can include a rest state in which no force is applied to the roll lever, and two fully actuated states in opposite directions. For example, a roll lever can be configured to move left and right. In some instances, the position of the roll lever can be described as between -1 (fully actuated left) and 1 (fully actuated right). For example, a roll lever moved halfway to the left can include a position of -0.5, while a roll lever moved one-third of the way to the right can include a position of 0.333. In some instances, the velocity component (e.g., horizontal velocity component) superimposed to the UAV as a result of actuation of the roll lever can be described by the equation: (1) superimposed velocity component = (roll lever position) x velocity factor. In some instances, the velocity factor can be a predetermined velocity value, e.g., a factory-set or user-determined value. For example, the predetermined velocity value can be equal to or less than about 2 m / s, 4 m / s, 6 m / s, 8 m / s, 10 m / s, 12 m / s, 15 m / s, 20 m / s, or 50 m / s. In some instances, the velocity factor can depend on the forward velocity of the UAV. For example, the forward velocity can refer to the velocity component of the UAV along the autonomous flight path. In these instances, the velocity component superimposed to the UAV as a result of actuation of the roll lever can be described by the equation: (2) superimposed velocity component = (roll lever position) x (forward velocity component) x factor. In some instances, the factor can be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, or 4. In some instances, the factor can be equal to about 0.5. Figure 5 In some instances, the forward velocity component can refer to the velocity component of the UAV along a direction parallel to the autonomous flight path 508. In some instances, the forward velocity component can refer to the velocity component of the UAV along the roll axis of the UAV. In these instances, the velocity component superimposed to the UAV as a result of actuation of the roll lever can be described by the equation: (2) superimposed velocity component = (roll lever position) x (forward velocity component) x factor. In some instances, the factor can be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, or 4. In some instances, the factor can be equal to about 0.5.

[0113] In some instances, user input can modify the flight path of the UAV to cause the UAV to fly in a curved trajectory toward the direction of one or more dial levers. Figure 13The user input can affect the UAV to rotate about a yaw axis of the UAV to cause it to fly in a curved trajectory. For example, dial 1312 can be an example of a yaw dial. Alternatively, dial 1313 can be an example of a yaw dial. In some instances, the speed at which the UAV flies along a pre-set curved trajectory (e.g., trajectories 1306, 1308, 1310, etc.) can depend on or be proportional to the user input on the yaw dial. Alternatively, the curved radius or degree of rotation of the UAV about the yaw axis can inversely depend on or be inversely proportional to the user input on the yaw dial. In some instances, the position of the yaw dial can be described as between -1 (fully actuated to the left) and 1 (fully actuated to the right), substantially as described with respect to the roll dial. If the user fully actuates the yaw dial to the left, the UAV can fly along trajectory 1308 with a small radius of curvature, while if the user actuates the yaw dial to the left by a small amount, the UAV can fly along trajectory 1306 with a large radius of curvature. Similarly, if the user fully actuates the yaw dial to the right, the UAV can fly along trajectory 1310 with a small radius of curvature.

[0114] As Figure 5 and Figure 13 As provided, the flight path of an autonomously operated UAV can be intuitively modified. For example, actuation of one or more dials 510, 512 to the left can modify the flight path of the UAV to move to the left, e.g., while maintaining autonomous flight of the UAV, causing it to remain moving toward a target. For example, actuation of one or more dials to the right (e.g., by user input) can modify the flight path of the UAV to move to the right.

[0115] In some instances, the above-mentioned superimposed components (e.g., direction, velocity, acceleration, etc.) can include a vertical component. Figure 6 A side view of an autonomous flight path of a movable object modified by user input is shown, in accordance with an embodiment. In some instances, UAV 604 can autonomously fly toward target 606, e.g., according to instructions from a user. The autonomous flight can include autonomous flight path 608. In some instances, a user can provide input (e.g., at a user terminal) to modify the autonomous flight of the UAV. For example, the user can provide input on remote controller 602, which includes one or more dials 610, 612. In some instances, the dials can be configured to affect the altitude of the UAV without affecting the rotation of the UAV about one or more axes of the UAV. Alternatively, the dials can be configured to affect the altitude of the UAV through rotation of the UAV about one or more axes.

[0116] As generally described above, a directional component can be superimposed on the autonomous flight path of the unmanned vehicle. In some cases, as shown in embodiment 620, an acceleration component 614 (e.g., a vertical velocity component) can be superimposed on the unmanned vehicle to modify the autonomous flight path of the unmanned vehicle.

[0117] In some cases, as shown in embodiment 630, an acceleration component 616 (e.g., a vertical acceleration component) can be superimposed on the unmanned vehicle to modify the autonomous flight path of the unmanned vehicle.

[0118] In some cases, as shown in embodiments 620 and 630, actuation of the one or more dials upward (e.g., away from the user) can modify the flight path of the unmanned vehicle to move it vertically upward. Alternatively, actuation of the one or more dials downward can modify the flight path of the unmanned vehicle to move it vertically upward. In some cases, actuation of the one or more dials downward (e.g., toward the user) can modify the flight path of the unmanned vehicle to move it vertically downward. Alternatively, actuation of the one or more dials upward can modify the flight path of the unmanned vehicle to move it vertically downward.

[0119] In some instances, a dial lever configured to affect the altitude of the UAV can be a throttle lever. For example, dial lever 610 can be a throttle lever. Alternatively, dial lever 612 can be an example of a throttle lever. Actuation of a throttle lever can superimpose a vertical velocity component to the UAV. In some instances, the superimposed velocity component can correspond to the degree of actuation of the throttle lever. For example, a throttle lever can include a rest state in which no force is applied to the throttle lever, and two fully actuated states in opposite directions. For example, a throttle lever can be configured to be able to (1) move upward as indicated by direction 609, and (2) move downward. In some instances, the position of a throttle lever can be described as being between -1 (fully actuated downward) and 1 (fully actuated upward). For example, a throttle lever moved halfway down can include a position of -0.5, while a roll lever moved one-third of the way up can include a position of 0.333. In some instances, the velocity component (e.g., vertical velocity component) superimposed to the UAV as a result of actuation of the throttle lever can be described by the equation: (1) superimposed velocity component = (throttle lever position) x velocity factor. A negative velocity can indicate that the UAV is moving downward, while a positive velocity component can indicate that the UAV is moving upward. In some instances, the velocity factor can be a predetermined velocity value, e.g., a factory-set or user-determined value. For example, the predetermined velocity value can be equal to or less than about 2 m / s, 4 m / s, 6 m / s, 8 m / s, 10 m / s, 12 m / s, 15 m / s, 20 m / s, or 50 m / s. In some instances, the predetermined velocity value can be equal to about 3 m / s. In some instances, the velocity factor can depend on the forward velocity of the UAV. For example, the forward velocity can refer to the velocity component of the UAV along the autonomous flight path. In Figure 6 In some instances, the forward velocity component can refer to the velocity component of the UAV along the roll axis of the UAV. In these instances, the velocity component superimposed to the UAV as a result of actuation of the throttle lever can be described by the equation: (2) superimposed velocity component = (roll lever position) x (forward velocity component) x factor. A negative velocity can indicate that the UAV is moving downward, while a positive velocity component can indicate that the UAV is moving upward. In some instances, the factor can be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, or 4. In some instances, the factor can be equal to about 0.5.

[0120] In some instances, a component (e.g., velocity component, acceleration component, etc.) can be superimposed without affecting the flight path of the UAV. Figure 14A top view showing a UAV moving along an autonomous flight path at an increased or decreased speed according to an embodiment is shown. In some instances, user input can modify the speed or acceleration of an autonomously flying UAV without affecting its trajectory or flight path. In some instances, user input affecting the speed of the UAV can be received through one or more dials.

[0121] In some instances, a dial configured to affect the speed of a UAV without affecting the flight path of the UAV can be a pitch dial. For example, dial 1402 can be an example of a pitch dial. Alternatively, dial 1403 can be an example of a pitch dial. Actuation of a pitch dial can affect or modify the speed of a UAV, e.g., along the roll axis of the UAV. In some instances, the speed of the UAV can be increased or decreased according to the degree of pitch dial actuation. For example, a pitch dial can include a rest state in which no force is applied to the pitch dial, and two fully actuated states in opposite directions. For example, a pitch dial can be configured to move downward as indicated by direction 1404, and upward. In some instances, the position of a pitch dial can be described as between -1 (fully actuated downward) and 1 (fully actuated upward). For example, a pitch dial moved halfway down can include a position of -0.5, while a roll dial moved one-third up can include a position of 0.333. In some instances, the speed component (e.g., along the autonomous flight path or along its roll direction) superimposed on the UAV due to actuation of the pitch dial can be described by the equation (1) Speed Component = (Pitch Dial Position) x Speed Factor + Base Flight Speed, as shown in embodiment 1406. In some instances, the speed factor can be a predetermined speed value, e.g., a factory-set or user-determined value. For example, the predetermined speed value can be equal to or less than about 2 m / s, 4 m / s, 6 m / s, 8 m / s, 10 m / s, 12 m / s, 15 m / s, 20 m / s, or 50 m / s. In some instances, the base flight speed can be a predetermined base flight speed, e.g., a factory-set or user-determined value. For example, the predetermined base flight speed (predetermined speed value) can be equal to or less than about 2 m / s, 4 m / s, 6 m / s, 8 m / s, 10 m / s, 12 m / s, 15 m / s, 20 m / s, or 50 m / s. In some instances, the speed component (e.g., along the autonomous flight path or along its roll direction) superimposed on the UAV due to actuation of the pitch dial can be described by the equation (2) Speed Component = (Pitch Dial Position + 1) x (Base Flight Speed) x Factor, as shown in embodiment 1408. In some instances, the factor can be equal to or less than about 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, or 4. In some instances, the factor can be equal to about 1.

[0122] In some instances, different dials can modify the flight path of the movable object in different ways. For example, actuation of a roll dial can affect rotation of the unmanned vehicle about a roll axis, which can superimpose a horizontal velocity component onto the unmanned vehicle to modify the autonomous flight path of the unmanned vehicle, while actuation of a yaw dial can affect rotation of the unmanned vehicle about a yaw axis, which can superimpose an acceleration (e.g., centripetal acceleration) component onto the unmanned vehicle to modify the autonomous flight path of the unmanned vehicle. For example, actuation of a throttle dial can superimpose a vertical velocity component onto the unmanned vehicle to modify the autonomous flight path of the unmanned vehicle, while actuation of a pitch dial can affect the velocity of the unmanned vehicle about its roll axis.

[0123] In some instances, the degree of user input can correspond to the degree to which the autonomous flight of the unmanned vehicle is modified. The degree of user input can correspond to the duration of the user input. For example, in response to a sustained input from the user (e.g., sustained actuation of one or more dials), the rotation of the unmanned vehicle about one or more of its axes can increase. For example, depending on the amount of time the one or more dials are actuated, the superimposed directional, velocity, and / or acceleration components can continue to increase. In some instances, the degree of user input can correspond to the force with which the user provides the user input. Figure 7 The force of user input proportionally modifying autonomous flight of a movable object is shown in accordance with an embodiment. In some instances, the degree of user input can correspond to the amount of force exerted by the user in providing the user input. In some instances, the degree of user input can correspond to the degree to which the input device (e.g., one or more dials) is actuated. During a first time period 701, the degree of user input can be a first level 703. Accordingly, the autonomous flight path 704 of the unmanned vehicle 706 can be modified by a first degree, as shown by the modified flight path 705. During a second time period 707, the degree of user input can be a second level 709. For example, the user can exert more force on the dial or can further actuate the dial. Accordingly, the autonomous flight path of the unmanned vehicle can be modified by a second degree, as shown by the second modified flight path 711. In some instances, the correspondence can be linear. For example, if the user exerts twice the force, or actuates the dial twice as far, the velocity component superimposed onto the unmanned vehicle can be twice as much, and the flight path of the unmanned vehicle can be modified by twice the amount. Alternatively, the correspondence can be non-linear. In some instances, the relationship between actuation of the one or more dials and the behavior of the unmanned vehicle can be as described above, for example, with respect to the roll dial, the pitch dial, and the yaw dial.

[0124] In some instances, various other factors (e.g., other than user input) can be considered to modify the autonomous flight of the UAV. In some instances, the various other factors can include environmental factors. The environmental factors can be determined based on one or more sensors on the UAV. For example, data from a distance or obstacle sensor can be considered to modify the autonomous flight path of the UAV to ensure that the modification required according to the user input does not endanger the UAV or other (e.g., place the UAV at risk of colliding with a detected obstacle). For example, data from other sensors can be considered to modify the autonomous flight path of the UAV to ensure that the modification required according to the user input does not upset the balance of the UAV, which otherwise can cause the UAV to become unstable or uncontrollable. Thus, the user input can be processed and interpreted by a flight controller on the UAV, for example, to ensure stability and safety, before a signal is generated and instructions are sent to one or more propulsion mechanisms to modify the autonomous flight path.

[0125] In some instances, the various factors mentioned above can include a threshold. The threshold can be a threshold time and / or a threshold distance. The threshold time and / or the threshold distance can be predetermined. In some instances, the threshold time and / or the threshold distance can be configured prior to or during operation of the UAV by the user. Figure 8 The behavior of the UAV according to embodiments when a threshold is reached is illustrated. In some instances, the threshold can be a threshold distance. For example, if the flight path of the UAV deviates from the original autonomous flight path by more than the threshold distance as a result of user input, further modification of the autonomous flight path can be prevented even with the user input. In some instances, the threshold can be a threshold time. For example, if the flight path of the UAV deviates from the original autonomous flight path by more than the threshold time as a result of user input, further deviation can be prevented even with the user input.

[0126] For example, autonomous flight of the UAV can be implemented 812. The autonomous flight can include an autonomous flight path 814 toward a target 816. While the UAV is autonomously flying toward the target, a user (e.g., a manipulator of the UAV) can modify the autonomous flight, e.g., by actuating one or more dials. As a result of the user input, the UAV can follow a modified flight path 818. In some cases, once the UAV is a threshold distance 819 away from the original autonomous flight path, the user input can no longer instruct the UAV to deviate from the autonomous flight path 814. For example, the UAV can only maintain the threshold distance 819 from the autonomous flight path 814, even if the user input continues, as illustrated by embodiment 810. For example, once the UAV reaches the threshold distance as a result of the user input, the UAV can begin moving toward the target 826, as illustrated by embodiment 820. In some cases, once the UAV reaches the threshold distance as a result of the user input, the UAV can begin moving toward the original autonomous flight path 824 before moving toward the target. In some cases, once the UAV reaches the threshold distance as a result of the user input, the autonomous flight of the UAV can be interrupted, as illustrated by embodiment 830. Subsequently, the UAV can hover at a location 832, or land or approach the location at which the distance threshold was reached. In some cases, the user can be required to manually control the UAV after reaching the threshold distance from the autonomous flight path.

[0127] In some cases, a warning can be sent to the user when the threshold distance is reached. The warning can be a visual, audible, and / or tactile warning. In some cases, the warning can be sent to a user terminal. The warning can inform the user that they are at the threshold distance. In some cases, the warning can inform the user that continuing the user input and / or deviating from the original autonomous flight path will result in termination of the autonomous flight. In some cases, the warning can be sent when the threshold distance is reached, but the user input can continue to modify the autonomous flight of the UAV. In some cases, the warning can be sent when the threshold distance is reached, and further modification of the flight path can be prevented, as described in embodiments 810, 820, or 830 above. In some cases, the behavior of the UAV can depend on more than one threshold. For example, the behavior of the UAV upon release of the user input can depend on both a threshold time and a threshold distance. For example, a first threshold distance (or time) and a second threshold distance (or time) can affect the behavior of the UAV during the user input. In some cases, the UAV can send a warning to the user when the autonomous flight path is deviated by a first threshold distance, and further deviation can be prevented when a second threshold distance greater than the first threshold distance is reached.

[0128] Although this discussion focuses on a threshold distance, it should be understood that the above discussion can equally apply to a threshold time. For example, if the UAV deviates from the autonomous flight path for longer than the threshold time, the UAV can be forced to return to the original autonomous flight path before it is allowed to deviate again. In some cases, if the flight path of the UAV deviates from the autonomous flight path for longer than the threshold time, a warning can be sent to the user as described above.

[0129] The modification of the autonomous flight path can only be maintained for a period of time, which is the time that the user input is maintained. Alternatively, the modification of the autonomous flight path can be maintained after the user input. Figure 9 The behavior of the UAV after the user input modifying the autonomous flight of the movable object is released according to embodiments is shown. For each configuration 910, 920, 930, 940, and 950, a user input modifying the autonomous flight path can be received for a period of time Tl. The user input can be released after this period of time. After the release, the movable object can operate autonomously to fly towards a target (e.g., a target object and / or a target destination), as shown in embodiments 910 and 920. In some cases, the UAV can calculate (e.g., autonomously calculate) a shortest flight path between the UAV and the target, and generate a new autonomous flight path 912, to then move towards the target. In some cases, the UAV can return to the original autonomous flight path 922 and continue its autonomous flight towards the target. Alternatively, after the release, the UAV can continue its flight on the modified flight path 932. For example, after a yaw stick actuation and subsequent release, the UAV can fly in its new roll direction. In some cases, after the release, the UAV can remain at the location where the user input was released 942 until further input. For example, the UAV can hover at the location, or land at or near the location where the user input was released. In some cases, after the release, the UAV can take a new autonomous flight path 952. The new autonomous flight path can be parallel to the original autonomous flight path. Alternatively, the new autonomous flight path can be at an arbitrary angle with respect to the original autonomous flight path.

[0130] In some cases, the behavior of the UAV after the user input (e.g., modifying the autonomous flight path) is released can depend on a threshold. The threshold can be a threshold time and / or a threshold distance. For example, if the user input is provided for a duration longer than the threshold time, the UAV can continue the modified autonomous flight path after the user input is released, as illustrated in configuration 930. However, if the user input is provided for a duration shorter than the threshold time, the UAV can autonomously operate towards the target, as illustrated in configuration 910 or 920. In some cases, if the user input is provided for a duration shorter than the threshold time, the UAV can remain at the location where the user input was released until further instructions are given, as illustrated in configuration 940. The threshold distance and / or threshold time can be predetermined. In some cases, the threshold distance and / or threshold time can be configured by the user prior to or during operation of the UAV.

[0131] Although primarily a threshold time is discussed, it should be understood that the above discussion can equally apply to a threshold distance. For example, if the user input provided causes the UAV to deviate from the original autonomous flight path by more than a distance threshold, the UAV can continue the modified autonomous flight path after the user input, as illustrated in configuration 930. However, if the user input provided causes the UAV to deviate from the original autonomous flight path by less than the distance threshold, the UAV can autonomously operate towards the target (as illustrated in configuration 910 or 920), or can remain at the location where the user input was released (as illustrated in configuration 940).

[0132] Any combination of the various configurations provided herein is possible. For example, the UAV can continue the modified autonomous flight path if the duration of the user input provided is shorter than a threshold time, as illustrated in configuration 930, while the UAV can autonomously operate towards the original target if the duration of the user input is maintained for longer than the threshold time, as illustrated in configuration 910 or 920. Furthermore, the behavior of the UAV can depend on more than one threshold. For example, the behavior of the UAV after the user input is released can depend on both a threshold time and a threshold distance. For example, the behavior of the UAV after the user input is released can depend on whether the duration of the user input is maintained for longer than a first threshold time (or distance) or a second threshold time (or distance).

[0133] Once the UAV begins following the new autonomous flight path, subsequent user input during the autonomous flight can modify the new autonomous flight path. Alternatively, subsequent user input during the autonomous flight can modify the UAV with respect to the original autonomous flight path. Figure 10Autonomous flight paths of a user input modified unmanned vehicle are shown in accordance with embodiments. Autonomous flight of an unmanned vehicle 1002 can be implemented substantially as described herein. Autonomous flight of the unmanned vehicle can include an original autonomous flight path 1004. During a time period Tl, user input can modify the autonomous flight path of the unmanned vehicle. Upon release of the user input, the unmanned vehicle can continue its autonomous flight. In some cases, the unmanned vehicle can continue its autonomous flight with a new autonomous flight path 1006. During a time period T2, new user input can modify the new autonomous flight path of the unmanned vehicle. In some cases, the modification 1008 can be relative to the new autonomous flight path as shown in embodiment 1010. For example, a directional component can be superimposed on the new autonomous flight path. The directional component can be perpendicular to the new autonomous flight path. In some cases, the directional component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane. Alternatively or additionally, a velocity component can be superimposed on the unmanned vehicle to modify the new autonomous flight path of the unmanned vehicle. The velocity component can be perpendicular to the new autonomous flight path of the unmanned vehicle. In some cases, the velocity component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane. Alternatively or additionally, an acceleration component can be superimposed on the unmanned vehicle to modify the new autonomous flight path of the unmanned vehicle. The acceleration component can be perpendicular to the new autonomous flight path of the unmanned vehicle. In some cases, the acceleration component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane.

[0134] In some cases, the modification 1022 can be relative to the original autonomous flight path 1024 as shown in embodiment 1020. For example, a directional component can be superimposed on the original autonomous flight path. The directional component can be perpendicular to the original autonomous flight path. In some cases, the directional component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane. Alternatively or additionally, a velocity component can be superimposed on the unmanned vehicle to modify the new autonomous flight path of the unmanned vehicle. The velocity component can be perpendicular to the original autonomous flight path of the unmanned vehicle. In some cases, the velocity component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane. Alternatively or additionally, an acceleration component can be superimposed on the unmanned vehicle to modify the new autonomous flight path of the unmanned vehicle. The acceleration component can be perpendicular to the original autonomous flight path of the unmanned vehicle. In some cases, the acceleration component can be superimposed along a reference plane, e.g., a lateral plane of the unmanned vehicle, or along a horizontal plane.

[0135] In some instances, one or more of the dials described throughout can be used in combination to affect the operation of the UAV. For example, one or more dials can be used simultaneously or sequentially. In some instances, one or more dials can be used together to adjust the autonomous flight of the UAV. In some instances, the yaw dial can be used with the roll dial, the pitch dial, and / or the throttle dial. In some instances, the roll dial can be used with the pitch dial, the throttle dial, and / or the yaw dial. In some instances, the pitch dial can be used with the throttle dial, the yaw dial, and / or the roll dial. In some instances, the throttle dial can be used with the yaw dial, the roll dial, and / or the pitch dial. In some instances, a user can operate two input means (e.g., two dials) to affect or change the behavior of the UAV during autonomous flight. In some instances, a user can operate 3, 4, 5, 6, 7, 8, 9, 10, or more input means to affect or change the behavior of the UAV during autonomous flight of the UAV. The input means can include, but are not limited to, dials, buttons, accelerometers, sound input devices, etc., substantially as described elsewhere herein. The combination of user inputs provided at the user terminal (e.g., dials) in implementing autonomous flight of the UAV can further modify the autonomous flight of the UAV in new ways.

[0136] For example, when the UAV is in an autonomous flight state (e.g., tap-to- follow mode) and flying in a particular direction, a user of the UAV can operate a dial that affects the rotation of the UAV about the yaw dial (e.g., yaw dial), and can operate a dial that affects the altitude of the UAV (e.g., throttle dial). In this case, the UAV can loop up or down. If the user's inputs are maintained on each dial, the UAV can spiral up or down. For example, when the UAV is in an autonomous flight state (e.g., tap-to-follow mode), a user of the UAV can operate a dial that affects the rotation of the UAV about the yaw dial (e.g., yaw dial), and can operate a dial that affects the speed of the UAV (e.g., pitch dial). In this case, the turning of the UAV can be precisely controlled. For example, in an autonomous flight state, the yaw dial can be fully actuated (e.g., to the left) when the pitch dial is fully actuated downward to reduce the speed of the UAV. The radius of curvature can be further reduced than when the yaw dial is actuated.

[0137] The systems provided herein can enable a user to quickly change the trajectory of a UAV while maintaining overall autonomous control (e.g., by a flight controller) so that the UAV can continue to accomplish its purpose or continue to fly toward a target. There can be seamless transitions between autonomous flight and modification of autonomous flight so that the burden on the user to manually navigate the vehicle can be significantly reduced while the user can still have some level of control when needed or beneficial. Alternatively or additionally, different user interfaces can enable a user to quickly and intuitively react to emergency or unexpected situations (e.g., by interrupting autonomous flight or modifying autonomous flight).

[0138] The systems, devices, and methods described herein can be applied to a wide variety of movable objects. As previously described, any description of a vehicle herein can apply to and be used for any movable object. The movable objects of the present disclosure can be configured to move in any suitable environment, such as in the air (e.g., a fixed-wing aircraft, a rotary-wing aircraft, or an aircraft with neither fixed wings nor rotary wings), in water (e.g., a ship or a submarine), on the ground (e.g., a motor vehicle, such as a car, a truck, a bus, a van, a motorcycle; a movable structure or frame, such as a rod, a fishing pole; or a train), under the ground (e.g., a subway), in space (e.g., a space vehicle, a satellite, or a probe), or any combination of these environments. The movable object can be a vehicle, such as the vehicles described elsewhere herein. In some embodiments, the movable object can be mounted to a living being, such as a human or an animal. Suitable animals can include a cow, a dog, a cat, a horse, a cow, a sheep, a pig, a flea, a rodent, or an insect.

[0139] The movable object can be free to move in an environment with respect to six degrees of freedom (e.g., three degrees of freedom for translation and three degrees of freedom for rotation). Alternatively, the movement of the movable object can be constrained with respect to one or more degrees of freedom, such as by a predetermined path, trajectory, or direction. The movement can be actuated by any suitable actuation mechanism, such as an engine or a motor. The actuation mechanism of the movable object can be powered by any suitable energy source, such as electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy, or any suitable combination thereof. As described elsewhere herein, the movable object can be self-propelled by a propulsion system. The propulsion system can optionally use, for example, electrical energy, magnetic energy, solar energy, wind energy, gravitational energy, chemical energy, nuclear energy, or any suitable combination thereof. Alternatively, the movable object can be carried by a living being.

[0140] In some instances, the movable object can be a vehicle. Suitable vehicles can include water-borne vehicles, aerial vehicles, space vehicles, or ground vehicles. For example, an aerial vehicle can be a fixed-wing aircraft (e.g., an airplane, a glider), a rotary-wing aircraft (e.g., a helicopter, a gyrocopter), an aircraft having both fixed- and rotary- wings, or none of these (e.g., a dirigible, a hot air balloon). A vehicle can be self-propelled, for example, in air, on or in water, in space, or on or under a ground surface. A self-propelled vehicle can utilize a propulsion system, for example, including one or more engines, motors, wheels, axles, magnets, rotors, propellers, blades, nozzles, or any suitable combination thereof. In some instances, a propulsion system can be used to enable the movable object to take off from a surface, land on a surface, maintain its current position and / or orientation (e.g., hover), change orientation, and / or change position.

[0141] A movable object can be remotely controlled by a user or locally controlled by an occupant within or on the movable object. In some embodiments, a movable object is an unmanned movable object, for example, a drone. An unmanned movable object, for example, a drone, can have no occupants on the movable object. A movable object can be controlled by a human or an autonomous control system (e.g., a computer control system), or any suitable combination thereof. A movable object can be an autonomous or semi-autonomous robot, for example, a robot configured with artificial intelligence.

[0142] A movable object can have any suitable size and / or dimensions. In some embodiments, a movable object can have a size and / or dimensions to accommodate a human occupant within or on the vehicle. Alternatively, a movable object can have a size and / or dimensions that are smaller than to accommodate a human occupant within or on the vehicle. A movable object can have a size and / or dimensions suitable to be lifted or carried by a human. Alternatively, a movable object can have a size and / or dimensions that are larger than suitable to be lifted or carried by a human. In some instances, a movable object can have a maximum dimension (e.g., length, width, height, diameter, diagonal) that is less than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. A maximum dimension can be greater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. For example, a distance between axes of opposing rotors of a movable object can be less than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. Alternatively, a distance between axes of opposing rotors can be greater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.

[0143] In some embodiments, the movable object can have a volume less than 100 cm x 100 cm x 100 cm, less than 50 cm x 50 cm x 30 cm, or less than 5 cm x 5 cm x 3 cm. The total volume of the movable object can be less than or equal to about: 1 cm 3 , 2 cm 3 , 5 cm 3 , 10 cm 3 , 20 cm 3 , 30 cm 3 , 40 cm 3 , 50 cm 3 , 60 cm 3 , 70 cm 3 , 80 cm 3 , 90 cm 3 , 100 cm 3 , 150 cm 3 , 200 cm 3 , 300 cm 3 , 500 cm 3 , 750 cm 3 , 1000 cm 3 , 5000 cm 3 , 10,000 cm 3 , 100,000 cm 3 , 1 m 3 , or 10 m 3 . Conversely, the total volume of the movable object can be greater than or equal to about: 1 cm 3 , 2 cm 3 , 5 cm 3 , 10 cm 3 , 20 cm 3 , 30 cm 3 , 40 cm 3 , 50 cm 3 , 60 cm 3 , 70 cm 3 , 80 cm 3 , 90 cm 3 , 100 cm 3 , 150 cm 3 , 200 cm 3 , 300 cm 3 , 500 cm 3 , 750 cm 3 , 1000 cm 3 , 5000 cm 3 , 10,000 cm 3 , 100,000 cm 3 , 1 m 3 , or 10 m 3 .

[0144] In some embodiments, the movable object can have a footprint (which can refer to a lateral cross-sectional area encompassed by the movable object) that is less than or equal to about: 32,000 cm 2 , 20,000 cm 2 , 10,000 cm 2 , 1,000 cm 2 , 500 cm 2 , 100 cm 2 , 50 cm 2 , 10 cm 2 , or 5 cm 2 . Conversely, the footprint can be greater than or equal to about: 32,000 cm 2 , 20,000 cm 2 , 10,000 cm 2 , 1,000 cm 2 , 500 cm 2 , 100 cm 2 , 50 cm 2 , 10 cm 2 , or 5 cm 2 .

[0145] In some cases, the movable object can have a weight that is no more than 1000 kg. The weight of the movable object can be less than or equal to about: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg, or 0.01 kg. Conversely, the weight can be greater than or equal to about: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg, or 0.01 kg.

[0146] In some embodiments, the payload carried by the movable object relative to the movable object can be small. As described in further detail below, the payload can include a payload and / or a carrier. In some examples, the ratio of the weight of the movable object to the weight of the payload can be greater than, less than, or equal to about 1 : 1. In some cases, the ratio of the weight of the movable object to the weight of the payload can be greater than, less than, or equal to about 1 : 1. Alternatively, the ratio of the weight of the carrier to the weight of the payload can be greater than, less than, or equal to about 1 : 1. When desired, the ratio of the weight of the movable object to the weight of the payload can be less than or equal to: 1 :2, 1 :3, 1 :4, 1 :5, 1 : 10, or even less. Conversely, the ratio of the weight of the movable object to the weight of the payload can also be greater than or equal to: 2: 1, 3: 1, 4: 1, 5: 1, 10: 1, or even more.

[0147] In some embodiments, the movable object can have low power consumption. For example, the movable object can use less than about: 5 W / h, 4 W / h, 3 W / h, 2 W / h, 1 W / h, or less. In some cases, the carrier of the movable object can have low power consumption. For example, the carrier can use less than about: 5 W / h, 4 W / h, 3 W / h, 2 W / h, 1 W / h, or less. Alternatively, the payload of the movable object can have low power consumption, for example, less than about: 5 W / h, 4 W / h, 3 W / h, 2 W / h, 1 W / h, or less.

[0148] Figure 11 An unmanned aerial vehicle (UAV) 1100 is shown in accordance with an embodiment. The UAV can be an example of a movable object as described herein to which the methods and apparatus for discharging a battery assembly can be applied. The UAV 1100 can include a propulsion system having four rotors 1102, 1104, 1106, and 1108. Any number of rotors can be provided (e.g., 1, 2, 3, 4, 5, 6, or more). The rotors, rotor assemblies, or other propulsion systems of the UAV can cause the UAV to hover / hold position, change direction, and / or change location. The distance between the axes of the opposing rotors can be any suitable length 1110. For example, the length 1110 can be less than or equal to 2 m, or less than or equal to 5 m. In some embodiments, the length 1110 can be in a range from 40 cm to 1 m, from 10 cm to 2 m, or from 5 cm to 5 m. Any description herein of a UAV can apply to a movable object, e.g., a movable object of a different type, and vice versa. The UAV can use the assisted takeoff system or method described herein.

[0149] Figure 12is a block diagram schematic of a system 1200 for controlling a movable object according to embodiments. The system 1200 can be used in combination with any appropriate system, device, and method disclosed herein. The system 1200 can include a sensing module 1202, a processing unit 1204, a non-transitory computer readable medium 1206, a control module 1208, and a communication module 1210.

[0150] The sensing module 1202 can utilize different types of sensors that collect movable object related information in different ways. Different types of sensors can sense different types of signals or signals from different sources. For example, the sensors can include inertial sensors, GPS sensors, proximity sensors (e.g., lidar), or vision / image sensors (e.g., cameras). The sensing module 1202 can be coupled to the processing unit 1204 having multiple processors. In some embodiments, the sensing module can be coupled to a transmission module 1212 (e.g., a Wi-Fi image transmission module) configured to directly transmit sensing data to an appropriate external device or system. For example, the transmission module 1212 can be used to transmit images captured by a camera of the sensing module 1202 to a remote terminal.

[0151] The processing unit 1204 can have one or more processors, such as programmable processors (e.g., central processing units (CPUs)). The processing unit 1204 can be coupled to the non-transitory computer readable medium 1206. The non-transitory computer readable medium 1206 can store logic, code, and / or program instructions executable by the processing unit 1204 for performing one or more steps. The non-transitory computer readable medium can include one or more memory units (e.g., removable media such as SD cards or external memory such as random access memory (RAM)). In some embodiments, data from the sensing module 1202 can be directly transferred and stored in the memory units of the non-transitory computer readable medium 1206. The memory units of the non-transitory computer readable medium 1206 can store logic, code, and / or program instructions executable by the processing unit 1204 to perform any appropriate embodiment of the methods described herein. For example, the processing unit 1204 can be configured to execute instructions that cause one or more processors of the processing unit 1204 to analyze sensing data generated by the sensing module. The memory units can store sensing data from the sensing module to be processed by the processing unit 1204. In some embodiments, the memory units of the non-transitory computer readable medium 1206 can be used to store processing results generated by the processing unit 1204.

[0152] In some embodiments, the processing unit 1204 can be coupled to a control module 1208 configured to control a state of the movable object. For example, the control module 1208 can be configured to control a propulsion mechanism of the movable object to adjust a spatial disposition, velocity, and / or acceleration of the movable object with respect to six degrees of freedom. Alternatively or in combination, the control module 1208 can control one or more states of the carrier, payload, or sensing module.

[0153] The processing unit 1204 can be coupled to a communication module 1210 configured to transmit and / or receive data with one or more external devices, such as a terminal, display device, or other remote controller. Any suitable communication means can be used, such as wired or wireless communication. For example, the communication module 1210 can utilize one or more local area networks (LANs), wide area networks (WANs), infrared, radio, Wi-Fi, peer-to-peer (P2P) networks, telecommunication networks, cloud communications, etc. Optionally, a relay station, such as a tower, satellite, or mobile station can be used. The wireless communication can be proximity-dependent or proximity-independent. In some embodiments, a line-of-sight can or can not be required for communication. The communication module 1210 can transmit and / or receive sensing data from the sensing module 1202, processing results generated by the processing unit 1204, predetermined control data, user commands from a terminal or remote controller, etc.

[0154] The components of the system 1200 can be arranged in any suitable configuration. For example, one or more components of the system 1200 can be located on the movable object, carrier, payload, terminal, sensing system, or additional external devices in communication with one or more of the above. Additionally, although Figure 12 A single processing unit 1204 and a single non-transitory computer-readable medium 1206 are depicted, but one of skill in the art will understand that this is not intended to be limiting, and that the system 1200 can include multiple processing units and / or non-transitory computer-readable media. In some embodiments, one or more of the multiple processing units and / or non-transitory computer-readable media can be located in different locations, such as on the movable object, carrier, payload, terminal, sensing module, additional external devices in communication with one or more of the above, or suitable combinations thereof, such that any suitable aspect of the processing and / or storage functions performed by the system 1200 can occur at one or more of the above locations.

[0155] As used herein, A and / or B includes one or more of A or B, and combinations thereof, e.g., A and B. It will be understood that, although the terms “first,” “second,” “third,” etc. can be used herein to describe various elements, components, regions, and / or sections, these elements, components, regions, and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, or section from another element, component, region, or section. Thus, a first element, component, region, or section discussed below could be termed a second element, component, region, or section without departing from the teachings of the present application.

[0156] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0157] In addition, relative terms such as "lower" or "bottom" and "upper" or "top" can be used herein to describe one element's or portion's relationship to another element or portion as illustrated in the Figures. It will be understood that relative terms are intended to encompass different orientations of an element in addition to the orientation depicted in the Figures. For example, if an element is depicted as being "lower" or "bottom" relative to another element, it will be understood that the element can also be "upper" or "top" relative to the other element. The exemplary term "lower" can therefore encompass both an orientation of below and above. Likewise, the exemplary term "upper" can encompass both an orientation of above and below. The exemplary terms "below" or "beneath" or "lower" and "above" or "upper" can thus encompass both an orientation of below and above. The exemplary term "vertical" can encompass both an orientation of vertical and horizontal.

[0158] While the preferred embodiments of the application have been illustrated and described herein, it will be appreciated that various changes can be made to the embodiments by those skilled in the art without departing from the scope of the application. It should be understood that various alternatives to the embodiments of the application described herein can be employed in practicing the application. Many different combinations of the embodiments described herein can be used to advantage. Similarly, the various features described or recited above can be practiced in various combinations, and the disclosure is to be understood to include all such combinations. Furthermore, the above description is intended to be illustrative only and not restrictive. The scope of the application should be given by the appended claims along with their full scope of equivalents, and the application is not limited to the embodiments described herein.

Claims

1. A system for modifying the autonomous flight of an unmanned aerial vehicle, the system comprising: A flight controller configured to: generate a first set of signals in response to a first user input received at a first user interface, the first set of signals enabling autonomous flight of the unmanned aerial vehicle; and generate a second set of signals in response to a second user input received at a second user interface, the second set of signals modifying the autonomous flight of the unmanned aerial vehicle, the autonomous flight including an autonomous flight trajectory or an autonomous flight path; The flight controller modifies the autonomous flight of the unmanned aerial vehicle during the duration of the second user input, wherein the second user input is superimposed with a directional component on the autonomous flight of the unmanned aerial vehicle, or by superimposing a velocity component or an acceleration component on the unmanned aerial vehicle to modify the autonomous flight of the unmanned aerial vehicle; Modifying the autonomous flight of the unmanned aerial vehicle includes: modifying the autonomous flight path or trajectory of the unmanned aerial vehicle while maintaining autonomous flight.

2. The system of claim 1, wherein the second user input modifies at least one of the trajectory, altitude, speed, and acceleration of the unmanned aerial vehicle.

3. The system of claim 1, wherein the second user interface comprises one or more actuable mechanisms.

4. The system of claim 3, wherein the one or more mechanisms include one or more levers.

5. The system of claim 4, wherein the one or more levers include a roll bar configured to influence the rotation of the unmanned aerial vehicle about a roll axis, the actuation of the roll bar being configured to superimpose a directional component, a velocity component, or an acceleration component perpendicular to the autonomous flight path of the unmanned aerial vehicle.

6. The system of claim 5, wherein the degree of actuation corresponds to the magnitude of the directional component, the magnitude of the velocity component, or the magnitude of the acceleration component.

7. The system of claim 4, wherein the one or more levers include a yaw lever configured to influence the rotation of the unmanned aerial vehicle about a yaw axis, and the actuation of the yaw lever is configured to superimpose centripetal acceleration onto the unmanned aerial vehicle.

8. The system of claim 7, wherein the degree of actuation corresponds inversely to the radius of the trajectory of the unmanned aerial vehicle.

9. The system of claim 4, wherein the one or more levers are used to stop autonomous flight.

10. The system of claim 1, wherein the flight controller further modifies the autonomous flight of the unmanned aerial vehicle by taking into account environmental factors related to the second user input.

11. The system of claim 1, wherein the first user interface is further configured to display the autonomous flight path of the unmanned aerial vehicle and the modified flight path of the unmanned aerial vehicle.

12. The system according to claim 1, wherein, The flight controller is further configured to generate a third set of signals that instruct the unmanned aerial vehicle (UAV) to fly along a new flight path that is different from the flight path the UAV had followed before the second set of signals were generated.

13. The system according to claim 1, wherein, The autonomous flight includes at least one of the following: Flight toward a target, which is either a target object or a target destination; Flight to the designated location; The unmanned aerial vehicle returned to its home location autonomously. Autonomous flight to points of interest; Flight along a preset trajectory or in a preset direction; Flight along a self-planned trajectory; Flight along a user-configured trajectory; Fly to the location corresponding to the clicked point on the map of the first user interface; Unmanned aerial vehicles (UAVs) track target objects.

14. A method for modifying the autonomous flight of an unmanned aerial vehicle, the method comprising: In response to a first user input received at a first user interface, a first set of signals is generated by the flight controller, and the first set of signals enables the autonomous flight of the unmanned aerial vehicle. and In response to a second user input received on a second user interface, a second set of signals is generated by the flight controller. The second set of signals modifies the autonomous flight of the unmanned aerial vehicle, including an autonomous flight trajectory or autonomous flight path. The flight controller modifies the autonomous flight of the unmanned aerial vehicle (UAV) during the duration of the second user input. The second user input is used to superimpose a directional component onto the autonomous flight of the UAV, or to superimpose a velocity component or an acceleration component onto the UAV to modify the autonomous flight. Modifying the autonomous flight of the UAV includes modifying the autonomous flight path or trajectory of the UAV while maintaining autonomous flight.

15. An unmanned aerial vehicle, the unmanned aerial vehicle comprising: A flight controller configured to generate: a first set of signals indicating autonomous flight of an unmanned aerial vehicle, wherein the first set of signals is generated based on a first user input received at a first user interface; and a second set of signals indicating modifications to the autonomous flight of the unmanned aerial vehicle, wherein the second set of signals is generated based on a second user input received at a second user interface. and One or more propulsion units are configured to: realize autonomous flight of the unmanned aerial vehicle in response to a first set of signals, and modify the autonomous flight of the unmanned aerial vehicle in response to a second set of signals, wherein the autonomous flight includes an autonomous flight trajectory or an autonomous flight path; The flight controller modifies the autonomous flight of the unmanned aerial vehicle (UAV) during the duration of the second user input. The second user input is used to superimpose a directional component onto the autonomous flight of the UAV, or to superimpose a velocity component or an acceleration component onto the UAV to modify the autonomous flight. Modifying the autonomous flight of the UAV includes modifying the autonomous flight path or trajectory of the UAV while maintaining autonomous flight.

16. A system for modifying the autonomous flight path of an unmanned aerial vehicle, the system comprising: A first user interface configured to receive first user input, wherein the first user input provides one or more instructions to enable autonomous flight of the unmanned aerial vehicle; and A second user interface configured to receive input from a second user, wherein the second user input provides one or more instructions to modify the autonomous flight of the unmanned aerial vehicle, the autonomous flight including an autonomous flight trajectory or autonomous flight path; Wherein, the second user input modifies the autonomous flight of the unmanned aerial vehicle during the duration of the second user input, wherein the second user input superimposes a directional component onto the autonomous flight of the unmanned aerial vehicle, or modifies the autonomous flight of the unmanned aerial vehicle by superimposing a velocity component or an acceleration component onto the unmanned aerial vehicle, and modifying the autonomous flight of the unmanned aerial vehicle includes: modifying the autonomous flight path or autonomous flight trajectory of the unmanned aerial vehicle while maintaining autonomous flight.

17. A method for modifying the autonomous flight path of an unmanned aerial vehicle, the method comprising: The first user interface receives first user input, wherein the first user input provides one or more instructions to enable the autonomous flight of the unmanned aerial vehicle. and The second user interface receives input from a second user, wherein the second user input provides one or more instructions to modify the autonomous flight of the unmanned aerial vehicle, the autonomous flight including an autonomous flight trajectory or an autonomous flight path; Wherein, the second user input modifies the autonomous flight of the unmanned aerial vehicle during the duration of the second user input, wherein the second user input superimposes a directional component onto the autonomous flight of the unmanned aerial vehicle, or modifies the autonomous flight of the unmanned aerial vehicle by superimposing a velocity component or an acceleration component onto the unmanned aerial vehicle, and modifying the autonomous flight of the unmanned aerial vehicle includes: modifying the autonomous flight path or autonomous flight trajectory of the unmanned aerial vehicle while maintaining autonomous flight.

18. A system for controlling the flight of an unmanned aerial vehicle, the system comprising one or more processors, the one or more processors being configured individually or collectively to: To achieve autonomous flight of the unmanned aerial vehicle, wherein the autonomous flight includes an autonomous flight trajectory or autonomous flight path; and The autonomous flight path or trajectory can be modified in response to user input, wherein the autonomous flight path or trajectory is modified while maintaining the autonomous flight. in, The modification of the autonomous flight path is maintained only for the duration of the user input, wherein the autonomous flight path of the unmanned aerial vehicle is modified by superimposing a directional component onto the autonomous flight path of the unmanned aerial vehicle, or by superimposing a velocity component or an acceleration component onto the unmanned aerial vehicle.

19. A method for controlling the flight of an unmanned aerial vehicle, the method comprising: To achieve autonomous flight of the unmanned aerial vehicle, wherein the autonomous flight includes an autonomous flight trajectory or an autonomous flight path; and The autonomous flight path or trajectory can be modified in response to user input, wherein the autonomous flight path or trajectory is modified while maintaining the autonomous flight. Specifically, the modification of the autonomous flight path is maintained only during the duration of the user input, wherein the autonomous flight path of the unmanned aerial vehicle is modified by superimposing a directional component onto the autonomous flight path of the unmanned aerial vehicle, or by superimposing a velocity component or an acceleration component onto the unmanned aerial vehicle.

20. An unmanned aerial vehicle, the unmanned aerial vehicle comprising: A flight controller configured to generate: a first set of signals for autonomous flight of the unmanned aerial vehicle, wherein the autonomous flight includes an autonomous flight trajectory or autonomous flight path, and a second set of signals for modifying the autonomous flight path or autonomous flight trajectory, wherein the autonomous flight path or autonomous flight trajectory is modified while maintaining the autonomous flight; and One or more propulsion units, the one or more propulsion units being configured to: enable autonomous flight of the unmanned aerial vehicle in response to the first set of signals, and modify the autonomous flight path of the unmanned aerial vehicle in response to the second set of signals; The second set of signals is generated based on user input and maintains the modification of the autonomous flight path only for the duration of the user input. The autonomous flight of the UAV is modified by superimposing a directional component onto the autonomous flight of the UAV, or by superimposing a velocity component or an acceleration component onto the UAV.

Images