Remote controller and method for autonomous flight for modifying a predetermined trajectory of an unmanned aerial vehicle, and system including the remote controller and the unmanned aerial vehicle

Abstract

A remote controller for autonomous flight adapted to modify a predetermined trajectory of an unmanned aerial vehicle (UAV) is provided. The remote controller includes at least one movable control member for adjusting a setpoint of an adjustable control parameter of the UAV. In addition, the remote controller includes at least one actuator capable of controllably applying a torque to at least one control member. The remote controller further includes a processing circuit configured to determine a setpoint of the torque to be applied to at least one control member based on a reference setpoint of a control parameter. The reference setpoint of the control parameter is related to the predetermined trajectory. The processing circuit is further configured to control at least one actuator to apply the determined torque setpoint to at least one control member. The remote controller includes a sensing circuit configured to sense a position of at least one control member when the determined torque setpoint is applied to at least one control member by at least one actuator, so as to detect a user input for modifying the predetermined trajectory. In addition, the remote controller includes a wireless transmitter configured to transmit information about the sensed position of at least one control member to the UAV.

Description

Technical Field

[0001] The present disclosure relates to the active control of an unmanned aerial vehicle (UAV). In particular, examples relate to a remote controller and method for autonomous flight that modifies a predetermined trajectory of a UAV. Further examples relate to a system that includes a remote controller and a UAV. Background Art

[0002] In applications such as photography, video production, or entertainment shows, it is often desirable to be able to precisely control the trajectory of a robotic device, such as a drone equipped with a camera or light source. Typically, such a robotic device is controlled using an RC transmitter (RC control or wireless control), which allows a user to command changes in the acceleration, speed, position, angular acceleration, angular velocity, and / or orientation of the robotic device. For example, in the case of a quadcopter drone, an RC transmitter is typically used to control the combined thrust setpoints of all four propellers as well as the torque setpoints about the three axes of the device (i.e., roll, pitch, and yaw). This allows a professional human pilot to control the trajectory of the quadcopter by mentally calculating the setpoints required to achieve the desired motion of the quadcopter and commanding the corresponding values via the control sticks of the RC transmitter. However, professional drone pilots are not only expensive but also prone to fatigue, and they cannot precisely replicate a flight trajectory multiple times.

[0003] Another way to program a drone trajectory is to use a graphical user interface (GUI) on a computer or mobile device (e.g., a smartphone). This approach results in a fixed control program that the drone automatically follows, and the program can be precisely replicated multiple times as needed. However, it is difficult to fine-tune a trajectory generated in this way or to quickly change it in response to external influences.

[0004] Therefore, there is a need for improved control of UAVs. Summary of the Invention

[0005] The apparatus and method according to the independent claims meet this need. The dependent claims propose advantageous embodiments.

[0006] According to a first aspect, there is provided a remote controller for autonomous flight adapted to modify a predetermined trajectory of a UAV. The remote controller includes at least one movable control member for adjusting a setpoint of an adjustable control parameter of the UAV. In addition, the remote controller includes at least one actuator capable of controllably applying a torque to at least one control member. The remote controller further includes a processing circuit configured to determine a setpoint of the torque to be applied to at least one control member based on a reference setpoint of the control parameter. The reference setpoint of the control parameter is related to the predetermined trajectory. The processing circuit is further configured to control at least one actuator to apply the determined torque setpoint to at least one control member. The remote controller includes a sensing circuit configured to sense the position of at least one control member when the determined torque setpoint is applied to at least one control member by at least one actuator, so as to detect a user input for modifying the predetermined trajectory. In addition, the remote controller includes a wireless transmitter configured to transmit information about the sensed position of at least one control member to the UAV.

[0007] According to a second aspect, the present disclosure provides a system including: a UAV configured to autonomously fly a predetermined trajectory; and a remote controller as described herein. The UAV includes a wireless receiver configured to receive information about the sensed position of at least one control member. In addition, the UAV includes a processing circuit configured to determine an updated setpoint of the control parameter based on the information about the sensed position of at least one control member. The processing circuit of the UAV is further configured to use the updated setpoint of the control parameter to control the operation of the UAV so as to modify the autonomous flight of the predetermined trajectory of the UAV.

[0008] According to a third aspect, the present disclosure provides a method for modifying the autonomous flight of a predetermined trajectory of a UAV. The method includes determining a torque setpoint to be applied to at least one control member of a remote controller of the UAV based on a reference setpoint of an adjustable control parameter of the UAV. The reference setpoint of the control parameter is related to the predetermined trajectory, and wherein the setpoint of the control parameter can be adjusted via at least one movable control member. In addition, the method includes controlling at least one actuator of the remote controller to apply the determined torque setpoint to at least one control member. The method further includes sensing the position of at least one control member while the determined torque setpoint is applied to at least one control member by at least one actuator so as to detect a user input for modifying the predetermined trajectory. The method includes transmitting information about the sensed position of at least one control member to the UAV. Description of the Drawings

[0009] Some examples of the device and / or method will be described hereinafter only by way of example and with reference to the drawings, wherein

[0010] Figure 1An example of a system including a UAV and a remote controller is shown; and

[0011] Figure 2 A flowchart showing an example of a method for autonomous flight to modify a predetermined trajectory of a UAV is shown. Detailed Description

[0012] Various examples will now be described more fully with reference to the accompanying drawings, in which some examples are shown. In the drawings, for clarity, the thickness of lines, layers, and / or regions may be exaggerated.

[0013] Accordingly, although further examples can have various modifications and alternative forms, some specific examples are shown in the drawings and will subsequently be described in detail. However, this detailed description does not limit further examples to the specific forms described. Further examples may cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure. Throughout the description of the drawings, the same or similar numerals refer to the same or similar elements, which may be implemented in the same or modified forms while providing the same or similar functions when compared to each other.

[0014] It should be understood that when an element is referred to as "connected" or "coupled" to another element, these elements can be connected or coupled directly or via one or more intermediate elements. If two elements A and B are combined using "or", this should be understood to disclose all possible combinations, i.e., only A, only B, and A and B, if not otherwise defined explicitly or implicitly. Another wording for the same combination is "at least one of A and B" or "A and / or B". This also applies to combinations of more than two elements (with the necessary modifications).

[0015] The terms used herein to describe specific examples are not intended to limit further examples. Whenever singular forms such as "a", "an", and "the" are used and only a single element is neither explicitly nor implicitly defined as mandatory, further examples can also be implemented using multiple elements to achieve the same function. Similarly, when a function is subsequently described as being achieved using multiple elements, further examples can be implemented using a single element or processing entity to achieve the same function. It will be further understood that the terms "comprises", "comprising", "includes", and / or "including", when used, specify the presence of the stated features, integers, steps, operations, processes, actions, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, processes, actions, elements, components, and / or any combination thereof.

[0016] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the ordinary meaning ascribed to them in the field to which these examples belong.

[0017] Figure 1 A remote controller 100 and a UAV 190 are shown. In Figure 1 the UAV 190 is depicted as a quadcopter, i.e., a multi-rotor UAV including four rotors 194-1, …, 194-4. However, it should be noted that the UAV 190 is not limited thereto. Generally, the UAV 190 can be any type of UAV, e.g., a single-rotor aircraft, a dual-rotor aircraft, or a fixed-wing UAV (e.g., an airplane or a vertical takeoff and landing VTOL aircraft).

[0018] The UAV 190 is configured to autonomously fly a predetermined trajectory σ'(t). For example, data related to the predetermined trajectory σ' can be stored in a (e.g., non-volatile) memory 195 of the unmanned aerial vehicle 190. The processing circuit 191 of the UAV 190 can read the predetermined trajectory σ' from the memory 195 and control the rotors 194-1, …, 194-4 or individual control units for the rotors 194-1, …, 194-4 such that the rotors 194-1, …, 194-4 adjust their respective rotor speeds based on the predetermined trajectory σ'. Thus, the UAV 190 can autonomously fly the predetermined trajectory σ'. For example, the processing circuit 191 can be a single dedicated processor, a single shared processor, or multiple individual processors, some or all of which can be shared, digital signal processor (DSP) hardware, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The processing circuit 191 can optionally be coupled to, e.g., a read-only memory (ROM), a random access memory (RAM), and / or a non-volatile memory for storing software.

[0019] The remote controller 100 is adapted to modify the autonomous flight of the predetermined trajectory σ' of the UAV. In Figure 1 the example shown, the remote controller 100 includes two control members 110 and 120 (e.g., control sticks), each of which can be moved along two spatial axes (e.g., the horizontal axis and the vertical axis as Figure 1 shown). Each of the two control members 110 and 120 is used to adjust the set points of two different adjustable control parameters of the UAV 190. For example, the control member 110 allows adjustment of the set points of the adjustable control parameters μ 1 and μ 4 of the UAV 190, while the control member 120 allows adjustment of the set points of the adjustable control parameters μ 2 and μ 3 of the UAV 190. By changing the position of the control member 110 along the vertical axis, the adjustable control parameter μ of the UAV 1901 The setpoint of 1 can be adjusted by the user. Similarly, by changing the position of the control member 110 along the horizontal axis, the adjustable control parameter μ of the UAV 190 4 The setpoint of 4 can be adjusted by the user. Similarly, the adjustable control parameter μ of the UAV 190 2 and μ 3 can be adjusted by the user via the control member 120.

[0020] Although Figure 1 the remote controller 100 shown includes two control members 110 and 120, the remote controller according to the proposed technology may include any number of control members for adjusting the setpoint of the adjustable control parameter of the UAV 190. For example, the remote controller may include more or fewer Figure 1 than the two control members 110 and 120 shown. The number of control members may depend, for example, on the type of the UAV 190. In other words, the remote controller according to the proposed technology includes at least one movable control member for adjusting the setpoint of the adjustable control parameter of the UAV.

[0021] The four adjustable control parameters μ of the UAV 190 1 to μ 4 allow controlling the movement of the UAV 190. For example, the adjustable control parameter μ 1 may describe (represent) the total torque generated by all four rotors 194-1,..., 194-4 of the UAV 190. The adjustable control parameter μ 2 、μ 3 and μ 4 may describe the torques generated by the rotors 194-1,..., 194-4 of the UAV 190 about the pitch axis, roll axis and yaw axis of the UAV 190. The pitch axis, roll axis and yaw axis are represented by the coordinate system I in Figure 1 . It should be noted that the above examples of the four control parameters μ 1 to μ 4 of the UAV 190 are for illustrative purposes only, and more, fewer or other control parameters may be used according to the proposed technology (e.g., depending on the type of the UAV).

[0022] The remote controller 100 further includes at least one actuator that can controllably apply torque to at least one control member of the remote controller 100. For example, the remote controller 100 includes actuators 111 and 121 for the control members 110 and 120 such that torque can be applied to the control members 110 and 120 along each of the two spatial axes along which the control members 110 and 120 can move. The number of actuators may vary according to the number of control members and the number of spatial axes along which the control members can move.

[0023] The remote controller 100 further includes a processing circuit 130 configured to determine a torque setpoint to be applied to at least one control member based on a reference setpoint of an associated control parameter. The reference setpoint of the control parameter is related to a predetermined trajectory σ'. For example, the setpoint of the torque applied to the control member 110 along the vertical axis is based on the reference setpoint μ 1 ' of the associated control parameter μ 1 '. Similarly, the torque setpoint of the horizontal axis of the control member 110 and the torque setpoint of the spatial axis of the control member 120 are based on the associated control parameters μ 2 、μ 3 and μ 4 ' of the reference setpoints μ 2 '、μ 3 ' and μ 4 '. The reference setpoints μ 1 ' to μ 4 ' are related to the predetermined trajectory σ'. For example, the processing circuit 130 can be a single dedicated processor, a single shared processor, or some or all of them can be multiple separate processors that are shared, DSP hardware, ASIC, or FPGA. The processing circuit 130 can optionally be coupled to, for example, ROM, RAM, and / or non-volatile memory for storing software.

[0024] The processing circuit 130 is further configured to control at least one actuator to apply the determined torque setpoint to at least one control member of the remote controller 100. For example, the processing circuit 130 can control the actuator 111 of the control member 110 to apply to the control member 110 along the vertical axis the torque setpoint determined for the reference setpoint μ 1 ' of the associated control parameter μ 1 '. Thus, if the user does not apply a force to the control member 110, the control member 110 can change its position according to the applied torque. If the user applies a force to the control member 110, the user feels the applied torque.

[0025] By applying the determined torque setpoint to at least one actuator of the remote controller 100, the change over time of the corresponding reference setpoint of the predetermined trajectory σ' can be presented to the user operating the remote controller 100. Thus, the user can feel the change over time of the corresponding reference setpoint of the predetermined trajectory σ'. For example, an inexperienced user (i.e., a beginner) can quickly obtain an intuitive understanding of the dynamics of the UAV 190 and learn how to control the UAV 190.

[0026] In addition, the remote controller 100 includes at least one sensing circuit configured to sense the position of at least one control member when a determined torque setpoint is applied to the at least one control member by at least one actuator, in order to detect a user input for modifying a predetermined trajectory σ'. For example, the remote controller may include sensing circuits 112 and 122 for control members 110 and 120, such that when a determined torque setpoint (associated with reference setpoints μ 1 ' to μ 4 ') is applied to control members 110 and 120 by actuators 111 and 121, the positions of control members 110 and 120 are sensed. Thus, a user input for modifying the predetermined trajectory σ' can be determined. That is, the movement of torque-actuated control members 110 and 120 can be overridden by the user in order to affect the trajectory executed by UAV 190.

[0027] In some examples, the processing circuit 130 may be configured to determine the torque setpoint to be applied to at least one control member 110, 120 based on a reference setpoint of a control parameter and the additionally sensed position of at least one control member 110, 120. For example, the processing circuit 130 may determine the torque setpoint to be applied to control member 110 based on the reference setpoint μ 1 of the associated control parameter μ 1 ' and the additionally sensed position of control member 110.

[0028] Furthermore, the remote controller 100 includes a wireless transmitter 140 (e.g., a radio frequency transmitter) coupled to an antenna 150 of the remote controller 100. The wireless transmitter 140 is configured to transmit information about the sensed position of at least one control member to the UAV 190. For example, the wireless transmitter 140 may transmit information about the sensed positions of control members 110 and 120, such that the UAV 190 can update the setpoints of control parameters μ 1 to μ 4 in order to modify the predetermined trajectory σ'.

[0029] As Figure 1 shown, the UAV 190 includes a wireless receiver 192 configured to receive information about the sensed position of at least one control member of the remote controller 100. For example, the wireless receiver 192 may receive information about the sensed positions of control members 110 and 120.

[0030] The processing circuit 191 of the UAV 190 determines updated setpoints of the corresponding control parameters based on the information about the sensed position of at least one control member. For example, the processing circuit 191 may determine the control parameter μ 1updated setpoints. Similarly, the processing circuit 191 can determine other control parameters μ based on information about the sensed positions of the control members 110 and 120 2 to μ 4 updated setpoints.

[0031] The processing circuit 191 of the UAV 190 is also configured to use the updated setpoints of the control parameters to control the operation of the UAV 190 so as to modify the autonomous flight of the predetermined trajectory σ' of the UAV 190. For example, the processing circuit 191 can use the control parameters μ 1 to μ 4 updated setpoints to control the operation of the UAV 190. By using the control parameters μ 1 to μ 4 updated setpoints, the autonomous flight of the predetermined trajectory σ' can be modified. Thus, the UAV 190 flies along the trajectory modified by the user rather than the predetermined trajectory σ'.

[0032] Therefore, the remote controller 100 can allow the predetermined (pre-programmed) trajectory to be affected at any time by actively moving at least one control member of the remote controller 100. When one or more control members of the remote controller 100 are released, the reference setpoints μ 1 ' to μ 4 ' are not updated, so that the UAV 190 returns to the initially programmed trajectory σ', which allows for precise fine-tuning in multiple repetitions. For example, the predetermined trajectory σ' can initially be predefined using a GUI on a computer or a mobile device, or (e.g., by default) includes a single fixed position always above the takeoff point. Thus, the proposed technique can bridge the gap between computer-designed trajectories that are difficult to modify and fully manual control that is difficult to replicate and not precise enough.

[0033] The remote controller 100 also includes a button 101 (start button) for selectively controlling the UAV 190 to start the autonomous flight of the predetermined trajectory σ'. The wireless transmitter 140 is configured to transmit a command to start the autonomous flight of the predetermined trajectory σ' to the UAV 190 if the user presses the button 101. Once the command is received, the UAV 190 starts the autonomous flight of the predetermined trajectory σ'.

[0034] In addition, the remote controller 100 includes another button 102 (stop button) for selectively controlling the UAV 190 to stop the autonomous flight of the predetermined trajectory σ'. The wireless transmitter 140 is configured to transmit a command to stop the autonomous flight of the predetermined trajectory σ' to the UAV 190 if the user presses the button 102. After receiving the command, the UAV 190 stops the autonomous flight of the predetermined trajectory σ'.

[0035] The remote controller 100 further includes another button 103 (reset button) for selectively updating a predetermined trajectory σ'. The wireless transmitter 140 is configured to transmit, if the user presses the button 103, a command to update the stored predetermined trajectory σ' to a trajectory modified by the user to the UAV 190. In the UAV 190, the predetermined trajectory σ' is stored in the memory 195. If the wireless receiver 192 receives from the remote controller 100 a command to update the stored predetermined trajectory σ', the processing circuit 191 of the UAV 190 is configured to update the stored predetermined trajectory σ' to a trajectory modified by the user, i.e., a trajectory derived from the predetermined trajectory σ' based on the user input sensed at the remote controller 100. That is, according to the user's selection, the changed (updated) trajectory can be stored as a new reference trajectory, which can allow for an incremental refinement of the predetermined trajectory σ' in an intuitive manner.

[0036] It can be assumed that the UAV 190 has different flat dynamics. Based on four adjustable control parameters μ 1 to μ 4 as inputs for the motion control of the UAV 190, four output parameters x, y, z, and Ψ describing the current attitude of the UAV 190 are effectively obtained. The output parameters x, y, and z describe (represent) the centroid position of the UAV 190, and the output parameter Ψ describes (represents) the yaw angle of the UAV 190.

[0037] At any point in time, the UAV 190 has a state X defined by the centroid position of the UAV 190, the centroid velocity of the UAV 190, the orientation of the UAV, and the angular velocity of the UAV (about the pitch, roll, and yaw axes).

[0038] If the state X of the UAV 190 is known, the adjustable control parameters μ 1 to μ 4 can be derived from the predetermined trajectory σ'. The state X of the UAV 190 can be observed by sensors (e.g., an inertial measurement unit (IMU)) and a position sensor of a global navigation satellite system (GNSS) (e.g., a global positioning system (GPS)). Due to the different flatness of the dynamic system, the adjustable control parameters μ 1 to μ 4 can be derived from the predetermined trajectory σ' using the state X of the UAV 190. Generally, the adjustable control parameters μ 1 to μ 4 can be derived at the UAV 190 or the remote controller 100.

[0039] For example, the processing circuit 191 of the UAV 190 may be configured to derive a reference setpoint for the control parameters of the UAV 190 using information about at least one of the centroid position of the UAV 190, the centroid velocity of the UAV 190, the orientation of the UAV, and the angular velocity of the UAV (about the pitch axis, roll axis, and yaw axis). Subsequently, the wireless transmitter 193 of the UAV 190 is configured to transmit information about the reference setpoint of the control parameters to the remote controller 100. In some examples, the processing circuit 191 of the UAV 190 derives the control parameters μ 1 to μ 4 in this way for the reference setpoints μ 1 ' to μ 4 ', and the wireless transmitter 193 transmits the reference setpoints μ 1 ' to μ 4 ' to the remote controller 100.

[0040] The wireless receiver 160 of the remote controller 100 is coupled to the antenna 150 and is configured to receive information about the (corresponding) reference setpoints of the control parameters from the UAV 190. Thus, this information can be provided to the processing circuit 130 of the remote controller 100 for controlling the actuators 111 and 121.

[0041] To control the UAV 190, the UAV 190 and the remote controller 100 may thus be equipped with means for two-way communication.

[0042] During the autonomous flight of the UAV 190 along a predetermined trajectory σ', the (corresponding) reference setpoints of the control parameters of the UAV 190 are sent to the remote controller 100 and are used as setpoints for torque control actuators that control one or more (e.g., four) degrees of freedom of the remote controller 100.

[0043] To provide a good user experience, the torque applied to one or more control members of the remote controller may be scaled appropriately. For example, the processing circuit 130 of the remote controller 100 may be configured to determine the torque setpoint to be applied to at least one control member by selecting a torque value from a predetermined range of torque values as the torque setpoint to be applied to at least one control member based on the reference setpoint of the control parameter associated with the control member. For example, the processing circuit 130 may be based on the reference setpoint μ 1 of the control parameter μ 1'(and optionally the sensing position of the control member 110), select a torque value from a predetermined range of torque values as the set point of the torque applied along the vertical axis to the control member 110. Thus, torque within reasonable limits can be applied to one or more control members of the remote controller 100. Similarly, the torque set point for the horizontal axis of the control member 110 and the torque set point for the spatial axis of the control member 120 can be determined (the range of torque values can be the same or different for different control members and / or different spatial axes).

[0044] The actual positions of one or more control members of the remote controller 100 are continuously sensed and sent back to the UAV 190, so that the control system of the UAV 190 (e.g., the processing circuit 191) can accordingly set its control output (e.g., the set point of the UAV motor angular velocity) for the rotors 194-1,..., 194-4. The communication delay between the UAV 190 and the remote controller 100 can be compensated by prediction at the UAV 190 and / or the remote controller 100.

[0045] The reference set point of the control parameter for the remote controller 100 to control the applied torque can be, for example, the predicted set point of the control parameter at a future time point. For example, the control parameter μ 1 for controlling the torque applied to the control member 110 along the vertical axis 1 ' can be the predicted set point of the control parameter at a future time point. The predicted set point of the control parameter can be determined by the processing circuit 130 of the remote controller 100. For example, the processing circuit 130 can be configured to predict the future dynamics of the UAV 190 based on a model of the UAV 190 and information received from the UAV 190 regarding the reference set point of the control parameter. In addition, the processing circuit 130 can be configured to determine the predicted set point of the control parameter at a future time point based on the predicted future dynamics of the UAV 190 and the received information regarding the reference set point of the control parameter. For example, the processing circuit 130 can predict the future dynamics of the UAV based on a model of the UAV 190 and information received from the UAV 190 regarding the reference set points μ 1 ' to μ 4 '. Based on the predicted future dynamics of the UAV 190 and the received information regarding the reference set points μ 1 ' to μ 4 ', the processing circuit 130 can determine the corresponding predicted set points of the control parameters μ 1 to μ 4 at a future time point.

[0046] At the UAV 190, a (corresponding) updated setpoint that can be used to determine a control parameter is predicted. For example, to determine an updated setpoint for a control parameter, the processing circuitry 191 of the UAV 190 can predict a future position of at least one control member (associated with the control parameter) based on information about the sensed position of at least one control member and a model of the user (e.g., first-order dynamics). Additionally, the processing circuitry 191 can determine an updated setpoint for the control parameter based on the predicted future position of at least one control member. For example, to determine an updated setpoint for the control parameter μ 1 , the processing circuitry 191 can predict the future position of the control member 110 along the vertical axis based on information about the sensed position of the control member 110 along the vertical axis and a model of the user. Subsequently, the processing circuitry 191 can determine an updated setpoint for the control parameter μ 1 .

[0047] To summarize the above aspects regarding controlling the UAV, Figure 2 a flowchart of a method 200 for autonomous flight to modify a predetermined trajectory of a UAV is further shown. The method 200 includes determining 200 a torque setpoint for at least one control member of a remote controller of the UAV to be applied based on a reference setpoint of an adjustable control parameter of the UAV. The reference setpoint of the control parameter is associated with the predetermined trajectory, and wherein the setpoint of the control parameter can be adjusted via at least one movable control member. Additionally, the method 200 includes controlling 204 at least one actuator of the remote controller to apply the determined torque setpoint to at least one control member. The method 200 further includes sensing 206 the position of at least one control member while the determined torque setpoint is applied to at least one control member by at least one actuator to detect a user input for modifying the predetermined trajectory. The method 200 includes transmitting 208 information about the sensed position of at least one control member to the UAV.

[0048] Similar to that described above with respect to Figure 1 the system shown, the method 200 can allow for improved control of the UAV.

[0049] Furthermore, the proposed technique can be combined with virtual reality (VR) or augmented reality (AR) techniques to further improve the driving experience. The method 200 can further include presenting an AR or VR view to the user, wherein the AR or VR view includes a visualization of information related to at least one of the predetermined trajectory and the trajectory modified by the user. For example, the predetermined trajectory and / or the trajectory modified by the user can be overlaid on a (two-dimensional or three-dimensional) graphical representation of the UAV environment. Additionally, additional information from the UAV can be visualized (e.g., status information).

[0050] More details and aspects of the method are explained in connection with the proposed technology or one or more of the above - mentioned exemplary embodiments. The method may include one or more additional optional features corresponding to one or more aspects of the proposed technology or the above - mentioned one or more examples.

[0051] The proposed technology relates to a control interface with control members that are torque - controlled and provide feedback of the current setpoint (which can be derived from the state of the UAV and a pre - programmed reference trajectory) to the user. By touching and actively moving the control members, the user can override the reference trajectory at any time, which provides an intuitive way to influence and fine - tune the UAV trajectory. The proposed technology even allows non - experts to teach, replay, and modify UAV trajectories, obtaining (very) precise results.

[0052] The following examples relate to further embodiments:

[0053] (1) A remote controller for autonomously flying a UAV adapted to modify a predetermined trajectory. The remote controller includes at least one movable control member for adjusting a setpoint of an adjustable control parameter of the UAV. In addition, the remote controller includes at least one actuator capable of controllably applying torque to at least one control member. The remote controller further includes a processing circuit configured to determine a setpoint of the torque to be applied to at least one control member based on a reference setpoint of the control parameter. The reference setpoint of the control parameter is related to the predetermined trajectory. The processing circuit is also configured to control at least one actuator to apply the determined torque setpoint to at least one control member. The remote controller includes a sensing circuit configured to sense the position of at least one control member when the determined torque setpoint is applied to at least one control member by at least one actuator, in order to detect a user input for modifying the predetermined trajectory. In addition, the remote controller includes a wireless transmitter configured to transmit information about the sensed position of at least one control member to the UAV.

[0054] (2) The remote controller according to (1), further comprising a wireless receiver configured to receive information about the reference setpoint of the control parameter from the UAV.

[0055] (3) The remote controller according to (1) or (2), wherein the processing circuit is configured to determine the torque setpoint to be applied to at least one control member based on the reference setpoint of the control parameter and the sensed position of at least one control member.

[0056] (4) The remote controller according to any one of (1) to (3), wherein the processing circuit is configured to determine the torque setpoint to be applied to at least one control member by selecting a torque value from a predetermined range of torque values based on the reference setpoint of the control parameter as the torque setpoint to be applied to at least one control member.

[0057] (5) The remote controller according to any one of (1) to (4), wherein the remote controller includes two control members, each control member being capable of moving along two spatial axes, and wherein each of the two control members is for adjusting the setpoints of two different adjustable control parameters of the UAV.

[0058] (6) The remote controller according to any one of (1) to (5), wherein, using information on at least one of the centroid position of the UAV, the centroid velocity of the UAV, the orientation of the UAV, and the angular velocity of the UAV, a reference setpoint of the control parameter is derived from a predetermined trajectory.

[0059] (7) The remote controller according to any one of (1) to (6), wherein the reference setpoint of the control parameter is a predicted setpoint of the control parameter for a future time point.

[0060] (8) The remote controller according to (7), wherein the processing circuit is further configured to: predict the future dynamics of the UAV based on the model of the UAV and the information on the reference setpoint of the control parameter received from the UAV; and determine the predicted setpoint of the control parameter for the future time point based on the predicted future dynamics of the UAV and the received information on the reference setpoint of the control parameter.

[0061] (9) The remote controller according to any one of (1) to (8) further includes a button for selectively controlling the UAV to start autonomous flight of a predetermined trajectory, wherein the wireless transmitter is configured to transmit a command to start autonomous flight of the predetermined trajectory to the UAV if the user presses the button.

[0062] (10) The remote controller according to any one of (1) to (9) further includes a button for selectively controlling the UAV to stop autonomous flight of a predetermined trajectory, wherein the wireless transmitter is configured to transmit a command to stop autonomous flight of the predetermined trajectory to the UAV if the user presses the button.

[0063] (11) The remote controller according to any one of (1) to (10) further includes a button for selectively updating the predetermined trajectory, wherein the wireless transmitter is configured to transmit a command to update the stored predetermined trajectory to the trajectory modified by the user to the UAV if the user presses the button.

[0064] (12) The remote controller according to any one of (1) to (11), wherein the UAV is a multi-rotor unmanned aerial vehicle.

[0065] (13) A system, comprising: a UAV configured to autonomously fly a predetermined trajectory; and a remote controller according to any one of (1) to (12). The UAV includes a wireless receiver configured to receive information about the sensed position of at least one control member. In addition, the UAV includes a processing circuit configured to determine an updated setpoint of a control parameter based on the information about the sensed position of at least one control member. The processing circuit of the UAV is further configured to use the updated setpoint of the control parameter to control the operation of the UAV so as to modify the autonomous flight of the predetermined trajectory of the UAV.

[0066] (14) The system according to (13), wherein the processing circuit of the UAV is further configured to derive a reference setpoint of a control parameter from the predetermined trajectory, and wherein the UAV includes a wireless transmitter configured to transmit information about the reference setpoint of the control parameter to the remote controller.

[0067] (15) The system according to (13) or (14), wherein the processing circuit of the UAV is further configured to derive a reference setpoint of a control parameter using information about at least one of the centroid position of the UAV, the centroid velocity of the UAV, the orientation of the UAV, and the angular velocity of the UAV.

[0068] (16) The system according to any one of (13) to (15), wherein the processing circuit of the UAV is further configured to determine an updated setpoint of a control parameter by: predicting the future position of at least one control member based on the information about the sensed position of at least one control member and a model of the user; and determining the updated setpoint of the control parameter based on the predicted future position of at least one control member.

[0069] (17) The method according to any one of (13) to (16), wherein the predetermined trajectory is stored in the memory of the UAV, and wherein the processing circuit of the UAV is further configured to update the stored predetermined trajectory to a user-modified trajectory if a command to update the stored predetermined trajectory is received from the remote controller.

[0070] (18) A method for autonomous flight to modify a predetermined trajectory of a UAV. The method includes determining a torque setpoint to be applied to at least one control member of a remote controller of the UAV based on a reference setpoint of an adjustable control parameter of the UAV. The reference setpoint of the control parameter is related to the predetermined trajectory, and wherein the setpoint of the control parameter can be adjusted via at least one movable control member. Further, the method includes controlling at least one actuator of the remote controller to apply the determined torque setpoint to the at least one control member. The method further includes sensing the position of the at least one control member while the determined torque setpoint is applied to the at least one control member by the at least one actuator to detect a user input for modifying the predetermined trajectory. The method includes transmitting information about the sensed position of the at least one control member to the UAV.

[0071] (19) The method according to (18), further comprising: determining, by the UAV, an updated setpoint of the control parameter based on the information about the sensed position of the at least one control member; and controlling, by the UAV, the operation of the UAV using the updated setpoint of the control parameter to modify the autonomous flight of the predetermined trajectory of the UAV.

[0072] (20) The method according to (18) or (19), further comprising presenting an augmented view or a virtual reality view to the user, wherein the augmented view or the virtual reality view includes a visualization of information related to at least one of the predetermined trajectory and the trajectory modified by the user.

[0073] (21) The method according to any one of (18) to (20), wherein the UAV is a multi-rotor unmanned aerial vehicle.

[0074] Aspects and features mentioned and described in connection with one or more of the previously detailed examples and the appended Figure 1 drawings may also be combined with one or more other examples in order to replace similar features of other examples or to introduce that feature additionally into other examples.

[0075] The description and the drawings merely illustrate the principles of the present disclosure. Further, all examples enumerated herein are mainly explicitly for illustrative purposes only to help the reader understand the principles of the present disclosure and the concepts contributed by the inventors to advance the art. All statements of the principles, aspects and examples of the present disclosure and their specific examples enumerated herein are intended to include their equivalents.

[0076] For example, a block diagram may illustrate a high-level circuit diagram implementing the principles of the present disclosure. Similarly, flowcharts, process diagrams, state transition diagrams, pseudocode, etc. may represent various processes, operations, or steps, which may be substantially represented, for example, on a non-transitory machine-readable medium (e.g., a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, and EEPROM, or flash memory), and executed by a processor or programmable hardware, whether or not such a processor or programmable hardware is explicitly shown. The methods disclosed in the specification or claims may be implemented by an apparatus having means for performing each corresponding action of these methods.

[0077] It should be understood that the disclosure of a plurality of actions, processes, operations, steps, or functions in the specification or claims should not be construed as in a particular order unless explicitly or implicitly stated otherwise, e.g., for technical reasons. Thus, the disclosure of a plurality of actions or functions does not limit these to a particular order unless these actions or functions are not interchangeable for technical reasons. Additionally, in some examples, a single action, function, process, operation, or step may separately include or may be divided into multiple sub-actions, functions, processes, operations, or steps. Such sub-actions may be included and are part of the disclosure of that single action unless explicitly excluded.

[0078] Furthermore, the following claims are hereby incorporated into the detailed description, where each claim may stand alone as a separate example. While each claim may stand alone as a separate example, it should be noted that although a dependent claim may refer to a particular combination with one or more other claims in the claims, other examples may also include combinations of the dependent claim with the subject matter of each other dependent or independent claim. Such combinations are explicitly set forth herein unless a particular combination is not intended. Additionally, it is also intended to include the features of any other independent claim in a claim, even if that claim does not directly depend on the independent claim.

Claims

1. A remote controller (100) for autonomous flight adapted to modify a predetermined trajectory of an unmanned aerial vehicle (190), the remote controller comprises: at least one movable control member (110, 120) for adjusting a setpoint of an adjustable control parameter of the unmanned aerial vehicle (190); at least one actuator (111, 121) capable of controllably applying a torque to at least one control member (110, 120); a processing circuit (130) configured to: determine a torque setpoint to be applied to at least one control member (110, 120) based on a reference setpoint of a control parameter, wherein the reference setpoint of the control parameter is related to the predetermined trajectory and the reference setpoint of the control parameter is a predicted setpoint of the control parameter at a future time point; predict future dynamics of the unmanned aerial vehicle (190) based on a model of the unmanned aerial vehicle (190) and information received from the unmanned aerial vehicle (190) regarding the reference setpoint of the control parameter; determine a predicted setpoint of the control parameter at a future time point based on the predicted future dynamics of the unmanned aerial vehicle (190) and the received information regarding the reference setpoint of the control parameter; and control at least one actuator (111, 121) to apply the determined torque setpoint to at least one control member (110, 120); a sensing circuit (112, 122) configured to sense a position of at least one control member (110, 120) when a determined torque setpoint is applied to at least one control member (110, 120) by at least one actuator (111, 121) in order to detect a user input for modifying the predetermined trajectory; and a wireless transmitter (140) configured to transmit information regarding the sensed position of at least one control member (110, 120) to the unmanned aerial vehicle (190).

2. The remote controller according to claim 1, further comprises: a wireless receiver (160) configured to receive information regarding the reference setpoint of the control parameter from the unmanned aerial vehicle (190).

3. The remote controller according to claim 1, wherein, the processing circuit (130) is configured to determine a torque setpoint to be applied to at least one control member (110, 120) based on the reference setpoint of the control parameter and the sensed position of at least one control member (110, 120).

4. The remote controller according to claim 1, wherein, the processing circuit (130) is configured to determine a torque setpoint to be applied to at least one control member (110, 120) by selecting a torque value from a predetermined torque value range based on the reference setpoint of the control parameter as the torque setpoint to be applied to at least one control member (110, 120).

5. The remote controller according to claim 1, wherein, The remote controller includes two control members (110, 120), each of which is capable of moving along two spatial axes, wherein each of the two control members (110, 120) is used to adjust the setpoints of two different adjustable control parameters of the unmanned aerial vehicle (190).

6. The remote controller according to claim 1, wherein, using information about at least one of the centroid position of the unmanned aerial vehicle, the centroid velocity of the unmanned aerial vehicle, the orientation of the unmanned aerial vehicle (190), and the angular velocity of the unmanned aerial vehicle (190), a reference setpoint of the control parameter is derived from a predetermined trajectory.

7. The remote controller according to claim 1, further comprising a button (101) for selectively controlling the unmanned aerial vehicle (190) to start autonomous flight of a predetermined trajectory, wherein, the wireless transmitter (140) is configured to transmit a command to start autonomous flight of a predetermined trajectory to the unmanned aerial vehicle (190) if the user presses the button (101).

8. The remote controller according to claim 1, further comprising a button (102) for selectively controlling the unmanned aerial vehicle (190) to stop autonomous flight of a predetermined trajectory, wherein, the wireless transmitter (140) is configured to transmit a command to stop autonomous flight of a predetermined trajectory to the unmanned aerial vehicle (190) if the user presses the button (102).

9. The remote controller according to claim 1, further comprising a button (103) for selectively updating a predetermined trajectory, wherein, the wireless transmitter (140) is configured to transmit a command to update the stored predetermined trajectory to the trajectory modified by the user to the unmanned aerial vehicle (190) if the user presses the button (103).

10. The remote controller according to claim 1, wherein, the unmanned aerial vehicle (190) is a multi-rotor unmanned aerial vehicle.

11. A system for an unmanned aerial vehicle, comprising: an unmanned aerial vehicle (190) configured to autonomously fly a predetermined trajectory; and the remote controller (100) according to claim 1, wherein the unmanned aerial vehicle (190) includes a wireless receiver (192) configured to receive information about the sensed position of at least one control member (110, 120), and wherein the unmanned aerial vehicle (190) includes a processing circuit (191) configured to: determine an updated setpoint of a control parameter based on information about the sensed position of at least one control member (110, 120); and use the updated setpoint of the control parameter to control the operation of the unmanned aerial vehicle (190) so as to modify the autonomous flight of the predetermined trajectory of the unmanned aerial vehicle (190).

12. The system according to claim 11, wherein, The processing circuit (191) of the unmanned aerial vehicle (190) is further configured to derive a reference setpoint of the control parameter from a predetermined trajectory, and wherein the unmanned aerial vehicle (190) includes a wireless transmitter (193) configured to transmit information about the reference setpoint of the control parameter to the remote controller (100).

13. The system according to claim 11, wherein, the processing circuit (191) of the unmanned aerial vehicle (190) is further configured to derive the reference setpoint of the control parameter using information about at least one of the centroid position of the unmanned aerial vehicle, the centroid velocity of the unmanned aerial vehicle, the orientation of the unmanned aerial vehicle (190), and the angular velocity of the unmanned aerial vehicle (190).

14. The system according to any one of claims 11, wherein, the processing circuit (191) of the unmanned aerial vehicle (190) is further configured to determine an updated setpoint of the control parameter by: predicting the future position of at least one control member (110, 120) based on information about the sensed position of at least one control member (110, 120) and a model of the user; and determining the updated setpoint of the control parameter based on the predicted future position of at least one control member (110, 120).

15. The system according to claim 11, wherein, the predetermined trajectory is stored in a memory (195) of the unmanned aerial vehicle (190), and wherein the processing circuit (191) of the unmanned aerial vehicle (190) is further configured to update the stored predetermined trajectory to a user-modified trajectory if a command to update the stored predetermined trajectory is received from the remote controller.

16. A method (200) for modifying the autonomous flight of a predetermined trajectory of an unmanned aerial vehicle, the method comprising: determining (202) a torque setpoint to be applied to at least one control member of a remote controller of the unmanned aerial vehicle based on a reference setpoint of an adjustable control parameter of the unmanned aerial vehicle, wherein the reference setpoint of the control parameter is related to a predetermined trajectory, and the reference setpoint of the control parameter is a predicted setpoint of the control parameter at a future time point, and wherein the setpoint of the control parameter can be adjusted via at least one movable control member; predicting the future dynamics of the unmanned aerial vehicle (190) based on a model of the unmanned aerial vehicle (190) and information about the reference setpoint of the control parameter received from the unmanned aerial vehicle (190); determining a predicted setpoint of the control parameter at a future time point based on the predicted future dynamics of the unmanned aerial vehicle (190) and the received information about the reference setpoint of the control parameter; controlling (204) at least one actuator of the remote controller to apply the determined torque setpoint to at least one control member; Sense the position of at least one control member while a determined torque setpoint is applied by at least one actuator to at least one control member to detect a user input for modifying a predetermined trajectory; and Transmit (208) information about the sensed position of at least one control member to the unmanned aerial vehicle.

17. The method according to claim 16, further comprising: Determining, by the unmanned aerial vehicle, an updated setpoint of the control parameter based on the information about the sensed position of at least one control member; and Controlling, by the unmanned aerial vehicle, the operation of the unmanned aerial vehicle using the updated setpoint of the control parameter to modify the autonomous flight of the predetermined trajectory of the unmanned aerial vehicle.

18. The method according to claim 16, further comprising presenting an augmented view or a virtual reality view to the user, wherein The augmented view or the virtual reality view includes a visualization of information related to at least one of the predetermined trajectory and the trajectory modified by the user.

Images