Unmanned aerial vehicle tactile teleoperation control method, device, equipment and medium

Abstract

The invention relates to an unmanned aerial vehicle tactile teleoperation control method, device and equipment and a medium, and the method comprises the steps that a position increment type motion controller obtains the current position information of a target unmanned aerial vehicle, the displacement increment of a control lever and the maximum radius of the working space of the control lever; determining a displacement increment corresponding to the target unmanned aerial vehicle according to the displacement increment of the operating lever and the ratio of the maximum radius of the operating lever working space to the displacement increment of the operating lever; the position increment type motion controller calculates and determines the obstacle avoidance pose of the target unmanned aerial vehicle according to the initial position of the control lever and the displacement increment of the control lever, and determines the target position and the yaw angle of the target unmanned aerial vehicle according to the current position information; and the differential flat controller generates a flight control instruction corresponding to the target unmanned aerial vehicle according to the obstacle avoidance pose, the target position and the yaw angle to drive the target unmanned aerial vehicle to perform automatic obstacle avoidance so as to complete control of tactile teleoperation of the unmanned aerial vehicle. The operation safety of the unmanned aerial vehicle in a complex environment is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of drone control, and in particular to a drone tactile remote control method, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] UAV tactile teleoperation is widely used in complex environments, especially in mission scenarios that are difficult to achieve with full automation, such as collapsed building risk inspection after natural disasters, dangerous area detection, and industrial equipment maintenance. Generally, teleoperation systems can be divided into two parts: motion control and force feedback. The motion control part includes methods for mapping the position information or position change rate information of the tactile handle to the position command and velocity command of the UAV, as well as direct control methods that directly map to the torque and Euler angle of the UAV.

[0003] In terms of tactile force feedback, traditional force feedback methods are mainly based on some physical models, including force-stiffness feedback, artificial potential field force feedback, etc. These methods provide more intuitive feedback effects through physical models, but their disadvantage is that they are less adaptable to complex environments, especially in changeable or dynamic environments, and their feedback effects are not ideal.

[0004] In summary, the tactile force feedback in the prior art has poor adaptability to complex environments, especially in changeable or dynamic environments, and its feedback effect is not ideal. The applicant has made corresponding explorations to solve this problem. Summary of the invention

[0005] The purpose of the present application is to solve the above-mentioned problems and to provide a method for tactile remote operation control of a drone, a corresponding device, an electronic device and a computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A method for controlling a drone by tactile teleoperation is proposed to meet one of the purposes of the present application, comprising:

[0008] When a target UAV is detected to enter a non-safe area and responds to a UAV tactile teleoperation control instruction, a model predictive controller in the tactile control device uses a preset second-order derivative to determine a feedback force that drives a joystick in the tactile control device to change its displacement according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV, and determines a joystick displacement increment of the joystick according to the feedback force;

[0009] The position increment type motion controller obtains the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space, and determines the displacement increment corresponding to the target UAV according to the joystick displacement increment and the ratio between the maximum radius of the joystick working space and the joystick displacement increment;

[0010] The position increment motion controller determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the displacement increment of the joystick, and determines the target position and yaw angle of the target UAV according to the current position information;

[0011] The differential flat controller generates a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile teleoperation.

[0012] Optionally, the model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force driving the joystick in the tactile control device to change displacement, comprising:

[0013] First, construct the global coordinate system of the system as F W :{x W ,y W ,z W}, the body reference system is F B :{x B ,y B ,z B}, where B represents the variable in the body coordinate system, and the other variables are expressed in the world (inertial) coordinate system. w To the body coordinate system F B The rotation matrix is ​​composed of the quaternion q = [q w ,q x ,q y ,q z ]∈SO(3) indicates;

[0014] The expression of the position kinematic model is:

[0015]

[0016] Among them, ξ w is the displacement of the UAV in the world coordinate system, v w is the speed of the drone in the world coordinate system;

[0017] The expression of the posture kinematic model is:

[0018] w=WΩ B ,

[0019] Among them, w represents the three attitude angles of the target UAV, Ω B =[p,q,r] T , W represents the angular velocity in the world coordinate system.

[0020] Optionally, the step of using a preset second-order derivative of a model predictive controller in the tactile control device to determine a feedback force for driving a joystick in the tactile control device to change its displacement according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV includes:

[0021] The expression of the function model corresponding to the model predictive controller includes:

[0022]

[0023] X k =x k -x k，r ,

[0024] U k =u k -u k，r ,

[0025] X N =x n -x n，r ,

[0026] stu∈[u min ,u max ],

[0027] x0=x init

[0028] x k+1 =f(x k ,u k ),

[0029] in, Expressed as state space, k is the time step, T is the transpose, and u is defined as v w and Ω B Set, x k+1 =f(x k ,u k ) is the kinematic model of the drone, Q u is the weight matrix of input u, Q and Q n It is a diagonal matrix with the weight matrix as its diagonal elements, and the subscript r represents the expected value.

[0030] Optionally, the step of using a preset second-order derivative of a model predictive controller in the tactile control device to determine a feedback force for driving a joystick in the tactile control device to change its displacement according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV includes:

[0031] The expression of the barrier function is:

[0032] d(x)-p sec ≥0,

[0033] Where d(x) represents the physical area of ​​the safe space, p sec Indicates the safety zone threshold;

[0034] The expression of the second-order derivative is:

[0035]

[0036] Among them, K f1 , K f2 and K f3 is a constant parameter used to adjust the amplitude of tactile feedback.

[0037] Optionally, determining the displacement increment corresponding to the target UAV according to the joystick displacement increment and the ratio between the maximum radius of the joystick working space and the joystick displacement increment includes:

[0038] The expression of the displacement increment corresponding to the target UAV is:

[0039]

[0040] Where a represents the input signal of the tactile device button, K m is a scaling matrix with positive diagonal, which is used to control the displacement increment of the drone in different directions; q j (t) represents the joystick displacement increment, which represents the distance between the end of the joystick and the origin of the joystick workspace, Δq s (t) represents the displacement increment of the drone, r represents the maximum radius of the joystick working space, when the joystick displacement increment q j When (t) is less than the maximum radius r of the joystick working space, the ratio between the maximum radius of the joystick working space and the joystick displacement increment is greater than 1, then the corresponding displacement increment of the UAV is the product of the scaling matrix with a positive diagonal and the joystick displacement increment.

[0041] Optionally, the step of calculating and determining the obstacle avoidance posture of the target UAV by the position increment motion controller according to the initial position of the joystick and the joystick displacement increment includes:

[0042] The expression of the obstacle avoidance posture of the target UAV is:

[0043]

[0044] Among them, ξ n represents the initial position of the joystick, N represents the number of control times of the joystick, and ξ(t) represents the obstacle avoidance posture of the target UAV.

[0045] Optionally, the step of using a preset second-order derivative of a model predictive controller in the tactile control device to determine a feedback force for driving a joystick in the tactile control device to change its displacement according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV includes:

[0046] In response to the safety obstacle avoidance warning instruction, the feedback force is output to a force feedback controller in the driving tactile control device so that the operator can sense the feedback force, so as to warn the operator to drive the target drone away from the unsafe area.

[0047] A tactile remote control device for an unmanned aerial vehicle provided for another purpose of the present application comprises:

[0048] A feedback force determination module is configured to detect that a target UAV enters a non-safe area and respond to a UAV tactile teleoperation control instruction, wherein a model predictive controller in a tactile control device uses a preset second-order derivative according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV to determine a feedback force for driving a joystick in the tactile control device to change its displacement, and determines a joystick displacement increment of the joystick according to the feedback force;

[0049] a displacement increment determination module, configured to obtain the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space by using a position increment type motion controller, and determine the displacement increment corresponding to the target UAV according to the ratio between the joystick displacement increment and the maximum radius of the joystick working space and the joystick displacement increment;

[0050] The obstacle avoidance posture determination module is configured as a position increment type motion controller to determine the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the joystick displacement increment calculation, and determine the target position and yaw angle of the target UAV according to the current position information;

[0051] The remote operation control module is configured as a differential flat controller to generate a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle, so as to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile remote operation.

[0052] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the drone tactile teleoperation control method described in the present application.

[0053] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the drone tactile remote operation control method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0054] Compared with the prior art, the present application aims to solve the problems that the tactile force feedback in the prior art is poorly adaptable to complex environments, especially in changeable or dynamic environments, and its feedback effect is not ideal. The present application includes but is not limited to the following beneficial effects:

[0055] First, the UAV tactile remote control method of the present application improves the system's control performance in complex environments and is suitable for tasks that require high-precision and delicate operations, such as precise positioning, inspection, and transportation. It significantly improves the operational safety of UAVs in complex environments, reduces the risks caused by control errors or environmental uncertainties, and provides operators with higher safety guarantees, especially in dangerous areas or highly uncertain environments.

[0056] Secondly, the UAV tactile remote control method of the present application reduces the operator's cognitive burden on control through position incremental motion control and smooth force feedback, allowing the operator to focus more on the task itself without having to pay too much attention to complex control details. At the same time, the system optimizes intelligent obstacle avoidance and dynamic feedback, allowing the operator to easily complete operations when faced with difficult tasks.

[0057] Third, the tactile remote control method of the drone in this application can not only improve the control accuracy, but also improve the reliability of the drone in performing tasks in complex mission environments through precise control and force feedback optimization. For example, in tasks such as medical transportation and dangerous area reconnaissance, precise control and reliable feedback can ensure the smooth completion of the task and minimize the risk of accidents.

[0058] Furthermore, the UAV tactile remote operation control method based on position increment and model predictive control optimization of the present application effectively improves the controllability, perception ability, safety and operation accuracy of the system through precise control, smooth force feedback, intelligent obstacle avoidance and other innovations. It not only enhances the operational performance of the UAV in complex environments, but also significantly reduces the burden on operators and improves work efficiency, thereby providing a more superior solution for various high-precision and high-risk remote operation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0060] Figure 1 An exemplary network architecture used in the tactile teleoperation control method of the drone of this application;

[0061] Figure 2 This is a flow chart of a method for controlling the tactile teleoperation of a drone in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of displacement changes of a joystick in a tactile control device in an embodiment of the present application;

[0063] Figure 4 This is a principle block diagram of the UAV tactile teleoperation control device in the embodiment of the present application;

[0064] Figure 5 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0066] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0067] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0068] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0069] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0070] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0071] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0072] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0073] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0074] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0075] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0076] See also Figure 1 , Figure 2 as well as Figure 3 In one embodiment, the drone tactile teleoperation control method of the present application includes:

[0077] Step S10: When a target UAV is detected to enter a non-safe area and responds to a UAV tactile teleoperation control instruction, a model predictive controller in the tactile control device uses a preset second-order derivative according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV to determine a feedback force for driving a joystick in the tactile control device to change its displacement, and determines a joystick displacement increment of the joystick according to the feedback force;

[0078] The UAV tactile teleoperation control system can detect that the target UAV enters a non-safe area and responds to the UAV tactile teleoperation control command. The model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force that drives the joystick in the tactile control device to change its displacement, and determines the joystick displacement increment of the joystick according to the feedback force;

[0079] Specifically, the UAV tactile teleoperation control system includes a system master and a system slave. The system master is a tactile control device, which includes a three-degree-of-freedom Falcon (Novint) tactile device, etc. The force feedback frequency of the three-degree-of-freedom Falcon (Novint) tactile device is 1kHz; the system slave is a target UAV, which can be a four-rotor UAV, and the sensor sampling frequency is 1kHz. The master and slave use the ROS system under the Linux system for communication, and use C++ and Python languages ​​to write the control algorithm.

[0080] First, establish the system master end coordinate system and the system slave end coordinate system. In the world coordinate system [x W ,y W ,z W ], the main end coordinate system is [x j ,y j ,z j ], the slave end coordinate system is [x B ,y B ,z B ], the rotation matrix from the world coordinate system to the slave coordinate system is R(q), where q is the quaternion [q x ,q y ,q z ,q w ].

[0081] In some embodiments, the three-axis displacement increment of the center point of the joystick is mapped to the position increment of the drone. When the control button in the tactile control device is pressed, the joystick no longer changes the position of the drone, and the drone can no longer be controlled. At this time, the joystick can be moved to any suitable position. When the control button in the tactile control device is released, new position increment information will continue to be transmitted to the drone, realizing position superposition, so that the drone can move without boundaries.

[0082] In some embodiments, the model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematics model, attitude kinematics model, function model corresponding to the model predictive controller, and obstacle function corresponding to the target UAV to determine the feedback force for driving the joystick in the tactile control device to change displacement, including:

[0083] The drone tactile teleoperation control system can respond to a safety obstacle avoidance warning instruction and output the feedback force to a force feedback controller in a driving tactile control device so that the operator can sense the feedback force to warn the operator to drive the target drone away from an unsafe area.

[0084] Specifically, the model predictive controller (MPC controller) receives obstacle constraint information and target drone sensor status information, makes plans when the drone enters a complex, unsafe area, and its output is transmitted to the force feedback controller and ultimately sensed by the operator, thereby making correct behavior; the present application uses a differential flat controller as an intermediate converter to achieve direct control of the joystick to the drone position. In addition, based on the force feedback method of the model predictive controller, its constraint conditions introduce control barrier constraints. When the operator flies the drone to an unsafe area, the controller will generate a repulsive force away from the obstacle, alerting the operator to react, and will generate an autonomous obstacle avoidance response when extremely close to the unsafe area.

[0085] In the specific embodiment, the global coordinate system of the system is first constructed as F W :{x W ,y W ,z W}, the body reference system is F B :{x B ,y B ,z B}, where B represents the variable in the body coordinate system, and the other variables are expressed in the world (inertial) coordinate system. w To the body coordinate system F B The rotation matrix is ​​composed of the quaternion q = [q w ,q x ,q y ,q z]∈SO(3) indicates;

[0086] The expression of the position kinematic model is:

[0087]

[0088] Among them, ξ w is the displacement of the UAV in the world coordinate system, v w is the speed of the drone in the world coordinate system;

[0089] The expression of the posture kinematic model is:

[0090] w=W·Ω B ,

[0091] Among them, w represents the three attitude angles (Euler angles) of the target UAV, W represents the angular velocity in the world coordinate system, and its expression is:

[0092]

[0093] The expression of the function model corresponding to the model predictive controller includes:

[0094]

[0095] X k =x k -x k，r ,

[0096] U k =u k -u k，r ,

[0097] X N =x n -x n，r ,

[0098] stu∈[u min ,u max ],

[0099] x0=x init

[0100] x k+1 =f(x k ,u k ),

[0101] in, Expressed as state space, k is the time step, T is the transpose, and u is defined as v w and Ω B Set, x k+1 =f(x k ,uk ) is the kinematic model of the drone, Q u is the weight matrix of input u, Q and Q n It is a diagonal matrix with the weight matrix as its diagonal elements, and the subscript r represents the expected value.

[0102] The expression of the barrier function is:

[0103] d(x)-p sec ≥0,

[0104] Where d(x) represents the physical area of ​​the safe space, p sec Indicates the safety zone threshold;

[0105] Specifically, referring to the control obstacle function, an obstacle constraint region is defined for the model predictive controller, which is expressed as:

[0106]

[0107] Function d(x) defines the physical area of ​​the safe space, and the obstacle is equivalent to an ellipsoid;

[0108] For UAV systems, the general expression for the obstacle surface is:

[0109]

[0110] In order to adapt to the constraints of the differential flat controller, this controller only uses the position as a constraint and designs the safety area threshold p sec , the final barrier function expression is:

[0111] d(x)-p sec ≥0.

[0112] In order to meet the requirements of differential flat controller, this application only considers the position constraint of the target drone. However, when designing such safety constraints for dynamic systems, it is more advantageous to include higher-order derivatives of the obstacle function. To this end, this application proposes a compensation measure to generate force feedback based on the second-order derivative of the input, which has advantages for the force feedback output. The expression of the second-order derivative is:

[0113]

[0114] Among them, K f1 , K f2 and K f3 is a constant parameter used to adjust the amplitude of tactile feedback.

[0115] When the operator enters an unsafe area, the model predictive controller records the current entry position as the target point, reversely plans and generates force feedback to drive the drone out of the unsafe area. The tactile feedback is generated by uMPC The decision produces smoother force feedback, and the force feedback coupled with the joystick can realize the function of autonomous obstacle avoidance.

[0116] It can be seen from the above embodiments that the model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force that drives the joystick in the tactile control device to change its displacement, and determines the joystick displacement increment of the joystick according to the feedback force.

[0117] Step S20, the position increment type motion controller obtains the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space, and determines the displacement increment corresponding to the target UAV according to the ratio between the joystick displacement increment and the maximum radius of the joystick working space and the joystick displacement increment;

[0118] After determining the feedback force that drives the joystick in the tactile control device to change its displacement, and determining the joystick displacement increment of the joystick according to the feedback force, the position increment type motion controller obtains the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space, and determines the corresponding displacement increment of the target UAV according to the joystick displacement increment and the ratio between the maximum radius of the joystick working space and the joystick displacement increment.

[0119] In a specific embodiment, the expression of the displacement increment corresponding to the target UAV is:

[0120]

[0121] Where a represents the input signal of the tactile device button, K m is a scaling matrix with positive diagonal, which is used to control the displacement increment of the drone in different directions; q j (t) represents the joystick displacement increment, which represents the distance between the end of the joystick and the origin of the joystick workspace, Δq s (t) represents the displacement increment of the drone, r represents the maximum radius of the joystick working space, when the joystick displacement increment q j When (t) is less than the maximum radius r of the joystick working space, the ratio between the maximum radius of the joystick working space and the joystick displacement increment is greater than 1, then the corresponding displacement increment of the UAV is the product of the scaling matrix with a positive diagonal and the joystick displacement increment.

[0122] More specifically, when the input signal a of the control button of the tactile device is 0, that is, the control button in the tactile control device is in a pressed state, the joystick no longer changes the position of the drone, and the drone can no longer be controlled at this time. The joystick can be moved to any suitable position at this time; when the input signal a of the control button of the tactile device is 1, when the control button in the tactile control device is in a released state, it will continue to transmit new displacement increments to the drone, realize position superposition, and thus realize boundless movement of the drone.

[0123] Step S30, the position increment type motion controller determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the displacement increment of the joystick, and determines the target position and yaw angle of the target UAV according to the current position information;

[0124] After determining the corresponding displacement increment of the target UAV according to the joystick displacement increment and the ratio between the maximum radius of the joystick workspace and the joystick displacement increment, a position increment motion controller (DFBC controller) determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the joystick displacement increment, and determines the target position and yaw angle of the target UAV according to the current position information; specifically, the target UAV transmits its corresponding current position information to the position increment motion controller in the tactile control device, and the position increment motion controller determines the target position and yaw angle of the target UAV according to the current position information to avoid obstacles, and transmits the obstacle avoidance posture of the target UAV, the target position and yaw angle of the target UAV to the differential flat controller to generate a control command to control the movement of the UAV.

[0125] In a specific embodiment, the step of calculating and determining the obstacle avoidance posture of the target UAV by the position increment motion controller according to the initial position of the joystick and the joystick displacement increment includes:

[0126] The expression of the obstacle avoidance posture of the target UAV is:

[0127]

[0128] Among them, ξ n represents the initial position of the joystick, N represents the number of times the joystick is controlled, and ξ(t) represents the obstacle avoidance posture of the target UAV. The obstacle avoidance posture of the target UAV is expressed as the displacement increment caused by operating the joystick and its initial position ξ n The sum of .

[0129] Step S40: The differential flat controller generates a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile teleoperation.

[0130] The position-increment motion controller determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the displacement increment of the joystick, and determines the target position and yaw angle of the target UAV according to the current position information. Then, the differential flat controller generates a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle and sends it to the target UAV to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile teleoperation.

[0131] Specifically, when there is no operator operating, when the target UAV enters a complex and unsafe area, the model predictive controller uses a preset second-order derivative to determine the feedback force that drives the joystick to change its displacement according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV, and drives the joystick to move so that its displacement changes, and determines the joystick displacement increment of the joystick according to the feedback force; the incremental motion controller determines the displacement increment corresponding to the target UAV according to the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space, according to the joystick displacement increment and the ratio between the maximum radius of the joystick working space and the joystick displacement increment; the position incremental motion controller calculates and determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the joystick displacement increment, and determines the target position and yaw angle of the target UAV according to the current position information; the differential flat controller generates the corresponding flight control instruction of the target UAV according to the obstacle avoidance posture, the target position and the yaw angle, so as to drive the target UAV to automatically avoid obstacles, thereby realizing the autonomous obstacle avoidance function.

[0132] It can be seen from the above embodiments that, compared with the prior art, the present application aims to solve the problems that the tactile force feedback in the prior art is poorly adaptable to complex environments, especially in changeable or dynamic environments, and its feedback effect is not ideal. The present application includes but is not limited to the following beneficial effects:

[0133] First, the UAV tactile remote control method of the present application improves the system's control performance in complex environments and is suitable for tasks that require high-precision and delicate operations, such as precise positioning, inspection, and transportation. It significantly improves the operational safety of UAVs in complex environments, reduces the risks caused by control errors or environmental uncertainties, and provides operators with higher safety guarantees, especially in dangerous areas or highly uncertain environments.

[0134] Secondly, the UAV tactile remote control method of the present application reduces the operator's cognitive burden on control through position incremental motion control and smooth force feedback, allowing the operator to focus more on the task itself without having to pay too much attention to complex control details. At the same time, the system optimizes intelligent obstacle avoidance and dynamic feedback, allowing the operator to easily complete operations when faced with difficult tasks.

[0135] Third, the tactile remote control method of the drone in this application can not only improve the control accuracy, but also improve the reliability of the drone in performing tasks in complex mission environments through precise control and force feedback optimization. For example, in tasks such as medical transportation and dangerous area reconnaissance, precise control and reliable feedback can ensure the smooth completion of the task and minimize the risk of accidents.

[0136] Furthermore, the UAV tactile remote operation control method based on position increment and model predictive control optimization of the present application effectively improves the controllability, perception ability, safety and operation accuracy of the system through precise control, smooth force feedback, intelligent obstacle avoidance and other innovations. It not only enhances the operational performance of the UAV in complex environments, but also significantly reduces the burden on operators and improves work efficiency, thereby providing a more superior solution for various high-precision and high-risk remote operation tasks.

[0137] See also Figure 4, a tactile teleoperation control device for a drone provided to meet one of the purposes of the present application, includes a feedback force determination module 1100, a displacement increment determination module 1200, an obstacle avoidance posture determination module 1300 and a teleoperation control module 1400. Among them, the feedback force determination module 1100 is configured to detect that the target drone enters a non-safe area and respond to the tactile teleoperation control instruction of the drone, and the model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target drone to determine the feedback force that drives the joystick in the tactile control device to change its displacement, and determines the joystick displacement increment of the joystick according to the feedback force; the displacement increment determination module 1200 is configured to be a position increment type motion controller to obtain the current position information of the target drone, the joystick displacement increment and the maximum radius of the joystick working space, according to the The ratio between the joystick displacement increment and the maximum radius of the joystick working space and the joystick displacement increment is used to determine the displacement increment corresponding to the target UAV; the obstacle avoidance posture determination module 1300 is configured as a position increment type motion controller to determine the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the joystick displacement increment, and determine the target position and yaw angle of the target UAV according to the current position information; the remote operation control module 1400 is configured as a differential flat controller to generate the corresponding flight control instruction of the target UAV according to the obstacle avoidance posture, the target position and the yaw angle, so as to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile remote operation.

[0138] Based on any embodiment of this application, please refer to Figure 5 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 5 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for tactile remote operation control of a drone. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the method for tactile remote operation control of a drone of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In this embodiment, the processor is used to execute Figure 4 The memory stores the program code and various data required to execute the above modules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / sub-modules in the drone tactile teleoperation control device of the present application, and the server can call the server's program code and data to execute the functions of all sub-modules.

[0140] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the drone tactile remote operation control method described in any embodiment of the present application.

[0141] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the drone tactile remote operation control method described in any embodiment of the present application.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0143] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0144] To summarize, the UAV tactile remote operation control method based on position increment and model predictive control optimization of the present application effectively improves the controllability, perception ability, safety and operation accuracy of the system through precise control, smooth force feedback, intelligent obstacle avoidance and other innovations. It not only enhances the operational performance of the UAV in complex environments, but also significantly reduces the burden on operators and improves work efficiency, thereby providing a more superior solution for various high-precision and high-risk remote operation tasks.

Claims

1. A tactile teleoperation control method for an unmanned aerial vehicle, characterized in that: include: When a target UAV is detected to enter a non-safe area and responds to a UAV tactile teleoperation control instruction, a model predictive controller in the tactile control device uses a preset second-order derivative to determine a feedback force that drives a joystick in the tactile control device to change its displacement according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV, and determines a joystick displacement increment of the joystick according to the feedback force; The position increment type motion controller obtains the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space, and determines the displacement increment corresponding to the target UAV according to the ratio between the joystick displacement increment and the maximum radius of the joystick working space and the joystick displacement increment; The position increment motion controller determines the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the displacement increment of the joystick, and determines the target position and yaw angle of the target UAV according to the current position information; The differential flat controller generates a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile teleoperation.

2. The method for controlling the tactile teleoperation of a drone according to claim 1, characterized in that: The model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force driving the joystick in the tactile control device to change displacement, including: First, construct the global coordinate system of the system as F W :{x W ,y W ,z W }, the body reference system is F B :{x B ,y B ,z B }, where B represents the variable in the body coordinate system, and the other variables are expressed in the world (inertial) coordinate system. w To the body coordinate system F B The rotation matrix is ​​composed of the quaternion q = [q w ,q x ,q y ,q z ]∈SO(3) indicates; The expression of the position kinematic model is: Among them, ξ w is the displacement of the UAV in the world coordinate system, v w is the speed of the drone in the world coordinate system; The expression of the posture kinematics model is: w=W·Ω B , Among them, w represents the three attitude angles of the target UAV, Ω B =[p,q,r] T , W represents the angular velocity in the world coordinate system.

3. The method for controlling the tactile teleoperation of a drone according to claim 1, characterized in that: The model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force driving the joystick in the tactile control device to change displacement, including: The expression of the function model corresponding to the model predictive controller includes: X k =x k -x k，r , IN k =in k -in k，r , X N =x n -x n，r , s.t.u∈[u min ,u max ], x0=x init , x k+1 =f(x k ,u k ), in, Expressed as state space, k is the time step, T is the transpose, and u is defined as v w and Ω B Set, x k+1 =f(x k ,u k ) is the kinematic model of the drone, Q u is the weight matrix of input u, Q and Q n It is a diagonal matrix with the weight matrix as its diagonal elements, and the subscript r represents the expected value.

4. The method for controlling the tactile teleoperation of a drone according to claim 1, characterized in that: The model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force driving the joystick in the tactile control device to change displacement, including: The expression of the barrier function is: d(x)-p sec ≥0, Where d(x) represents the physical area of ​​the safe space, p sec Indicates the safety zone threshold; The expression of the second-order derivative is: Among them, K f1 , K f2 and K f3 is a constant parameter used to adjust the amplitude of tactile feedback.

5. The method for controlling the tactile teleoperation of a drone according to claim 1, characterized in that: Determining the displacement increment corresponding to the target UAV according to the joystick displacement increment and the ratio between the maximum radius of the joystick working space and the joystick displacement increment includes: The expression of the displacement increment corresponding to the target UAV is: Where a represents the input signal of the tactile device button, K m is a scaling matrix with positive diagonal, which is used to control the displacement increment of the drone in different directions; q j (t) represents the joystick displacement increment, which represents the distance between the end of the joystick and the origin of the joystick workspace, Δq s (t) represents the displacement increment of the drone, r represents the maximum radius of the joystick working space, when the joystick displacement increment q j When (t) is less than the maximum radius r of the joystick working space, the ratio between the maximum radius of the joystick working space and the joystick displacement increment is greater than 1, then the corresponding displacement increment of the UAV is the product of the scaling matrix with a positive diagonal and the joystick displacement increment.

6. The method for controlling the tactile teleoperation of a drone according to claim 1, characterized in that: The step of calculating and determining the obstacle avoidance posture of the target UAV by the position increment motion controller according to the initial position of the joystick and the joystick displacement increment includes: The expression of the obstacle avoidance posture of the target UAV is: Among them, ξ n represents the initial position of the joystick, N represents the number of control times of the joystick, and ξ(t) represents the obstacle avoidance posture of the target UAV.

7. The method for controlling the tactile teleoperation of a drone according to claims 1 to 6, characterized in that: The model predictive controller in the tactile control device uses a preset second-order derivative according to the position kinematic model, attitude kinematic model, function model corresponding to the model predictive controller and obstacle function corresponding to the target UAV to determine the feedback force driving the joystick in the tactile control device to change displacement, including: In response to the safety obstacle avoidance warning instruction, the feedback force is output to a force feedback controller in the driving tactile control device so that the operator can sense the feedback force, so as to warn the operator to drive the target drone away from the unsafe area.

8. A tactile remote control device for an unmanned aerial vehicle, characterized in that: include: A feedback force determination module is configured to detect that a target UAV enters a non-safe area and respond to a UAV tactile teleoperation control instruction, wherein a model predictive controller in a tactile control device uses a preset second-order derivative according to a position kinematic model, an attitude kinematic model, a function model corresponding to the model predictive controller, and an obstacle function corresponding to the target UAV to determine a feedback force for driving a joystick in the tactile control device to change its displacement, and determines a joystick displacement increment of the joystick according to the feedback force; a displacement increment determination module, configured to obtain the current position information of the target UAV, the joystick displacement increment and the maximum radius of the joystick working space by using a position increment type motion controller, and determine the displacement increment corresponding to the target UAV according to the ratio between the joystick displacement increment and the maximum radius of the joystick working space and the joystick displacement increment; The obstacle avoidance posture determination module is configured as a position increment type motion controller to determine the obstacle avoidance posture of the target UAV according to the initial position of the joystick and the joystick displacement increment calculation, and determine the target position and yaw angle of the target UAV according to the current position information; The remote operation control module is configured as a differential flat controller to generate a flight control instruction corresponding to the target UAV according to the obstacle avoidance posture, the target position and the yaw angle, so as to drive the target UAV to automatically avoid obstacles, so as to complete the control of the UAV tactile remote operation.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images