Sensory fusion aerial robot teleoperation system and following assistance control method

Abstract

The present application relates to the field of aerial robot teleoperation, and discloses a vision-tactile fusion aerial robot teleoperation system and a following auxiliary control method. The system comprises a master end, a slave end, a control strategy module and a network communication module. The master end comprises a joystick signal acquisition module, a tactile feedback module, a visual feedback module and a signal processing module. The slave end comprises a leader and a follower. The leader is provided with a positioning module, and the follower is an aerial robot. The aerial robot comprises a motion control module, a positioning mapping module, a motion planning module and a visual acquisition module. The present application provides following auxiliary control guidance by using the tactile feedback method, so that the teleoperation system can safely complete complex tasks. Compared with controlling the aerial robot in an unstructured environment by relying on visual feedback signals alone, the present application reduces the workload of the operator and improves the safety of the control.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aerial robot teleoperation, in particular to a visual and tactile feedback integrated aerial robot teleoperation system and a following auxiliary control method. BACKGROUND

[0002] With the maturity of aerial robot teleoperation technology, it is increasingly widely used in dangerous, complex, and unstructured environments that are difficult for humans to enter, such as high altitude, fire sites, and nuclear radiation areas. In addition, multi-agent cooperation technology can take advantage of the heterogeneity between humans and machines or multiple machines in unstructured environments, and complete tasks by operators operating cooperatively in unstructured environments.

[0003] Teleoperation technology is a technology for remote interaction between humans and robots, which captures the control intention of the operator and transmits the operator's control command to the remote robot controller, and at the same time, through the multi-sensor device installed on the robot, the state information of the environment and the robot is fed back to the operator, so that the operator can obtain the basis for control decision.

[0004] Current aerial robot teleoperation methods mainly rely on single-sided teleoperation based on human eye or visual sensor feedback images. In highly unstructured environments, such as environments with poor line of sight, operators have a high cognitive load. In addition, multi-agent technology regards each mobile agent in a human-robot formation or a multi-robot formation as an intelligent agent. In the process of formation movement, a human or a robot as a leader often leads the rest of the robots to move. For the rest of the robots, whether they can safely follow the leader is a concern in recent years. However, current robots can rely on target tracking and autonomous navigation technology to complete the tasks of following and obstacle avoidance along the optimal path, but this autonomous method is difficult to adapt to unexpected accidents and touchable dangers. SUMMARY

[0005] To solve the problems existing in the prior art, the present application provides a visual and tactile feedback integrated aerial robot teleoperation system for operators to provide following auxiliary control, combined with visual first perspective information, and a visual and tactile feedback integrated aerial robot following auxiliary control method is proposed according to the state information of the rest of the agents in the formation, the environment information and the operator command of each agent, to improve the safety of teleoperation and reduce the workload of the operator.

[0006] According to the visual and tactile feedback integrated aerial robot teleoperation system provided by the present application, comprising:

[0007] The master end comprises a joystick signal acquisition module, a tactile feedback module, a visual feedback module and a signal processing module, wherein the signal processing module is connected with the joystick signal acquisition module, the tactile feedback module and the visual feedback module respectively;

[0008] The control strategy module;

[0009] The network communication module;

[0010] The slave end comprises a leader and a follower, the leader is provided with a positioning module, and the follower is an aerial robot; the aerial robot comprises a motion control module, a positioning mapping module, a motion planning module and a visual acquisition module; the positioning module is connected with the network communication module and the motion planning module respectively;

[0011] The joystick signal acquisition module acquires the rotation amount of the rotating shaft of the mobile tactile feedback device joystick and corresponding time information, and outputs the tactile feedback device signal to the signal processing module;

[0012] The tactile feedback module receives the expected tactile feedback device joystick displacement signal calculated by the signal processing module, controls the tactile feedback device joystick to render the corresponding tactile sensation, and assists in controlling the movement of the tactile device joystick;

[0013] The signal processing module receives the information of the joystick signal acquisition module and processes it into an aerial robot speed control instruction; receives the follow-up auxiliary control speed signal calculated by the control strategy module and processes it into an expected tactile feedback device joystick displacement signal for the tactile feedback module; receives the first perspective image signal collected by the visual acquisition module of the slave end aerial robot transmitted via the network communication module, and transmits it to the visual feedback module;

[0014] The visual feedback module displays and outputs the first perspective image signal of the aerial robot transmitted by the signal processing module;

[0015] The positioning module acquires and calculates the three-dimensional position coordinates of the leader and the follower and corresponding time information;

[0016] The motion control module receives the positioning information from the positioning mapping module and the final slave end aerial robot speed control instruction calculated from the control strategy module, so as to control the motion of the aerial robot;

[0017] The positioning mapping module obtains the point cloud map information and the positioning information of the slave end aerial robot and outputs them to the motion planning module, and transmits the positioning information to the control strategy module via the network communication module;

[0018] The motion planning module receives point cloud map information and positioning information of the follower aerial robot from the aerial robot positioning mapping module, and receives the positioning information of the leader as a following target; in combination with a dynamic model of the follower aerial robot, a motion trajectory satisfying dynamics and obstacle avoidance is derived, and the trajectory information is transmitted to the control strategy module via the network communication module;

[0019] The visual acquisition module acquires a first visual angle image signal from a visual sensor in real time and transmits the signal to the signal processing module via the network communication module;

[0020] The control strategy module receives a follower aerial robot speed control instruction from the signal processing module, follower aerial robot positioning information, and desired trajectory information, calculates a final follower aerial robot speed control instruction output to the motion control module, and outputs a following auxiliary control speed signal to the signal processing module.

[0021] The following auxiliary control method provided by the application is based on the above-mentioned visual-tactile fusion aerial robot teleoperation system, and specifically includes the following steps:

[0022] The signal processing module matches the spatial displacement amount of the end point of the haptic feedback device handlebar and the rotation amount of the end rotation shaft as the desired spatial movement speed instruction of the follower aerial robot and the deflection angle speed instruction thereof.

[0023] According to the guiding auxiliary force of the follower aerial robot following and the repulsion auxiliary force of aerial robot formation cluster collision avoidance, the auxiliary resultant force on the haptic feedback device is calculated.

[0024] Compared with the prior art, the technical effects obtained by the application include:

[0025] 1、The visual-tactile fusion aerial robot teleoperation system and the following auxiliary control method provided by the application combine the experience of the operator and the self-capability of the aerial robot, adopt the method of haptic feedback to provide the operator with following auxiliary control guidance, so that the teleoperation system can safely complete complex tasks; compared with the method of simply relying on visual feedback signals to control the aerial robot in an unstructured environment, the workload of the operator is reduced, and the safety of the control is improved.

[0026] 2、The haptic feedback force is divided into a guiding force for following the unmanned aerial vehicle and a repulsion force for dynamic obstacle avoidance between multiple machines, so that the accuracy of the aerial robot trajectory following is improved, and the safety between multiple machines is improved.

[0027] 3. The present invention adopts a multi-robot formation remote operating system structure. By giving different aerial robots basic intelligence and configuring different equipment adapted to different tasks, compared with the solution of equipping all functional modules in one aerial robot, the present invention makes each robot in the formation lightweight, and completes complex tasks through multi-robot cooperation, thereby improving the system's flexibility.

[0028] 4. The follow-assisted control method adopted in this invention integrates human autonomy with the auxiliary prompts of the automated system, making full use of human experience and the advantages of the autonomous system, and improving the system's ability to adapt to different tasks. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the teleoperation system in an embodiment of the present invention;

[0030] Figure 2 This is an example of a conceptual diagram of a velocity obstacle in the position space according to an embodiment of the present invention;

[0031] Figure 3 This is an example of a conceptual diagram of a velocity obstacle in velocity space in an embodiment of the present invention. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0033] Example

[0034] like Figure 1 As shown, this embodiment provides a visual-tactile fusion teleoperation system for an aerial robot. This teleoperation system includes a master terminal, a slave terminal, a wireless network communication module, and a control strategy module. The master terminal includes a joystick signal acquisition module, a tactile feedback module, a visual feedback module, and a signal processing module, wherein the signal processing module is connected to the joystick signal acquisition module, the tactile feedback module, and the visual feedback module, respectively. The slave terminal includes a leader and a follower. The leader is a person or a leader robot in a remote environment, equipped with a positioning module; the follower is an aerial robot, including a special task module, a motion control module, a positioning and mapping module, a motion planning module, and a visual acquisition module. The positioning and mapping module can be implemented using a series of positioning and mapping sensors, and the visual acquisition module can be implemented using a visual sensor. The motion planning module, the positioning and mapping module, the motion control module, and the motion control module are connected sequentially; the positioning module is connected to the network communication module and the motion planning module, respectively.

[0035] A joystick signal acquisition module is configured to acquire the rotation amounts of the six rotation axes of the joystick of the mobile haptic feedback device and corresponding time information, and output a haptic feedback device signal to the signal processing module; the joystick of the mobile haptic feedback device can be controlled by an operator;

[0036] A haptic feedback module is configured to receive the expected haptic feedback device joystick displacement amount signal calculated by the signal processing module, control the haptic feedback device joystick to render a corresponding tactile sensation, and assist the operator in controlling the haptic device joystick to move;

[0037] The signal processing module is configured to receive the information of the joystick signal acquisition module and process it into aerial robot speed control instructions, wherein the aerial robot speed control instructions include aerial robot spatial movement speed instructions and aerial robot deflection angle speed instructions; receive the recommended follow-up assistance control speed signal calculated by the control strategy module, and process it into an expected haptic feedback device joystick displacement amount signal for the haptic feedback module; receive the first perspective image signal acquired by the vision acquisition module of the slave aerial robot transmitted via the wireless network communication module, and transmit the first perspective image signal of the aerial robot to the vision feedback module at a specified frame rate and format;

[0038] The vision feedback module is configured to display and output the first perspective image signal of the aerial robot transmitted by the signal processing module; generally, the first perspective image signal can be displayed and output on a display screen;

[0039] A positioning module is configured to acquire and calculate the three-dimensional position coordinates and corresponding time information of the leader operator / leader robot and other followers in the formation; the position coordinates calculated by the positioning module of each robot are in the same coordinate system;

[0040] A special task module is configured to receive instructions for running special functions of the motion control module, and is responsible for executing special tasks;

[0041] The motion control module is configured to receive the positioning information from the positioning and mapping module and the final slave aerial robot speed control instructions calculated by the control strategy module, process the received information, and output a PWM signal to control the motor speed of the power system of the slave aerial robot, thereby controlling the motion of the slave aerial robot; the motion control module is also configured to send a control command signal to control the operation of the special task module;

[0042] The positioning and mapping module is configured to obtain point cloud information of a positioning and mapping series of sensors of the aerial robot, inertial measurement sensor (IMU) information, and information of other positioning sensors, and output the point cloud map information and the positioning information to the motion planning module, and output the positioning information of the aerial robot in the world coordinate system to the control strategy module via the wireless network communication module;

[0043] A motion planning module is configured to receive the point cloud map information and the positioning information of the aerial robot positioning mapping module, receive the positioning information of the leader / follower robot as a following target, combine the dynamics model of the aerial robot, and derive a motion trajectory meeting the dynamics and obstacle avoidance via online processing of the aerial robot on-board computer to obtain a trajectory with path point position information; and transmit the trajectory information to the control strategy module via a wireless network communication module;

[0044] A visual acquisition module is configured to acquire a first perspective image signal from a visual sensor in real time and transmit the first perspective image signal to the signal processing module via the wireless network communication module;

[0045] A wireless network communication module is configured to be responsible for signal transmission between the master, the follower aerial robot, the leader / follower robot and other intelligent agents.

[0046] A control strategy module is configured to receive the aerial robot speed control instruction from the signal processing module, the positioning information of the positioning module of the other follower, the aerial robot positioning information and the desired trajectory information, perform calculation according to the following auxiliary control method of the aerial robot teleoperation system of the application, output the final aerial robot speed control instruction to the motion control module, and output the following auxiliary control speed signal to the signal processing module.

[0047] In the embodiment, the positioning module is a UWB positioning tag or a motion capture system tag arranged at the center of the leader / follower robot; and the special task module is a sensor and / or an actuator, such as an infrared detector and a mechanical arm, arranged on the aerial robot.

[0048] In the embodiment, the aerial robot includes a flight controller and an on-board computer; the motion control module is arranged in the flight controller, acquires the top-level control instruction and the positioning information from the on-board computer, analyzes the motion state information of the robot according to the sensors such as the gyroscope, the accelerometer and the barometer on the flight controller, generates a PWM signal to control the motor speed of the power system, and further controls the spatial pose of the robot. In addition, the motion control module is responsible for information interaction with the special task module, sends information to the special task module through the on-board computer or the flight controller, and receives the information collected by the special task module.

[0049] In the embodiment, the positioning mapping module runs a simultaneous localization and mapping algorithm; the positioning mapping series sensors can use the following sensor schemes including but not limited to: multi-line laser radar+IMU module, multi-line laser radar, monocular / binary camera+IMU, binary camera+depth camera+IMU, real sense tracking camera, etc.; the positioning mapping module obtains a three-dimensional point cloud map, a three-dimensional positioning coordinate information of the robot and a four-element number attitude information of the robot through the simultaneous localization and mapping algorithm.

[0050] In a preferred embodiment, the visual acquisition module is an RGB camera arranged on the aerial robot.

[0051] Based on the same inventive concept, the embodiment also provides a following assistance control method, which is based on the above-mentioned visual-haptic fusion aerial robot teleoperation system, and specifically comprises the following steps:

[0052] S1, the signal processing module matches the spatial displacement of the haptic feedback device joystick end point and the rotation of the end rotating shaft to the spatial movement speed instruction and the deflection angular velocity instruction expected by the from-end aerial robot, and the displacement-velocity matching formula is:

[0053]

[0054] wherein the spatial movement speed instruction and the deflection angular velocity instruction expected by the aerial robot represent the linear speed instruction of the aerial robot, and ω z，com represents the deflection angular velocity instruction, represents the displacement of the haptic feedback device joystick end position, and Ψ HD represents the rotation of the haptic feedback device joystick end rotating shaft, and the coefficient is a displacement-velocity matching parameter;

[0055] S2, the guidance assistance force of the from-end aerial robot following and the repulsion assistance force of the aerial robot formation cluster collision avoidance are calculated to obtain the auxiliary resultant force on the haptic feedback device, and the formula is:

[0056] F = F at + F re

[0057] wherein, is the guidance assistance force of the aerial robot following, is the repulsion assistance force of the aerial robot formation cluster collision avoidance, is the auxiliary resultant force generated by the haptic feedback device.

[0058] The guidance assistance force F at of the aerial robot following can be further decomposed as:

[0059] F at = F ⊥ + F ||

[0060] wherein, is the regression assistance force in the vertical direction towards the trajectory calculated by the motion planning module, is the following assistance force in the forward direction towards the trajectory calculated by the motion planning module;

[0061] The regression auxiliary force F ⊥ The calculation formula is:

[0062]

[0063] K HD，P The elastic coefficient of the haptic feedback device joystick, K HD，D The damping coefficient of the haptic feedback device joystick, The spatial velocity of the aerial robot expected by the motion planning module defined in the aerial robot body coordinate system O B，F The moving speed of the end of the haptic feedback device joystick; wherein, The calculation formula is:

[0064]

[0065] O F represents the coordinate system O The aerial robot moving speed calculated by the control strategy module towards the vertical direction of the expected trajectory, the calculation formula is:

[0066]

[0067] K ⊥，P , K ⊥，I and K ⊥，D are the proportional, integral and derivative parameters of the path regression PID controller, x F The position coordinates of the aerial robot in the coordinate system O F The path point closest to the aerial robot on the expected path calculated by the motion planning module.

[0068] The following auxiliary force F || The purpose is to guide the operator to control the aerial robot to follow the leader (i.e. the leader or the leader robot) at a safe speed in the direction of the trajectory planned by the motion planning module through the haptic feedback device, and the calculation method is:

[0069] When the aerial robot is very close to the leader, the safe following speed formula in the trajectory direction is:

[0070]

[0071] Wherein, K PP , K PI and K PD are the proportional, integral and derivative parameters of the PID controller, D​​rel D represents the current distance between the aerial robot and the leader. default The expected relative distance between the aerial robot and the leader when they are stationary;

[0072] When the aerial robot is relatively far from the leader, its safe following acceleration formula in the trajectory direction is:

[0073]

[0074] Among them, K VP K VI and K VD These are the proportional, integral, and derivative parameters of the path-following PID controller, v f，tra The velocity of the aerial robot in the trajectory direction. For the leader in the inertial coordinate system O F Given the moving speed, we can derive the speed formula as follows:

[0075]

[0076] Therefore, the speed is determined based on the distance between the aerial robot and the leader. A coordinate system O for the aerial robot body can be defined along the trajectory direction. B，F velocity vector Its mold length is

[0077] Therefore, the following auxiliary force F || The calculation formula is:

[0078]

[0079] The aerial robot follows the repulsive assist force F re The calculation method is as follows:

[0080] F re The core of the computational method is to construct a velocity obstacle (VO) for each agent other than the current aerial robot. Ultimately, the velocity obstacle space of the current aerial robot is the superposition of each velocity obstacle VO.

[0081] For each velocity obstacle VO, the currently airborne robot shrinks to a point mass with radius 0, while the remaining agents, including the leader, expand into a sphere with radius r. Le The calculation method is as follows:

[0082] r Le =r L +r e

[0083] rL Let r be the sum of the radii of the smallest sphere enclosing the aerial robot and the radii of the smallest sphere enclosing the corresponding intelligent agent. e To extend the safety error radius.

[0084] like Figure 2 As shown, taking a formation consisting of an aerial robot and a leader as an example, they are assigned the numbers F and L respectively. This means the aerial robot is shrunk to a point mass. This indicates that the leader has become bloated and inflated. Indicates the speed of the aerial robot. The speed of the leader is represented by the collision cone VO′ of the speed obstacle VO. F|L It can be represented as:

[0085]

[0086]

[0087] like Figure 3 As shown, a further optimization scheme for the velocity obstacle VO is to... F|L Based on this, a feasible region is set so that the control strategy only focuses on a future time period T. h Collisions within the feasible region VO H for:

[0088]

[0089] Where, d m A point mass shrunk to the size of an aerial robot To the leader's swollen ball The closest distance to the surface. The optimized velocity obstacle VO space region is:

[0090]

[0091] Define the repulsive auxiliary force F re direction Rapidly Destroying Aerial Robots The vector endpoint is perpendicular to the velocity obstacle space region. The direction of the surface and the calculation method for the repulsive auxiliary force Fre are as follows:

[0092]

[0093] Among them, F set The appropriate force modulus value is adjusted manually.

[0094] In this embodiment, by Figure 2 The speed of the aerial robot can be visually observed in position space. Leader velocity The position of the superimposed velocity in the velocity obstacle VO is determined, and it is determined whether the superimposed velocity is in the velocity obstacle VO, and whether the aerial robot and the leader will collide in the future. By Figure 3 In the velocity space, the velocity obstacle VO can be moved according to the leader velocity direction and size, and it is determined whether the aerial robot velocity points to a position in the velocity obstacle VO, and whether the aerial robot and the leader will collide in the future.

[0095] The above embodiments are preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, etc. made without departing from the spirit and principles of the present application should be equivalent replacement methods and should be included in the protection scope of the present application.

Claims

1. A visuo-haptics fusion aerial robot teleoperation system, characterized in that, The system comprises: a master end comprising a joystick signal acquisition module, a tactile feedback module, a visual feedback module and a signal processing module, wherein the signal processing module is connected with the joystick signal acquisition module, the tactile feedback module and the visual feedback module; a control strategy module; a network communication module; a slave end comprising a leader and a follower, the leader being provided with a positioning module, and the follower being an aerial robot; the aerial robot comprising a motion control module, a positioning mapping module, a motion planning module and a visual acquisition module; the positioning module being connected with the network communication module and the motion planning module; wherein the joystick signal acquisition module acquires the rotation amount of the rotating shaft of the mobile tactile feedback device joystick and corresponding time information, and outputs the tactile feedback device signal to the signal processing module; the tactile feedback module receives the expected tactile feedback device joystick displacement amount signal calculated by the signal processing module, controls the tactile feedback device joystick to render the corresponding tactile sensation, and assists in controlling the movement of the tactile device joystick; the signal processing module receives the information of the joystick signal acquisition module and processes it into an aerial robot speed control instruction; receives the follow-up auxiliary control speed signal calculated by the control strategy module and processes it into an expected tactile feedback device joystick displacement amount signal for the tactile feedback module; receives the first perspective image signal acquired by the visual acquisition module of the slave end aerial robot transmitted via the network communication module, and transmits it to the visual feedback module; the visual feedback module displays and outputs the first perspective image signal of the aerial robot transmitted by the signal processing module; the positioning module acquires and calculates the three-dimensional position coordinates of the leader and the follower and corresponding time information; the motion control module receives the positioning information from the positioning mapping module and the final slave end aerial robot speed control instruction calculated by the control strategy module to control the motion of the aerial robot; the positioning mapping module obtains the point cloud map information and positioning information of the slave end aerial robot and outputs them to the motion planning module, and transmits the positioning information to the control strategy module via the network communication module; the motion planning module receives the point cloud map information and positioning information of the slave end aerial robot positioning mapping module, and receives the positioning information of the leader as a follow-up target; in combination with the dynamics model of the slave end aerial robot, it processes to obtain a motion trajectory that meets the dynamics and obstacle avoidance, and transmits the trajectory information to the control strategy module via the network communication module; the visual acquisition module obtains the first perspective image signal from the visual sensor in real time and transmits it to the signal processing module via the network communication module; the control strategy module receives the slave end aerial robot speed control instruction from the signal processing module, the positioning information of the slave end aerial robot and the expected trajectory information, calculates and outputs the final slave end aerial robot speed control instruction to the motion control module, and outputs the follow-up auxiliary control speed signal to the signal processing module. The signal processing module matches the spatial displacement amount of the haptic feedback device joystick end point and the rotation amount of the end rotating shaft as the spatial movement speed instruction and the deflection angular velocity instruction expected by the from-end aerial robot; and calculates the auxiliary resultant force on the haptic feedback device according to the guidance auxiliary force followed by the from-end aerial robot and the repulsion auxiliary force for avoiding collision between the aerial robot formation clusters.

2. The visuo-haptics fused aerial robotic teleoperation system of claim 1, wherein, The positioning module is a UWB positioning tag or a motion capture system tag.

3. The visuo-haptics fused aerial robotic teleoperation system of claim 1, wherein, The aerial robot further comprises a special task module connected with the motion control module, and the special task module is a sensor and / or an actuator mounted on the aerial robot.

4. The visuo-haptics fused aerial robotic teleoperation system of claim 1, wherein, The positioning mapping module obtains three-dimensional point cloud map information, robot three-dimensional positioning coordinate information and robot quaternion attitude information through a simultaneous localization and mapping algorithm.

5. The visuo-haptics fused aerial robotic teleoperation system of claim 1, wherein, The matching formula of the spatial movement speed instruction and the deflection angular velocity instruction is: wherein the spatial movement velocity command and the deflection angle velocity command of the end-of-arm aerial robot are expected from the end-of-arm aerial robot represents the linear velocity command of the end-of-arm aerial robot, ω z,com represents the deflection angle velocity command, P HD represents the displacement of the haptic feedback device joystick end position, Ψ HD represents the rotation amount of the haptic feedback device joystick end rotation axis, and the coefficient α is a displacement-velocity matching parameter.

6. The visuo-haptics fused aerial robotic teleoperation system of claim 1, wherein, The calculation formula of the auxiliary resultant force on the haptic feedback device is: F = F at + F re where F at is the guidance assistance force from the aerial robot follower, F re is the repulsion assistance force for aerial robot formation flocking collision avoidance, and F is the assistance resultant force generated by the haptic feedback device.

7. The visuo-haptics fused aerial robotic teleoperation system of claim 6, wherein, The guidance assist force F from the endo-aerial robot following at Breaks down into: F at = F ⊥ + F ‖ where F ⊥ is a regression assist force in the perpendicular direction to the trajectory calculated by the motion planning module, F ‖ is a follow assist force in the forward direction to the trajectory calculated by the motion planning module; The regression auxiliary force F ⊥ The calculation formula is: K HD,P a spring coefficient of the haptic feedback device joystick HD,D a damping coefficient of the haptic feedback device joystick HD a displacement representing the end position of the haptic feedback device joystick a spatial velocity of the end aerial robot desired by the motion planning module defined in the end aerial robot body coordinate system O B,F HD a moving speed of the end of the haptic feedback device joystick​ The following assistance force F ‖ The calculation formula is: The velocity vector of the end-effector in the trajectory direction from the aerial robot body coordinate system O B,F The velocity vector of the end-effector in the trajectory direction from the aerial robot body coordinate system O 8. The visuo-haptics fused aerial robotic teleoperation system of claim 6, wherein, repulsive assistance force F from the endo-aerial robot re The calculation method is: A speed obstacle is constructed for each intelligent agent except the current aerial robot, and finally the speed obstacle space of the current aerial robot is the superposition of each speed obstacle VO; For each speed obstacle VO, the current aerial robot is reduced to a mass point with a radius of 0, and the remaining intelligent agents including the leader are inflated into a ball; Let denotes the aerial robot shrinking into a point, denotes the leader inflating into a sphere, denotes the velocity of the aerial robot, denotes the velocity of the leader, the collision cone of the velocity obstacle VO F|L denotes that: Optimizing on velocity obstacles VO, on the basis of collision cones VO' F|L setting a feasible region, so that the control strategy only concerns collisions in a future time T h The feasible region VO H is: where d m airborne robot shrunk mass point ball inflated to leader the nearest distance of the surface; the optimized velocity obstacle VO space region is: The direction of the repulsive assistance force F re is defined as The direction of the aerial robot velocity The direction of the velocity obstacle spatial region The direction of the surface on which the repulsive assistance force F re is calculated as where F set is a suitable force mode length value adjusted by a human.

Images