Teleoperation control system and method based on isomorphic force feedback hand controller

By introducing a combination of isomorphic force feedback manual and robotic arm in the remote operating system, a variety of control methods are provided, which solves the problem of limited information transmission in remote operation, improves the accuracy and safety of operation, and is suitable for Tasks in complex and dynamic environments.

CN118977244AActive Publication Date: 2024-11-19SOUTHEAST UNIV

Patent Information

Application Number
CN202411197596.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-19
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The existing remote operation technology has limited information transmission in complex and dynamic environments, which makes it impossible for operators to accurately judge obstacles, spatial distance or equipment status in the environment, which in turn causes operation errors, equipment damage or task failure, especially in dangerous environments. Increase accident risk, affecting the success rate of the task and the overall reliability of the operation.

Method used

The remote operation control system based on isomorphic force feedback manual control is adopted. Through the combination of the master and slave, the force feedback manual control and the robot arm are used for position control, providing rough joint adjustment, fine joint adjustment and end control. The force sensor coordinate system coincides with the end coordinate system of the force feedback hand controller to achieve feedback and mapping of the force information collected by the end force sensor of the robot arm, and enhance the operator's perception and control ability.

Benefits of technology

Improve the adaptability of remote operating systems and efficiency and accuracy during operation, increase the operator's sense of presence and reality, reduce operation difficulty, enhance safety and reliability in complex tasks, and avoid the limitations of a single control method , suitable for complex job tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118977244A_ABST
    Figure CN118977244A_ABST
Patent Text Reader

Abstract

The invention discloses a teleoperation control system and method based on an isomorphic force feedback hand controller. The teleoperation control system is composed of a master end and a slave end. The main end at least comprises an interactive interface and a force feedback hand controller; the slave end at least comprises a mechanical arm, a camera and a force sensor; the force feedback hand controller and the mechanical arm are isomorphic; force mapping from the slave-end mechanical arm to the master-end force feedback hand controller is set, stress information collected by a force sensor at the tail end of the mechanical arm is fed back to the force feedback hand controller, and position control is conducted on the slave-end mechanical arm by operating the master-end hand controller or an interactive interface; the position control comprises joint coarse adjustment control, joint fine adjustment control and tail end control; the coordinate system of the force sensor coincides with the coordinate system of the tail end of the force feedback hand controller. According to the method, rapid movement and fine operation can be simultaneously met through three control modes, the adaptability of the teleoperation system and the efficiency and the accuracy in the operation process are improved, and the immediacy sense and the reality sense of an operator are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of robot teleoperation control, and mainly relates to a teleoperation control system and method based on an isomorphic force feedback hand controller. Background Art

[0002] In recent years, teleoperation technology has been increasingly used in various fields. Autonomous robots face many challenges in dealing with complex and dynamic environments, such as perception errors and decision uncertainty. However, teleoperation technology makes up for the shortcomings of autonomous robots in these aspects through human-machine collaboration. Teleoperation allows human operators to control robots in real time in a remote and safe environment, directly intervene in the execution of complex tasks, and ensure accuracy and flexibility. Teleoperation technology can quickly adapt to changing environments and uncertain conditions, and achieve tasks that are difficult for autonomous robots to complete. At the same time, teleoperation technology can improve the safety and efficiency of operations. It enables people to operate complex equipment in a remote and safe environment, and perform tasks in dangerous or extreme environments such as deep sea, space, and nuclear energy, thereby avoiding personnel risks. At the same time, teleoperation technology is also widely used in other fields such as medical and power operations. Operators improve work efficiency and safety by remotely operating robots. Teleoperation technology not only promotes scientific and technological progress and innovation, but also brings significant economic and social benefits, and improves people's lives and working methods.

[0003] During teleoperation, due to the physical isolation between the operator and the remote environment and limited information transmission, the operator may lack a comprehensive understanding of the environment. This lack of cognition may cause the operator to be unable to accurately judge obstacles, spatial distances or equipment status in the environment, which may lead to operational errors, equipment damage or mission failure. Especially in dangerous environments or high-precision tasks, insufficient cognition may also bring safety hazards, increase the risk of accidents, and affect the success rate of the mission and the overall reliability of the operation. Therefore, how to increase information feedback during teleoperation is an important research topic.

[0004] In robot control and teleoperation, joint angle space and Cartesian space are two commonly used description and control methods. Joint angle space refers to the set of angles or positions of each joint of the robot, and joint angle control directly controls the angle of each joint of the robot. Cartesian space refers to the position and posture of the robot's end effector in three-dimensional space, and end control directly controls the position and posture of the robot's end effector. Single joint angle control method and end control each have certain limitations and cannot be universally applied to complex teleoperation tasks. Although joint angle control is simple and real-time, it is not intuitive in complex path planning, and it is difficult for operators to recognize. It cannot be applied to operations in dynamic environments and complex paths. End control operation is intuitive and flexible, but it requires inverse kinematics calculations, and the control algorithm is relatively complex. It has obvious defects in tasks that require high-precision and repetitive operations. Summary of the invention

[0005] The present invention is aimed at the problem that the existing technology is difficult to be generally applied to complex remote operation tasks, and provides a remote operation control system and method based on an isomorphic force feedback hand controller, which is composed of a master end and a slave end; the master end at least includes an interactive interface and a force feedback hand controller; the slave end at least includes a mechanical arm, a camera and a force sensor; the force feedback hand controller and the mechanical arm are isomorphic; a force mapping is set from the mechanical arm of the slave end to the force feedback hand controller of the master end, and the force information collected by the force sensor at the end of the mechanical arm is fed back to the force feedback hand controller, and the position of the mechanical arm of the slave end is controlled by operating the hand controller of the master end or the interactive interface; the position control includes coarse joint control, fine joint control and end control; the force sensor coordinate system and the force feedback hand controller end coordinate system coincide. The present invention can simultaneously meet the requirements of fast movement and fine operation through three control methods, improve the adaptability of the remote operation system and the efficiency and accuracy during the operation process, and increase the operator's sense of presence and reality.

[0006] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is: a remote operation control system based on an isomorphic force feedback hand controller, which is composed of a master end and a slave end;

[0007] The master end at least comprises an interactive interface and a force feedback hand controller; the interactive interface is used to fine-tune joint control and display the current state of the robot arm, and the force feedback hand controller is used for the operator to remotely control the robot and provide force feedback;

[0008] The slave end at least comprises a robotic arm, a camera and a force sensor; the robotic arm is used to perform operations, the camera is used to obtain a global view, and the force sensor is used to obtain interaction force information between the robotic arm and the environment;

[0009] Among them, the force feedback hand controller and the robotic arm are isomorphic.

[0010] In order to achieve the above-mentioned purpose, the technical solution also adopted by the present invention is: a remote operation control method based on an isomorphic force feedback hand controller, setting a force mapping from a slave-end robotic arm to a master-end force feedback hand controller, feeding back the force information collected by the force sensor at the end of the robotic arm to the force feedback hand controller, and performing position control on the slave-end robotic arm by operating the master-end hand controller or the interactive interface; the position control includes coarse joint control, fine joint control and end control; wherein the force sensor coordinate system coincides with the end coordinate system of the force feedback hand controller.

[0011] As an improvement of the present invention, the following steps are executed cyclically in each time step to complete the force mapping from the slave end manipulator to the master end force feedback hand controller:

[0012] A1: Read the current sensor posture, that is, the end posture R of the robot arm sc , and read the output F of the sensor in its own coordinate system s ;

[0013] A2: Calculate the transformation matrix of the sensor coordinate system relative to the hand controller coordinate system is the transformation matrix of the robot base coordinate system relative to the hand controller base coordinate system;

[0014] A3: Convert the force information to the hand controller coordinate system.

[0015] A4: F m The feedback force F is obtained through the nonlinear mapping function t , and sent to the force feedback hand controller.

[0016] As an improvement of the present invention, in the coarse joint control, the changes of the joint angles of the force feedback hand controller are mapped to the joint angles of the slave end mechanical arm;

[0017] In the fine-tuning joint control, the value of a single joint angle is changed by clicking the plus and minus buttons of each joint in the interactive interface;

[0018] In the end control, the relative position change of the end of the force feedback hand controller is mapped to the end of the slave end robot arm;

[0019] Set the control mode S c , when the control mode S c =1, the operator's current control mode is the coarse joint control mode; when the control mode S c =0, the operator's current control mode is the terminal control mode; the coarse joint control and terminal control are incrementally controlled by the buttons on the force feedback hand controller; set the button state S b , button status S b=1 means the button is pressed and the robot arm is controlled by the operator; button status S b =0 means that the button is not pressed, the robot arm is not controlled by the hand controller, the coarse joint control and the end control are ineffective, and the operator can move the hand controller freely; the fine joint control can selectively work simultaneously with the coarse joint control or the end control.

[0020] As another improvement of the present invention, the specific method of the coarse joint control is as follows:

[0021] S1: Read the current force feedback hand controller joint angle A mc and the current robot arm joint angle information A sc ;

[0022] S2: Button status S b Make a judgment, when S b = 0, no operation is performed, and steps S1 and S2 are looped; when the key is pressed, S b When it changes from 0 to 1, execute step S3a; when S b =1, execute S3b;

[0023] S3a: Record the current hand controller joint angle A mc and the current robot arm joint angle information A sc , respectively, are the initial hand controller joint angle A m0 and the initial robot arm joint angle information A s0 , that is, A m0 =A mc ,A s0 =A sc , return to step S1 and loop;

[0024] S3b: Calculate the hand controller joint angle increment ΔA=A according to the current hand controller joint angle and the initial hand controller joint angle mc -A m0 ;

[0025] S4: Calculate the target joint angle A of the robot arm based on the joint angle increment of the hand controller and the current joint angle of the robot arm st =A s0 +ΔA*k A , where k A It is the angle scaling factor and is sent to the robot arm to make the robot arm joints follow the movement of the hand controller.

[0026] As another improvement of the present invention, the fine-tuning joint control specifically comprises the following steps:

[0027] S1': Read the current joint angle information of the robot arm A sc ;

[0028] S2': Obtain the operator's operation buttons on the interactive interface, determine the target robot arm joint i and the corresponding angle increment as ±Δα;

[0029] S3': According to the current robot arm joint angle and the corresponding angle increment, add them to get the robot arm target joint angle A st , and sent to the robotic arm.

[0030] As another improvement of the present invention, the terminal control specifically includes the following steps:

[0031] S1": Read the position X of the current force feedback hand controller end relative to the force feedback hand controller base coordinate system mc and posture R mc , the current position of the end of the robot relative to the robot base coordinate system X sc and posture R sc ;

[0032] S2": for key status S b Make a judgment, when S b = 0, no operation is performed, and steps S1 and S2 are looped; when the key is pressed, S b When it changes from 0 to 1, execute S3a”; when S b =1, execute S3b";

[0033] S3a”: Record the current hand controller end position and the current robot arm end position, which are the initial hand controller end position and the initial robot arm end position, i.e. X m0 =X mc ,R m0 =R mc ,X s0 =X sc ,R s0 =R sc , return to step S1″ for loop operation;

[0034] S3b”: Calculate the current change in the end position of the hand controller ΔX m and attitude change ΔR m ;

[0035] S4”: Change the end position of the hand controller by ΔX m Transform to the robot base coordinate system The hand controller end posture change ΔR m Transform to the robot base coordinate system through similarity transformation

[0036] S5”: Calculate the target position X of the robot arm according to the change in the robot arm posture and the initial posture st =X s0 +ΔXs *k x , where k x is the displacement scaling factor, and the target posture of the robot arm R st =ΔR s *R s0 ;

[0037] S6”: According to the target position of the robot arm, use the analytical method to perform inverse kinematics analysis to solve the target joint angle A of the robot arm st , and sent to the robotic arm.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) The present invention discloses a remote control system and method based on an isomorphic force feedback hand controller, designs different master-slave mapping methods, and simultaneously meets the operational requirements of rapid movement and fine operation of the robotic arm, improves the applicability and flexibility of the remote control system, avoids the limitations of single joint angle control and end control, and can be applied to complex operational tasks to achieve efficient and accurate remote operation.

[0040] (2) The present invention is based on an isomorphic hand controller, which is highly consistent with the controlled robotic arm in structure and kinematics, making the operation more intuitive and natural. The operator can control the robotic arm in a manner similar to his own limb movements, which reduces the difficulty of learning and operation and improves the accuracy and efficiency of operation.

[0041] (3) The present invention introduces force feedback mapping to enhance the telepresence of the remote operation system and broaden the dimension of human-machine interaction. The introduction of force feedback can more accurately simulate the actual operation scenario, enhance the operator's perception and control ability of the remote environment, and thus provide higher safety and reliability in complex tasks.

[0042] (4) The present invention realizes incremental control by force controlling the hand controller button, thereby increasing the motion space and control accuracy of the control end. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a control principle diagram of the remote operation control method based on the isomorphic force feedback hand controller of the present invention;

[0044] Figure 2 is a flow chart of the steps of the coarse joint control mode in the method of the present invention;

[0045] Figure 3 is a flow chart of the steps of fine-tuning the joint control mode in the method of the present invention;

[0046] Figure 4 It is an interactive graphical interface diagram used for fine-tuning joint control of the present invention;

[0047] Figure 5 is a flow chart of steps in the terminal control mode of the present invention;

[0048] Figure 6 is a schematic structural diagram of the robot arm Universal Robot 10 used in the present invention;

[0049] Figure 7 It is a flow chart of the steps of force feedback mapping in the method of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0051] Example 1

[0052] A teleoperation control system based on an isomorphic force feedback hand controller is suitable for situations where both rapid movement and precise adjustment are required. It consists of a master end and a slave end. The master end includes an interactive interface and a force feedback hand controller, and the slave end includes a robotic arm, a camera, and a force sensor. The hand controller and the robotic arm are isomorphic. During the operation, the operator performs teleoperation based on a global field of view, which is provided by a camera.

[0053] The slave robot arm is controlled by operating the master hand controller or the interactive interface, which specifically includes three control modes: coarse joint control, fine joint control and end control. Fine joint control is achieved through the buttons on the interactive interface, while coarse joint control and end control use the master hand controller. At the same time, the force mapping of the slave robot arm is fed back to the master hand controller.

[0054] Based on the teleoperation control method of the isomorphic force feedback hand controller, the operator can choose to use coarse joint control or end control through the interactive interface. Fine joint control can act simultaneously with coarse joint control or end control. End control is used in scenarios where fast movement is required, while joint control is used when accurate control of the robot arm is required, that is, coarse joint control and fine joint control.

[0055] Example 2

[0056] The control principle of the teleoperation control method based on isomorphic force feedback hand controller is as follows: Figure 1 In the coarse joint control mode, the operator controls the main hand controller to change the angle value A of each joint. m The computer multiplies the joint angle change by the angle scaling factor k A As the slave robot joint angle increment; if the operator sends a fine-tuning control command for a specific joint angle through the graphical interface, the fine-tuning increment is added on the basis of the increment, and the new slave robot joint angle value A after the increment is introduced stSent to the slave robot via the communication network.

[0057] In the end control mode, the operator controls the main hand control to change its end position X m and posture R m The computer maps the position change and posture change of the end of the hand controller to the base coordinate system of the slave robot, and then superimposes them with the initial posture of the robot to obtain the target posture of the robot. The new target posture of the slave robot after the increment is analyzed by inverse kinematics to obtain the joint angle value A. st ; If the operator sends a fine-tuning control instruction for a specific joint angle through the graphical interface, the fine-tuning increment is added and sent to the slave robot arm through the communication network.

[0058] The force of the slave manipulator is mapped to the master hand controller. The force sensor coordinate system coincides with the hand controller end coordinate system, that is, it changes with the hand controller end position. After reading the current sensor output, the rotation matrix between the sensor coordinate system and the hand controller base coordinate system is calculated to transform the force information into the hand controller coordinate system to represent F m , and then the feedback force F is obtained through the nonlinear mapping function t , and sent to the force feedback hand controller.

[0059] In this embodiment, the 3D SYSTEMS Touch hand controller and Universal Robot 10 are used as slave robotic arms, and the ROS robot operating system is used. When sending instructions to control the robotic arms, the joint angle controller is used uniformly. The hand controller and the robotic arm are isomorphic, have the same structural configuration, and both have six degrees of freedom.

[0060] The operator selects to use coarse joint control or end control mode through the interactive interface. Fine joint control can be used simultaneously with coarse joint control or end control. End control is used in scenarios where fast movement is required, while joint control is used when accurate control of the robot arm is required. Set control mode S c When the control mode S c =1, the operator's current control mode is the coarse joint control mode. c When =0, the operator’s current control mode is the terminal control mode.

[0061] Coarse joint control and end control rely on incremental control on the buttons on the hand controller to increase the hand controller's workspace. The control amount is only related to the relative change after pressing the hand controller button, and has nothing to do with the current absolute position and joint angle state. Set button state S b , button status S b =1 means the button is pressed. At this time, the robot arm is controlled by the hand controller. The button status S b=0 means that the button is not pressed. At this time, the robot arm is not controlled by the hand controller, and the operator can freely move the hand controller to a comfortable position.

[0062] Joint angle control can precisely control each joint and provide high-precision operation. Joint control has a clearer control effect on posture. Coarse joint control and incremental control map the change in the joint angle of the isomorphic hand controller to the joint space of the robot arm. The following steps are included: Figure 2 As shown:

[0063] S1: Read the current hand controller joint angle A through the hand controller interface and the robot arm interface mc , and the current robot arm joint angle information A sc , A mc and A sc They are all six-dimensional vectors;

[0064] S2: Current button status of the hand controller S b Make a judgment, when S b = 0, no operation is performed, and S1 and S2 are looped; when S b When it changes from 0 to 1, that is, the moment the key is pressed, step S3a is executed; when S b =1, execute S3b;

[0065] S3a: Record the current hand controller joint angle A mc and the current robot arm joint angle information A sc , respectively, are the initial hand controller joint angle A m0 and the initial robot arm joint angle information A s0 , that is, A m0 =A mc ,A s0 =A sc , return to step S1 and loop;

[0066] S3b: Calculate the hand controller joint angle increment ΔA=A according to the current hand controller joint angle and the initial hand controller joint angle mc -A m0 ;

[0067] S4: Calculate the target joint angle A of the robot arm based on the joint angle increment of the hand controller and the current joint angle of the robot arm st =A s0 +ΔA*k A , where k A is the angle scaling factor; the target joint angle is sent to the robot arm so that the robot arm joints follow the movement of the hand controller. To maintain accuracy, k A Set to less than 1.

[0068] Fine-tuning joint control can further achieve more precise control based on coarse-tuning joint control. The operator sets the target angle of the robot arm's specified joint through the buttons on the interactive interface. Figure 4 As shown in the figure, the addition and subtraction of each joint on the interface is used to set the joint change amount. Set the angle change amount Δα corresponding to each button click in advance, which specifically includes the following steps, such as Figure 3 As shown:

[0069] S1': Read the current joint angle information of the robot arm A sc .

[0070] S2': Obtain the operator's operation buttons on the interactive interface, determine the target robot arm joint i, and the corresponding angle increment is ±Δα.

[0071] S3': Calculate the target joint angle A of the robot arm based on the current robot arm joint angle and the corresponding angle increment st , and sent to the robotic arm.

[0072] Assume that there are six robotic arm joints in total, and the current robotic arm joint angles are 10°, 20°, 0°, 0°, 0°, 0°. If the robotic arm joint 2 adds 5°, then the target joint angles are 10°, 25°, 0°, 0°, 0°, 0°.

[0073] End control is more intuitive, but the overall posture of the robot arm is not unique. End control and incremental control specifically map the posture change of the hand controller end to the robot arm end coordinate system, where the coordinate system mapping relationship is fixed and does not change with the change of the robot arm end posture. In this embodiment, the hand controller base coordinate system is set as M, and the robot arm base coordinate system is set as S. The system involves the hand controller base coordinate system M and the robot arm base coordinate system S, and uses Represents the transformation matrix of the robot base coordinate system relative to the hand controller base coordinate system. The system involves the hand controller base coordinate system M and the robot base coordinate system S. Represents the transformation matrix of the robot base coordinate system relative to the hand controller base coordinate system. Figure 5 As shown, the following steps are performed in each time step loop:

[0074] In this embodiment, the robot arm base coordinate system is obtained by rotating 45 degrees around the Z axis of the hand controller base coordinate system. At this time:

[0075]

[0076] The terminal control mode includes the following steps:

[0077] S1”: Read the position X of the current hand controller end relative to the hand controller base coordinate system through the hand controller interface and the robot arm interface mc and posture R mc, the current position of the end of the robot relative to the robot base coordinate system X sc and posture R sc ; To unify with the coarse joint control mode, the end position of the robot arm can also be obtained by reading the joint angle of the robot arm and then solving the kinematics positive solution of UR10;

[0078] S2": for key status S b Make a judgment, when S b = 0, no operation is performed, and S1" and S2" are looped; when S b When it changes from 0 to 1, that is, when the key is pressed, S3a is executed; when S b =1, execute S3b";

[0079] S3a”: Record the current hand controller end position and the current robot arm end position, which are the initial hand controller end position and the initial robot arm end position, i.e. X m0 =X mc ,R m0 =R mc ,X s0 =X sc ,R s0 =R sc , return to step S1" and repeat the operation;

[0080] S3b”: Calculate the current change in the end position of the hand controller ΔX m and attitude change ΔR m ; For the hand controller position change ΔX m =X mc -X m0 , for the hand controller posture change ΔR m =R m0 -1 *R mc ;

[0081] S4”: Change the end position of the hand controller by ΔX m Transform to the robot base coordinate system The hand controller end posture change ΔR m Transform to the robot base coordinate system through similarity transformation

[0082] S5”: Calculate the target position X of the robot arm according to the change in the robot arm posture and the initial posture st =X s0 +ΔX s *k, where k is the displacement scaling factor. Since the hand controller’s working space is much smaller than the robot’s motion space, k is set to 5. The robot’s target posture R st =ΔR s *Rs0 ;

[0083] S6": Get the target position X from the robot arm st and R st Finally, for a robot arm without redundant degrees of freedom, an analytical method is often used to directly calculate the joint angle values. The solution process is a typical nonlinear equation solution process.

[0084] This example uses Universal Robot 10 as the slave robot arm. The coordinate relationship and structure diagram of each joint are as follows: Figure 6 Assume that the target pose matrix is

[0085]

[0086] Combining the robot arm structure and analytical method, the joint angle values ​​from the base to the end are θ1, θ2, θ3, θ4, θ5, and θ6 respectively.

[0087]

[0088] θ5=atan2(s5,ax×s1-ay×c1)

[0089]

[0090] in, A=2a 2 E, B = 2a 2 D, C = D 2 +E 2 +a2 2 +a3 2 , D=px×c1+py×s1+d5×s 234 , E=pz–d1+d5×c 234 ,s 234 = sin(θ2+θ3+θ4), c 234 =cos(θ2+θ3+θ4).

[0091] The solution set of nonlinear equations is not unique. Due to the existence of invalid solution sets caused by singular points and other problems, the number of valid solution sets varies with the input, and the optimal solution needs to be screened out. According to the analytical method, the n solution sets are obtained as q i (i=1,2,...,n), the current joint angle of the slave robot is A s , select a set of solutions i with the smallest sum of the absolute values ​​of the differences between the joint angles of the current robot arm * As the new joint angle, and send it to the robot arm.

[0092]

[0093] In the overall teleoperation process, force mapping from the end manipulator to the master hand controller is set, and the force information collected by the end force sensor of the manipulator is fed back to the force feedback hand controller. Among them, the force sensor coordinate system coincides with the end coordinate system of the hand controller, that is, it changes with the end position of the hand controller. The following steps are executed in each time step loop, such as Figure 7 As shown:

[0094] A1: Read the current sensor posture, that is, the end posture R of the robot arm sc , and read the output F of the sensor in its own coordinate system s ;

[0095] A2: Calculate the transformation matrix of the sensor coordinate system relative to the hand controller coordinate system

[0096] A3: Convert the force information to the hand controller coordinate system.

[0097] A4: F m The feedback force F is obtained through the nonlinear mapping function t , and send it to the force feedback hand controller; calculate F m The resultant force F magnitude , after a nonlinear function Get new combined force F new , through the nonlinear function, the fluctuation caused by zero drift is eliminated and the upper limit of the force feedback force is 2N. Calculate the force proportionality coefficient k ratio =F new / F magnitude The original three-dimensional force is scaled to obtain the force feedback F acting on the hand controller. t =F m *k ratio Finally, F t Acts on the hand control.

[0098] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A teleoperation control system based on an isomorphic force feedback hand controller, characterized in that : Consists of a master end and a slave end; The master end at least comprises an interactive interface and a force feedback hand controller; the interactive interface is used to fine-tune joint control and display the current state of the robot arm, and the force feedback hand controller is used for the operator to remotely control the robot and provide force feedback; The slave end at least comprises a robotic arm, a camera and a force sensor; the robotic arm is used to perform operations, the camera is used to obtain a global view, and the force sensor is used to obtain interaction force information between the robotic arm and the environment; Among them, the force feedback hand controller and the robotic arm are isomorphic.

2. A remote operation control method based on an isomorphic force feedback hand controller using the system as claimed in claim 1, characterized in that: A force mapping is set from a slave-end robot arm to a master-end force feedback hand controller, and the force information collected by the force sensor at the end of the robot arm is fed back to the force feedback hand controller. The position of the slave-end robot arm is controlled by operating the master-end hand controller or the interactive interface; the position control includes coarse joint control, fine joint control and end control; wherein the force sensor coordinate system coincides with the end coordinate system of the force feedback hand controller.

3. A remote operation control method based on a homogeneous force feedback hand controller as claimed in claim 2, characterized in that: The following steps are executed in each time step loop to complete the force mapping from the end manipulator to the master end force feedback hand controller: A1: Read the current sensor posture, that is, the end posture R of the robot arm sc , and read the output F of the sensor in its own coordinate system s ; A2: Calculate the transformation matrix of the sensor coordinate system relative to the hand controller coordinate system is the transformation matrix of the robot base coordinate system relative to the hand controller base coordinate system; A3: Convert the force information to the hand controller coordinate system. A4: F m The feedback force F is obtained through the nonlinear mapping function t , and sent to the force feedback hand controller.

4. A remote operation control method based on a homogeneous force feedback hand controller as claimed in claim 3, characterized in that: In the coarse joint control, the changes of the joint angles of the force feedback hand controller are mapped to the joint angles of the slave end robot arm; In the fine-tuning joint control, the value of a single joint angle is changed by clicking the plus and minus buttons of each joint in the interactive interface; In the end control, the relative position change of the end of the force feedback hand controller is mapped to the end of the slave end robot arm; Set the control mode S c , when the control mode S c =1, the operator's current control mode is the coarse joint control mode; when the control mode S c =0, the operator's current control mode is the terminal control mode; the coarse joint control and terminal control are incrementally controlled by the buttons on the force feedback hand controller; set the button state S b , button status S b =1 means the button is pressed and the robot arm is controlled by the operator; button status S b =0 means that the button is not pressed, the robot arm is not controlled by the hand controller, the coarse joint control and the end control are ineffective, and the operator can move the hand controller freely; the fine joint control can selectively work simultaneously with the coarse joint control or the end control.

5. A remote operation control method based on a homogeneous force feedback hand controller as claimed in claim 4, characterized in that: The specific method of the coarse joint control is as follows: S1: Read the current force feedback hand controller joint angle A mc and the current robot arm joint angle information A sc ; S2: For button status S b Make a judgment, when S b = 0, no operation is performed, and steps S1 and S2 are looped; when the key is pressed, S b When it changes from 0 to 1, execute step S3a; when S b =1, execute S3b; S3a: Record the current hand controller joint angle A mc and the current robot arm joint angle information A sc , respectively, are the initial hand controller joint angle A m0 and the initial robot arm joint angle information A s0 , that is, A m0 =A mc , A s0 =A sc , return to step S1 and loop; S3b: Calculate the hand controller joint angle increment ΔA=A according to the hand controller current joint angle and the initial hand controller joint angle mc -A m0 ; S4: Calculate the target joint angle A of the robot arm based on the joint angle increment of the hand controller and the current joint angle of the robot arm st =A s0 +ΔA*k A , where k A It is the angle scaling factor and is sent to the robot arm to make the robot arm joints follow the movement of the hand controller.

6. A remote operation control method based on a homogeneous force feedback hand controller as claimed in claim 4, characterized in that: The fine-tuning joint control specifically comprises the following steps: S1′: Read the current joint angle information of the robot arm A sc ; S2′: Obtain the operator's operation buttons on the interactive interface, determine the target robot arm joint i and the corresponding angle increment as ±Δα; S3′: According to the current robot arm joint angle and the corresponding angle increment, add them together to get the robot arm target joint angle A st , and sent to the robotic arm.

7. A remote control system method based on a homogeneous force feedback hand controller as claimed in claim 4, characterized in that: The terminal control specifically includes the following steps: S1″: Read the position X of the current force feedback hand controller end relative to the force feedback hand controller base coordinate system mc and posture R mc , the current position of the end of the robot relative to the robot base coordinate system X sc and posture R sc ; S2″: for key status S b Make a judgment, when S b =0 does not perform any operation, and steps S1 and S2 are looped; when the key is pressed, S b When it changes from 0 to 1, execute S3a″; when S b =1, execute S3b"; S3a″: Record the current hand controller end position and the current robot arm end position, which are the initial hand controller end position and the initial robot arm end position, i.e., X m0 =X mc , R m0 =R mc , X s0 =X sc , R s0 =R sc , return to step S1″ for loop operation; S3b″: Calculate the current change in the end position of the hand controller ΔX m and attitude change ΔR m ; S4″: Change the end position of the hand controller by ΔX m Transform to the robot base coordinate system The hand controller end posture change ΔR m Transform to the robot base coordinate system through similarity transformation S5″: Calculate the target position X of the robot arm according to the change in the robot arm posture and the initial posture st =X s0 +ΔX s *k x , where k x is the displacement scaling factor, and the target posture of the robot arm R st =ΔR s *R s0 ; S6″: According to the target position of the robot arm, use the analytical method to perform inverse kinematics analysis to solve the target joint angle A of the robot arm st , and sent to the robotic arm.

Citation Information

Patent Citations

  • Control system and method for mechanical arm joint

    CN106182002A

  • Operation method of human-machine cooperative ophthalmic microsurgery auxiliary robot system

    CN109602499A

  • Testing device and method for force feedback function of teleoperation equipment

    CN112077816A

  • Multi-mode sharing teleoperation system and method for three-arm space robot

    CN118061176A

  • Testing device for force feedback function of teleoperation equipment

    CN212859444U

Cited By

  • Robot motion control method, system, device, medium and program product

    CN119910660A

  • Master-slave isomorphic teleoperation force feedback control method based on double force sensors

    CN119927900A