A master-slave teleoperation method and system for live working robots based on heterogeneous master hands

Abstract

A master-slave teleoperation method and system for a live working robot based on a heterogeneous master hand. The live working robot is a master-slave heterogeneous robot, and the method includes the following steps: First, obtain the motion information of the heterogeneous master and slave hands respectively; then perform mapping of the motion information of the heterogeneous master and slave hands. Based on the base coordinate system, map and convert the motion of the master hand into the motion of the slave hand. When mapping, decouple the teleoperation, and decompose it into position teleoperation of the ends of the master and slave hands and attitude teleoperation of the master and slave hands; finally, perform motion state monitoring to detect whether the motion of the slave hand is close to a singular position or a limit position. If it is close to a singular position, give an alarm and stop the motion. The method of the present invention is not restricted by the configuration of the master hand, is suitable for the heterogeneous master hand to complete the teleoperation task, and has a high adaptability; optimizes the motion mapping between the heterogeneous master and slave hands through a decoupling method, ensuring that the teleoperation of the heterogeneous master and slave hands is simple, easy to understand, and efficient; the present invention adds motion state monitoring during the teleoperation process, improving the safety of the teleoperation of the heterogeneous master and slave hands.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, relates to a live-working robot and is a master-slave teleoperation control method of the live-working robot based on a heterogeneous master hand. Background Art

[0002] Most of the power distribution operations in the power grid are performed with insulated gloves, requiring the live working team to climb the high-voltage iron tower or use the insulated boom truck to perform uninterrupted maintenance on the high-voltage power devices on the power poles. Manual high-voltage live working requires extremely high quality of workers, and personal injury accidents are very likely to occur if they are not careful. In response to this phenomenon, the existing technology has proposed a live working robot, a special robot for distribution network line operations, which uses robots to replace manual work to complete live working.

[0003] When the live working robot is working, the operator monitors the working scene and remotely controls the robot to ensure that the operator is isolated from the high voltage electric field. There are two main remote control schemes for live working robots:

[0004] 1. Remote control based on three degrees of freedom uses a visual tablet or remote control for remote control. The remote control is pushed in the specified direction, and the algorithm identifies the push direction of the remote control and sends a movement command to the robotic arm. In this solution, the tablet or remote control has fewer degrees of freedom than the robot body, and can only remotely control the robot for simple three-degree-of-freedom position movement. Since the robotic arm often has six degrees of freedom, the three-degree-of-freedom remote control often cannot accurately control the rotation of the robotic arm and cannot meet the operational requirements.

[0005] 2. Based on teleoperation of isomorphic master hands, the master hand remotely controls the slave hand to perform operations, and the degrees of freedom are synchronized. This often requires the master hand and the slave hand to have the same mechanical configuration and the same reachable space of the joint angle. When the master hand moves, the movement of the master hand is calculated as the change of each joint angle, which is mapped one-to-one to the joint angle of the slave hand. This solution has high requirements for the master hand and requires a customized master hand that is proportionally smaller than the slave hand. The master hand is difficult to obtain and expensive, and the algorithm has poor versatility.

[0006] Therefore, there is an urgent need to provide a new master-slave remote control algorithm method for live working robots to solve the above problems. Summary of the invention

[0007] The technical problem to be solved by the present invention is: the existing remote control method of live working robots has insufficient operating freedom of ordinary remote controllers. Although the master-slave hand solution ensures accurate freedom, it has high requirements on the master hand, and each pair of master-slave hands requires a corresponding control algorithm, which has low versatility and increases costs.

[0008] The technical solution of the present invention is as follows: A master-slave teleoperation method for an on-line working robot based on a heterogeneous master hand. The on-line working robot is a master-slave heterogeneous robot, and the method includes the following steps:

[0009] 1) Obtaining the motion information of the heterogeneous master-slave hands: By remotely operating the slave hand with the master hand, the motion information of the master-slave hands is obtained, including the end pose and end motion speed of the master hand, as well as the end pose and the angle information of each joint of the slave hand. The end poses of the master hand and the slave hand include the end position and rotation information;

[0010] 2) Mapping the motion information of the heterogeneous master-slave hands: Based on the motion information of the master-slave hands, the motion of the master hand is mapped and converted into the motion of the slave hand. When mapping, the teleoperation is decoupled and decomposed into the position teleoperation of the ends of the master-slave hands and the pose teleoperation of the master-slave hands;

[0011] 3) Monitoring the motion state: After the motion of the master hand is mapped and converted into the motion of the slave hand, the motion information is sent to the slave hand, and it is detected in real time whether the motion of the slave hand is approaching the singular position or the limit position. If it is approaching the singular position, the motion is stopped.

[0012] The present invention also provides a master-slave teleoperation system for an on-line working robot based on a heterogeneous master hand, including an interaction subsystem, a perception subsystem, a control subsystem, a computing subsystem, and a communication subsystem.

[0013] The interaction subsystem includes a human-machine interaction device for human-machine interaction for teleoperation;

[0014] The perception subsystem includes a robot sensor for collecting information on the robot operation site;

[0015] The control subsystem includes a robot actuator device for controlling the motion of the robot and collecting the motion information of the robot;

[0016] The computing subsystem includes a deployable computing platform and software modules. The software modules store computer programs, and when executed, implement the above-mentioned teleoperation method;

[0017] The communication subsystem includes network communication devices for communication between the master and slave hands of the robot.

[0018] The present invention provides a teleoperation method based on a heterogeneous master-slave hand. The master hand and the slave manipulator both have six degrees of freedom, enabling remote control operations to get rid of the remote controller with fewer degrees of freedom and allowing the charged operation robot to move and rotate in any direction with high teleoperation sensitivity. In addition, the teleoperation method of the present invention is not restricted by the configuration of the six-degree-of-freedom master hand. The master hand and the slave hand of any configuration perform motion mapping through the method of the present invention to complete the teleoperation task, with high adaptability. Existing master-slave teleoperation algorithms often require the configurations and joint angle reach ranges of the master hand and the slave manipulator to be highly similar or even exactly the same to complete the motion mapping between the master hand and the slave hand. However, the master-slave hand motion mapping proposed by the present invention can be used for master-slave hands with different configurations and different joint reach ranges to complete the remote control operation, getting rid of the requirement that the master-slave hands must be isomorphic. Through a decoupling method, the motion mapping between heterogeneous master-slave hands is optimized to ensure that the teleoperation method of heterogeneous master-slave hands is simple, easy to understand, and efficient. Finally, the present invention adds the monitoring of the motion states of the master-slave hands during the teleoperation process, real-time detects the motion state of the slave hand, and alarms when the posture of the slave hand approaches singularity to ensure that the slave hand stays away from dangerous poses, improving the safety of the teleoperation of heterogeneous master-slave hands. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the teleoperation method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] For the master-slave heterogeneous robot of the present invention, a heterogeneous teleoperation control method and system that are not restricted by the configurations of the master-slave hands and the reachable spaces of each joint are proposed, which are particularly applicable to charged operation robots. As Figure 1 shown, it includes the following steps:

[0021] 1) Acquisition of heterogeneous master-slave hand motion information: The master hand is used to remotely operate the slave hand to obtain the motion information of the master-slave hands, including the end pose and end motion speed of the master hand, as well as the end pose and the angle information of each joint of the slave hand. The end poses of the master hand and the slave hand include the end position and rotation information.

[0022] 2) Mapping of heterogeneous master-slave hand motion information: Based on the motion information of the master-slave hands, the motion mapping of the master hand is converted into the motion of the slave hand. During the mapping, the teleoperation is decoupled and decomposed into the position teleoperation of the master-slave hand ends and the attitude teleoperation of the master-slave hands.

[0023] 3) Motion state monitoring: After the motion of the master hand is converted into the motion of the slave hand through mapping, the motion information is sent to the slave hand, and it is continuously detected whether the motion of the slave hand approaches the singular position or the limit position. If it approaches the singular position, the motion is stopped.

[0024] The present invention is applicable to a live working robot for a heterogeneous master-slave hand. The mechanical structures of the master hand and the slave hand are different, and the reachable ranges of the joint angles are different, thereby reducing the requirements for the master hand and realizing a low-cost and high-precision remotely controlled live working robot for high-altitude live working. The specific implementation manners are described as follows.

[0025] The specific steps for obtaining the motion information of the master-slave hand are as follows:

[0026] 1) Obtain the motion information of the master hand. The motion information of the master hand includes the position information P (Px, Py, Pz) of the master hand end in its base coordinate system Baset, the attitude information R (Rx, Ry, Rz) of the master hand end in its base coordinate system Baset, and the motion speed information V (Vx, Vy, Vz) of the master hand end in the base coordinate system Baset. First, process the position information P (Px, Py, Pz) of the master hand end in its base coordinate system Baset and the attitude information R (Rx, Ry, Rz) of the master hand end in its base coordinate system Baset. Through the RPY to quaternion formula, convert the obtained attitude information R (Rx, Ry, Rz) into a quaternion Q (Qx, Qy, Qz, Qw), and then combine the quaternion Q and the position information P to generate the master hand pose T.

[0027] 2) Use the second-order difference to process the speed information v (vx, vy, vz) of the master hand end to obtain the master hand motion speed information V (Vtx, Vty, Vtz). The specific formula is:

[0028]

[0029] where vel_buff = (position * 3 - 4 * pos_hist1 + pos_hist2) / 0.002. pos_hist2 is the position of the robot end one cycle ago, pos_hist1 is the position of the robot end in the current cycle, inp_vel3 is the second-order difference coefficient of the robot end moving speed two cycles ago, inp_vel2 is the second-order difference coefficient of the robot end moving speed one cycle ago, inp_vel1 is the second-order difference coefficient of the robot end moving speed in the current cycle, out_vel3 is the robot end moving speed two cycles ago, out_vel2 is the robot end moving speed one cycle ago, out_vel1 is the robot end moving speed before the current cycle, and the sampling frequency is 20 Hz. Processing the speed information through the second-order difference can remove speed mutations and obtain stable speed information.

[0030] 3) Obtain the motion information of the slave hand. The motion information of the slave hand includes the position information P’(Px, Py, Pz) of the slave hand end in its base coordinate system Baser, the attitude information R’(Rx, Ry, Rz) of the master hand end in its base coordinate system Baser, and the current joint angle information Theta(q1, q2, q3, q4, q5, q6) of the slave hand.

[0031] In this step, various motion information of the master and slave hands is obtained, and the motion speed information of the master hand is processed to obtain accurate speed information, preparing for the subsequent master-slave hand motion mapping.

[0032] The specific method of master-slave hand motion mapping is as follows. To make the master-slave hand teleoperation simple and easy to operate, first decouple the teleoperation, which is decomposed into master-slave hand position teleoperation and master-slave hand attitude teleoperation. The steps of master-slave hand position teleoperation are as follows:

[0033] 1) Record the position information Pti(initialTouchX, initialTouchY, initialTouchZ) of the master hand end teleoperation starting point in its base coordinate system Baset

[0034] 2) Record the position information Prn(initialRobotX, initialRobotY, initialRobotZ) of the slave hand end teleoperation starting point in its base coordinate system Baser

[0035] 3) Obtain the current position information Ptn(Ptnx, Ptny, Ptnz) of the master hand end in its base coordinate system Baset.

[0036] 4) Subtract Pti from Ptn to obtain the position movement information Tchange(TchangeX, TchangeY, TchangeY) of the master hand end, that is, TchangeX = Ptnx - Ptix, TchangeY = Ptny - Ptiy, TchangeZ = Ptnz - Ptiz.

[0037] 5) Obtain the current position information Prn(Prnx, Prny, Prnz) of the slave hand;

[0038] 6) Compare the velocity values (Vtx, Vty, Vtz) in each direction of the master hand, and take the direction with the maximum velocity value as the current motion direction D. The slave hand target position Prt (Prtx, Prty, PrtZ) in direction D is the master hand position movement information TchangeD plus the initial position of the slave hand at the end of the remote operation in direction D, initialRobotD. The slave hand target positions in other directions are the current slave hand positions. For example, if the master hand movement direction is the X direction, i.e., D = X, then the slave hand target position coordinates are Prtx = TchangeX + initialRobotX, Prty = Prny, Prtz = Prnz. In this way, the decoupling of position movement is achieved, ensuring that each movement is only in one direction, and guaranteeing the operability and intuitiveness of the remote operation.

[0039] 7) Read the current end pose Rrn (Rrnx, Rrny, Rrnz) of the slave hand as the target end pose Rrt (Rrtx, Rrty, Rrtz) of the slave hand, i.e., Rrtx = Rrnx, Rrty = Rrny, Rrtz = Rrnz. Rrt + Prt is the target pose Tt of the slave hand in the current cycle.

[0040] 8) Use the analytical method to solve the target joint angles of the slave hand. To ensure the smooth movement of the robot and avoid the influence of singular positions inside the reachable workspace, plan the robot motion trajectory in the joint space. Therefore, it is necessary to solve the target joint angles Thetat of the robot. Using the analytical method to solve the target joint angles Thetat of the robot, eight groups of robot joint angles Thetat1…Theta8 can be obtained, and each group of joint angles contains six joint angle values (qi1, qi2, qi3, qi4, qi5, qi6), where i = 1, 2, …, 8.

[0041] 9) Solution selection: Since the geometric method obtains eight groups of inverse solutions, solution selection is required. To ensure the smoothness and continuity of the motion and improve the response efficiency of the remote operation, the minimum distance method is used to select the target solution. Read the current joint angles Thetac (qc1, qc2, qc3, qc4, qc5, qc6) of the slave hand, and calculate the difference Differentjoint (d1, d2, d3, d4, d5, d6) between each of the eight groups of solutions and the current joint angles Thetac, simply referred to as the di value:

[0042] di = abs(qi1 - qc1) + abs(qi2 - qc2) + abs(qi3 - qc3) + abs(qi4 - qc4) + abs(qi5 - qc5) + abs(qi6 - qc6)

[0043] Take the solution with the smallest difference from the current joint angle Thetac as the target solution, that is, the joint angle Thetai(qi1,qi2,qi3,qi4,qi5,qi6) when the di value is the smallest is the target joint angle Thetatarget.

[0044] The master-slave hand attitude teleoperation is similar to the master-slave hand position teleoperation, and the specific steps are as follows:

[0045] 1) To make the teleoperation more controllable and specific, first align the ends of the master and slave hands, that is, make the orientation of the Zt coordinate system at the end of the master hand coincide with the orientation of the Zr coordinate system at the end of the slave hand.

[0046] 2) Record the attitude change Rchange(RchangeX, RchangeY, RchangeZ) of the end of the master hand, that is, RchangeX = Rtnx - Rtix, RchangeY = Rtny - Rtiy, RchangeZ = Rtnz - Rtiz, where Rti(Rtix, Rtiy, Rtiz) is the initial attitude of the master hand and Rtn(Rtnx, Rtny, Rtnz) is the current attitude of the master hand.

[0047] 3) Add the attitude change Rchange(RchangeX, RchangeY, RchangeZ) of the end of the master hand to the initial attitude Rri(Rrix, Rriy, Rriz) of the slave hand to obtain the target attitude Rrt(Rrtx, Rrty, Rrtz) of the slave hand.

[0048] 4) Solve, and the specific steps are the same as step 8) of the position teleoperation.

[0049] 5) Select the solution, and the specific steps are the same as step 9) of the position teleoperation.

[0050] The mapping between the master-slave hand movements in the present invention decouples the position teleoperation and attitude teleoperation of the master-slave hands, making this teleoperation scheme easy to operate and ensuring the intuitiveness and simplicity of the teleoperation. Moreover, the motion mapping focuses on the end movements of the master and slave hands, getting rid of the dependence on the same master-slave hand mechanical configuration, enabling remote control operations to be completed between heterogeneous master-slave hands.

[0051] The teleoperation motion monitoring of the present invention is mainly for monitoring the motion state of the slave hand. To avoid the slave hand moving to a singular position or reaching the limit reachable position during teleoperation, which may cause damage to the slave hand and affect the conduct of the business, the specific steps of the motion monitoring are as follows:

[0052] 1) Read the joint angle values Thetac(qc1, qc2, qc3, qc4, qc5, qc6) of the slave hand in the current cycle

[0053] 2) Solve the current motion flexibility index k of the slave hand:

[0054]

[0055] In the formula, J N is the Jacobian matrix of the robot, and tr is to find the trace of the matrix. This index can evaluate whether the current pose of the end-effector is close to the singular pose. When k(J N ) → 0, the current robot is close to the singular point. When k(J N ) → 1, the end of the current robot is far from the singular point.

[0056] 3) After testing, when the k value is less than 0.1, the motion performance drops sharply and it approaches the singular pose. Therefore, when the k value is less than 0.1, an alarm is sent and the teleoperation motion is stopped.

[0057] The motion state of the slave hand is monitored through the master-slave hand motion monitoring module. When the slave hand approaches the singular position or the limit position, the user is reminded that the limit position is about to be reached, and please remotely operate the slave hand away from the singular position. This module improves the safety of the heterogeneous teleoperation system and makes the operation safer.

[0058] Through the above steps, the teleoperation of the live working robot based on the heterogeneous master-slave hand is realized, making this teleoperation scheme have the advantages of simple, intuitive, efficient, strong flexibility, and not being restricted by the mechanical configuration of the master-slave hands.

[0059] Based on the above method, the present invention also provides a master-slave teleoperation system for a live working robot based on a heterogeneous master hand, including an interaction subsystem, a perception subsystem, a control subsystem, a computing subsystem, and a communication subsystem.

[0060] The interaction subsystem includes a human-machine interaction device for human-machine interaction for teleoperation.

[0061] The perception subsystem includes a robot sensor for collecting information on the robot operation site.

[0062] The control subsystem includes a robot actuator device for controlling the robot motion and collecting robot motion information.

[0063] The computing subsystem includes a deployable computing platform and software modules. The software modules store computer programs, and when executed, implement the teleoperation method of the present invention.

[0064] The communication subsystem includes network communication devices for communication between the master and slave hands of the robot.

[0065] The specific implementation of each of the above subsystems is achieved through hardware and corresponding software modules. The following shows the teleoperation system of the present invention through an embodiment.

[0066] I. Human-Machine Interaction Subsystem

[0067] 1. Hardware Devices:

[0068] A. A touch display screen, which is convenient for presenting the internal and external sensor data and execution status sensed by the robot to the user.

[0069] 2. Software modules:

[0070] A. User interface module

[0071] 1. Display the system sensing data.

[0072] 2. Display the system calculation data.

[0073] 3. Display the system control data.

[0074] 4. Display the system communication data.

[0075] B. Data visualization module

[0076] a. Visualize the control information read from the interactive device driver module into the user interface.

[0077] b. Visualize the images, video streams, and point clouds parsed by the sensing subsystem into the user interface.

[0078] c. Visualize the data such as obstacles, planned trajectories, and monitored states parsed by the computing subsystem into the user interface.

[0079] d. Map the coordinate change data from the control subsystem to the simulated robot and then visualize it into the user interface.

[0080] C. Interactive device driver module

[0081] e. Read physical quantities such as the position P, attitude [], displacement [], velocity [], acceleration [], acceleration [], and angular acceleration [] in the Cartesian coordinate system at the end of the device.

[0082] f. After preprocessing the data, send it to the coordinate conversion module.

[0083] g. Read the torque information fed back by the robot and return it to the master hand in a proportional manner.

[0084] h. Obtain the above data, and proportionally amplify the current data change to the position of the slave arm of the robotic arm through Cartesian space mapping.

[0085] i. Send the expected coordinate attitude and velocity of the user to the motion planning module.

[0086] D. Distributed communication module

[0087] a. Implement peer discovery and establish a trusted connection with the sensing subsystem, control subsystem, and computing subsystem.

[0088] b. Compress the data of the main hand drive and user interaction, and send the data to the computing subsystem.

[0089] c. Receive the sensor data such as images, videos, and point clouds from the perception subsystem, decompress, parse, and distribute them to the data visualization module.

[0090] d. Receive the data such as the feedback coordinate changes from the control subsystem, parse the custom protocol, and distribute them to the data visualization module.

[0091] e. Receive the data such as obstacles, trajectory planning data, and simulation operations from the computing subsystem, and distribute them to the data visualization module.

[0092] II. Perception Subsystem

[0093] 1. Hardware Devices:

[0094] A. 2 pan-tilt cameras

[0095] For the type of camera, to increase the field of view angle, a wide-angle camera of the same model can be replaced; to increase the success rate of night operations, an infrared camera can be replaced; to increase more accurate obstacle depth information, multiple depth cameras can be replaced; to increase the stereoscopic vision function of the VR headset, a binocular camera may be replaced.

[0096] B. Network video recording device NVR

[0097] C. Other sensor devices

[0098] 2. Software Modules:

[0099] A. Data Persistence Module

[0100] a. Use the NVR interface to directly connect it to the NVR device and store the video through the NVR device.

[0101] B. Data Preprocessing Module

[0102] a. For cameras that support network pan-tilt, use the NVR interface to directly connect them to the NVR device, store and forward the video through the NVR device, and send it to the computing platform subsystem.

[0103] b. After converting the point cloud format, send it to the computing platform subsystem and push the data to the data persistence module for storage.

[0104] C. Data Driving Module

[0105] a. For cameras that do not support the network flow protocol, use the camera driver module to obtain image data from the camera. After encoding and compressing the image data, actively push it to the interaction subsystem for display or save it in data persistence.

[0106] b. For RGBD cameras, use the camera driver module to read depth data and obtain image data from the camera. After encoding and compressing the image data, actively push it to the interaction subsystem for display or save it in data persistence.

[0107] D, Distributed communication module

[0108] a. Implement peer discovery and establish a trusted connection with the perception subsystem and the computing subsystem.

[0109] b. After parsing the user interaction, send the data to the sensor module to drive some movable sensors.

[0110] c. Drive the compression of sensor data such as images, videos, and point clouds of the perception subsystem during driving and send them to the interaction subsystem and the computing subsystem.

[0111] III. Computing subsystem

[0112] 1. Hardware device:

[0113] A. An industrial control computer with an x86_64 architecture CPU, which has the most powerful computing power and can perform real-time calculations including point cloud, image, and other data calculations.

[0114] Alternatively, the computing subsystem can support multiple computing platforms to achieve computing power expansion and iterative upgrade of mass-produced models;

[0115] Alternatively, the computing subsystem can support processing of multiple architectures to achieve core computing cost control of mass-produced models;

[0116] Alternatively, after adding a cellular network module to the computing subsystem, it can be combined with various cloud computing platforms to achieve the function of enhancing the robot's body networking and the OTA upgrade function.

[0117] 2. Software module

[0118] A, Master-slave teleoperation module, to achieve the following operations:

[0119] a. Obtain the teleoperation instructions of the user through the master hand.

[0120] b. Convert the movement of the master hand into the movement of the slave hand through the mapping algorithm.

[0121] B, Environment modeling module:

[0122] a. Obtain the registration of the point cloud of each object and the image from the perception subsystem.

[0123] b. Obtain the images from the perception subsystem for instance segmentation and obtain the obstacle mask information.

[0124] c. Segment the obstacle data of each frame of point cloud according to the mask information of the image to obtain the pose information of the point cloud.

[0125] d. Repeat the above steps to complete the splicing of the object data of the whole scene until the data meets the requirements of the operation scenario.

[0126] f. Send all the obstacle data to the human-computer interaction subsystem interface module for display.

[0127] g. Send all the obstacle data to the motion planning module of this subsystem for obstacle avoidance.

[0128] C. System Monitoring Module

[0129] a. Obtain information such as the number of CPU cores, main frequency, temperature, and load in the computing platform.

[0130] b. Obtain information such as the main frequency, temperature, and load of the memory in the computing platform.

[0131] c. Obtain information such as the hard disk capacity, temperature, and load in the computing platform.

[0132] d. Obtain information such as the network bandwidth and load in the computing platform.

[0133] D. Distributed Communication Module

[0134] a. Implement peer discovery and establish a trusted connection with the interaction subsystem, perception subsystem, and control subsystem.

[0135] b. After parsing the user interaction, send the data to the control subsystem to drive some motion sensors.

[0136] c. After parsing the sensor data such as the original images, videos, and point clouds of the perception subsystem, send them to the environment modeling module for modeling.

[0137] d. After parsing the data such as obstacles, path planning, and monitoring information processed by the environment modeling, motion planning, and system monitoring modules, send them to the corresponding control subsystem and interaction subsystem.

[0138] IV. Control Subsystem

[0139] 1. Hardware Devices:

[0140] A. Six-degree-of-freedom robotic arm.

[0141] B. End effector of the robotic arm.

[0142] C. Robotic arm controller.

[0143] 2. Software modules:

[0144] A. Robotic arm actuator module

[0145] a. This program runs within a control unit containing an x86_64 architecture processor.

[0146] b. Receives instruction data from the computing platform subsystem and achieves the instructions by controlling the movement of the robotic arm.

[0147] B. End effector module

[0148] a. This program runs within a control unit containing an x86_64 architecture processor.

[0149] b. Receives instruction data from the computing platform subsystem and achieves the instructions by the movement of the end effector.

[0150] C. Status feedback module

[0151] a. Performs real-time recording of the robotic arm status.

[0152] b. Performs real-time recording of the end effector status.

[0153] D. Distributed communication module

[0154] a. Implements peer discovery and establishes a trusted connection with the interaction subsystem and the control subsystem.

[0155] b. Parses the robotic arm control instructions sent from the computing subsystem and sends the data to the robotic arm control module.

[0156] c. Parses the end effector control instructions sent from the computing subsystem and sends the data to the end effector control module.

[0157] d. Packages and sends the status returned by the robotic arm and the end effector controller to the interaction subsystem and the control subsystem.

[0158] V. Communication subsystem

[0159] 1. Hardware devices:

[0160] A. Optoelectronic hybrid switch

[0161] Alternatively, on the premise of ensuring a 1000MB network bandwidth and a network latency of less than 20ms, a wireless hotspot can be replaced.

[0162] B. Tool wireless hotspot device

[0163] Alternatively, on the premise of ensuring a network bandwidth of 100MB and a network latency of less than 10ms, the Bluetooth protocol 5.0 can be replaced.

[0164] C. User mobile device interaction hotspot

[0165] Alternatively, on the premise of ensuring a network bandwidth of 1000MB and a network latency of less than 30ms, the 4G / 5G cellular network can be replaced.

[0166] 2. Software modules:

[0167] A. Peer discovery module

[0168] a. The end-to-end discovery of both sides of the message can be achieved through a centralized registration mechanism, that is, the communication subsystem acts as the server, and other subsystems act as clients, through the service routing mechanism. This embodiment is implemented through the ROS core mechanism.

[0169] Alternatively, peer discovery can be achieved through a broadcast mechanism, that is, all communication ends are equal. In the broadcast domain, the messages that need to be sent and received are broadcast, and other subsystems determine whether to reply to the sender by listening to network packets and identify the source address and destination address of the data packets.

[0170] B. Message sending and receiving queue

[0171] a. In this embodiment, functions such as message reception, serialization, parsing, deserialization, and sending are implemented through the ROS publish-subscribe mechanism.

[0172] Similarly, message middleware such as ActiveMQ, RabbitMQ, RocketMQ, Kafka, ZeroMQ, and MQTT can achieve the same effect.

[0173] Similarly, the publish-subscribe mechanism can also be implemented using open source or closed source data set distribution systems such as DDS (Data Distribution System) openDDS, Fast-RTPS, and opensplice.

[0174] C. Custom protocol module

[0175] a. Implemented through custom messages

[0176] This embodiment is implemented by customizing the ROS msg method.

[0177] Alternatively, protobuf or thrift can also be used.

[0178] The degrees of freedom of the master hand of the present invention are the same as those of the live working robot, both being six degrees of freedom, which can control the live working robot to move and rotate in any direction, and has high teleoperation sensitivity. It is not restricted by the master-slave hand configuration. The master hand and the slave hand of any configuration perform motion mapping through the method of the present invention to complete the teleoperation task, and the algorithm has high fitness; a singular value detection algorithm is added during the teleoperation process to detect the motion state of the master hand in real time. When the posture of the slave hand approaches singularity, an alarm is given to ensure that the master hand is away from dangerous postures, improving the safety of heterogeneous master-slave hand teleoperation.

Claims

1. A master-slave teleoperation method for live working robots based on heterogeneous master hands, characterized in that The live working robot is a master-slave heterogeneous robot, including the following steps: 1) Acquisition of heterogeneous master-slave hand motion information: The slave hand is remotely operated by the master hand to obtain the master-slave hand motion information, including the end pose and end motion speed of the master hand, as well as the end pose and joint angle information of each joint of the slave hand. The end poses of the master hand and the slave hand include end position and rotation information; 2) Mapping of heterogeneous master-slave hand motion information: Based on the master-slave hand motion information and with the base coordinate system as the basis, the motion of the master hand is mapped and converted into the motion of the slave hand. When mapping, the remote operation is decoupled and decomposed into the position remote operation of the master-slave hand ends and the attitude remote operation of the master-slave hand. The steps of the master-slave hand position remote operation are as follows: 2.1.1) Record the position information Pti (initialTouchX, initialTouchY, initialTouchZ) of the master hand end remote operation start point in its base coordinate system Baset; 2.1.2) Record the position information Prn (initialRobotX, initialRobotY, initialRobotZ) of the slave hand end remote operation start point in its base coordinate system Baser; 2.1.3) Obtain the current position information Ptn (Ptnx, Ptny, Ptnz) of the master hand end in its base coordinate system Baset; 2.1.4) Subtract Pti from Ptn to obtain the master hand end position movement information Tchange (TchangeX, TchangeY, TchangeY), that is, TchangeX = Ptnx - Ptix, TchangeY = Ptny - Ptiy, TchangeZ = Ptnz - Ptiz; 2.1.5) Obtain the current position information Prn (Prnx, Prny, Prnz) of the slave hand; 2.1.6) Compare the velocity values V (Vtx, Vty, Vtz) in each direction of the master hand, and take the direction with the largest velocity value as the current motion direction D. Then, the slave hand target position Prt (Prtx, Prty, Prtz) in the direction D is the master hand position movement information TchangeD plus the end remote operation start point position initialRobotD in the D direction of the slave hand, and the slave hand target positions in other directions are the current slave hand positions; 2.1.7) Take the current slave hand end attitude Rrn (Rrnx, Rrny, Rrnz) as the slave hand target end attitude Rrt (Rrtx, Rrty, Rrtz), that is, Rrtx = Rrnx, Rrty = Rrny, Rrtz = Rrnz, and Rrt + Prt is the slave hand current cycle target pose Tt; 2.1.8) Use the analytical method to solve the slave hand target joint angle Thetat, and obtain eight groups of robot joint angles Thetat1…Theta8, and each group of joint angles contains six joint angle values (q1, q2, q3, q4, q5, q6); 2.1.9) Solution selection: Use the minimum distance method to select the target solution. Read the current hand joint angle Thetac(qc1, qc2, qc3, qc4, qc5, qc6), calculate the difference between each of the eight groups of solutions and the current joint angle Thetac one by one, and select the solution with the smallest difference from the current joint angle Thetac as the target solution; The master-slave hand attitude teleoperation is similar to the master-slave hand position teleoperation, and the specific steps are as follows: 2.2.1) First, align the ends of the master and slave hands, that is, make the orientation of the Zt coordinate system at the end of the master hand coincide with the orientation of the Zr coordinate system at the end of the slave hand; 2.2.2) Record the attitude change of the master hand end Rchange(RchangeX, RchangeY, RchangeZ). Denote Rti(Rtix, Rtiy, Rtiz) as the initial attitude of the master hand and Rtn(Rtnx, Rtny, Rtnz) as the current attitude of the master hand. Then RchangeX = Rtnx - Rtix, RchangeY = Rtny - Rtiy, RchangeZ = Rtnz - Rtiz; 2.2.3) The attitude change of the master hand end Rchange(RchangeX, RchangeY, RchangeZ) plus the initial attitude of the slave hand Rri(Rrix, Rriy, Rriz) is the target attitude of the slave hand Rrt(Rrtx, Rrty, Rrtz); 2.2.4) Solution solving: The specific steps are the same as step 2.1.8) of the position teleoperation; 2.2.5) Solution selection: The specific steps are the same as step 2.1.9) of the position teleoperation; 3) Motion state monitoring: After the motion of the master hand is converted into the motion of the slave hand through mapping, the motion information is sent to the slave hand, and it is continuously detected whether the motion of the slave hand is close to the singular position or the limit position. If it is close to the singular position, the motion is stopped.

2. A master-slave teleoperation method for an on-line working robot based on a heterogeneous master hand according to claim 1, characterized in that the heterogeneity The acquisition of the master-slave hand motion information is as follows: 1.1) Obtain the master hand motion information. The master hand motion information includes the position information P(Px, Py, Pz) of the end of the master hand in the base coordinate system Baset of the master hand, the attitude information R(Rx, Ry, Rz) of the end of the master hand in its base coordinate system Baset, and the motion speed information V(Vx, Vy, Vz) of the end of the master hand in the base coordinate system Baset. First, process P(Px, Py, Pz) and R(Rx, Ry, Rz). Through the RPY angle to quaternion formula, convert the obtained attitude information R(Rx, Ry, Rz) into quaternion Q(Qx, Qy, Qz, Qw). Then combine the quaternion Q(Qx, Qy, Qz, Qw) and the position information P(Px, Py, Pz) to generate the pose T of the master hand; 1.2) Use the second-order difference to process the velocity information v(vx, vy, vz) of the end of the master hand to obtain the master hand motion speed information V(Vtx, Vty, Vtz). The specific formula is: where vel_buff = (position * 3 - 4 * pos_hist1 + pos_hist2) / 0.002, pos_hist2 is the position of the end - effector of the robot one cycle ago, pos_hist1 is the position of the end - effector of the robot in the current cycle, inp_vel3 is the second - order difference coefficient of the moving speed of the end - effector of the robot two cycles ago, inp_vel2 is the second - order difference coefficient of the moving speed of the end - effector of the robot one cycle ago, inp_vel1 is the second - order difference coefficient of the moving speed of the end - effector of the robot in the current cycle, out_vel3 is the moving speed of the end - effector of the robot two cycles ago, out_vel2 is the moving speed of the end - effector of the robot one cycle ago, out_vel1 is the moving speed of the end - effector of the robot before the current cycle, the sampling frequency is 20Hz, and the speed information is processed by second - order difference to remove speed mutations and obtain stable speed information; 1.3) Obtain the motion information of the slave hand. The motion information of the slave hand includes the position information P'(Px, Py, Pz) of the end - effector of the slave hand in its base coordinate system Baser, the attitude information R'(Rx, Ry, Rz) of the end - effector of the slave hand in its base coordinate system Baser, and the current joint - angle information Theta(q1, q2, q3, q4, q5, q6) of the slave hand.

3. A master-slave teleoperation method for live working robots based on heterogeneous master hands according to claim 1, characterized in that the master and slave hands The specific steps of motion - state monitoring are as follows: 3.1) Read the joint - angle values Thetac(qc1, qc2, qc3, qc4, qc5, qc6) of the slave hand in the current cycle; 3.2) Solve the current motion - flexibility index k of the slave hand; where J N is the Jacobian matrix of the robot, tr is to find the trace of the matrix, and the index k is used to evaluate whether the current pose of the end - effector is close to the singular pose. When k(J N ) → 0, the current robot is close to the singular point. When k(J N ) → 1, the end - effector of the current robot is far from the singular point; 3.3) When the value of k is less than 0.1, the motion performance drops sharply, approaching the singular pose, send an alarm, and stop the teleoperation motion.

4. A master-slave teleoperation system for live working robots based on a heterogeneous master hand, characterized in that It includes an interaction subsystem, a perception subsystem, a control subsystem, a computing subsystem, and a communication subsystem. The interaction subsystem includes a human - machine interaction device for human - machine interaction in teleoperation. The perception subsystem includes robot sensors for collecting information on the robot's working site. The control subsystem includes robot actuator devices for controlling the robot's motion and collecting the robot's motion information. The computing subsystem includes a deployable computing platform and software modules. The software modules store computer programs, and when executed, implement the teleoperation method described in any one of claims 1 - 3. The communication subsystem includes network communication devices for communication between the master and slave hands of the robot.

Images