Upper body teleoperation force feedback device for dual-arm robot
By designing an upper-body teleoperation force feedback device for dual-arm robots, and utilizing a wearable exoskeleton and an ESP8266 development board to achieve linkage between the exoskeleton and the robotic arm, the collaborative problem of remote robot operation was solved, enabling efficient remote task execution and human-robot collaboration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- RES INST OF ZHEJIANG UNIV TAIZHOU
- Filing Date
- 2026-06-22
- Publication Date
- 2026-07-31
AI Technical Summary
In existing technologies, remote operation and control methods for robots are difficult to achieve remote collaboration of robotic arms, and the signal acquisition of existing exoskeletons is easily affected by external interference, affecting signal reliability and making it difficult to accurately collect hand posture and joint data changes.
A force feedback device for upper-body teleoperation of a dual-arm robot was designed. It adopts a wearable exoskeleton and an ESP8266 development board. By quickly reading the joint angles and exoskeleton link speeds, a linkage platform between the exoskeleton and the robotic arm is established. The back support device and control unit are used to realize signal transmission and processing. It is combined with a WIFI module to link with the robot.
It achieves efficient collaboration between robots and exoskeletons, accurately mimicking the postures and movements of real human hands, and is applicable to service, medical, and criminal investigation fields, enabling the completion of dangerous tasks and reducing the risk of personal injury.
Smart Images
Figure CN122480912A_ABST
Abstract
Description
Technical Field
[0005]
[0001] The present invention belongs to the field of teleoperation, and particularly relates to an upper body teleoperation force feedback device for a dual-arm robot. Background Art
[0002] Robots can replace humans to enter many dangerous situations, complete corresponding tasks and avoid possible personal injuries. However, the current main methods for remotely operating and controlling robots are computer instruction control or controller instructions such as remote control handles. The main problems faced by these two control methods are that the robotic arms cannot well achieve remote collaboration or imitate the postures and movements of real human hands.
[0003] In terms of human-robot collaborative cooperation, existing exoskeletons mainly capture arm postures through flexible sensors arranged on the forearm link and the upper arm. The main problems are that a large amount of calculations are required and the collected signals are extremely vulnerable to external factors, such as zero drift, which may affect signal reliability. Many existing control and design schemes are difficult to accurately collect data changes such as the hand postures of users, different joint angles, speeds, and displacements of wearable exoskeletons. Summary of the Invention
[0004] To solve the problems and requirements in the background art, the present invention proposes an upper body teleoperation force feedback device for a dual-arm robot. The present invention designs a wearable exoskeleton and uses a new way to control the robotic arm to use the exoskeleton to control the machine and achieve a linkage effect. In addition, based on the exoskeleton, in order to achieve more effective human-robot cooperation, the present invention realizes the functions of quickly reading joint angles and different exoskeleton link rates. At the signal transmission end, an ESP8266 development board is used to control the exoskeleton-linked robot, establishes a platform for the linkage of the exoskeleton and the robotic arm, and provides a control system for the coordinated operation of the exoskeleton + robot.
[0005] The present invention adopts the following technical solutions: In the first aspect, the present invention proposes an upper body teleoperation force feedback device for a dual-arm robot, and the upper body teleoperation force feedback device includes: A back support device for placing a control unit and a power supply; A wearable exoskeleton connected to the back support device for wearing on an operator and collecting the operator's joint information and transmitting the collected joint information to a control board; A control unit for processing the joint information transmitted by all wearable exoskeletons and sending the processed data to a remote robotic arm.
[0006] Furthermore, at least one wearable exoskeleton is provided, and each wearable exoskeleton includes, from top to bottom, a first connecting rod, a first link assembly, an upper arm link assembly, a forearm link assembly, a fourth link assembly, a fifth link assembly, an adjustable hand grip, and a grasping assembly. The first connecting rod is connected to the back support device, and a micro switch is installed at the grasping assembly. The first link assembly, the upper arm link assembly, the forearm link assembly, the fourth link assembly, the fifth link assembly, and the adjustable hand grip all include corresponding encoders.
[0007] Furthermore, each wearable exoskeleton has multiple microswitches at its gripping assembly.
[0008] Furthermore, the control unit includes a left control board and a right control board. The left control board is used to receive joint information transmitted from the left wearable exoskeleton, as well as to process and transmit the data. The right control board is used to receive joint information transmitted from the right wearable exoskeleton, as well as to process and transmit the data.
[0009] Furthermore, the control unit also includes a WIFI module for sending the data processed by the control board to the exoskeleton robot.
[0010] Furthermore, the wearable exoskeleton is connected to the operator via elastic straps.
[0011] Furthermore, a silicone pad is provided on the side of the back support device closest to the operator.
[0012] Secondly, this invention proposes a method for controlling an exoskeleton-linked robot using an upper-body teleoperation force feedback device for a dual-arm robot, the method comprising the following steps: Step 1: Wear the upper body remote force feedback device on the operator, execute the feasibility procedure, and proceed to Step 2 after verification; Step 2: Perform the calibration process. After successful calibration, proceed to Step 3. Step 3: The upper body teleoperation force feedback device collects the operator's joint information and generates pose information, which is then sent to the exoskeleton linkage robot. The exoskeleton linkage robot executes operation commands based on the received pose information.
[0013] The present invention has the following beneficial effects: 1. This invention enables a novel method of robot control: exoskeleton control. This control method can effectively achieve remote collaboration; furthermore, through machine learning, the robot can mimic the posture and movements of a real human hand using joint data transmitted from the exoskeleton.
[0014] 2. This invention is applicable to numerous situations, and this linkage device can be used to complete many tasks. Firstly, in the service sector, it can assist in tasks such as moving heavy objects, saving users' physical strength and improving work efficiency. Secondly, this invention also has wide applications in the medical and criminal investigation fields, because by using robots in linkage, they can replace humans in many dangerous situations, completing corresponding tasks and avoiding potential personal injury. For example, in wards treating patients with infectious diseases, the entry and exit of medical staff and family members may increase the risk of infection. This linkage device can provide patients with necessary nursing services, such as basic assistance like medication delivery and replacement. Furthermore, this robot can also perform bomb disposal tasks in the field, using its robotic arm to defuse explosives, which can be of great help in the field of criminal investigation. Attached Figure Description
[0015] Figure 1 This is a front view of the upper body remote force feedback device. Figure 2 This is a control framework diagram of the upper body remote force feedback device; Figure 3 This is a control flowchart for human-machine collaborative remote operation; In the figure: 1. Back support device, 2. First connecting rod, 3. First link assembly, 4. Upper arm link assembly, 5. Encoder, 6. Forearm connection assembly, 7. Fourth link assembly, 8. Fifth link assembly, 9. Adjustable hand grip, 10. Grip assembly, 11. Micro switch. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.
[0017] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.
[0018] like Figure 1 As shown, the present invention proposes an upper-body teleoperation force feedback device for dual-arm robots, comprising: The back support device 1 is used to house the control unit and the power supply; the power supply is used to power the control unit and the encoder, micro switch, etc. in the wearable exoskeleton.
[0019] A wearable exoskeleton connected to a back support device is worn by the operator to collect the operator's joint information and transmit the collected joint information to the control panel. The control unit is used to process the joint information transmitted by all wearable exoskeletons and send the processed data to the remote robotic arm.
[0020] In one feasible implementation, the overall design of the wearable exoskeleton conforms to ergonomics and has a good design correlation with the exoskeleton linkage robot. The length of each joint is proportional to the length of the corresponding joint of the slave robot, which facilitates maintaining the consistency of their postures.
[0021] In one feasible implementation, at least one wearable exoskeleton is provided. Each wearable exoskeleton includes, from top to bottom, a first connecting rod 2, a first link assembly 3, an upper arm link assembly 4, a forearm connecting assembly 6, a fourth link assembly 7, a fifth link assembly 8, an adjustable hand grip 9, and a grasping assembly 10. The first connecting rod 2 is connected to the back support device 1. A microswitch 11 is installed at the grasping assembly 10 to sense the movement of each finger. After sensing pressure, the microswitch reads the data and transmits it to the Arduino Mega development board. Specifically, the microswitch is used to control the start and stop of the robotic arm, the opening and closing of two fingers of the mechanical gripper, and the opening and closing of three fingers of the mechanical gripper. The first link assembly 3, the upper arm link assembly 4, the forearm connecting assembly 6, the fourth link assembly 7, the fifth link assembly 8, and the adjustable hand grip 9 each contain a corresponding encoder 5. There is one wearable exoskeleton on each side, with a total of 12 encoders 5. Specifically, encoders are installed on the shoulder, upper arm, forearm, and wrist of the upper limb exoskeleton to acquire real-time information on the wearer's elbow angle, elbow joint, and forearm displacement.
[0022] In one feasible implementation, multiple microswitches 11 are provided at the gripping component 10 of each wearable exoskeleton. Optionally, three microswitches 11 are provided at the gripping component 10 of each wearable exoskeleton, which can be linked with the robotic hand fingers to control the gripping of the robotic arm.
[0023] In one feasible implementation, the control unit includes a left control board and a right control board. The left control board is used to receive joint information and process and transmit data transmitted from the left wearable exoskeleton, while the right control board is used to receive joint information and process and transmit data transmitted from the right wearable exoskeleton. Optionally, the control board is an Arduino Mega development board. Specifically, the Arduino Mega development board connects to an encoder for joint signal input, used to acquire and collect information on the corresponding joint angular velocity and angle parameters in real time. After filtering, organizing, and calculating the information, relevant characteristic parameters of the wearer and the wearable device are obtained. The collected parameters are then transmitted to the robot to clarify the operator's movement intention. By changing the robot's posture through the program, its posture reaches a specified position, achieving synchronization and linkage with the exoskeleton.
[0024] In one feasible implementation, the control unit further includes a left WIFI module and a right WIFI module, used to send the data processed by the control board to the exoskeleton robot. Optionally, the WIFI module is an ESP8266 module, which uses serial communication to receive signals sent by the Arduino Mega development board.
[0025] In one feasible implementation, to better adapt to different user groups and facilitate better data collection and analysis, the exoskeleton utilizes an adjustable strap groove structure to accommodate users with varying arm lengths. Furthermore, the straps connecting the exoskeleton to the user also employ elastic straps to accommodate operators with different waist circumferences, shoulder widths, and arm circumferences.
[0026] In one feasible implementation, both the back support device 1 and the wearable exoskeleton are 3D printed, and aluminum flange couplings are used at each joint of the wearable exoskeleton.
[0027] In one possible implementation, a silicone pad is provided on the side of the back support device 1 closest to the operator to increase comfort.
[0028] like Figure 2 and Figure 3 As shown, this invention proposes a method for controlling an exoskeleton-linked robot using an upper-body teleoperation force feedback device for dual-arm robots. The method includes the following steps: Step 1: Wear the upper body remote force feedback device on the operator's body using straps, execute the feasibility procedure, and proceed to Step 2 after verification; In one feasible implementation, the feasibility process includes: 1. Power on the Arduino Mega development board and ESP8266 module. Start the exoskeleton robot and the ROS (Robot Operating System) system used to work with the exoskeleton robot.
[0029] 2. Check the reliability of the connection between the Arduino Mega development board and the ESP8266 module, as well as the reliability of the communication between the ESP8266 module and the exoskeleton robot.
[0030] 3. Verify the data transmission by slightly rotating a few joints and pressing microswitches. This includes reading the arm joint angles and finger opening / closing states.
[0031] Arm joint angle reading: The user moves the joints of the exoskeleton by using straps, which in turn causes the joints to rotate; the encoders on each joint read the joint angle changes in real time and send them to the Arduino Mega development board.
[0032] Finger opening / closing state reading: The user controls the voltage state read by the Arduino Mega development board by controlling the opening and closing state of the switch installed on the exoskeleton hand; a high level represents the finger being closed, and a low level represents the finger being relaxed.
[0033] Step 2: Perform the calibration process. After successful calibration, proceed to Step 3. In one feasible implementation, the calibration process includes: 1. Manually plan the exoskeleton pose to ensure consistency between the initial pose and the pose of the exoskeleton-linked robot, and ensure that the two are synchronized after the exoskeleton-linked robot returns to zero.
[0034] 2. Perform zeroing operation. The ROS nodes inside the robot subscribe to the target position of the exoskeleton joint angle to solve for the robot's initial pose.
[0035] 3. The operator wears the exoskeleton and performs preset actions to observe the reliability of the linkage between the robot and the exoskeleton.
[0036] Step 3: The operator observes the robot's external environment, and the upper body telescopic force feedback device collects the operator's joint information and generates pose information. This pose information is then sent to the exoskeleton robot. The exoskeleton robot executes operation commands based on the received pose information, such as controlling the robot to deliver medical supplies, packages, or other items.
[0037] Optionally, the exoskeleton robot is a Kinova Gen2 robotic arm. The robot's controller receives joint angle information from the ESP8266 module and then sends the joint angle information to the joint angle receiving node of the Kinova Gen2 robotic arm.
[0038] Specifically: The Arduino Mega development board transmits pose information to the ESP8266 module via serial communication; the ESP8266 module transmits the data to the ROS node inside the robot via WIFI; the ROS node inside the robot obtains the joint angle difference by subscribing to the target position of the exoskeleton joint angle and the actual joint angle of the controlled robotic arm, and uses the difference to perform PID-based closed-loop control until the target position of the exoskeleton matches the actual joint angle of the robotic arm.
[0039] This invention achieves closed-loop control of the exoskeleton and robotic arm, realizing the correspondence between the control end and the output end. The required rotational speed of the corresponding joint is calculated by comparing the difference between the target angle and the actual angle. The target joint is controlled to rotate to the target posture by changing the rotational speed of each joint.
[0040] The entire operation process obtains the target state information of the mechanical exoskeleton through encoders and microswitches, compares it with the actual state information of the robotic arm returned by the Kinova Gen2 robotic arm, and adjusts the control signal until the target state matches the actual state, forming a closed-loop system.
[0041] The working process and principle of this invention are as follows: The wearable exoskeleton collects upper limb movements and converts these signals into control signals, which are then transmitted to a remote robotic arm for remote control. The wearable exoskeleton employs intuitive joint space mapping control, reducing reliance on traditional remote control and computer input for robotic arm remote operation, thus improving the convenience and accuracy of remote operation. This invention also boasts advantages such as simple structure, convenient operation, and ease of implementation.
Claims
1. A force feedback device for upper body teleoperation of a dual-arm robot, characterized by, include: Back support device (1) for housing the control unit and power supply; A wearable exoskeleton connected to a back support device is worn by the operator to collect the operator's joint information and transmit the collected joint information to the control panel. The control unit is used to process the joint information transmitted by all wearable exoskeletons and send the processed data to the remote robotic arm.
2. The upper body teleoperation force feedback device for dual-arm robot according to claim 1, wherein At least one wearable exoskeleton is provided. Each wearable exoskeleton includes, from top to bottom, a first connecting rod (2), a first link assembly (3), an upper arm link assembly (4), a forearm link assembly (6), a fourth link assembly (7), a fifth link assembly (8), an adjustable hand grip (9), and a grip assembly (10). The first connecting rod (2) is connected to the back support device (1), and a micro switch (11) is installed at the grip assembly (10). The first link assembly (3), the upper arm link assembly (4), the forearm link assembly (6), the fourth link assembly (7), the fifth link assembly (8), and the adjustable hand grip (9) all include a corresponding encoder (5).
3. The upper body teleoperation force feedback device for dual-arm robot according to claim 1, wherein Multiple microswitches (11) are provided at the gripping assembly (10) of each wearable exoskeleton.
4. The upper body teleoperation force feedback device for dual-arm robot according to claim 1, wherein The control unit includes a left control board and a right control board. The left control board is used to receive joint information transmitted from the left wearable exoskeleton, as well as to process and send the data. The right control board is used to receive joint information transmitted from the right wearable exoskeleton, as well as to process and send the data.
5. The upper body teleoperation force feedback device for dual-arm robot according to claim 4, wherein The control unit also includes a WIFI module, which is used to send the data processed by the control board to the exoskeleton linkage robot.
6. The upper body teleoperation force feedback device for dual-arm robot according to claim 1, wherein The wearable exoskeleton is connected to the operator via elastic straps.
7. The upper body teleoperation force feedback device for dual-arm robot according to claim 1, wherein The back support device (1) has a silicone pad on the side closest to the operator.
8. The method for controlling an exoskeleton-linked robot using an upper-body teleoperation force feedback device for a dual-arm robot as described in claim 1, characterized in that... Includes the following steps: Step 1: Wear the upper body remote force feedback device on the operator, execute the feasibility procedure, and proceed to Step 2 after verification; Step 2: Perform the calibration process. After successful calibration, proceed to Step 3. Step 3: The upper body teleoperation force feedback device collects the operator's joint information and generates pose information, which is then sent to the exoskeleton linkage robot. The exoskeleton linkage robot executes operation commands based on the received pose information.