A teleoperation method using a combination of IMU and tethered traction, with the human upper limb as the master teleoperation controller.

Abstract

This invention relates to the field of teleoperation technology, and in particular to a teleoperation method that uses an IMU and rope traction in combination, with the human upper limb as the master teleoperation controller. The method uses the combination of IMU and rope traction for teleoperation force feedback, adopts a reasonable rope layout to generate tactile force, and uses a self-calibration method to automatically learn the wearer's arm length to optimize the IMU teleoperation effect.

Description

Technical Field

[0001] This invention relates to the field of teleoperation technology, specifically to a teleoperation method that uses a combination of IMU and cable traction, with the human upper limb as the master teleoperation controller. Background Technology

[0002] In addition to position control of the slave device, teleoperable force sensing devices can also sense the interaction force of the slave device. Current teleoperable force sensing devices, such as Geomagic Touch and Omega, are fixed to the operating platform, requiring changes in people's operating habits for teleoperation. Vision-based teleoperation lacks strong tactile feedback, while exoskeleton-based solutions are expensive and bulky. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a teleoperation method that uses a combination of IMU and rope traction, with the human upper limb as the master teleoperation controller.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a teleoperation method using a combination of IMU and rope traction, with the human upper limb as the master teleoperation controller, comprising the following steps:

[0005] Step 1: Wear a backpack on the back of the human body, wear an IMU on the arm of the human body, wear an IMU on the forearm of the human body, and wear flexible rope anchors on the arm and wrist of the human body.

[0006] Step 2: Connect the flexible rope located in the backpack to the flexible rope anchor point at the wrist of the upper limb;

[0007] Step 3: The motor in the backpack drives the flexible cable to pull the upper limb of the human body, causing the upper limb to move. The IMU module collects the data of the upper limb, the encoder collects the change in cable length, and then the optimization algorithm is used to obtain the arm length parameters of the human upper limb, thus completing the calibration of the arm length of the human upper limb.

[0008] Step 4: Based on the calibrated upper limb arm length and the joint rotation angle of the upper limb obtained after IMU attitude calculation, construct the forward kinematic model of the upper limb using the improved DH method, and calculate the trajectory of the upper limb end.

[0009] Step 5: Transmit the trajectory data of the human upper limb end to the slave robot, so that the slave robot can move to the corresponding position and perform teleoperation.

[0010] Step 6: Collect robot force information through the end-effector force sensor and transmit the force information to the master controller. The motor drives the flexible rope to pull the human upper limb to achieve force feedback.

[0011] In some embodiments, the backpack consists of an aluminum frame, a motor bracket, a motor, a rope shaft, a flexible rope, an IMU bracket, an IMU module, and a strap. The strap, motor bracket, and IMU bracket are mounted on the aluminum frame, the IMU module is mounted on the IMU bracket, the motor is mounted on the motor bracket, the rope shaft is mounted on the motor, and the flexible rope is wrapped around the rope shaft.

[0012] In some embodiments, according to step one, an IMU is worn on any part of the upper arm of the human body, and the position of the IMU1 is aligned with the position of the IMU on the backpack;

[0013] An IMU is worn on the forearm of the upper limb, and its position is aligned with the IMU on the backpack.

[0014] Flexible rope anchors are worn on the upper limbs of the human body, including flexible rope anchors at the arm and wrist.

[0015] In some embodiments, according to step two, the motor power is turned on, and the encoder in the motor enters the working state. The end of the flexible rope starts from the shoulder and is stretched to the wrist flexible rope anchor point of the human upper limb when the upper limb is in a natural vertical position. At this time, the encoder records the change in the length of the flexible rope as L1.

[0016] In some embodiments, according to step three, the motor is started, causing it to rotate slowly. The connected flexible wrist cord slowly pulls the upper limb, causing it to gradually change from a natural vertical position to a bent position. The change in cord length L2 during this process is recorded by an encoder in the motor. The change in the upper limb position is recorded by two IMUs worn on the upper limb. After data acquisition, the quaternion data q1 and q2 collected by IMU1 and IMU2 are used to calculate the upper limb posture and obtain the change in elbow angle. Based on the collected rope length changes L1 and L2 and the calculated elbow angle change... The function f(x) is solved using the nonlinear programming function fmincon() in Matlab software, thereby determining the arm length x1 and forearm length x2 of the human upper limb and completing the forward kinematic calibration of the human upper limb.

[0017] In some embodiments, the specific method is as follows:

[0018] 1) Transform quaternions q1 and q2 into rotation matrices R1 and R2:

[0019] q=w+xi+yj+zk

[0020]

[0021] 2) The rotation matrix R1 transforms into R2. tra:

[0022] R tra =R2 -1 R1

[0023] 3).R tra The matrix is ​​decomposed into three Euler angles based on the fixed frame, where The change in angle corresponding to the elbow:

[0024]

[0025]

[0026] θ=atan2(R tra (2, 1), R tra (1,1))

[0027] ψ=atan2(R tra (3,2),R tra (3,2))

[0028] 4). The functional relationship f(x) is derived from... Figure 3 have to:

[0029]

[0030]

[0031]

[0032]

[0033] L2 2 =x2 2 +x3 2 -2x2x3cosβ

[0034]

[0035] Use the nonlinear programming function fmincon() in Matlab to find the optimal lengths of the upper limbs x1 and forearm x2.

[0036] In some embodiments, according to step four, the rotation matrix obtained between the IMU on the backpack and the IMU worn on the upper arm is decomposed into three Euler angles according to the fixed system, corresponding to the three degrees of freedom of the upper arm shoulder, where angle θ corresponds to adduction-abduction of the shoulder, and angle ψ corresponds to pronation-supination of the shoulder. The angle corresponds to the flexion-extension of the shoulder. It is obtained through the rotation matrix between the IMU worn on the upper arm and the IMU worn on the forearm, and decomposed into three Euler angles according to the fixed frame, corresponding to the two degrees of freedom of the elbow and forearm respectively. The angle corresponds to the flexion-extension of the elbow, and the ψ angle corresponds to the pronation-supination of the forearm. Based on the obtained arm length x1 and forearm length x2 of the human upper limb, the improved DH method is used to construct the forward kinematic model of the human upper limb.

[0037] In some embodiments, the homogeneous transformation matrix of the distal end position of the human upper limb is:

[0038]

[0039] Where matrix A is:

[0040]

[0041] The coordinates and posture of the human upper limb end are obtained.

[0042] In some embodiments, according to step five, the trajectory data of the human upper limb end is transmitted to the slave robot using the TCP / IP communication mechanism, the end and the robot end are matched for position, and the slave robot moves to the corresponding position to perform teleoperation.

[0043] In some embodiments, according to step six, the robot force information is collected by the end-effector force sensor of the end robot, the force information is transmitted to the master controller, and the force feedback is achieved by driving the flexible rope to pull the human upper limb through the rotation of the motor.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] 1. By adopting a backpack-like design and an IMU worn on the upper limb, the human upper limb is used as a remote operation platform, avoiding the limitations of remote operation devices that require a fixed platform. Furthermore, using the upper limb as the master remote operation platform allows control of slave robots without altering the user's operating habits.

[0046] 2. By using rope to pull the upper limbs of the human body, the remote operation force feedback function is realized, which avoids the disadvantage of lacking tactile feedback in visual remote operation schemes.

[0047] 3. By using a backpack and an IMU worn on the upper limbs, it is simpler to wear, more convenient to operate, and cheaper than teleoperation solutions based on exoskeletons.

[0048] 4. It can automatically learn the wearer's arm length through a self-calibration method, without the need to measure the wearer's upper limb data in advance for kinematic calculations, and can adapt to people with various arm lengths.

[0049] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. The embodiments of this application will provide a detailed description and understanding of the application. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the remotely operated backpack of the present invention;

[0051] Figure 2 This is a schematic diagram of the human upper limb IMU module and the flexible rope anchor point arrangement of the present invention;

[0052] Figure 3 This is a simplified diagram of the human upper limb arm length calibration according to the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] This invention provides a technical solution: a teleoperation method using a combination of IMU and cable traction, with the human upper limb as the master teleoperation controller, comprising the following steps:

[0055] Step 1: Wear a backpack on the back of the human body, wear an IMU on the arm of the human body, wear an IMU on the forearm of the human body, and wear flexible rope anchors on the arm and wrist of the human body.

[0056] Step two: Connect the flexible rope located in the backpack to the flexible rope anchor point at the wrist of the upper limb.

[0057] Step 3: The motor in the backpack drives the flexible cable to pull the upper limb of the human body, causing the upper limb to move. The IMU module collects the data of the upper limb, the encoder collects the change in cable length, and then the optimization algorithm is used to obtain the arm length parameters of the human upper limb, thus completing the calibration of the arm length of the human upper limb.

[0058] Step 4: Based on the calibrated upper limb arm length and the joint rotation angle of the upper limb obtained after IMU attitude calculation, construct the forward kinematic model of the upper limb using the improved DH method, and calculate the trajectory of the upper limb end.

[0059] Step 5: Transmit the trajectory data of the human upper limb end to the slave robot, so that the slave robot can move to the corresponding position and perform teleoperation.

[0060] Step six: Collect robot force information through the end-effector force sensor of the robot and transmit the force information to the master controller. The motor drives the flexible rope to pull the human upper limb to achieve force feedback.

[0061] Implementation scheme of the present invention:

[0062] 1. In step one, the backpack worn on the back of the body is like... Figure 1 As shown, the backpack consists of an aluminum frame, a motor bracket, a motor, a rope shaft, a flexible rope, an IMU bracket, an IMU0, and a strap. The strap, motor bracket, and IMU bracket are mounted on the aluminum frame. The IMU0 is mounted on the IMU bracket, the motor is mounted on the motor bracket, the rope shaft is mounted on the motor, and the flexible rope is wrapped around the rope shaft. An IMU1 worn on the upper arm is shown... Figure 2 As shown, the IMU1 can be worn on any part of the upper arm, and its position is aligned with the IMU0 on the backpack. The IMU2 worn on the forearm of the upper limb is as follows... Figure 2 As shown, the IMU2 position is aligned with the backpack IMU0 position. The flexible rope anchor point worn on the wrist of the upper limb is as follows... Figure 2 As shown, it is divided into flexible rope anchor points at the arm and flexible rope anchor points at the wrist.

[0063] 2. In step two, turn on the motor power to enable the encoder in the motor to enter the working state. With the shoulder as the starting point, and the upper limb of the human body in a natural vertical position, stretch the flexible rope to the wrist flexible rope anchor point worn by the upper limb of the human body. At this time, the encoder records the change in the length of the flexible rope as L1.

[0064] 3. In step three, the motor is started and slowly rotated, gently pulling the upper limb of the human body through the connected flexible wrist cord. This causes the upper limb to gradually change from a natural vertical position to a bent position. The change in cord length L2 during this process is recorded by the encoder in the motor, and the change in the position of the upper limb is recorded by two IMUs worn on the upper limb. After data acquisition, the quaternion data q1 and q2 collected by IMU1 and IMU2 are used to calculate the posture of the upper limb and obtain the change in elbow angle. Based on the collected rope length changes L1 and L2 and the calculated elbow angle change... The function f(x) is solved using the nonlinear programming function fmincon() in Matlab software, thereby determining the arm length x1 and forearm length x2 of the human upper limb and completing the forward kinematic calibration of the human upper limb.

[0065] The specific method is as follows:

[0066] 1) Transform quaternions q1 and q2 into rotation matrices R1 and R2:

[0067] q=w+xi+yj+zk

[0068]

[0069] 2) The rotation matrix R1 transforms into R2. tra :

[0070] R tra =R2 -1 R1

[0071] 3).R tra The matrix is ​​decomposed into three Euler angles based on the fixed frame, where The change in angle corresponding to the elbow:

[0072]

[0073]

[0074] θ=atan2(R tra (2, 1), R tra (1,1))

[0075] ψ=atan2(R tra (3,2),R tra (3,2))

[0076] 4). The functional relationship f(x) is derived from... Figure 3 have to:

[0077]

[0078]

[0079]

[0080]

[0081] L2 2 =x2 2 +x3 2 -2x2x3cosβ

[0082]

[0083] Use the fmincon() function in Matlab software for nonlinear programming to find the optimal solution for the arm length x1 and forearm length x2 of the human upper limb.

[0084] 4. In step 4, the rotation matrix obtained between IMU0 and IMU1 is decomposed into three Euler angles according to the fixed frame, corresponding to the three degrees of freedom of the human upper limb shoulder. Among them, angle θ corresponds to adduction-abduction of the shoulder, and angle ψ corresponds to pronation-supination of the shoulder. The angle corresponds to the flexion-extension of the shoulder. The rotation matrix obtained between IMU1 and IMU2 is decomposed into three Euler angles in a fixed frame, corresponding to the two degrees of freedom of the elbow and forearm. The angle corresponds to elbow flexion-extension, and the angle ψ corresponds to forearm pronation-supination. Based on the obtained arm length x1 and forearm length x2, a forward kinematic model of the human upper limb can be constructed using the improved DH method. The homogeneous transformation matrix for the distal position of the human upper limb is:

[0085]

[0086] Where matrix A is:

[0087]

[0088] The coordinates and posture of the human upper limb end are obtained.

[0089] 5. In step 5, the trajectory data of the human upper limb end is transmitted to the slave robot using the TCP / IP communication mechanism. The position of the end and the robot end is matched so that the slave robot moves to the corresponding position to perform teleoperation.

[0090] 6. In step 6, the robot force information is collected by the end-effector force sensor of the robot and transmitted to the master controller. The force feedback is achieved by driving the flexible rope to pull the human upper limb through the rotation of the motor.

[0091] This technical solution utilizes a backpack-like design and an IMU worn on the upper limb to achieve remote operation using the human upper limb as the control platform, avoiding the limitations of remote operation devices that require a fixed platform. Furthermore, by using the upper limb as the master remote operation platform, the slave robots can be controlled without altering the user's operating habits.

[0092] By using ropes to pull the upper limbs of the human body, the remote operation force feedback function is realized, avoiding the disadvantage of lacking tactile feedback in visual remote operation schemes.

[0093] Using a backpack-style design and an IMU worn on the upper limbs, this method is simpler to wear, more convenient to operate, and cheaper than exoskeleton-based teleoperation solutions.

[0094] The self-calibration method can automatically learn the wearer's arm length without prior measurement of the wearer's upper limb data for kinematic calculations, and can adapt to people with various arm lengths.

[0095] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0096] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A teleoperation method using a combination of IMU and tethered traction, with the human upper limb as the master teleoperation controller, characterized in that: The steps are as follows: Step 1: Wear a backpack on the back of the human body, wear an IMU on the arm of the human body, wear an IMU on the forearm of the human body, and wear flexible rope anchors on the arm and wrist of the human body. Step 2: Connect the flexible rope located in the backpack to the flexible rope anchor point at the wrist of the upper limb; Step 3: The motor in the backpack drives the flexible cable to pull the upper limb of the human body, causing the upper limb to move. The IMU module collects the data of the upper limb, the encoder collects the change in cable length, and then the optimization algorithm is used to obtain the arm length parameters of the human upper limb, thus completing the calibration of the arm length of the human upper limb. Step 4: Based on the calibrated upper limb arm length and the joint rotation angle of the upper limb obtained after IMU attitude calculation, construct the forward kinematic model of the upper limb using the improved DH method, and calculate the trajectory of the upper limb end. Step 5: Transmit the trajectory data of the human upper limb end to the slave robot, so that the slave robot can move to the corresponding position and perform teleoperation. Step 6: Collect robot force information through the end-effector force sensor and transmit the force information to the master controller. The motor drives the flexible rope to pull the human upper limb to achieve force feedback.

2. The teleoperation method according to claim 1, which uses a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, is characterized in that: The backpack consists of an aluminum frame, a motor bracket, a motor, a rope shaft, a flexible rope, an IMU bracket, an IMU module, and a pack strap. The pack strap, motor bracket, and IMU bracket are mounted on the aluminum frame, the IMU module is mounted on the IMU bracket, the motor is mounted on the motor bracket, the rope shaft is mounted on the motor, and the flexible rope is wrapped around the rope shaft.

3. The teleoperation method according to claim 2, which uses a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, is characterized in that: According to step one, wear an IMU on any part of the upper arm of the human body and set it as IMU1. The position of IMU1 is aligned with the position of the IMU on the backpack. An IMU is worn on the forearm of the upper limb and designated as IMU2. The position of IMU2 is aligned with the position of the IMU on the backpack. Flexible rope anchors are worn on the upper limbs of the human body, including flexible rope anchors at the arm and wrist.

4. The teleoperation method according to claim 3, which uses a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, is characterized in that: According to step two, turn on the motor power to enable the encoder in the motor to enter the working state. With the shoulder as the starting point, and the upper limb of the human body in a natural vertical position, stretch the flexible rope to the wrist flexible rope anchor point worn by the human body. At this time, the encoder records the change in the length of the flexible rope as L1.

5. A teleoperation method according to claim 4, using a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, characterized in that: According to step three, the motor is started and slowly rotated, and the upper limb of the human body is slowly pulled by the flexible rope connected to the wrist, so that the upper limb of the human body slowly changes from a natural vertical state to a bent state. The change in rope length L2 during the process is recorded by the encoder in the motor, and the change in position of the human body's upper limb is recorded by two IMUs worn on the human body's upper limb. After data acquisition, the posture of the human body's upper limb is calculated by using the quaternion data q1 and q2 collected by IMU1 and IMU2 to obtain the change in elbow angle φ. Based on the collected changes in rope length L1 and L2 and the calculated change in elbow angle φ, the function f(x) is solved using the nonlinear programming fmincon() function in Matlab software, thereby determining the arm length x1 and forearm length x2 of the human body's upper limb, and completing the positive kinematic calibration of the human body's upper limb.

6. A teleoperation method according to claim 5, using a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, characterized in that: The specific method is as follows: 1) Transform quaternions q1 and q2 into rotation matrices R1 and R2: 2) The rotation matrix R1 transforms into R2. tra : 3).R tra The matrix is ​​decomposed into three Euler angles based on the fixed frame, where angle φ corresponds to the angular change at the elbow: 4) The functional relationship is f(x): Use the nonlinear programming function fmincon() in Matlab to find the optimal lengths of the upper limbs x1 and forearm x2.

7. A teleoperation method according to claim 6, using a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, characterized in that: Based on step four, the rotation matrix obtained between the IMU on the backpack and the IMU worn on the upper arm is decomposed into three Euler angles according to the fixed system, corresponding to the three degrees of freedom of the upper arm shoulder. Among them, the angle θ corresponds to the adduction-abduction of the shoulder, the angle ψ corresponds to the pronation-supination of the shoulder, and the angle φ corresponds to the flexion-extension of the shoulder. The rotation matrix obtained between the IMU worn on the upper arm and the IMU worn on the forearm is also decomposed into three Euler angles according to the fixed system, corresponding to the two degrees of freedom of the elbow and forearm. Among them, the angle φ corresponds to the flexion-extension of the elbow, and the angle ψ corresponds to the pronation-supination of the forearm. Then, based on the obtained upper arm length x1 and forearm length x2, the improved DH method is used to construct the forward kinematic model of the upper limb.

8. A teleoperation method according to claim 7, using a combination of IMU and cable traction with the human upper limb as the master teleoperation controller, characterized in that: The homogeneous transformation matrix of the distal end position of the human upper limb is: Where matrix A is: The coordinates and posture of the human upper limb end are obtained.

9. A teleoperation method using a combination of IMU and cable traction, with the human upper limb as the master teleoperation controller, as described in claim 1 or 8, characterized in that: According to step five, the trajectory data of the human upper limb end is transmitted to the slave robot using the TCP / IP communication mechanism. The position of the end and the robot end is matched so that the slave robot moves to the corresponding position to perform teleoperation.

10. A teleoperation method using a combination of IMU and cable traction, with the human upper limb as the master teleoperation controller, as described in claim 9, characterized in that: According to step six, the robot force information is collected by the end-effector force sensor of the robot and transmitted to the master controller. The motor drives the flexible rope to pull the human upper limb to achieve force feedback.

Images