Upper extremity exoskeleton control method

Abstract

This invention discloses an upper limb exoskeleton control method, comprising: fixing a monocular vision sensor to the user's wrist with the lens facing the user's torso to collect spatial position information at the wrist; using the EPnP algorithm to inversely calculate the sensor's pose based on feature points at the wrist to obtain a rotation matrix and translation vector; simultaneously collecting surface electromyography (EMG) signals and predicting intent through feature extraction and classification models; synchronizing the visual data and EMG data in time and performing multimodal fusion to predict the end-effector position and velocity at future moments; mapping three-dimensional motion to a two-dimensional plane, simplifying it into a two-link inverse kinematics problem, and analytically calculating joint angles and angular velocities; and adjusting the torque of the exoskeleton motors in real time based on joint angles, angular velocities, and EMG activation intensity. This upper limb exoskeleton control method of the present invention installs the vision sensor backwards at the wrist facing the body and solves the problem of scale uncertainty in monocular vision by adding kinematic constraints through monocular vision.

Description

Technical Field

[0001] This invention specifically relates to a method for controlling an upper limb exoskeleton. Background Technology

[0002] Currently, common exoskeleton control methods mainly include electromyography (EMG) control, electroencephalography (EEG) control, hybrid control methods, and torque sensing control. For example, EMG control collects electrical signals (EMG signals) during muscle contraction and decodes the user's movement intentions. Hybrid control uses multi-sensor fusion (sEMG + IMU + force sensor) data for control, ultimately sending drive signals to drive the upper limb exoskeleton. Drives typically use motor drive modules, including servo motors and stepper motors, which receive output signals from the analysis and planning module through a control module to drive each joint to complete the corresponding task. Existing methods using visual sensors for human posture recognition and prediction typically involve cameras mounted on body parts or the head, detecting skeletal points in both arms with the body as the center. Alternatively, a camera array can be deployed in a specific space to capture the user's posture. The main drawback of existing technologies is that...

[0003] Upper limb exoskeleton control systems require a trade-off between accuracy, real-time performance, cost, and comfort. Different exoskeleton control methods are needed for different application scenarios. Hybrid control systems are complex and expensive. Furthermore, the upper limbs differ from the lower limbs; their movement patterns are more complex, and the flexibility between the upper limbs and the exoskeleton is insufficient. User-centered visual solutions typically mount cameras on the chest or head. Constrained by mounting location and lens focal length, posture recognition is limited to areas captured by cameras on the front of the body. When the arm is raised or moves to the side, positional information cannot be captured, and positioning is impossible when objects obstruct the view during handling or production work. Spatial vision-based motion capture systems utilize multiple high-quality cameras to record the spatial position of markers attached to the human body. These systems are accurate but complex, expensive, and inconvenient. Summary of the Invention

[0004] This invention provides an upper limb exoskeleton control method to solve the aforementioned technical problems, specifically adopting the following technical solution:

[0005] A method for controlling an upper limb exoskeleton includes the following steps:

[0006] S1: Pre-measure the user's upper limb size parameters, including upper arm length L1 and forearm length L2, and establish a four-degree-of-freedom kinematic model of the human upper limb and exoskeleton;

[0007] S2: Fix a monocular vision sensor to the user's wrist with the lens facing the user's torso to collect spatial position information at the wrist.

[0008] S3: The EPnP algorithm is used to inversely calculate the sensor pose based on the feature points at the wrist, obtain the rotation matrix and translation vector, and perform scale anchoring through the kinematic model to restore the absolute scale;

[0009] S4: Synchronously acquire surface electromyography signals, and predict muscle activation direction and intensity through feature extraction and classification models;

[0010] S5: Time-align visual data with electromyographic data, perform multimodal fusion, and predict the end position and velocity at future moments;

[0011] S6: Map the three-dimensional motion to a two-dimensional plane, simplify it into a two-link inverse kinematics problem, and analytically calculate the joint angles and angular velocities;

[0012] S7: Adjusts the torque of the exoskeleton motor in real time based on joint angle, angular velocity, and electromyographic activation intensity to achieve compliant assistance.

[0013] Furthermore, in step S3, the EPnP algorithm is used to inversely calculate the sensor pose based on feature points at the wrist, thereby obtaining the rotation matrix R. W C e st and translation vector t W C e st; The output of the EPnP algorithm is scale-anchored by a kinematic model to ensure that the pose estimation of the visual sensor is consistent with the actual kinematic model of the human upper limb, thereby restoring the absolute scale;

[0014] The constraint formula is:

[0015]

[0016] Among them, T c→b The kinematic model of the upper limb established in step S1, T w→b The calibration matrix is ​​used to convert the relationship between the world coordinate system and various calibration points on the body into fixed position coordinates. c→w This is the rotation matrix from the world coordinate system to the camera coordinate system.

[0017] Furthermore, in step S5, the time alignment of visual data and electromyographic data is achieved through the following steps:

[0018] Calculate the visual signal timestamp t v and electromyographic signal timestamp t e Time difference between them:

[0019] Δt=|t v -t e |

[0020] If Δt ≤ 50ms, the data is considered valid, and a two-way time-weighted average alignment is performed. The alignment formula is as follows:

[0021]

[0022] Here, w1 and w2 are the weighting coefficients of the visual signal and the electromyographic signal, respectively, used to adjust the contribution ratio of the two in the time alignment.

[0023] Furthermore, in step S6, the method for mapping three-dimensional motion to a two-dimensional plane is as follows:

[0024] Keeping the upper arm length L1 constant, the forearm length L2′ is calculated based on the abduction angle q3:

[0025]

[0026] Wherein, q3 is the abduction angle of the forearm relative to the upper arm, which is obtained by subtracting the angle between the two-dimensional plane obtained by the shoulder angle sensor and the body from the angle information of the visual sensor, thereby simplifying the complex three-dimensional motion into a planar two-bar motion.

[0027] Furthermore, in step S6, the analytical solution of the inverse kinematics of the planar two-link is given by the following formula:

[0028] Assuming the end-effector coordinates (x, y) are known, along with the upper arm length L1 and the converted forearm length L2′, the analytical solutions for joint angles θ1 and θ2 are:

[0029]

[0030] Where θ1 is the shoulder joint angle and θ2 is the elbow joint angle, the joint angles are obtained by analyzing the above formula to ensure the uniqueness of the inverse kinematics solution, thereby achieving precise control of the upper limb exoskeleton.

[0031] Furthermore, in step S6, the end velocity mapping formula is:

[0032]

[0033] Where the Jacobian matrix J is:

[0034]

[0035] The end velocities ẋ and ẏ are mapped to joint angular velocities using the Jacobian matrix J. and It is used for speed feedforward control of motors.

[0036] Furthermore, in step S7, the motor torque τ is obtained by linear mapping of the electromyography RMS percentage:

[0037]

[0038] Where k is the gain coefficient, and RMS is the real-time root mean square value of electromyography. max τ is the root mean square value of electromyography during maximal voluntary contraction. max The maximum torque of the motor is represented by this formula, which maps the electromyographic activation intensity to the motor torque to achieve compliant assistance.

[0039] Furthermore, in step S3, the translation vector t output by the EPnP algorithm W C e st is used to calculate velocity information in various directions from the vision sensor:

[0040]

[0041]

[0042]

[0043] Where Δt is the sampling period, t x t y t z v represents the components of the translation vector. x v y v z This represents the speed at which the sensor moves in each direction.

[0044] Furthermore, in step S4, the feature extraction of the electromyographic signal includes calculating the root mean square value and waveform length feature of the electromyographic signal, and using the BioSPPy feature extraction tool to transform the original signal into a description that can characterize the movement intention.

[0045] Furthermore, in step S7, the control signal of the exoskeleton motor predicts the joint angle and joint velocity at future moments through the covariance of the state equation of the extended Kalman filter. Combined with the real-time calculated joint angle, angular velocity and electromyographic torque command, the control signal of the motor is generated, so that the exoskeleton can be driven to the human upper limb according to the target displacement and velocity, achieving a compliant and low-latency assistive effect.

[0046] The advantage of this invention lies in the upper limb exoskeleton control method provided, which mounts the visual sensor in reverse to the wrist facing the body, and solves the problem of scale uncertainty in monocular vision by adding kinematic constraints through monocular vision. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the upper limb exoskeleton structure of this application;

[0049] Figure 2 This is a schematic diagram of the upper limb exoskeleton control method of this application;

[0050] Figure 3 This is a schematic diagram of a user wearing an upper limb exoskeleton according to this application;

[0051] Figure 4 This is a schematic diagram of the upper limb exoskeleton sensor of this application;

[0052] Figure 5 This is a simplified workspace diagram of this application. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0054] The specific hardware design of the exoskeleton, such as Figure 1As shown, the device consists of three main parts: a shoulder support section, two four-bar linkages, and a back support. Specifically, the shoulder support section comprises a motor support block 6, a motor mounting position 7, a slide rail fixing base 8, a lightweight slide rail 9, a lightweight slider 10, and an upper arm support 11. The four shaft holes on the top of the motor support block are interconnected with the four-bar linkage structure. The four-bar linkage structure (first link 2, second link 3, third link 4, and fourth link 5) is assembled with the four shaft holes of the four-bar support base 1 and finally fixed to the back support section. The interaction of the four links allows the motor center point above the motor support block 6 to move around a planned curve during movement, ensuring that the motor's rotation plane always matches the rotation center during shoulder internal and external rotation. In the shoulder section, the upper arm is worn with the upper arm support 11 via an elastic strap. After the support is assembled with the lightweight slider 10, it is mounted on the lightweight slide rail 9, allowing free up-and-down movement. Combined with the up-and-down movement of the shoulder strap section, the height can also be adjusted according to different body sizes. The back support consists of two parts: the back support 13 and the waist back panel 14, which are connected by the support structure beam 12. The waist side back panel 15 is connected to the waist back panel 14 by a rotating hinge, which can accommodate different waist widths.

[0055] like Figure 2 The image shows a method for controlling an upper limb exoskeleton according to this application, comprising the following steps:

[0056] S1: Pre-measure the user's upper limb size parameters, including upper arm length L1 and forearm length L2, and establish a four-degree-of-freedom kinematic model of the human upper limb and exoskeleton.

[0057] Specifically, such as Figure 3 As shown, after the user wears the upper limb exoskeleton and the visual sensors at the wrist, the values ​​of L1 and L2 are measured to prepare for modeling the overall system. In this application, the forearm length L2 is specifically the length from the elbow joint to the sensor (described later).

[0058] The overall system modeling process involves mathematically modeling the upper limb movements of the human body after wearing an upper limb exoskeleton. This transforms the complex upper limb and exoskeleton movements into computable kinematic formulas, aiding in the calculation of joint angles and related kinematic parameters. After wearing the exoskeleton, the human upper limb (unilateral) is constrained as a four-degree-of-freedom link. The kinematic modeling of this link is described using a DHtable, as shown in the table below.

[0059] Table 1. Kinematic Modeling DH Table

[0060]

[0061] The human upper limb exoskeleton, once restrained, is connected by 4-DOF linkages. A DH table is used to record the rotational and displacement operations between each linkage. θi Let a be the angle of the i-th joint. i α is the distance between the axes of rotation. i d is the torsional angle between the connecting shafts. i This represents the joint offset. Since joints 1, 2, and 4 are rotational joints, their offset is 0. Link 3 represents the upper arm L1. Since the length of L2 is the end-effector offset value, it is not included in this kinematic DHtable expression. The value of L2 is used in conjunction with simplified planar kinematics. The measured upper limb dimensions are substituted into the kinematic formula after the user is wearing the device, and kinematic constraints are applied using monocular vision to prevent the problem of monocular visual scale offset.

[0062]

[0063] A1-A4 represent the transformation relationships between the various links, T tool Using the tool coordinate system, the position of the wearable sensor is represented in the external skeleton, and the overall transformation matrix is ​​finally calculated.

[0064]

[0065]

[0066] Wherein, q 234 Let q2 + q3 + q4 be the values, where q represents the rotation angle of each joint in the DHtable. The purpose of establishing overall exoskeleton kinematic constraints is to solve for uncertainties in the position of the end effector through forward and inverse kinematics calculations, using data from the end effector to calculate the rotation angle of the shoulder joint. This kinematic data is then substituted into the spatial position acquired by the vision sensor.

[0067] S2: Fix a monocular vision sensor to the user's wrist with the lens facing the user's torso to collect spatial position information at the wrist.

[0068] like Figure 4 As shown, a monocular vision sensor is fixed to the user's wrist with the lens facing the user's torso. The user extends both arms forward, allowing the sensor to identify feature points for calibration. The feature points are configured by determining five control points in the world coordinate system, with one as a backup. The positions of the control points are bound to the exoskeleton dimensions, and a new coordinate system is established.

[0069] S3: Using the EPnP algorithm, the sensor pose is inferred from the feature points at the wrist to obtain the rotation matrix R. W C e st and translation vector t W C e st, and scale anchoring is performed through kinematic models to restore the absolute scale.

[0070] The formula for calculating the camera's own pose is as follows:

[0071]

[0072] Parameter T w C e `st` is the transformation matrix estimated by the camera from the world coordinate system W to the camera coordinate system C, where the world coordinate system is the human-centered coordinate system. EPnP provides visual estimation, with the visual sensor at the wrist serving as the camera coordinate system C, relative to the world coordinate system (human body) W in terms of pose. Visual estimation using the EPnP algorithm includes the rotation matrix R. W C e st and translation vector t W C e st. Because at this point the camera pose is unaware of the kinematic constraints it is subject to, the camera pose only knows its own position relative to the body. Since monocular vision has scale ambiguity, it is necessary to constrain the rotation matrix calculated by EPnP with the known upper limb kinematics.

[0073]

[0074] In the constraint equation, T c→b For the kinematic model of the upper limb established in step S1, T c→b equal to T total T w→b The calibration matrix is ​​used to convert the relationship between the world coordinate system and various calibration points on the body into fixed position coordinates. c→w This is the rotation matrix from the world coordinate system to the camera coordinate system, output in real time by the EPnP algorithm. When the kinematic model experiences joint angle shifts due to accumulated errors, the EPnP observations (from monocular vision) on the right side of the constraint equation force the kinematic model on the left side to output a pose that matches them. In optimization problems, we usually don't directly require the equation to hold strictly, but rather transform it into a least-squares problem. The residuals are defined to solve for the joint angle θ that satisfies the visual observation. This operation is equivalent to providing the kinematic model with absolute spatial anchor points, solving the shift problem in monocular vision.

[0075] The translation vector t output by the EPNP algorithm W C e `st` represents the velocity information in each direction from the computational vision sensor, and `t` represents the position in each direction of the output. The specific formula is as follows:

[0076]

[0077] The translation of frame k and frame (k+1) are as follows:

[0078]

[0079]

[0080] Calculate the velocity components in each direction:

[0081]

[0082]

[0083]

[0084] Where Δt is the sampling period, t x t y t z v represents the components of the translation vector. x v y v z The moving speed of the sensor in the X-axis, Y-axis and Z-axis directions.

[0085] S4: Simultaneously acquire surface electromyography signals, and predict muscle activation direction and intensity through feature extraction and classification models.

[0086] Feature extraction of electromyography (EMG) signals includes calculating the root mean square value and waveform length features of the EMG signals, and then using the BioSPPy feature extraction tool to transform the raw signals into descriptions that can characterize the movement intention.

[0087] Specifically, while the wrist position information is obtained visually, the exoskeleton control also requires prediction of future posture. This application's control method uses electromyography (EMG) signals to predict arm muscle activation in advance, thereby enhancing the exoskeleton's compliance.

[0088] Electromyography (EMG) signals are acquired by Figure 4 The electromyography (EMG) sensors worn by the participant acquire signals from the biceps brachii muscle in the upper arm and the flexor carpi ulnaris muscle in the forearm. EMG feature extraction is performed using the existing BioSPPy feature extraction tool to transform the raw signals into descriptions representing movement intentions. A movement direction classification model is used, with inputs of a multi-channel feature vector (e.g., 8 channels × 10 features = 80 dimensions) and outputs of discrete movement direction labels (e.g., "upper arm flexion," "forearm supination"). By collecting EMG signals from the participant's forward and backward arm movements and lateral elevation, a classification algorithm predicts arm muscle activation, identifies the approximate movement direction, and records the activation of individual muscles.

[0089] S5: Time-align visual data with electromyographic data, perform multimodal fusion, and predict the end position and velocity at future moments.

[0090] EMG (electromyography) data only has intention prediction capabilities and cannot record relative spatial movement. Therefore, it is necessary to fuse EMG data with intention prediction capabilities with a visual sensor capable of recording relative spatial position. To synchronize the information between the two sensors, the EMG signals and visual signals need to be time-aligned. In the embodiments of this application, the time alignment of visual data and EMG data is achieved through the following steps:

[0091] Calculate the visual signal timestamp t v and electromyographic signal timestamp t e Time difference between them:

[0092] Δt=|t v -t e |

[0093] If signal loss and communication fluctuations occur between the two sensors during signal acquisition, leading to signal asynchrony and a mismatch between joint angle and muscle activation, the data is considered invalid if the time difference is too large (Δt > 50ms). If Δt ≤ 50ms, the data is considered valid, and a bidirectional time-weighted average alignment is performed. The alignment formula is as follows:

[0094]

[0095] Here, w1 and w2 are the weighting coefficients of the visual signal and the electromyographic signal, respectively, used to adjust the contribution ratio of the two in the time alignment.

[0096] S6: Map the three-dimensional motion to a two-dimensional plane, simplify it into a two-link inverse kinematics problem, and analytically obtain the joint angles and angular velocities.

[0097] like Figure 5As shown, D is the virtual working plane where the instantaneous motor is located, E is the trajectory of the upper arm at that instantaneous moment, F is the rotation center of the instantaneous motor, G is the movement trajectory of the motor's rotation center, and P is the rotation center of the glenohumeral joint. The two-dimensional mapping step is to match the posture analysis data with the exoskeleton structure, mapping the complex motion in three-dimensional space onto the two-dimensional plane D where the upper limb exoskeleton motor is located. According to the exoskeleton's structural design, the motor plane is fixed on the shoulder structure around the virtual rotation center. According to the four-degree-of-freedom upper limb kinematics description, the rotation angle of the motor is q2, which is also the angle value that needs to be solved in inverse kinematics. q1 is the rotation of the virtual rotation center of the shoulder around the axis perpendicular to the ground at point P. According to the exoskeleton design, this degree of freedom is a passive degree of freedom. The forearm's left and right abduction and adduction movements around the upper arm are q3, and q4 is the forearm's up and down flexion and extension movements around the upper arm. The two-dimensional mapping method simplifies the upper limb motion into a passive rotation of the motor plane around q1. Since the forearm moves left and right abduction and adduction around the upper arm, trigonometric functions are used to map the motion of q3 onto the plane. After mapping, the complex four-degree-of-freedom inverse kinematics solution is simplified to the kinematics of a two-linkage linkage. Furthermore, by omitting the passive degrees of freedom, the inverse kinematics solution is unique, avoiding multiple solutions that would require secondary verification.

[0098] In the embodiments of this application, the method for mapping three-dimensional motion to a two-dimensional plane is as follows:

[0099] Keeping the upper arm length L1 constant, L1′=L1, the forearm length L2′ is calculated based on the abduction angle q3:

[0100]

[0101] Here, q3 is the abduction angle of the forearm relative to the upper arm, obtained by subtracting the angle between the two-dimensional plane obtained by the shoulder angle sensor and the body from the x-angle information of the visual sensor, thus simplifying the complex three-dimensional motion into a planar two-link motion. As shown in the table below, by simultaneously mapping the upper limb as a planar two-link and performing kinematic modeling on it, solving its inverse kinematic formula and velocity Jacobian, and performing inverse kinematic calculation on the spatial coordinates of future time, the exoskeleton can respond in advance.

[0102] Table 2. Kinematic Modeling DH Table

[0103]

[0104] In the embodiments of this application, the analytical solution of the inverse kinematics of the planar two-link is given by the following formula:

[0105] Assuming the end-effector coordinates (x, y) are known, along with the upper arm length L1 and the converted forearm length L2′, the analytical solutions for joint angles θ1 and θ2 are:

[0106]

[0107] Where θ1 is the shoulder joint angle and θ2 is the elbow joint angle, the joint angles are obtained by analyzing the above formula to ensure the uniqueness of the inverse kinematics solution, thereby achieving precise control of the upper limb exoskeleton. It equals q2 in Table 2. It equals q4 in Table 2, so by calculation This tells us the motor angle that needs to be controlled.

[0108] In the embodiments of this application, in step S6, the terminal velocity mapping formula is:

[0109]

[0110] ẋ and ẏ are the velocities of the terminal points in Cartesian space. and It is the joint angular velocity. The Jacobian matrix J is:

[0111]

[0112] Specifically, the forward kinematic equations of the planar two-link linkage are:

[0113]

[0114] Taking the derivative with respect to time, we obtain the terminal velocity:

[0115]

[0116] Organize into matrix form:

[0117]

[0118] Jacobian matrix of a planar bilink:

[0119]

[0120] S7: Adjusts the torque of the exoskeleton motor in real time based on joint angle, angular velocity, and electromyographic activation intensity to achieve compliant assistance.

[0121] The system identifies and controls the motor's start and stop based on the activation direction of electromyographic (EMG) signals. Forearm supination and external swing movements do not activate the motor, while upper arm forward and backward movements activate it, providing initial velocity. Simultaneously, the angle calculated from inverse kinematics and the velocity derived from Jacobian are sent to the motor controller after time alignment. Combining the strength of the EMG signal at that moment, the system maps the EMG RMS percentage to the motor torque (0-20 Nm) to control the motor torque. This controls the upper limb exoskeleton actuator to drive the upper limb to the target displacement, angle, and velocity, ensuring compliant movement between the human and mechanical joints. While the user is exercising wearing the exoskeleton, the processor calculates the real-time posture to determine the user's posture at time T and predicts future joint angles and velocities based on the extended Kalman kinetic equation covariance to generate control signals for the motor.

[0122] In the embodiments of this application, the motor torque τ is obtained by linear mapping of the percentage of electromyography RMS:

[0123]

[0124] Where k is the gain coefficient, and RMS is the real-time root mean square value of electromyography. max τ is the root mean square value of electromyography during maximal voluntary contraction. max The maximum torque of the motor is represented by this formula, which maps the electromyographic activation intensity to the motor torque to achieve compliant assistance.

[0125] In step S7, the control signal of the exoskeleton motor predicts the joint angle and joint velocity at future moments through the covariance of the state equation of the extended Kalman filter. Combined with the real-time calculated joint angle, angular velocity and electromyographic torque command, the control signal of the motor is generated, so that the exoskeleton can be driven to the human upper limb according to the target displacement and velocity, achieving a compliant and low-latency assistive effect.

[0126] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A method for controlling an upper limb exoskeleton, characterized in that, Includes the following steps: S1: Pre-measure the user's upper limb size parameters, including upper arm length L1 and forearm length L2, and establish a four-degree-of-freedom kinematic model of the human upper limb and exoskeleton; S2: Fix a monocular vision sensor to the user's wrist with the lens facing the user's torso to collect spatial position information at the wrist. S3: The EPnP algorithm is used to inversely calculate the sensor pose based on the feature points at the wrist, obtain the rotation matrix and translation vector, and perform scale anchoring through the kinematic model to restore the absolute scale; S4: Synchronously acquire surface electromyography signals, and predict muscle activation direction and intensity through feature extraction and classification models; S5: Time-align visual data with electromyographic data, perform multimodal fusion, and predict the end position and velocity at future moments; S6: Map the three-dimensional motion to a two-dimensional plane, simplify it into a two-link inverse kinematics problem, and analytically calculate the joint angles and angular velocities; S7: Adjusts the torque of the exoskeleton motor in real time based on joint angle, angular velocity, and electromyographic activation intensity.

2. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S3, the EPnP algorithm is used to inversely calculate the sensor pose based on the feature points at the wrist, and the rotation matrix R is obtained. W C e st and translation vector t W C e st; The output of the EPnP algorithm is scale-anchored by a kinematic model to ensure that the pose estimation of the visual sensor is consistent with the actual kinematic model of the human upper limb, thereby restoring the absolute scale; The constraint formula is: ； Among them, T c→b The kinematic model of the upper limb established in step S1, T w→b The calibration matrix is ​​used to convert the relationship between the world coordinate system and various calibration points on the body into fixed position coordinates. c→w Output the rotation matrix from the world coordinate system to the camera coordinate system.

3. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S5, the time alignment of visual data and electromyographic data is achieved through the following steps: Calculate the visual signal timestamp t v and electromyographic signal timestamp t e Time difference between them: ; If Δt ≤ 50ms, the data is considered valid, and a two-way time-weighted average alignment is performed. The alignment formula is as follows: ; Here, w1 and w2 are the weighting coefficients of the visual signal and the electromyographic signal, respectively, used to adjust the contribution ratio of the two in the time alignment.

4. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S6, the method for mapping three-dimensional motion to a two-dimensional plane is as follows: Keeping the upper arm length L1 constant, the converted upper arm length L1′ is: L1′=L 1; Forearm length L2′ is calculated based on abduction angle q3: ; Wherein, q3 is the abduction angle of the forearm relative to the upper arm, which is obtained by subtracting the angle between the two-dimensional plane obtained by the shoulder angle sensor and the body from the angle information of the visual sensor, thereby simplifying the complex three-dimensional motion into a planar two-bar motion.

5. The upper limb exoskeleton control method according to claim 4, characterized in that, In step S6, the analytical solution of the inverse kinematics of the planar two-link system is given by the following formula: Assuming the end-effector coordinates (x, y) are known, along with the adjusted upper arm length L1′ and forearm length L2′, the analytical solutions for joint angles θ1 and θ2 are: ; Where θ1 is the shoulder joint angle and θ2 is the elbow joint angle, the joint angles are obtained by analyzing the above formula to ensure the uniqueness of the inverse kinematics solution, thereby achieving precise control of the upper limb exoskeleton.

6. The upper limb exoskeleton control method according to claim 5, characterized in that, In step S6, the end velocity mapping formula is: ; Where the Jacobian matrix J is: ; The terminal velocity is represented by the Jacobian matrix J. and Mapped to joint angular velocity and It is used for speed feedforward control of motors.

7. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S7, the motor torque τ is obtained by linear mapping of the percentage of electromyography (EMG) RMS: ; Where k is the gain coefficient, and RMS is the real-time root mean square value of electromyography. max τ is the root mean square value of electromyography during maximal voluntary contraction. max The maximum torque of the motor is represented by this formula, which maps the electromyographic activation intensity to the motor torque to achieve compliant assistance.

8. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S3, the translation vector t output by the EPnP algorithm W C e st is used to calculate velocity information in various directions from the vision sensor: ; ; ; Where Δt is the sampling period, t x t y t z v represents the components of the translation vector. x v y v z This represents the speed at which the sensor moves in each direction.

9. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S4, the feature extraction of the electromyographic signal includes calculating the root mean square value and waveform length features of the electromyographic signal, and using the BioSPPy feature extraction tool to transform the original signal into a description that can characterize the movement intention.

10. The upper limb exoskeleton control method according to claim 1, characterized in that, In step S7, the control signal of the exoskeleton motor predicts the joint angle and joint velocity at future moments through the covariance of the state equation of the extended Kalman filter. Combined with the real-time calculated joint angle, angular velocity and electromyographic torque command, the control signal of the motor is generated, so that the exoskeleton can be driven to the human upper limb according to the target displacement and velocity, achieving a smooth and low-latency assistive effect.

Images