Hand exoskeleton robot and synchronous perception method thereof

Abstract

The application relates to the technical field of medical devices, and provides a hand exoskeleton robot and a synchronous sensing method thereof. The robot comprises a hand fixing assembly, at least one flexible execution module, a motion compatibility mechanism, a control and data processing unit, a driving assembly and a multi-modal sensing module. One end of the flexible execution module is connected to the hand fixing assembly. The driving assembly is connected to the flexible execution module through a flexible transmission member. The motion compatibility mechanism is arranged on the flexible execution module and is used for adaptively compensating for the relative displacement and angle deviation between the flexible execution module and the human finger joint during the flexing and stretching movement of the finger. The multi-modal sensing module is used for synchronously collecting multi-modal sensing information related to the movement and stress. The hand exoskeleton robot is matched with the driving assembly through the flexible execution module, and the portability and auxiliary performance of the device are considered. The integrated motion compatibility mechanism can dynamically compensate for the movement deviation between the exoskeleton and the human joint during the flexing and stretching of the finger.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a hand exoskeleton robot and its synchronous sensing method. Background Technology

[0002] As a wearable assistive and assessment device, hand exoskeletons have broad application prospects in rehabilitation training after stroke or spinal cord injury, helping patients restore or maintain their daily living activities, and assisting the elderly in fine motor skills. Their goal is to effectively restore or enhance hand grasping and manipulation functions while ensuring user safety and comfort.

[0003] Currently, most hand exoskeletons on the market rely on rigid links, parallel cables, or inflatable soft structures for drive and transmission. While these technologies can assist finger flexion and extension to some extent, they generally suffer from the following prominent drawbacks, limiting their clinical application and user experience: First, portability and assistive force are difficult to achieve simultaneously. Hand exoskeleton robots based on rigid link structures can typically provide a large assistive torque, but their complex structure, large overall size, and heavy weight result in poor portability, making them unsuitable for continuous wear and use in daily life. Conversely, while soft hand exoskeletons driven by hydraulics or pneumatics are lightweight and portable, the assistive force they can generate is limited, making it difficult to meet the force requirements of substantial grasping or functional activities. Second, there are shortcomings in human-machine motion compatibility and safety. The physiological structure of the human hand has joint motion coupling characteristics, and the skin of the fingers undergoes significant stretching and slippage during flexion and extension. Traditional rigid exoskeletons, or structures that can only bend in a single plane, struggle to adaptively conform to the complex three-dimensional movements of the fingers. This can easily lead to shear friction between the exoskeleton and skin at joints, causing pressure pain or discomfort for the user. Furthermore, the lack of smoothness in human-computer interaction poses safety risks. Additionally, the system's weak sensing capabilities limit rehabilitation assessment and adaptive control. Existing devices integrate sensors with limited dimensions, typically only measuring joint angles. This makes it difficult to provide adaptive, personalized assistance based on changes in the patient's rehabilitation progress, resulting in a low level of intelligence. Summary of the Invention

[0004] This invention provides a hand exoskeleton robot and its synchronous sensing method to solve the problems of existing hand exoskeletons, such as difficulty in achieving both portability and assistive force, insufficient human-machine motion compatibility and safety, and weak sensing capabilities.

[0005] This invention provides a hand exoskeleton robot, comprising: The device includes a hand fixation component, at least one flexible actuation module, a control and data processing unit, a motion-compatible mechanism, a drive component, and a multimodal sensing module. One end of the flexible actuation module is connected to the hand fixing component and extends along the back of the human finger; the driving component is connected to the flexible actuation module through a flexible transmission component to drive it to produce biomimetic bending deformation; The motion-compatible mechanism is disposed on the flexible actuation module and is used to adaptively compensate for the relative displacement and angular deviation between the flexible actuation module and the human finger joint during finger flexion and extension movements. The multimodal sensing module is used to synchronously acquire multimodal sensing information related to motion and force to achieve synchronous perception of the finger joint state. The multimodal sensing module includes a deformation sensing unit, an attitude sensing unit, and a driving force sensing unit. The deformation sensing unit is attached and fixed to the outer surface of the flexible actuation module. The attitude sensing unit is fixed to the end of the flexible actuation module. The driving force sensing unit is connected in series and fixed to the power transmission link between the output end of the driving component and the flexible transmission component. The control and data processing unit is electrically connected to the deformation sensing unit, the attitude sensing unit, and the driving force sensing unit, respectively. The control and data processing unit has a pre-set multimodal perception algorithm model based on deep learning. The control and data processing unit includes a signal preprocessing module and a sliding time window extraction module. The signal preprocessing module performs time alignment, low-pass filtering, and standardization processing on the input sensing signals. The sliding time window extraction module extracts continuous multi-channel data into multimodal time series samples and inputs them into the multimodal perception algorithm model.

[0006] According to the present invention, a hand exoskeleton robot is provided, wherein the flexible execution module includes: a plurality of guide supports and at least two elastic members; The at least two elastic members include a fixed member and a sliding member, both of which are inserted through the guide support. The fixed component is fixedly connected to each of the guide supports; one end of the sliding component is fixedly connected to the guide support located at the end of the flexible execution module, the other end of the sliding component is connected to the flexible transmission component, and the sliding component is slidably connected to the remaining guide supports.

[0007] According to the present invention, a hand exoskeleton robot is provided, wherein the guide support is provided with non-parallel guide holes or non-parallel grooves for the sliding component to pass through.

[0008] According to the present invention, a hand exoskeleton robot is provided, wherein the drive component is disposed outside the hand fixing component, and the output end of the flexible transmission component is fixedly connected to the input end of the sliding component.

[0009] According to the present invention, a hand exoskeleton robot is provided, wherein the fixing component is an elastic reset component.

[0010] According to the present invention, a hand exoskeleton robot is provided, wherein the motion compatibility mechanism is disposed at the end of the flexible actuation module away from the hand fixing component, and the motion compatibility mechanism includes: a guide joint and a rotary joint; The guide pair and the rotary pair; the fixed end of the guide pair is fixedly connected to the end of the flexible actuator module, the movable end of the guide pair is hinged to one end of the rotary pair, and the other end of the rotary pair is fixedly connected to a ring-shaped or strip-shaped connector for fitting on the finger part of the human body.

[0011] According to the present invention, a hand exoskeleton robot is provided, wherein the posture sensing unit is an inertial sensor, the deformation sensing unit is a bending sensor, and the driving force sensing unit is an input force sensor.

[0012] The present invention also provides a synchronous sensing method for a hand exoskeleton robot, comprising: During the process of the flexible execution module driving the movement of human fingers, the multimodal sensing module synchronously collects the input force, bottom bending angle, top bending angle and triaxial acceleration, and performs time alignment, outlier removal and normalization through the signal preprocessing module in the control and data processing unit to form the input feature vector at time t. Using the sliding time window extraction module, with the current time t as the prediction time, a sliding time window containing data from the current time and previous times is extracted to form network input samples; The network input samples are input into the multimodal perception algorithm model, which is a multi-task temporal convolutional neural network, including multiple cascaded time blocks, angle estimation fully connected layers, and torque estimation fully connected layers; the network input samples extract multimodal time series features representing the dynamic hysteresis, force changes, and vibration information of the flexible structure through the multiple cascaded time blocks; The output features of the last time block at time t are input into two independent angle estimation fully connected layers and torque estimation fully connected layers, respectively, to simultaneously output the predicted values ​​of human finger joint angles and joint torques.

[0013] The hand exoskeleton robot provided by this invention, through the cooperation of a flexible execution module and a drive component, can output sufficient biomimetic bending drive force while ensuring a lightweight and portable structure, effectively balancing the portability and assistive performance of the device. Its integrated motion-compatible mechanism can dynamically compensate for the motion deviation between the exoskeleton and the human joint during finger flexion and extension, significantly enhancing the fit and compliance of human-computer interaction, thereby reducing the risk of skin shear friction and pressure pain, and improving the safety and comfort of use. In addition, the introduction of a multimodal sensing module enables synchronous and multidimensional perception of motion and force information, providing a data foundation for accurately assessing finger status, recognizing user intentions, and realizing adaptive and personalized intelligent assistive control, enabling this hand exoskeleton robot to simultaneously meet the needs of efficient assistance, comfortable wear, and intelligent interaction. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0015] Figure 1 This is a front view of the hand exoskeleton robot provided by the present invention.

[0016] Figure 2 This is a side view of the hand exoskeleton robot provided by the present invention.

[0017] Figure 3 This is a schematic diagram of the structure of the driving component provided by the present invention.

[0018] Figure 4 This is a schematic diagram of the flexible execution module provided by the present invention when it is not bent.

[0019] Figure 5 This is a schematic diagram of the flexible execution module provided by the present invention when it is bent.

[0020] Figure 6 This invention provides a deep learning network architecture for sensing joint angles and joint torques.

[0021] Figure label: 1. Hand restraint components; 2. Flexible execution module; 21. Guide support component; 22. Elastic component; 23. Fixed component; 24. Sliding component; 3. Motion compatible mechanism; 31. Rotary joint; 32. Slider; 33. Slide rail; 4. Drive components; 41. Flexible transmission components; 42. Motor; 43. Sliding components; 44. Sheaves; 5. Multimodal sensing module; 51. Deformation sensing unit; 52. Attitude sensing unit; 53. Driving force sensing unit. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The following is combined Figures 1 to 6 This invention describes the hand exoskeleton robot provided by the present invention.

[0024] In some embodiments, such as Figures 1 to 5 As shown, the hand exoskeleton robot includes: a hand fixation component 1, at least one flexible actuation module 2, a motion-compatible mechanism 3, a drive component 4, and a multimodal sensing module 5. One end of the flexible actuation module 2 is connected to the hand fixation component 1 and extends along the back of the human finger; the drive component 4 is connected to the flexible actuation module 2 via a flexible transmission component 41 to drive it to produce biomimetic bending deformation; the motion-compatible mechanism 3 is disposed on the flexible actuation module 2 and is used to adaptively compensate for the relative displacement and angular deviation between the flexible actuation module 2 and the human finger joint during finger flexion and extension movements; the multimodal sensing module 5 is used to synchronously collect multimodal sensing information related to motion and force to achieve synchronous perception of the finger joint state.

[0025] In this embodiment, the hand fixation component 1 is typically worn on the back of the user's hand or forearm, providing a stable attachment base for other functional modules. One end of the flexible actuation module 2 is connected to the fixation component, and its main body extends along the dorsal side of the human finger. Each flexible actuation module 2 drives one finger, enabling it to produce a biomimetic deformation that approximates the natural bending of a human finger under actuation.

[0026] The drive assembly 4 is typically located within the main body of the device, away from the hand. It is powered by the flexible transmission element 41 and connected to each flexible actuator module 2. When the drive assembly 4 is in operation, it can remotely pull or release the flexible actuator module 2 by extending or retracting the flexible transmission element 41, thereby causing it to produce the required bending or extending motion and providing auxiliary force for finger movement.

[0027] To improve wearing comfort and reduce motion interference, the motion-compatible mechanism 3 is directly integrated into the flexible actuator module 2, specifically corresponding to the position of the human finger joints. During finger flexion and extension, this mechanism can automatically and passively adapt to and compensate for the slight relative displacement and angular deviation between the rigid or semi-rigid structure of the exoskeleton and the soft finger joints that are subject to skin stretching and slippage. This effectively avoids the hard friction and compression of the joint skin caused by traditional rigid structures, significantly improving the compatibility and safety of human-machine motion.

[0028] To endow the hand exoskeleton robot with sensing and interactive intelligence, a multimodal sensing module 5 is also provided. The multimodal sensing module 5 is not a single sensor, but a collection of distributed, multifunctional sensors that can simultaneously collect various information related to motion and force. By fusing multimodal sensing information, the system can achieve synchronous and comprehensive perception of the user's finger joint movement state, interactive force, and control intention.

[0029] The hand exoskeleton robot provided by this invention, through the cooperation of the flexible execution module 2 and the drive component 4, can output sufficient biomimetic bending drive force while ensuring a lightweight and portable structure, effectively balancing the portability and assistive performance of the device. Its integrated motion-compatible mechanism 3 can dynamically compensate for the motion deviation between the exoskeleton and the human joint during finger flexion and extension, significantly enhancing the fit and compliance of human-computer interaction, thereby reducing the risk of skin shear friction and pressure pain, and improving the safety and comfort of use. In addition, the introduction of the multimodal sensing module 5 enables synchronous and multidimensional perception of motion and force information, providing a data foundation for accurately assessing finger status, recognizing user intentions, and realizing adaptive and personalized intelligent assistive control, enabling the hand exoskeleton robot to simultaneously meet the needs of efficient assistance, comfortable wear, and intelligent interaction.

[0030] like Figure 2 As shown, the multimodal sensing module 5 includes a deformation sensing unit 51, an attitude sensing unit 52, and a driving force sensing unit 53; the deformation sensing unit 51 is attached and fixed to the outer surface of the flexible execution module 2; the attitude sensing unit 52 is fixed to the end of the flexible execution module 2; and the driving force sensing unit 53 is connected in series and fixed to the power transmission link between the output end of the drive component 4 and the flexible transmission component 41.

[0031] Specifically, the deformation sensing unit 51 is attached to the fixed component 23 and is used to measure the bending deformation information of the flexible finger module. When the flexible actuator 2 undergoes bending deformation, the deformation sensing unit 51 attached to it deforms synchronously, and its output electrical or optical signals change accordingly, thereby enabling real-time and continuous measurement of the bending curvature or angle change information of the flexible finger module. The driving force sensing unit 53 is mounted on the slider 43 and is used to detect the driving input force information. It can directly detect the driving input force information, that is, the actual traction force transmitted by the motor 42 to the sliding component 24 through the Bowden cable and the slider 43. The posture sensing unit 52 is mounted on the guide support 21 and is used to collect the fingertip movement posture information. It can calculate the fingertip movement posture information, including its pitch angle, roll angle, etc. in space, thereby tracking the absolute or relative orientation and movement trajectory of the fingertip in three-dimensional space in real time.

[0032] In this embodiment, the attitude sensing unit 52 is an inertial sensor, typically including at least a three-axis accelerometer and a three-axis gyroscope. The deformation sensing unit 51 is a bending sensor, which can operate on a resistive, capacitive, or fiber optic grating principle. This bending sensor is tightly attached and fixed to the outer surface of the fixed member 23, which serves as an elastic reset member, ensuring that its deformation is highly consistent with the deformation of the fixed member 23. When the flexible actuator 2 bends under drive, the fixed member 23 undergoes elastic deformation, causing the bending sensor attached to it to also undergo synchronous mechanical bending. The driving force sensing unit 53 is an input force sensor, commonly including miniature strain gauge force sensors and piezoresistive force sensors. It is connected in series and fixed between the output end of the drive assembly 4 and the flexible transmission member 41, i.e., directly located on the power transmission path.

[0033] like Figure 6 As shown, the hand exoskeleton robot also includes a control and data processing unit. The control and data processing unit is electrically connected to the deformation sensing unit 51, the posture sensing unit 52, and the driving force sensing unit 53, respectively. The control and data processing unit has a pre-set multimodal perception algorithm model based on deep learning.

[0034] Specifically, the control and data processing unit synchronously acquires data through the deformation sensing unit 51, the attitude sensing unit 52, and the driving force sensing unit 53. The control and data processing unit performs low-pass filtering on the raw data to suppress high-frequency noise and transient interference. Then, it standardizes the data of each channel to improve the stability of the network training and inference process. After that, it uses a sliding time window to segment the continuous data to form multimodal time series samples for network input.

[0035] The control and data processing unit includes a signal preprocessing module and a sliding time window truncation module. The signal preprocessing module performs time alignment, low-pass filtering, and normalization on the input sensor signals. The sliding time window truncation module truncates continuous multi-channel data into multimodal time series samples and inputs them into the multimodal perception algorithm model. The multimodal perception algorithm model is a multi-task temporal convolutional neural network, including multiple cascaded temporal convolutional modules. Each temporal convolutional module consists of a dilated convolutional layer, a linear rectified activation layer, and a random deactivation layer. Residual connections are introduced at the end of the modules to enhance the extraction capability of long-term time-dependent features and improve the network training stability. After extracting multimodal time series features layer by layer, the network sets an angle sensing branch and a torque sensing branch at the output end, which output finger joint angle estimates and joint torque estimates through fully connected layers, respectively. During training, a multi-task weighted loss function is used to jointly optimize the angle estimation error and torque estimation error. The expression of this loss function is attached. Figure 6 As shown, the formula is: In the formula: L is the total loss value of multi-task joint training; The loss weighting coefficients for the joint angle estimation task; The estimated finger joint angles output by the angle-sensing branch; This represents the actual labeled value of the finger joint angle; The loss weighting coefficient for the joint torque estimation task; The estimated finger joint torque output by the torque sensing branch; This represents the true labeled value of the finger joint torque. During training, the gradient is calculated based on the loss function and backpropagated to update the network parameters. Through joint optimization of this multi-task weighted loss function, the network can improve the overall perception accuracy while taking into account two perception tasks.

[0036] In some embodiments, such as Figures 1 to 5 As shown, the flexible actuation module 2 includes: multiple guide supports 21 and at least two elastic members 22. In a typical configuration, a flexible actuation module 2 will have three guide supports 21 and two elastic members 22 connecting them to drive a finger.

[0037] In this embodiment, the guide supports 21 are spaced apart along the length of a finger, serving not only as the skeleton of the module but also providing necessary longitudinal support stiffness for the overall structure. At least two elastic members 22 are functionally divided into a fixing member 23 and a sliding member 24. Both are inserted through the aforementioned guide supports 21. The fixing member 23 is fixedly connected to all the guide supports 21, thereby connecting and fixing the individual guide supports 21 in series. One end of the sliding member 24 is fixedly connected to the guide support 21 located at the very end of the flexible execution module 2, while its other end extends out and connects to the flexible transmission member 41 from the drive assembly 4. The sliding member 24 has a sliding through-connection relationship with the remaining guide supports 21 between the fixing member 23 and the end guide support 21, meaning it can slide freely within the guide holes of these intermediate supports.

[0038] When the drive component 4 pulls the sliding member 24 via the flexible transmission member 41, the sliding member 24 is subjected to axial tension. Since the end of the sliding member 24 is fixed, this tension attempts to shorten the effective length of the sliding member 24 relative to the intermediate support member. Under the constraint of the fixed member 23 maintaining a relatively stable shape, this pushing behavior of the sliding member 24 forces relative rotation and displacement between the guide supports 21, thereby driving the entire flexible actuation module 2 to produce a smooth and continuous bending motion. This mode of buckling deformation of the overall structure caused by the sliding of the central drive member mimics the motion mechanism of biological structures such as octopus tentacles, and can maintain the lightweight and compactness of the structure while providing sufficient auxiliary force.

[0039] It should be noted that the guide support 21 has non-parallel guide holes or non-parallel grooves for the sliding member 24 to pass through. Specifically, these guide holes or grooves are not arranged parallel to each other, but are arranged with specific, gradually changing axial directions according to a pre-designed kinematic model. For example, on the guide support 21 from the proximal end to the distal end, the axial direction of the corresponding guide hole may exhibit a regular deflection. When the sliding member 24 passes through this series of non-parallel guide holes in sequence, its movement path is forcibly constrained by these holes. When the drive assembly 4 pulls the sliding member 24 through the flexible transmission member 41, the sliding member 24 slides in the non-parallel guide holes, applying a force to each guide support 21 with a direction collinear with the hole axis. Since these forces have different directions and the point of application has a specific lever arm relative to the rotation center of the support, a torque sequence that causes the support to rotate is formed. The orderly rotational displacement generated by all the supporting components under the action of torque eventually accumulates and manifests as the bending deformation of the entire module. Its bending curvature and shape are determined by the spatial orientation of the non-parallel guide holes.

[0040] In some embodiments, such as Figures 1 to 5As shown, the drive assembly 4 is disposed outside the hand fixing assembly 1, and the flexible transmission element 41 is a Bowden wire. The output end of the flexible transmission element 41 is fixedly connected to the input end of the sliding member 24. The power of the drive assembly 4 is transmitted through the flexible transmission element 41. The output end of the flexible transmission element 41 is fixedly connected to the input end of the sliding member 24 within the flexible actuation module 2. The fixed component 23 adopts an elastic reset component, which not only plays a structural role in connecting and fixing the various guide supports 21 and defining the basic shape of the module, but also has a significant elastic deformation capacity. Its function is to provide a passive elastic restoring force in the opposite direction to the driving direction: when the driving component 4 pulls the execution module to bend through the sliding component 24, the elastic reset component is stretched or compressed accordingly, storing elastic potential energy; once the driving component 4 releases or reduces the traction force, the energy stored in the elastic reset component will be released, and the driving execution module and the human fingers connected to it will actively and smoothly return to the extended state, making the flexion and extension movements of the fingers more natural, controllable and energy-saving, while avoiding the jamming that may be caused by unidirectional driving or the insufficiency of relying on the human hand's own muscle force to return to the original position.

[0041] In some embodiments, such as Figures 1 to 5 As shown, the motion-compatible mechanism 3 is located at the end of the flexible actuator module 2 away from the hand fixing component 1, and the motion-compatible mechanism 3 includes a guide pair and a rotary pair 31. The fixed end of the guide pair is fixedly connected to the end of the flexible actuator module 2, the movable end of the guide pair is hinged to one end of the rotary pair 31, and the other end of the rotary pair 31 is fixedly connected to a ring-shaped or strip-shaped connector for fitting onto the human finger area.

[0042] Specifically, the guide pair consists of a slide rail 33 and a slider 32. When the flexible finger module drives the human finger to bend, the slider 32 can slide naturally on the slide rail 33. The rotary pair provides relative rotational freedom, thereby effectively reducing the shearing force of the exoskeleton on the skin on the back of the human finger, improving human-machine kinematic compatibility, and reducing discomfort and potential damage risks during wear.

[0043] When the flexible actuator 2 is driven to bend, causing the human finger to flex or extend, significant sliding and stretching occur in the skin and soft tissue of the hand. This creates a dynamic relative motion between the exoskeleton's drive point and the actual rotation center of the finger joint. The relative slippage between the finger skin and bone in the guide joint is converted into axial traction or compression of the connector. This force pushes the slider 32 to slide naturally on the slide rail 33, absorbing and compensating for the axial relative displacement, thus preventing the exoskeleton structure from rigidly pulling or pushing the skin. Simultaneously, the rotary joint 31 also functions. The actual bending axis of the finger joint may not perfectly coincide with the theoretical bending axis of the exoskeleton. This slight angular deviation drives the rotary joint 31 to rotate accordingly, allowing the finger connector to adaptively follow the natural flexion and extension angle of the finger. Through the combined effect of the axial sliding degree of freedom provided by the guide joint and the rotational degree of freedom provided by the revolute joint 31, the motion-compatible mechanism 3 can dynamically and passively follow the movement of the finger joints, reducing the shear friction and local pressure on the skin on the back of the fingers that are difficult to avoid in traditional rigid exoskeletons, thereby significantly improving human-machine compatibility. Its direct benefit is that it effectively reduces skin discomfort experienced by users during wear and exercise, making long-term, repetitive rehabilitation training or assisted operations more feasible and comfortable.

[0044] In some embodiments, such as Figure 3 As shown, the drive assembly 4 includes a motor 42, a pulley 44, and a Bowden line. After the motor 42 outputs power, it pulls the sliding member 43 on the back of the hand through the pulley 44 and the Bowden line. The sliding member 43 then drives the sliding member 24 connected to it to move, thereby causing the elastic member 22 to bend and deform, thus realizing the drive assistance for the flexion action of the fingers.

[0045] In this embodiment, specifically, the motor 42 serves as the power source, and a pulley 44 is fixedly mounted on its output shaft. The pulley 44 is designed with winding grooves for neatly winding Bowden wire. After receiving a control signal, the output shaft of the motor 42 generates a forward or reverse rotational motion. The pulley 44 converts the rotational motion of the motor 42 into linear traction. When the motor 42 rotates, it drives the pulley 44 to rotate synchronously, thereby winding or unwinding the Bowden wire core wire wound on it, achieving control over the core wire length. The outer sheath of the Bowden wire is fixed to the drive assembly 4 housing and the hand fixing assembly 1, providing stable path guidance; its internal core wire acts as a traction cable, with one end fixed to the pulley 44 and the other end passing through the hand fixing assembly 1, extending to the back of the hand. There, the end of the core wire is fixed to a slider 43 located on the back of the hand. This slider 43 is typically constrained on a linear guide rail and can only slide in a direction parallel to the palm. The slider 43 is then connected to the input end of the slider 24 within the flexible actuator 2, which extends to the back of the finger, via a connecting structure, thereby transmitting the linear traction force to the flexible actuator 2.

[0046] The complete driving process is as follows: Motor 42 outputs power to drive pulley 44 to rotate. Pulley 44 winds up Bowden wire core, generating a linear traction force towards the proximal end. This traction force is transmitted through Bowden wire to slider 43 on the back of the hand, causing slider 43 to slide. The movement of slider 43 then causes displacement of sliding member 24, which is fixed to it and located within flexible actuator module 2. As mentioned above, the movement of sliding member 24 changes its relative positional relationship with elastic member 22, thereby forcing the entire flexible actuator module 2 to produce the expected, biomimetic bending deformation. This bending deformation acts directly on the human finger worn underneath, providing auxiliary torque for finger flexion and achieving the driving assistance function.

[0047] This invention also provides a synchronous sensing method for a hand exoskeleton robot, comprising the following steps: S1: During the process of the flexible actuator driving the human finger movement, the input force, bottom bending angle, top bending angle and triaxial acceleration are synchronously collected by the multimodal sensing module. The signal preprocessing module in the control and data processing unit performs time alignment, outlier removal and normalization to form the input feature vector at time t.

[0048] S2: Using the sliding time window extraction module, with the current time t as the prediction time, extract the sliding time window containing the current time and the data of the previous N-1 times to form the network input sample.

[0049] S3: Input the network input samples into the multimodal perception algorithm model. The multimodal perception algorithm model is a multi-task temporal convolutional neural network, which includes multiple cascaded time blocks, angle estimation fully connected layers, and torque estimation fully connected layers. The network input samples are processed through multiple cascaded time blocks to extract multimodal time series features that characterize the dynamic hysteresis, force changes, and vibration information of the flexible structure.

[0050] S4: Input the output features of the last time block at time t into two independent angle estimation fully connected layers and torque estimation fully connected layers respectively, and output the predicted values ​​of human finger joint angles and joint torques simultaneously.

[0051] Specifically, during the movement of the flexible finger exoskeleton, input force, bottom bending angle, top bending angle, and triaxial acceleration are simultaneously collected; at the same time, the actual finger joint angles and actual joint torques corresponding to each sampling moment are acquired as supervised learning labels. Preferably, the signals of each channel are subjected to time alignment, outlier removal, and normalization processing.

[0052] At time t, the acquired multimodal signals are configured as follows: ; Where x(t) is the input feature vector at time t; Fin(t) is the input force; ,and u (t) represents the bottom and top bending angles, respectively; a x (t), a_z(t) and a_z(t) are triaxial accelerations, respectively.

[0053] Using the current time t as the prediction time, a sliding window containing data from the current time and the N-1 previous times is extracted to form the network input sample X. t Preferably, the window duration is 300ms; N is determined by both the sensor sampling frequency and the window duration.

[0054] ; Among them, C in The input channel number is specified. The sliding time window allows the network to simultaneously obtain the current motion state and recent historical states, thereby characterizing the dynamic hysteresis, force changes, and vibration information of the flexible structure.

[0055] The multi-task temporal convolutional neural network comprises four cascaded temporal blocks, an angle estimation fully connected layer, and a moment estimation fully connected layer. Each temporal block includes two convolutional modules; each convolutional module includes a one-dimensional dilated causal convolutional layer, a ReLU activation layer, and a Dropout layer; residual connections are set at the end of each temporal block. Preferably, the convolutional modules have 256 channels and the kernel size is 7.

[0056] The expansion coefficients of the four time blocks increase exponentially; if the time block number l starts from 0, then: d (l) = 2 l , l ∈ {0, 1, 2, 3}; Therefore, the dilation coefficients for the four time blocks are 1, 2, 4, and 8, respectively. Dilated causal convolution only uses data from the current and past time points, not future data, thus it can be used for real-time online estimation.

[0057] For the j-th convolutional module within the l-th time block, the convolutional output of its o-th output channel at time t is: D o (lj) (t) = Σ i=0 k-1 Σ c =1 C in W o , i , c(lj) H (lj-1) (t - d (l) i, c); Where k is the kernel size, W o , i , c(lj) H represents the convolution kernel parameters. (lj-1) d represents the input features of the current convolutional module. (l) is the expansion coefficient for the l-th time block.

[0058] The convolution output is then processed sequentially through ReLU activation and Dropout to obtain: ; Among them, b (lj) This is the bias term. Different expansion coefficients correspond to historical information over different time spans: the earlier time blocks are used to extract local, rapidly changing features, while the later time blocks are used to extract flexural trends and stress evolution features over a longer time range.

[0059] The input features of the l-th time block are added to the output features of the second convolutional module in that time block to obtain the residual connection output: H (l) (t,o) = H (l-1) (t,o) + H (l , 2) (t,o); When the channel dimensions at both ends of the residual are inconsistent, the input features are aligned using a one-dimensional convolution with a kernel size of 1. Residual connections enable low-level sensing features to be directly passed to subsequent time blocks, while also facilitating stable backpropagation of training gradients.

[0060] Let the output feature of the last time block at time t be denoted as By inputting two independent fully connected layers, we obtain the predicted values ​​of joint angles and joint moments: Among them, W θ and b θ The parameters for the angle estimation branch; W τ and b τ The parameters for the torque estimation branch are specified. Both branches share the aforementioned time feature extraction module, but each retains its own independent output parameters.

[0061] For a training batch containing M samples, calculate the mean square error for the angle task and the torque task respectively: ; Construct the total loss function based on the pre-defined task weights α and β: L = α( Here, α is used to adjust the contribution of the angle task error to the total loss, and β is used to adjust the contribution of the torque task error to the total loss. α and β are hyperparameters used during the training phase, and their values ​​can be set according to the dual-task accuracy on the validation set and the numerical scale of the two types of labels. Preferably, α + β = 1 is set after standardizing the labels; when the two tasks are of equal importance and have similar error magnitudes, α = β can be set.

[0062] The network parameters are divided into shared-time feature extraction parameters ω. s The parameters are: angle branch parameter ωθ and torque branch parameter ωτ. Backpropagation is performed based on the total loss function L, where the shared parameter receives the weighted sum of the gradients from both tasks. ; Accordingly, a parameter update using the learning rate η can be expressed as: ; Furthermore, for the shared feature h(t) output in the last time block, its back gradient is: L / h(t) = 2α( θ(t)-yθ(t))Wθ + 2β( τ(t)-yτ(t))Wτ; This indicates that the effect of multi-task weighting lies in the gradient feedback of the shared feature extraction module: angle error and torque error, weighted by α and β, jointly act on the same set of convolutional parameters. If the angle task error or α is large, the parameter adjustment effect of the angle task on the shared network is enhanced; if the torque task error or β is large, the feedback effect of the torque task is enhanced.

[0063] The trained network parameters are then fixed. During actual operation, sensor data is used, and a 300 ms time window X is obtained using a sliding method. t ; X t Input the trained network to obtain joint angle estimates. θ(t) and estimated joint torque values τ(t). The inference phase no longer requires labels for the true angle and true torque, nor does it calculate the total loss function or perform parameter updates.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hand exoskeleton robot, characterized in that, include: The device includes a hand fixation component, at least one flexible actuation module, a control and data processing unit, a motion-compatible mechanism, a drive component, and a multimodal sensing module. One end of the flexible actuation module is connected to the hand fixing component and extends along the back of the human finger; the driving component is connected to the flexible actuation module through a flexible transmission component to drive it to produce biomimetic bending deformation; The motion-compatible mechanism is disposed on the flexible actuation module and is used to adaptively compensate for the relative displacement and angular deviation between the flexible actuation module and the human finger joint during finger flexion and extension movements. The multimodal sensing module is used to synchronously acquire multimodal sensing information related to motion and force to achieve synchronous perception of the finger joint state. The multimodal sensing module includes a deformation sensing unit, an attitude sensing unit, and a driving force sensing unit. The deformation sensing unit is attached and fixed to the outer surface of the flexible actuation module. The attitude sensing unit is fixed to the end of the flexible actuation module. The driving force sensing unit is connected in series and fixed to the power transmission link between the output end of the driving component and the flexible transmission component. The control and data processing unit is electrically connected to the deformation sensing unit, the attitude sensing unit, and the driving force sensing unit, respectively. The control and data processing unit has a pre-set multimodal perception algorithm model based on deep learning. The control and data processing unit includes a signal preprocessing module and a sliding time window extraction module. The signal preprocessing module performs time alignment, low-pass filtering, and standardization processing on the input sensing signals. The sliding time window extraction module extracts continuous multi-channel data into multimodal time series samples and inputs them into the multimodal perception algorithm model.

2. The hand exoskeleton robot according to claim 1, characterized in that, The flexible actuation module includes: multiple guide supports and at least two elastic components; The at least two elastic members include a fixed member and a sliding member, both of which are inserted through the guide support. The fixed component is fixedly connected to each of the guide supports; one end of the sliding component is fixedly connected to the guide support located at the end of the flexible execution module, the other end of the sliding component is connected to the flexible transmission component, and the sliding component is slidably connected to the remaining guide supports.

3. The hand exoskeleton robot according to claim 2, characterized in that, The guide support is provided with non-parallel guide holes or non-parallel grooves for the sliding component to pass through.

4. The hand exoskeleton robot according to claim 2, characterized in that, The drive component is disposed outside the hand fixing component, and the output end of the flexible transmission component is fixedly connected to the input end of the sliding component.

5. The hand exoskeleton robot according to claim 4, characterized in that, The fixing component is an elastic reset component.

6. The hand exoskeleton robot according to claim 1, characterized in that, The motion-compatible mechanism is located at the end of the flexible actuation module away from the hand fixing component, and the motion-compatible mechanism includes: a guide pair and a rotary pair; The guide pair and the rotary pair; the fixed end of the guide pair is fixedly connected to the end of the flexible actuator module, the movable end of the guide pair is hinged to one end of the rotary pair, and the other end of the rotary pair is fixedly connected to a ring-shaped or strip-shaped connector for fitting on the finger part of the human body.

7. The hand exoskeleton robot according to claim 1, characterized in that, The attitude sensing unit is an inertial sensor, the deformation sensing unit is a bending sensor, and the driving force sensing unit is an input force sensor.

8. A synchronous sensing method using a hand exoskeleton robot as described in any one of claims 1-7, characterized in that, include: During the process of the flexible execution module driving the movement of human fingers, the multimodal sensing module synchronously collects the input force, bottom bending angle, top bending angle and triaxial acceleration, and performs time alignment, outlier removal and normalization through the signal preprocessing module in the control and data processing unit to form the input feature vector at time t. Using the sliding time window extraction module, with the current time t as the prediction time, a sliding time window containing data from the current time and previous times is extracted to form network input samples; The network input samples are input into the multimodal perception algorithm model, which is a multi-task temporal convolutional neural network, including multiple cascaded time blocks, angle estimation fully connected layers, and torque estimation fully connected layers; the network input samples extract multimodal time series features representing the dynamic hysteresis, force changes, and vibration information of the flexible structure through the multiple cascaded time blocks; The output features of the last time block at time t are input into two independent angle estimation fully connected layers and torque estimation fully connected layers, respectively, to simultaneously output the predicted values ​​of human finger joint angles and joint torques.

Images