Fatigue perception and reinforcement learning fused hand exoskeleton control method and system

Abstract

The invention discloses a fatigue perception and reinforcement learning fused hand exoskeleton control method and system. The method comprises the steps that surface electromyogram signals, man-machine interaction force and position information of a user and hand posture data collected based on vision are collected; utilizing the pre-training fatigue recognition model to obtain a muscle fatigue level and constructing an enhanced state vector; optimized virtual stiffness and damping parameters are generated through a reinforcement learning parameter optimizer; updating the virtual hand model based on the hand posture and performing collision detection, and selecting an admittance or mixed admittance control mode according to the contact state to calculate an expected speed instruction; finally, the exoskeleton is driven to execute. By sensing muscle fatigue in real time and adjusting force feedback parameters in a self-adaptive mode, safe and personalized rehabilitation training is achieved; by means of virtual contact triggering and realistic force sense simulation, the immersion of training is improved; by fusing physiological, physical and visual information, an intelligent cooperative control closed loop is constructed.

Description

Technical Field

[0001] This invention relates to the field of rehabilitation robots and human-computer interaction control technology, and in particular to a hand exoskeleton control method and system that integrates fatigue perception and reinforcement learning. Background Technology

[0002] For hand motor dysfunction caused by stroke, spinal cord injury, and other diseases, hand rehabilitation robots have become an important auxiliary tool in clinical rehabilitation training. Currently, a typical control scheme for such systems involves the patient wearing a rigid exoskeleton driven by a motor. This exoskeleton, through a linkage mechanism, guides the patient's fingers to perform rehabilitation movements such as flexion and extension. The control system typically includes several preset training modes. For example, in passive training, the system strictly follows a preset trajectory, speed, and constant resistance to guide the patient's hand in repetitive movements; in active training, the patient is required to actively apply a force greater than a fixed threshold to trigger the robot's assisted movements. To enhance patient engagement, some systems are also equipped with displays that provide visual feedback on the movement process through simple progress bars or basic animations.

[0003] However, the aforementioned existing technical solutions have several significant drawbacks in practical applications, which restrict the safety, effectiveness, and user experience of rehabilitation training.

[0004] First, the control strategy is rigid and lacks the ability to dynamically respond to the user's physiological state. Once the parameters of motion trajectory, speed, and resistance in existing systems are preset, they usually remain unchanged throughout the training process, ignoring the dynamic changes of the patient as a physiological individual, especially failing to perceive and respond to muscle fatigue that occurs during training. When the patient's muscles are already fatigued, the system continues to force the training task at the original intensity, which may not only cause patient discomfort and reduce rehabilitation effects, but in severe cases may even lead to secondary injuries such as muscle strains, posing safety hazards.

[0005] Second, the human-computer interaction experience is monotonous and dull, lacking immersion and fidelity. Existing systems primarily provide fixed, programmed resistance feedback, which differs significantly from the rich and dynamic tactile experience of grasping different objects (such as sponges or hard balls) in the real world. Simultaneously, the visual feedback system is severely disconnected from physical interaction, failing to construct a meaningful and immersive interactive scenario. This mechanical and repetitive training mode easily bores patients, generating resistance and leading to poor training compliance, which is detrimental to stimulating cortical plasticity, crucial for neural function remodeling.

[0006] Third, the system's control logic is fragmented, and multi-source information fails to be deeply integrated. In the existing system architecture, the physical control loop for processing force and position, the visual feedback loop for displaying virtual scenes, and the module for monitoring physiological states are often independent of each other. There is a lack of deep information interaction and collaboration between these modules. In particular, the patient's real-time physiological state information (such as muscle fatigue level) is excluded from the main control loop, requiring active intervention and adjustment by rehabilitation therapists. The system cannot render matching physical force feedback in real time and accurately based on key events in the virtual scene (e.g., a virtual hand touching an elastic object), nor can it adaptively adjust the difficulty of virtual tasks and the intensity of physical assistance based on the patient's real-time physiological state. This makes the entire system more like a simple collection of functions than an intelligent, unified, and organically collaborative interactive whole, thus severely limiting the personalization level and overall effectiveness of rehabilitation training. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the problems of rigid control strategies of hand rehabilitation robots in the prior art, which cannot respond to changes in the user's physiological state, the single human-computer interaction experience and lack of immersion, and the difficulty in coordination caused by the fragmentation of multi-source information.

[0008] To address the aforementioned technical problems, this invention provides a hand exoskeleton control method that integrates fatigue perception and reinforcement learning, comprising the following steps: S1: Collects the user's raw surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data based on vision acquisition; S2: Input the original surface electromyography signal into a pre-trained fatigue recognition model to obtain the current muscle fatigue level; fuse the muscle fatigue level, the human-computer interaction force, and the position information; construct an attribute-decoupled time-series generative adversarial network based on the human-computer interaction information to synthesize a simulated hand human-computer interaction trajectory dataset; fuse the muscle fatigue level, the human-computer interaction force, and the position information to construct an enhanced state vector; S3: Input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters; S4: Update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, perform collision detection based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and obtain a judgment result including the contact state and contact position. S5: Based on the contact state in the judgment result, select the control mode and calculate the command. If the contact state indicates no contact, select the admittance control mode and calculate the desired speed command based on the human-machine interaction force. If the contact state indicates contact, select the hybrid admittance control mode and apply the optimized virtual stiffness parameter and virtual damping parameter, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired speed command. S6: Obtain motor control commands based on the desired speed commands, and drive the hand rehabilitation exoskeleton to execute them.

[0009] In one embodiment of the present invention, in step S2, the training method of the fatigue recognition model includes: preprocessing the original surface electromyography (EMG) signal, segmenting the preprocessed signal using a sliding window, and extracting time-domain and frequency-domain features for each signal window; randomly dividing the preprocessed EMG signal data into labeled and unlabeled datasets, and using a semi-supervised learning framework, inputting the labeled and unlabeled datasets into a convolutional neural network for training to obtain a recognition model capable of multi-level fatigue classification of real-time EMG signals.

[0010] In one embodiment of the present invention, in step S2, a time-series generative adversarial network with attribute decoupling is constructed based on human-computer interaction information to synthesize a dataset simulating human-computer interaction trajectories of the hand; the muscle fatigue level, the human-computer interaction force, and the position information are fused to construct an enhanced state vector, the method being as follows: The system collects human-computer interaction data generated by users when performing tasks under different fatigue states and different combinations of impedance parameters. The human-computer interaction data includes impedance parameters as static attributes and hand position trajectory, human-computer interaction force trajectory and real-time muscle fatigue level as dynamic time series features. Based on the collected human-computer interaction data, a time-series generative adversarial network (GAN) for attribute decoupling is constructed; the attribute-decoupling time-series GAN includes a generator and a discriminator, wherein: The generator maps random noise to static attributes and generates corresponding normalization factors based on the static attributes. Based on the static attributes, normalization factors, and noise, a long short-term memory network is used to synthesize dynamic time series data containing information on hand position, human-computer interaction force, and fatigue. The discriminator evaluates the static attributes and the authenticity of the overall time series in the samples generated by the generator, respectively. The generator and the discriminator are trained alternately so that the generator synthesizes simulated trajectory data that is consistent with the distribution of real human-computer interaction data, and constructs an enhanced state vector that is input to the reinforcement learning parameter optimizer.

[0011] In one embodiment of the present invention, in step S3, the method of inputting the enhanced state vector into a pre-trained reinforcement learning parameter optimizer to obtain the optimized virtual stiffness parameters and virtual damping parameters is as follows: inputting the enhanced state vector into the pre-trained reinforcement learning parameter optimizer to obtain an action vector containing virtual damping adjustment and virtual stiffness adjustment; updating the current virtual damping parameters and virtual stiffness parameters according to the adjustment in the action vector to obtain the optimized virtual stiffness parameters and virtual damping parameters.

[0012] In one embodiment of the present invention, in step S3, the training method of the reinforcement learning parameter optimizer is as follows: the reinforcement learning parameter optimizer is optimized based on a comprehensive reward function; the comprehensive reward function includes: a trajectory reward term associated with muscle fatigue level and tracking error of the user's motion trajectory, a smoothing penalty term that suppresses the adjustment range of the virtual stiffness parameter and the virtual damping parameter, and a sparse positive reward term triggered when the average tracking error of the motion trajectory is lower than a preset threshold.

[0013] In one embodiment of the present invention, in step S3, the reinforcement learning parameter optimizer is optimized using an actor-critic framework, wherein the critic network evaluates the value of a given state and updates it by minimizing the value prediction error; the actor network outputs an action based on the current state and updates it by optimizing the objective function.

[0014] In one embodiment of the present invention, in step S5, the desired velocity command in admittance control mode Calculated using the following formula: , in, Human-computer interaction power The preset admittance damping coefficient; The desired velocity command under the hybrid admittance control mode Calculated using the following formula: , in, The optimized virtual stiffness parameters, For the optimized virtual damping parameters, To determine the contact location in the results, This is the current location.

[0015] This invention also provides a hand exoskeleton control system that integrates fatigue perception and reinforcement learning, comprising the following modules: The information acquisition module includes a hand rehabilitation exoskeleton, which is used to acquire the user's original surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data acquired based on vision. The state processing module is used to input the original surface electromyography signal into a pre-trained fatigue recognition model to obtain the current muscle fatigue level; fuse the muscle fatigue level, the human-computer interaction force, and position information; construct an attribute-decoupled time-series generative adversarial network based on the human-computer interaction information to synthesize a simulated hand human-computer interaction trajectory dataset; and fuse the muscle fatigue level, the human-computer interaction force, and position information to construct an enhanced state vector. The parameter optimization module is used to input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters. The virtual interaction module is used to update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, collision detection is performed based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and a judgment result including the contact state and contact position is obtained. The instruction generation module is used to select a control mode and calculate instructions based on the contact state in the judgment result. If the contact state indicates no contact, the admittance control mode is selected, and the desired velocity instruction is calculated based on the human-machine interaction force. If the contact state indicates contact, the hybrid admittance control mode is selected, and the optimized virtual stiffness parameters and virtual damping parameters are applied, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired velocity instruction. The execution control module is used to obtain motor control commands based on the desired speed command and drive the hand rehabilitation exoskeleton to execute them.

[0016] The present invention also provides an electronic device comprising a processor, a memory, and a bus system, wherein the processor and the memory are connected via the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the hand exoskeleton control method integrating fatigue perception and reinforcement learning.

[0017] The present invention also provides a computer storage medium, characterized in that the computer storage medium stores a computer software product, the computer software product including several instructions for causing a computer device to execute the hand exoskeleton control method that integrates fatigue perception and reinforcement learning.

[0018] The technical solution of the present invention has the following advantages compared with the prior art: The hand exoskeleton control method integrating fatigue perception and reinforcement learning described in this invention obtains muscle fatigue levels by interpreting surface electromyography signals in real time and integrates human-computer interaction information. It utilizes reinforcement learning to dynamically optimize force feedback parameters, enabling the control strategy to adapt to changes in the user's physiological state and achieve safe and personalized rehabilitation training. This method uses collision detection in a virtual environment as a trigger signal to switch control modes and uses optimized parameters to simulate the tactile sensation of real objects, transforming mechanical training into an immersive interactive task and effectively improving user engagement. By constructing an enhanced state vector integrating physiological, physical, and visual information and a unified decision core, it achieves deep fusion and intelligent collaboration of multi-source information. Attached Figure Description

[0019] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating the hand exoskeleton control method that integrates fatigue perception and reinforcement learning provided in an embodiment of the present invention. Figure 2 This is an overall architecture block diagram of the hand exoskeleton control method that integrates fatigue perception and reinforcement learning in an embodiment of the present invention; Figure 3 This is a schematic diagram of the reinforcement learning architecture used in the embodiments of the present invention; Figure 4 This is a schematic diagram of the structure of the time series generative adversarial network model in an embodiment of the present invention; Figure 5 This is a schematic diagram of the hand exoskeleton control system that integrates fatigue perception and reinforcement learning provided in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0022] Example 1: like Figure 1 As shown, this invention provides a hand exoskeleton control method that integrates fatigue perception and reinforcement learning, comprising the following steps: S1: Collects the user's raw surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data based on vision acquisition; S2: Input the original surface electromyography signal into a pre-trained fatigue recognition model to obtain the current muscle fatigue level; fuse the muscle fatigue level, the human-computer interaction force, and the position information; construct an attribute-decoupled time-series generative adversarial network based on the human-computer interaction information to synthesize a simulated hand human-computer interaction trajectory dataset; fuse the muscle fatigue level, the human-computer interaction force, and the position information to construct an enhanced state vector; S3: Input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters; S4: Update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, perform collision detection based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and obtain a judgment result including the contact state and contact position. S5: Based on the contact state in the judgment result, select the control mode and calculate the command. If the contact state indicates no contact, select the admittance control mode and calculate the desired speed command based on the human-machine interaction force. If the contact state indicates contact, select the hybrid admittance control mode and apply the optimized virtual stiffness parameter and virtual damping parameter, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired speed command. S6: Obtain motor control commands based on the desired speed commands, and drive the hand rehabilitation exoskeleton to execute them.

[0023] This invention collects surface electromyography (EMG) signals, interaction forces, and positional posture information from users of rehabilitation exoskeletons. Based on the EMG signals, it obtains real-time muscle fatigue levels, fuses these with interaction information, and adaptively generates virtual stiffness and damping parameters using a reinforcement learning optimizer. Visual data drives a virtual hand model and performs collision detection to determine the contact state. Based on the contact state determination, it switches control modes, obtains speed commands, and drives the exoskeleton. This invention achieves safe and personalized training intensity control through fatigue perception and adaptive parameter adjustment; it transforms mechanical training into immersive interaction through virtual contact triggering and realistic force simulation; and it forms a collaborative control closed loop capable of intelligent adjustment based on the global state through multi-source information fusion and unified decision-making.

[0024] Specifically, such as Figure 2 As shown, in step S1, at the beginning of each control cycle, data from three dimensions—the user's physiology, exoskeleton physical interaction, and visual perception—are acquired in real time.

[0025] A surface electromyography (SEMG) sensor array, attached to relevant forearm muscle groups (such as the flexor carpi radialis and flexor digitorum superficialis), non-invasively collects raw SEMG signals related to hand movements. These raw SEMG signals are an important source of physiological data for real-time assessment of muscle fatigue. The sensor unit integrated into the hand rehabilitation exoskeleton reads the human-machine interaction force and the exoskeleton's position information in real time. The human-machine interaction force is measured by the force sensing capability of the exoskeleton's drive unit (such as a motor), directly reflecting the active force applied by the user. The position information is acquired by a position sensor, and its continuous sequence over time constitutes the user's movement trajectory in physical space. A visual information sensing unit captures the user's actual hand posture data, including its spatial position, posture, and movement information. Optionally, the visual information sensing unit can be a non-contact optical hand tracking device such as Leap Motion.

[0026] The three types of raw data, consisting of surface electromyography signals (physiological information), human-computer interaction force and position (physical information), and hand visual posture (visual information), are collected synchronously at the beginning of each control cycle, providing the necessary unified input data foundation.

[0027] Furthermore, in step S2, the collected multi-source raw information is preprocessed and key state is interpreted, including real-time assessment of muscle fatigue based on surface electromyography (sEMG) signals, and an enhanced state vector is constructed for subsequent reinforcement learning decision-making.

[0028] Specifically, the acquired raw electromyography (sEMG) signals are input in real time into a pre-trained fatigue recognition model. This model is trained based on a semi-supervised learning framework (e.g., using the FixMatch algorithm). The training process includes: bandpass filtering (e.g., 10-1000Hz) of the raw sEMG signals to remove noise and motion artifacts; segmenting the signal using a sliding window (e.g., window length 1 second, step size 0.25 seconds); extracting time-domain features including root mean square (RMS) and frequency-domain features including median frequency (MDF) and power spectral standard deviation (Power STD) for each signal window; and concatenating these feature vectors with the raw signal data of the corresponding window as input to a one-dimensional convolutional neural network.

[0029] The fatigue recognition model is trained using a small amount of labeled data (sEMG signals and subjective fatigue scale labels) and a large amount of unlabeled sEMG signals. This results in a model capable of classifying real-time sEMG signal windows into multiple levels of fatigue, including "no exertion," "no fatigue," "mild fatigue," "moderate fatigue," and "severe fatigue." In real-time operation, the fatigue recognition model performs the same feature extraction and concatenation operations on the sEMG data for each time window, and outputs the classification results through forward propagation. This yields a quantitative indicator representing the current physiological state of the user's hand muscles, namely, the muscle fatigue level. .

[0030] Furthermore, such as Figure 3 As shown, the system will interpret the muscle fatigue level. It is deeply fused with physical interaction information read in real time from the hand rehabilitation exoskeleton sensing unit to construct an 8-dimensional enhanced state vector. This serves as the decision-making basis for the adaptive parameter optimization of the reinforcement learning agent.

[0031] The enhanced state vector is specifically defined as follows: , in, The current level of muscle fatigue is a key bridge connecting physiological state and control strategy; It is the instantaneous position error in trajectory tracking; It is the instantaneous human-computer interaction force error; The target location of the current interactive task in the virtual environment; and These are the normalized virtual damping and virtual stiffness parameters, representing the instantaneous mechanical characteristics of the system. The last two dimensions of the vector... and This is forward-looking information provided by an error prediction model trained on a simulated trajectory database built on a time-series generative adversarial network (GAN).

[0032] Specifically, the aforementioned time-series generative adversarial network is used to generate large-scale simulated human-computer interaction trajectory data covering different fatigue states and multiple impedance parameter combinations, thus constructing a GAN trajectory database. Based on this database, an independent error prediction model is trained, which receives the current impedance parameter combination. Muscle fatigue level Using other contextual information as input, it outputs in real time the average position error of the entire future motion trajectory. Compared with the average human-computer interaction force error The predicted value. By embedding online error prediction information based on GAN simulation data, the state vector It not only integrates the current physical and physiological state, but also incorporates the ability to predict future trends in task execution, thus providing a decision-making basis with state enhancement characteristics for reinforcement learning parameter optimizers. This enables them to assess the long-term impact of parameter adjustments and achieve smoother and safer adaptive control.

[0033] In step S3, the enhanced state vector constructed in step S2 is... Input the pre-trained reinforcement learning parameter optimizer.

[0034] The pre-training of reinforcement learning parameter optimizers (i.e., PPO-PSA agents) relies on high-quality, diverse interaction trajectory data. To address the problem of sparse rehabilitation clinical data, especially the difficulty in obtaining large amounts of interaction data under different fatigue levels, an improved time-series generative adversarial network (GAN) model is used for data augmentation.

[0035] By designing a human-computer interaction experimental paradigm to collect real data, subjects were required to perform flexion and extension movements under multiple preset combinations of impedance parameters in four states: "no fatigue," "mild fatigue," "moderate fatigue," and "severe fatigue." The collected data were divided into two categories: static attributes and dynamic time series. Static attributes included impedance parameters and discrete fatigue level labels; dynamic time series included continuously changing hand position trajectories and human-computer interaction force trajectories.

[0036] Before inputting the data into the model, two levels of normalization are performed: global normalization scales all features to the [-1,1] interval, and fatigue level is encoded using one-hot encoding; intra-sample normalization preserves the dynamic shape of the trajectory, calculates the midpoint and half-range of each trajectory as normalization factors, and treats them as additional attributes that the model needs to learn.

[0037] In this embodiment, as Figure 4 As shown, a decoupled GAN architecture for time series adaptation is designed. Its generator consists of three sub-modules: an attribute generator, a scale attribute generator, and a time series generator. The attribute generator is composed of a multilayer perceptron, which takes noise as input and outputs normalized static attributes. The scale attribute generator is also composed of a multilayer perceptron, which receives noise and generated static attributes and outputs a normalization factor. The time series generator is composed of a long short-term memory network, which receives all generated attributes and noise and uses a recurrent structure to generate time-dependent position and force sequences.

[0038] The discriminator consists of an attribute discriminator and a master discriminator, which are used to evaluate the authenticity of the generated static attributes and the authenticity of the overall time series, respectively.

[0039] The model training uses the WGAN-GP framework, introducing a gradient penalty term into the discriminator loss function to constrain the discriminator to satisfy the 1-Lipschitz condition, as shown in the formula: , in To fabricate samples, For real samples, The interpolated sample is a point randomly sampled from the line connecting a real sample and a fake sample.

[0040] The generator loss function aims to minimize the discriminator score, and the formula is: , in This represents the complete sample generated by the generator. This indicates the generated attribute portion. This is the input noise.

[0041] The gradient signal is backpropagated to each sub-module of the generator through a feedback loop, achieving coordinated optimization of static properties and dynamic trajectory generation. This data augmentation method generates a large amount of simulation interaction data covering different fatigue states and impedance parameter combinations, providing ample and diverse data support for the pre-training of the reinforcement learning parameter optimizer.

[0042] The aforementioned data augmentation methods can generate a large amount of simulated interactive data covering different fatigue states and impedance parameter combinations, providing sufficient and diverse data support for the pre-training of the reinforcement learning parameter optimizer. Based on this, the reinforcement learning parameter optimizer adopts a proximal policy optimization algorithm and integrates predictive state augmentation (PPO-PSA), the core of which is a pre-trained policy network. The network receives the state vector. Then, output a 2D continuous action vector. ,in and These represent the adjustments made to the current virtual damping and virtual stiffness parameters, respectively. This output serves as an optimization suggestion for the impedance parameters, aiming to achieve smooth, adaptive parameter adjustments.

[0043] To guide the learning process of the agent, a comprehensive reward function is defined during the offline training phase. This function is calculated after a complete decision round (i.e., after executing an entire trajectory), and consists of three parts, as shown in the formula: , in, It is the average reward for the entire trajectory, which incorporates the fatigue reward at each point in time. (For example: +0.5 for no fatigue, +1.5 for slight fatigue, -1.5 for moderate fatigue, and -2 for severe fatigue) and an error penalty term, which is used to penalize the instantaneous errors in trajectory and force. Its calculation formula is: , in This represents the error penalty weight. This is a smoothing penalty term, used to penalize overly drastic parameter adjustments to ensure smooth output. Its calculation formula is: , in To adjust the penalty weights, It is a sparse positive reward that is triggered when the average error of the entire trajectory is less than a set threshold, in order to encourage high-precision completion.

[0044] The aforementioned reinforcement learning parameter optimizer uses the Actor-Critic method for optimization, and its optimization objective is divided into two parts.

[0045] Critic Network Used to assess status The value (i.e., expected return) is determined by minimizing the mean square loss of the time difference (TD) error. To update parameters : .

[0046] The Actor network is the pre-trained policy network mentioned above. Used to generate actions It achieves this by maximizing a pruning proxy objective function. To update parameters This function is designed to safely optimize the policy, and its expression is: , in It is the probability ratio of the new strategy to the old strategy. It is the dominant function. It is the cutting factor.

[0047] In real-time control, the reinforcement learning agent adjusts according to the output. Update the impedance parameters, that is, the current virtual damping parameters. and virtual stiffness parameters Adjusted to and This yields optimized virtual stiffness and virtual damping parameters, which are then used for the calculation and execution of subsequent control modes.

[0048] Furthermore, in step S4, the hand posture data collected in step S1 is used to drive the virtual environment and complete the real-time judgment of virtual interaction and contact status.

[0049] Optionally, in this embodiment, the user's actual hand spatial position and posture data captured by a non-contact optical hand tracking device such as Leap Motion are transmitted in real time to a virtual reality interaction module built on the Unity3D game engine. This module has a built-in virtual hand model corresponding to the user's hand shape. The system uses the acquired hand posture data to drive the virtual hand model in real time through the official SDK or a custom mapping algorithm, so that its posture and movement in the virtual 3D scene are synchronized with the user's real hand, thereby establishing an intuitive mapping and feedback at the visual level. In the virtual scene, interactive virtual objects are also preset, such as a virtual sphere with elasticity and deformation characteristics built based on a physics engine (such as local mesh deformation). In each control cycle, the system executes a precise collision detection algorithm based on the updated virtual hand model posture. This algorithm determines whether the two intersect or contact in space by calculating the geometric relationship between the key points of the virtual hand model (such as fingertips and palm) and the surface of the virtual object. The collision detection result is output as a Boolean contact status signal, a binary flag indicating "not in contact" or "in contact". Simultaneously, when contact is detected, the system records and outputs the spatial coordinates of the contacted point on the virtual object in real time, which are used as the contact position in the judgment result. The judgment result (including contact status and contact position) is immediately sent to the subsequent control command calculation module, providing a key visual event trigger signal for switching physical control strategies.

[0050] Step S5, based on the contact state in the judgment result, selects the control mode and calculates the desired speed command. Its core control law originates from the system's Cartesian space dynamics model. The dynamics of a general n-degree-of-freedom transmission structure (such as the hand rehabilitation exoskeleton in this embodiment) in Cartesian space... The middle can be represented as: , in, The inertia matrix, The matrix of Coriolis force and centrifugal force. For gravity, It is the control torque (i.e., control input) provided by the motor. It is an external input force, that is The objective is to calculate. This allows the complex nonlinear system to be represented as a simpler and more ideal target dynamics model in human-computer interaction. The trigger-type impedance controller switches between the two target dynamics models based on the contact state signal obtained in step S3.

[0051] If no contact occurs (free-space motion), the controller switches to pure admittance control mode. In this mode, the target dynamic model is a purely first-order admittance system, which is only affected by external forces. To produce a proportionally expected speed (Right now ), determined by the set admittance damping coefficient Define its dynamic relationship: , Therefore, the controller in each discrete control cycle The desired speed command calculated internally is: .

[0052] If contact occurs (interaction with a virtual object), the controller switches to hybrid admittance mode. At this time, the established target dynamics model responds to user forces. At the same time, a reaction force generated by the virtual environment must be countered. This virtual force is determined by adaptive virtual stiffness. and virtual damping Common definition. At this point, the expected dynamic relationship of the system (i.e., net force) The drive system is: , Among them, virtual reaction force It is defined to contain only the spring term (i.e., the position error term) to ensure the passivity and stability of the system: , Substituting this into the formula, we obtain the formula for calculating the desired speed command in hybrid mode: .

[0053] The key parameter in this mode is virtual stiffness. and virtual damping The parameters are adaptively adjusted by the reinforcement learning parameter optimizer in step S3. Wherein, To determine the contact location in the results, This is the current position of the exoskeleton.

[0054] Furthermore, in step S6, based on the calculated desired speed command... The target position command for the next moment is calculated through integration. This generates motor drive commands, which in turn drive the hand rehabilitation exoskeleton to perform corresponding movements, thus completing a full control cycle. Afterward, the process returns to step S1 and repeats the above steps, forming an adaptive closed loop that can intelligently and in real-time adjust the control strategy based on the user's physiological state, physical interaction, and virtual scene.

[0055] Optionally, the exoskeleton is a customized five-finger linkage exoskeleton, with the motor being a Maxon EC series micro motor that integrates an AMSI force sensor and a RENISHAW grating encoder.

[0056] The sEMG acquisition system uses the Trigno wireless surface electromyography system or similar products to ensure signal quality and transmission stability.

[0057] The central processing computer is equipped with an Intel Core i7 or higher processor, 32GB of memory, and an NVIDIA RTX series graphics card. It runs Ubuntu or Windows and integrates ROS (Robot Operating System) for inter-module communication.

[0058] The reinforcement learning optimizer is implemented based on the Stable Baselines3 library; the virtual reality module is developed based on the Unity3D engine.

[0059] Example 2: like Figure 5 As shown, the present invention also provides a hand exoskeleton control system that integrates fatigue perception and reinforcement learning, for implementing the hand exoskeleton control method integrating fatigue perception and reinforcement learning described in Embodiment 1, comprising the following modules: The information acquisition module includes a hand rehabilitation exoskeleton, which is used to acquire the user's original surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data acquired based on vision. The state processing module is used to input the original surface electromyography signal into a pre-trained fatigue recognition model to obtain the current muscle fatigue level; fuse the muscle fatigue level, the human-computer interaction force, and position information; construct an attribute-decoupled time-series generative adversarial network based on the human-computer interaction information to synthesize a simulated hand human-computer interaction trajectory dataset; and fuse the muscle fatigue level, the human-computer interaction force, and position information to construct an enhanced state vector. The parameter optimization module is used to input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters. The virtual interaction module is used to update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, collision detection is performed based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and a judgment result including the contact state and contact position is obtained. The instruction generation module is used to select a control mode and calculate instructions based on the contact state in the judgment result. If the contact state indicates no contact, the admittance control mode is selected, and the desired velocity instruction is calculated based on the human-machine interaction force. If the contact state indicates contact, the hybrid admittance control mode is selected, and the optimized virtual stiffness parameters and virtual damping parameters are applied, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired velocity instruction. The execution control module is used to obtain motor control commands based on the desired speed command and drive the hand rehabilitation exoskeleton to execute them.

[0060] This embodiment provides an adaptive rehabilitation control system for implementing the control method described in this invention. Through the collaborative work of multiple modules, the system achieves synchronous acquisition, fusion processing, and intelligent control of the user's physiological state, physical interaction, and virtual scene information, thereby completing safe, personalized, and immersive rehabilitation training.

[0061] Example 3: The present invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the hand exoskeleton control method that integrates fatigue perception and reinforcement learning as described in Embodiment 1.

[0062] Example 4: The present invention also provides a computer storage medium storing a computer software product, the computer software product including several instructions for causing a computer device to execute the hand exoskeleton control method integrating fatigue perception and reinforcement learning as described in Embodiment 1.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for controlling a hand exoskeleton that integrates fatigue perception and reinforcement learning, characterized in that, Includes the following steps: S1: Collects the user's raw surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data based on vision acquisition; S2: Input the original surface electromyography signal into the pre-trained fatigue recognition model to obtain the current muscle fatigue level; fuse the muscle fatigue level, the human-computer interaction force and position information; A time-series generative adversarial network with attribute decoupling is constructed based on human-computer interaction information to synthesize a dataset simulating human-computer interaction trajectories of the hand; the muscle fatigue level, the human-computer interaction force and position information are fused to construct an enhanced state vector; S3: Input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters; S4: Update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, perform collision detection based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and obtain a judgment result including the contact state and contact position. S5: Based on the contact state in the judgment result, select the control mode and calculate the command. If the contact state indicates no contact, select the admittance control mode and calculate the desired speed command based on the human-machine interaction force. If the contact state indicates contact, select the hybrid admittance control mode and apply the optimized virtual stiffness parameter and virtual damping parameter, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired speed command. S6: Obtain motor control commands based on the desired speed commands, and drive the hand rehabilitation exoskeleton to execute them.

2. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1, characterized in that: In step S2, the training method of the fatigue recognition model includes: preprocessing the original surface electromyography (EMG) signal, segmenting the preprocessed signal using a sliding window, and extracting time-domain and frequency-domain features for each signal window; randomly dividing the preprocessed EMG signal data into labeled and unlabeled datasets, and using a semi-supervised learning framework, inputting the labeled and unlabeled datasets into a convolutional neural network for training to obtain a recognition model capable of classifying multi-level fatigue in real-time EMG signals.

3. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1, characterized in that: In step S2, a time-series generative adversarial network with attribute decoupling is constructed based on human-computer interaction information to synthesize a dataset simulating hand-computer interaction trajectories; the muscle fatigue level, the human-computer interaction force, and the position information are fused to construct an enhanced state vector, the method being: The system collects human-computer interaction data generated by users when performing tasks under different fatigue states and different combinations of impedance parameters. The human-computer interaction data includes impedance parameters as static attributes and hand position trajectory, human-computer interaction force trajectory and real-time muscle fatigue level as dynamic time series features. Based on the collected human-computer interaction data, a time-series generative adversarial network for attribute decoupling is constructed. The attribute-decoupled time-series generative adversarial network includes a generator and a discriminator, wherein: The generator maps random noise to static attributes and generates corresponding normalization factors based on the static attributes. Based on the static attributes, normalization factors, and noise, a long short-term memory network is used to synthesize dynamic time series data containing information on hand position, human-computer interaction force, and fatigue. The discriminator evaluates the static attributes and the authenticity of the overall time series in the samples generated by the generator, respectively. The generator and the discriminator are trained alternately so that the generator synthesizes simulated trajectory data that is consistent with the distribution of real human-computer interaction data, and constructs an enhanced state vector that is input to the reinforcement learning parameter optimizer.

4. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1, characterized in that: In step S3, the method for inputting the enhanced state vector into the pre-trained reinforcement learning parameter optimizer to obtain the optimized virtual stiffness parameters and virtual damping parameters is as follows: input the enhanced state vector into the pre-trained reinforcement learning parameter optimizer to obtain an action vector containing virtual damping adjustment and virtual stiffness adjustment; update the current virtual damping parameters and virtual stiffness parameters according to the adjustment in the action vector to obtain the optimized virtual stiffness parameters and virtual damping parameters.

5. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1 or 4, characterized in that: In step S3, the training method of the reinforcement learning parameter optimizer is as follows: the reinforcement learning parameter optimizer is optimized based on the comprehensive reward function; the comprehensive reward function includes: a trajectory reward term associated with the muscle fatigue level and the tracking error of the user's motion trajectory, a smoothing penalty term that suppresses the adjustment range of the virtual stiffness parameter and the virtual damping parameter, and a sparse positive reward term triggered when the average tracking error of the motion trajectory is lower than a preset threshold.

6. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1 or 4, characterized in that: In step S3, the reinforcement learning parameter optimizer is optimized using an actor-critic framework, wherein the critic network evaluates the value of a given state and updates it by minimizing the value prediction error; the actor network outputs an action based on the current state and updates it by optimizing the objective function.

7. The hand exoskeleton control method integrating fatigue perception and reinforcement learning according to claim 1, characterized in that: In step S5, the desired velocity command in admittance control mode Calculated using the following formula: ， in, Human-computer interaction power The preset admittance damping coefficient; The desired velocity command under the hybrid admittance control mode Calculated using the following formula: ， in, The optimized virtual stiffness parameters, For the optimized virtual damping parameters, To determine the contact location in the results, This is the current location.

8. A hand exoskeleton control system integrating fatigue perception and reinforcement learning, characterized in that, Includes the following modules: The information acquisition module includes a hand rehabilitation exoskeleton, which is used to acquire the user's original surface electromyography signals, human-computer interaction force and position information from the hand rehabilitation exoskeleton, and hand posture data acquired based on vision. The state processing module is used to input the original surface electromyography signal into a pre-trained fatigue recognition model to obtain the current muscle fatigue level. The muscle fatigue level, the human-computer interaction force, and the position information are integrated; A time-series generative adversarial network with attribute decoupling is constructed based on human-computer interaction information to synthesize a dataset simulating human-computer interaction trajectories of the hand; the muscle fatigue level, the human-computer interaction force and position information are fused to construct an enhanced state vector; The parameter optimization module is used to input the enhanced state vector into a reinforcement learning parameter optimizer pre-trained based on a simulated human-computer interaction dataset to obtain optimized virtual stiffness parameters and virtual damping parameters. The virtual interaction module is used to update the virtual hand model in the virtual environment using the hand posture data; in the virtual environment, collision detection is performed based on the updated virtual hand model to determine whether the virtual hand is in contact with a virtual object, and a judgment result including the contact state and contact position is obtained. The instruction generation module is used to select a control mode and calculate instructions based on the contact state in the judgment result. If the contact state indicates no contact, the admittance control mode is selected, and the desired velocity instruction is calculated based on the human-machine interaction force. If the contact state indicates contact, the hybrid admittance control mode is selected, and the optimized virtual stiffness parameters and virtual damping parameters are applied, combined with the human-machine interaction force and the position error calculated from the contact position in the judgment result, to calculate the desired velocity instruction. The execution control module is used to obtain motor control commands based on the desired speed command and drive the hand rehabilitation exoskeleton to execute them.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the hand exoskeleton control method that integrates fatigue perception and reinforcement learning as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer software product, which includes several instructions for causing a computer device to execute the hand exoskeleton control method integrating fatigue perception and reinforcement learning as described in any one of claims 1 to 7.

Images