Forearm limb disability patient movement function rehabilitation training system based on vision and vibration touch collaborative feedback

Abstract

The invention discloses a motion function rehabilitation training system for a forearm limb disabled patient based on visual and vibratory tactile collaborative feedback. The system comprises a hardware layer, a data processing layer, an application layer and a user interface layer. The training process is guided through visual feedback, the task completion condition and the gripping strength are reflected through vibration stimulation, the forearm amputation patient is helped to achieve integration of visual information and tactile information, and the accuracy of exercise decision making and the training effect are remarkably improved. Besides, on the basis of the electromyographic signal level of the patient, a personalized rehabilitation training scheme is adopted, and the pertinence and the individual adaptability of rehabilitation training are effectively enhanced by evaluating the muscle activity intensity level and the antagonism coefficient of the patient with the forearm limb disability and customizing the difficulty level of a digital large-synthesis electromyographic game and the training types of life scenes and tasks; functional recovery of the patient is promoted.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback. Background Technology

[0002] Restoring motor function in the forearm stump hinges on recovering coordinated muscle movement patterns, particularly during the myoelectric prosthesis dexterity control and prosthesis adaptation training phases, which represent one of the greatest challenges for forearm amputees. Virtual Reality (VR) technology, as an emerging rehabilitation training tool, is widely used due to its ability to provide an immersive training experience. Virtual environments can enhance the interaction between the patient and the prosthesis and, through gamification, improve the patient's motivation and engagement in training.

[0003] Existing technologies, through diverse game scenarios and interactive designs, create an immersive rehabilitation experience for forearm amputees, enhancing the fun of training and user engagement. Simultaneously, personalized training settings allow users to flexibly adjust training content and difficulty according to their own needs and abilities, thereby improving training effectiveness. However, current virtual rehabilitation training systems for forearm amputees have the following problems:

[0004] 1) Limited Sensory Feedback Mode: Existing virtual rehabilitation training systems for forearm amputees primarily rely on visual feedback, meaning patients can only obtain prosthetic control information through a single visual pathway. This limits the robustness and fault tolerance of perception, affecting the precise adjustment of motor output by the brain's internal model. Furthermore, continuous visual stimulation can easily cause visual fatigue, thereby reducing training effectiveness and patient engagement.

[0005] 2) Insufficient Adaptability to Individual Differences: Traditional rehabilitation training programs typically rely on therapists to roughly assess a patient's initial electromyographic (EMG) function status through visual inspection of the amputation level and manual palpation, thereby predicting incremental rehabilitation goals. This approach lacks quantitative evidence, heavily depends on the therapist's experience, and struggles to guarantee the scientific validity and effectiveness of the rehabilitation program. Furthermore, forearm amputees often experience lower EMG intensity due to muscle atrophy, and their condition is significantly influenced by factors such as the cause and duration of amputation, resulting in substantial individual differences. Traditional qualitative rehabilitation programs often fail to personalize adjustments based on each patient's EMG signal characteristics and psychological adaptability, leading to significant fluctuations in training effectiveness due to individual variations. Summary of the Invention

[0006] The purpose of this invention is to provide a rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback, comprising: a hardware layer, a data processing layer, an application layer, and a user interface layer.

[0007] The hardware layer is used to collect electromyographic signals of the user's forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback to the user according to the control instructions of the data processing layer.

[0008] The data processing layer is used to process the acquired electromyographic signals and recognize gestures, and to generate control commands.

[0009] The application layer is used to dynamically map and simulate the electromyographic signals and recognized gestures processed by the data processing layer, and to build a rehabilitation training interface.

[0010] The user interface layer is used to display the gestures and rehabilitation training interface recognized by the data processing layer, providing visual feedback to the user.

[0011] Users conduct rehabilitation training based on vibration feedback provided by the hardware layer and visual feedback provided by the user interface layer.

[0012] The application layer plans a personalized rehabilitation path for the user based on the user's rehabilitation training data and the electromyographic signals processed by the data processing layer.

[0013] Furthermore, the hardware layer includes a wearable electromyography (EMG) acquisition device, a 9-axis inertial measurement unit, and a vibration meter.

[0014] The wearable electromyography (EMG) acquisition device is used to collect EMG signals from the user's forearm stump in real time.

[0015] The sites for collecting electromyographic signals of the forearm stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus.

[0016] The 9-axis inertial measurement unit and vibration meter are integrated into the wearable electromyography (EMG) acquisition device.

[0017] The 9-axis inertial measurement unit is used to monitor changes in the user's upper limb posture.

[0018] The user's upper limb posture changes include wrist rotation, arm raising, and hand angle adjustment.

[0019] The 9-axis inertial measurement unit includes an accelerometer, a gyroscope, and a magnetometer.

[0020] The accelerometer is used to capture the acceleration of the user's forearm in three-dimensional space.

[0021] The gyroscope is used to capture the angular velocity of the user's forearm in three-dimensional space.

[0022] The magnetometer is used to capture the geomagnetic vector data of the user's forearm in three-dimensional space.

[0023] The vibration meter provides vibration feedback to the user.

[0024] Furthermore, the data processing layer includes an electromyography signal preprocessing module, a feature extraction module, and a gesture recognition and classification module.

[0025] The electromyography signal preprocessing module includes an adaptive notch filter, a bandpass filter, a dynamic gain control module, an invalid rejection module, and a window processing module.

[0026] The adaptive notch filter is used to filter the acquired electromyographic signals to obtain electromyographic signals that have eliminated power frequency interference.

[0027] The bandpass filter is used to filter the electromyographic signal to eliminate power frequency interference, thereby obtaining an electromyographic signal with baseline drift and high-frequency noise removed.

[0028] The dynamic gain control module is used to adjust the amplitude of the electromyographic signal after filtering out baseline drift and high-frequency noise, so as to obtain an amplitude-adjusted electromyographic signal.

[0029] The invalid data removal module is used to remove invalid data from the amplitude-adjusted electromyographic signals.

[0030] The window processing module is used to perform windowing processing on the electromyographic signals after removing invalid data to obtain preprocessed data.

[0031] The feature extraction module is used to extract segmented features from the preprocessed data.

[0032] The features include time-domain features and frequency-domain features.

[0033] The time-domain features include the mean absolute value, root mean square value, and zero-crossing rate of the electromyographic signal.

[0034] The frequency domain characteristics include the proportion of mid-frequency energy.

[0035] The gesture recognition and classification module uses a lightweight convolutional neural network to perform gesture recognition on the extracted features, and obtains the gesture recognition result.

[0036] The gesture recognition results include, but are not limited to, clenching a fist, extending a palm, extending a wrist, flexing a wrist, and relaxing.

[0037] The gesture recognition and classification module generates control commands based on the gesture recognition results.

[0038] Furthermore, the rehabilitation training interface includes a digital synthetic electromyography game, life-like scenarios, and tasks.

[0039] The digital synthetic electromyography game is based on the classic 2048 rule, mapping the recognized gestures to the corresponding game operations, moving and merging virtual blocks, and displaying the game score and win / loss status in real time.

[0040] The classic 2048 rules use a 4x4 grid to dynamically manage the block state and generate new blocks using a two-dimensional array and a random algorithm. Players score points by merging adjacent blocks with the same number.

[0041] The game controls include moving up, moving down, moving left, and moving right.

[0042] The described everyday scenarios and tasks are constructed by creating virtual scenes, simulating the processed electromyographic signals and recognized gestures into corresponding game actions, and completing daily tasks in the virtual scenes.

[0043] The game actions include camera movement and virtual hand movements.

[0044] The virtual hand gestures include opening a door, closing a door, grabbing, and putting down.

[0045] The daily tasks include opening and closing doors, turning lights on and off, and grabbing items.

[0046] Furthermore, the steps for constructing the virtual hand in the aforementioned everyday scenarios and tasks are as follows:

[0047] S1 creates geometric models of the palm, fingers, and joints based on real hand anatomy.

[0048] S2 utilizes Blender's skeletal animation tools to achieve independent control of each joint, constructing a hierarchical skeletal system that covers the skeletal nodes of the palm and five finger joints.

[0049] S3 uses binding technology to bind the geometric model of the palm, fingers, and joints, as well as a hierarchical skeletal system, to drive the virtual hand to move.

[0050] Furthermore, in everyday scenarios and tasks, the mechanical feedback of virtual hand grasping actions is simulated based on the collision body dynamic adjustment method and the mass-damping physics engine.

[0051] The radius R of the collider is shown below:

[0052] R = 8 + 2(RMS / 0.5) (1)

[0053] In the formula, RMS is the root mean square value of the electromyography (EMG) signal. When the EMG signal is greater than a preset threshold, the radius of the collider is increased to assist in grasping; when the EMG signal is less than the preset threshold, the radius of the collider is decreased for grasping.

[0054] The forces acting on the virtual hand are shown below:

[0055] F = ma + cv (2)

[0056] In the formula, F represents the force on the virtual hand mapped by the electromyographic signal. m is the mass of the virtual hand, c is the damping coefficient of the virtual hand, a is the acceleration of the virtual hand, and v is the velocity of the virtual hand.

[0057] Furthermore, the visual feedback includes real-time motion feedback of the user's virtual hand and a real-time electromyography (EMG) intensity radar comparison between the user and a healthy user.

[0058] Furthermore, the vibration feedback includes feedback on the completion of daily tasks and games, as well as grip strength level feedback.

[0059] The completion status of daily tasks and games is fed back through different vibration durations.

[0060] The grip strength level is fed back through different vibration channels and vibration frequencies.

[0061] The grip strength level is characterized using a ratio-weighted method, as shown below:

[0062]

[0063] In the formula, D represents the grip strength level. MAV 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar flexor carpi radialis muscles in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given, and MVC represents the maximum voluntary contraction of the muscle.

[0064] Furthermore, the rehabilitation training data includes electromyographic indicators and rehabilitation training indicators.

[0065] The electromyographic parameters include muscle activity intensity and antagonistic coefficient.

[0066] The rehabilitation training indicators include the game score rate of the digital mesotherapy game and the task success rate of life-like scenarios and tasks.

[0067] The game score rates are shown below:

[0068]

[0069] In the formula, λ represents the game score rate. The average score in the game. This represents the average game time.

[0070] The success rates of the tasks are shown below:

[0071]

[0072] In the formula, γ represents the task success rate. q1 represents the number of times the gesture was successfully completed. q0 represents the total number of gesture attempts.

[0073] Furthermore, the application layer modules for planning personalized rehabilitation pathways include a muscle strength level adaptation module and an antagonist muscle synergy adaptation module.

[0074] The muscle strength level adaptation module classifies the user's muscle activity intensity into corresponding muscle strength levels and matches them with the difficulty level of the corresponding digital synthetic electromyography game.

[0075] The users' muscle strength levels are divided into four levels, from weak to strong.

[0076] The digital synthetic electromyography game is divided into four levels from easy to difficult.

[0077] When the root mean square value of the electromyography signal of some muscles is detected to be lower than that of other muscles, the muscle strength level adaptation module performs rehabilitation training on the detected muscles.

[0078] The antagonistic muscle synergistic adaptation module evaluates the coordination of wrist flexion and extension movements by calculating the antagonistic synergistic coefficients of extensor and flexor muscles, and increases the training ratio of everyday tasks in real-life scenarios based on the calculated antagonistic synergistic coefficients.

[0079] The antagonistic synergistic coefficients of the extensor and flexor muscles are shown below:

[0080]

[0081] In the formula, α is the wrist flexion antagonism coefficient, and β is the wrist extension antagonism coefficient. RMS 屈肌 RMS is the sum of the root mean square (RMS) values ​​of the electromyographic signals of the radial and ulnar wrist flexors in patients with forearm amputations. 伸肌 It is the sum of the root mean square values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations.

[0082] The technical effectiveness of this invention is undeniable. It employs a synergistic feedback approach combining visual and vibratory-tactile feedback, enriching the feedback modalities. Visual feedback provides cues and action guidance, while vibratory-tactile feedback activates the skin's tactile and proprioceptive senses, enabling forearm amputees to utilize sensory integration to improve decision-making accuracy and thus significantly enhance training effectiveness.

[0083] This invention calculates the muscle activity intensity level and antagonism coefficient based on the initial electromyographic signals collected from each forearm amputee patient, and adapts it to the game difficulty level and life scenario task type in the training program, reflecting the patient's personalized rehabilitation path, thereby improving the targeted nature of the rehabilitation effect.

[0084] This system breaks through the limitations of traditional rehabilitation programs in terms of individual adaptability. It can dynamically adjust the training content according to the patient's electromyographic control ability, and accurately improve the prosthesis control ability and the patient's rehabilitation compliance. Attached Figure Description

[0085] Figure 1 This is a schematic diagram of the structure of a motor function rehabilitation training system for patients with forearm limb disabilities based on visual and vibration-tactile feedback provided by the present invention.

[0086] Figure 2 This is a flowchart of the "Digital Synthesis" electromyography game implementation provided by the present invention;

[0087] Figure 3 This is a schematic diagram of the user interface provided by the present invention. Detailed Implementation

[0088] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0089] Example 1:

[0090] See Figures 1 to 3 The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback includes: hardware layer, data processing layer, application layer and user interface layer.

[0091] The hardware layer is used to collect electromyographic signals of the user's forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback to the user according to the control instructions of the data processing layer.

[0092] The data processing layer is used to process the acquired electromyographic signals and recognize gestures, and to generate control commands.

[0093] The application layer is used to dynamically map and simulate the electromyographic signals and recognized gestures processed by the data processing layer, and to build a rehabilitation training interface.

[0094] The user interface layer is used to display the gestures and rehabilitation training interface recognized by the data processing layer, providing visual feedback to the user.

[0095] Users conduct rehabilitation training based on vibration feedback provided by the hardware layer and visual feedback provided by the user interface layer.

[0096] The application layer plans a personalized rehabilitation path for the user based on the user's rehabilitation training data and the electromyographic signals processed by the data processing layer.

[0097] Example 2:

[0098] The rehabilitation training system for forearm limb amputees based on visual and vibration-tactile feedback is described in Example 1. The hardware layer further includes a wearable electromyography (EMG) acquisition device, a 9-axis inertial measurement unit, and a vibration meter.

[0099] The wearable electromyography (EMG) acquisition device is used to collect EMG signals from the user's forearm stump in real time.

[0100] The sites for collecting electromyographic signals of the forearm stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus.

[0101] The 9-axis inertial measurement unit and vibration meter are integrated into the wearable electromyography (EMG) acquisition device.

[0102] The 9-axis inertial measurement unit is used to monitor changes in the user's upper limb posture.

[0103] The user's upper limb posture changes include wrist rotation, arm raising, and hand angle adjustment.

[0104] The 9-axis inertial measurement unit includes an accelerometer, a gyroscope, and a magnetometer.

[0105] The accelerometer is used to capture the acceleration of the user's forearm in three-dimensional space.

[0106] The gyroscope is used to capture the angular velocity of the user's forearm in three-dimensional space.

[0107] The magnetometer is used to capture the geomagnetic vector data of the user's forearm in three-dimensional space.

[0108] The vibration meter provides vibration feedback to the user.

[0109] Example 3:

[0110] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 1 and 2. Furthermore, the data processing layer includes an electromyography signal preprocessing module, a feature extraction module, and a gesture recognition and classification module.

[0111] The electromyography signal preprocessing module includes an adaptive notch filter, a bandpass filter, a dynamic gain control module, an invalid rejection module, and a window processing module.

[0112] The adaptive notch filter is used to filter the acquired electromyographic signals to obtain electromyographic signals that have eliminated power frequency interference.

[0113] The bandpass filter is used to filter the electromyographic signal to eliminate power frequency interference, thereby obtaining an electromyographic signal with baseline drift and high-frequency noise removed.

[0114] The dynamic gain control module is used to adjust the amplitude of the electromyographic signal after filtering out baseline drift and high-frequency noise, so as to obtain an amplitude-adjusted electromyographic signal.

[0115] The invalid data removal module is used to remove invalid data from the amplitude-adjusted electromyographic signals.

[0116] The window processing module is used to perform windowing processing on the electromyographic signals after removing invalid data to obtain preprocessed data.

[0117] The feature extraction module is used to extract segmented features from the preprocessed data.

[0118] The features include time-domain features and frequency-domain features.

[0119] The time-domain features include the mean absolute value, root mean square value, and zero-crossing rate of the electromyographic signal.

[0120] The frequency domain characteristics include the proportion of mid-frequency energy.

[0121] The gesture recognition and classification module uses a lightweight convolutional neural network to perform gesture recognition on the extracted features, and obtains the gesture recognition result.

[0122] The gesture recognition results include, but are not limited to, clenching a fist, extending a palm, extending a wrist, flexing a wrist, and relaxing.

[0123] The gesture recognition and classification module generates control commands based on the gesture recognition results.

[0124] Example 4:

[0125] The rehabilitation training system for forearm limb amputees based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 1 to 3, further, the rehabilitation training interface includes digital synthetic electromyography games, life-like scenes and tasks.

[0126] The digital synthetic electromyography game is based on the classic 2048 rule, mapping the recognized gestures to the corresponding game operations, moving and merging virtual blocks, and displaying the game score and win / loss status in real time.

[0127] The classic 2048 rules use a 4x4 grid to dynamically manage the block state and generate new blocks using a two-dimensional array and a random algorithm. Players score points by merging adjacent blocks with the same number.

[0128] The game controls include moving up, moving down, moving left, and moving right.

[0129] The described everyday scenarios and tasks are constructed by creating virtual scenes, simulating the processed electromyographic signals and recognized gestures into corresponding game actions, and completing daily tasks in the virtual scenes.

[0130] The game actions include camera movement and virtual hand movements.

[0131] The virtual hand gestures include opening a door, closing a door, grabbing, and putting down.

[0132] The daily tasks include opening and closing doors, turning lights on and off, and grabbing items.

[0133] Example 5:

[0134] The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 1 to 4, further, the steps for constructing the virtual hand in the lifelike scenarios and tasks are as follows:

[0135] S1 creates geometric models of the palm, fingers, and joints based on real hand anatomy.

[0136] S2 utilizes Blender's skeletal animation tools to achieve independent control of each joint, constructing a hierarchical skeletal system that covers the skeletal nodes of the palm and five finger joints.

[0137] S3 uses binding technology to bind the geometric model of the palm, fingers, and joints, as well as a hierarchical skeletal system, to drive the virtual hand to move.

[0138] Example 6:

[0139] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 1 to 5. Furthermore, in everyday scenarios and tasks, the system simulates the mechanical feedback of virtual hand grasping actions based on the collision body dynamic adjustment method and the mass-damping physics engine.

[0140] The radius R of the collider is shown below:

[0141] R = 8 + 2(RMS / 0.5) (1)

[0142] In the formula, RMS is the root mean square value of the electromyography (EMG) signal. When the EMG signal is greater than a preset threshold, the radius of the collider is increased to assist in grasping; when the EMG signal is less than the preset threshold, the radius of the collider is decreased for grasping.

[0143] The forces acting on the virtual hand are shown below:

[0144] F = ma + cv (2)

[0145] In the formula, F represents the force on the virtual hand mapped by the electromyographic signal. m is the mass of the virtual hand, c is the damping coefficient of the virtual hand, a is the acceleration of the virtual hand, and v is the velocity of the virtual hand.

[0146] Example 7:

[0147] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 1 to 6. Furthermore, the visual feedback includes real-time motion feedback of the user's virtual hand and real-time electromyography intensity radar comparison feedback between the user and a healthy user.

[0148] Example 8:

[0149] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 1 to 7. Furthermore, the vibration feedback includes feedback on the completion of daily tasks and games, and feedback on the grip strength level.

[0150] The completion status of daily tasks and games is fed back through different vibration durations.

[0151] The grip strength level is fed back through different vibration channels and vibration frequencies.

[0152] The grip strength level is characterized using a ratio-weighted method, as shown below:

[0153]

[0154] In the formula, D represents the grip strength level. MAV 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar flexor carpi radialis muscles in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given, and MVC represents the maximum voluntary contraction of the muscle.

[0155] Example 9:

[0156] The rehabilitation training system for forearm limb amputees based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 1 to 8, and further, the rehabilitation training data includes electromyographic indicators and rehabilitation training indicators.

[0157] The electromyographic parameters include muscle activity intensity and antagonistic coefficient.

[0158] The rehabilitation training indicators include the game score rate of the digital mesotherapy game and the task success rate of life-like scenarios and tasks.

[0159] The game score rates are shown below:

[0160]

[0161] In the formula, λ represents the game score rate. The average score in the game. This represents the average game time.

[0162] The success rates of the tasks are shown below:

[0163]

[0164] In the formula, γ represents the task success rate. q1 represents the number of times the gesture was successfully completed. q0 represents the total number of gesture attempts.

[0165] Example 10:

[0166] The forearm limb disability rehabilitation training system based on visual and vibration-tactile feedback is described in any one of Examples 1 to 9. Furthermore, the application layer module for planning personalized rehabilitation pathways includes a muscle strength level adaptation module and an antagonistic muscle synergistic adaptation module.

[0167] The muscle strength level adaptation module classifies the user's muscle activity intensity into corresponding muscle strength levels and matches them with the difficulty level of the corresponding digital synthetic electromyography game.

[0168] The users' muscle strength levels are divided into four levels, from weak to strong.

[0169] The digital synthetic electromyography game is divided into four levels from easy to difficult.

[0170] When the root mean square value of the electromyography signal of some muscles is detected to be lower than that of other muscles, the muscle strength level adaptation module performs rehabilitation training on the detected muscles.

[0171] The antagonistic muscle synergistic adaptation module evaluates the coordination of wrist flexion and extension movements by calculating the antagonistic synergistic coefficients of extensor and flexor muscles, and increases the training ratio of everyday tasks in real-life scenarios based on the calculated antagonistic synergistic coefficients.

[0172] The antagonistic synergistic coefficients of the extensor and flexor muscles are shown below:

[0173]

[0174] In the formula, α is the wrist flexion antagonism coefficient, and β is the wrist extension antagonism coefficient. RMS 屈肌 RMS is the sum of the root mean square (RMS) values ​​of the electromyographic signals of the radial and ulnar wrist flexors in patients with forearm amputations. 伸肌 It is the sum of the root mean square values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations.

[0175] Example 11:

[0176] See Figures 1 to 3 A rehabilitation training system for forearm amputees based on visual and vibration-tactile feedback includes: a hardware layer, a data processing layer, an application layer, and a user interface layer.

[0177] The hardware layer is used to collect electromyographic signals of the user's forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback to the user according to the control instructions of the data processing layer.

[0178] The data processing layer is used to process the acquired electromyographic signals and recognize gestures, and to generate control commands.

[0179] The application layer is used to dynamically map and simulate the electromyographic signals and recognized gestures processed by the data processing layer, and to build a rehabilitation training interface.

[0180] The user interface layer is used to display the gestures and rehabilitation training interface recognized by the data processing layer, providing visual feedback to the user.

[0181] Users conduct rehabilitation training based on vibration feedback provided by the hardware layer and visual feedback provided by the user interface layer.

[0182] The application layer plans a personalized rehabilitation path for the user based on the user's rehabilitation training data and the electromyographic signals processed by the data processing layer.

[0183] Example 12:

[0184] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in Example 11. Furthermore, the hardware layer includes a wearable electromyography (EMG) acquisition device, a 9-axis inertial measurement unit, and a vibration meter.

[0185] The wearable electromyography (EMG) acquisition device is used to collect EMG signals from the user's forearm stump in real time.

[0186] The sites for collecting electromyographic signals of the forearm stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus.

[0187] The 9-axis inertial measurement unit and vibration meter are integrated into the wearable electromyography (EMG) acquisition device.

[0188] The 9-axis inertial measurement unit is used to monitor changes in the user's upper limb posture.

[0189] The user's upper limb posture changes include wrist rotation, arm raising, and hand angle adjustment.

[0190] The 9-axis inertial measurement unit includes an accelerometer, a gyroscope, and a magnetometer.

[0191] The accelerometer is used to capture the acceleration of the user's forearm in three-dimensional space.

[0192] The gyroscope is used to capture the angular velocity of the user's forearm in three-dimensional space.

[0193] The magnetometer is used to capture the geomagnetic vector data of the user's forearm in three-dimensional space.

[0194] The vibration meter provides vibration feedback to the user.

[0195] Example 13:

[0196] The forearm limb disability rehabilitation training system based on visual and vibration-tactile feedback is described in any one of Examples 11 to 12. Furthermore, the data processing layer includes an electromyography signal preprocessing module, a feature extraction module, and a gesture recognition and classification module.

[0197] The electromyography signal preprocessing module includes an adaptive notch filter, a bandpass filter, a dynamic gain control module, an invalid rejection module, and a window processing module.

[0198] The adaptive notch filter is used to filter the acquired electromyographic signals to obtain electromyographic signals that have eliminated power frequency interference.

[0199] The bandpass filter is used to filter the electromyographic signal to eliminate power frequency interference, thereby obtaining an electromyographic signal with baseline drift and high-frequency noise removed.

[0200] The dynamic gain control module is used to adjust the amplitude of the electromyographic signal after filtering out baseline drift and high-frequency noise, so as to obtain an amplitude-adjusted electromyographic signal.

[0201] The invalid data removal module is used to remove invalid data from the amplitude-adjusted electromyographic signals.

[0202] The window processing module is used to perform windowing processing on the electromyographic signals after removing invalid data to obtain preprocessed data.

[0203] The feature extraction module is used to extract segmented features from the preprocessed data.

[0204] The features include time-domain features and frequency-domain features.

[0205] The time-domain features include the mean absolute value, root mean square value, and zero-crossing rate of the electromyographic signal.

[0206] The frequency domain characteristics include the proportion of mid-frequency energy.

[0207] The gesture recognition and classification module uses a lightweight convolutional neural network to perform gesture recognition on the extracted features, and obtains the gesture recognition result.

[0208] The gesture recognition and classification module is based on a lightweight convolutional neural network (CNN) to achieve gesture classification. It combines offline training with online deployment: an 8-channel electromyography signal classification model is trained using PyTorch, converted to ONNX format, and integrated into Unity3D. The Barracuda engine is used to achieve real-time inference in less than 50ms. The dynamic calibration mechanism establishes a user-personalized feature model by collecting 30 seconds of baseline action data, and dynamically optimizes the classification threshold using the sliding window method to ensure stable cross-user recognition accuracy.

[0209] The gesture recognition results include, but are not limited to, clenching a fist, extending a palm, extending a wrist, flexing a wrist, and relaxing.

[0210] The gesture recognition and classification module generates control commands based on the gesture recognition results.

[0211] The system adjusts the duration, intensity, and mode of vibration in real time based on the control commands (specific actions performed by the user and the completion status of the task) obtained from the gesture recognition module and the intensity of directly acquired electromyographic signals.

[0212] The gesture recognition and classification module is responsible for converting the collected electromyographic signals into control commands that can be executed for virtual training and rehabilitation tasks. Here, "control commands" refers to mapping the recognized gestures into commands that the system can understand and respond to, such as "clench fist," "open hand," and "rotate wrist" action commands.

[0213] Example 14:

[0214] The rehabilitation training system for forearm limb amputees based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 11 to 13, further wherein the rehabilitation training interface includes digital synthetic electromyography games, life-like scenes and tasks.

[0215] The digital synthetic electromyography game is based on the classic 2048 rule, mapping the recognized gestures to the corresponding game operations, moving and merging virtual blocks, and displaying the game score and win / loss status in real time.

[0216] The classic 2048 rules use a 4x4 grid to dynamically manage the block state and generate new blocks using a two-dimensional array and a random algorithm. Players score points by merging adjacent blocks with the same number.

[0217] The game uses a 4x4 grid to dynamically manage the state of blocks, generating new blocks through a two-dimensional array and a random algorithm, and implementing movement and merging logic based on row / column detection (such as doubling adjacent blocks with the same number). Unity coroutines are used to control animation delays and smooth transitions, combined with real-time score updates and victory / defeat status feedback, to enhance patients' hand-eye coordination, electromyographic signal control accuracy, and reaction speed.

[0218] The game controls include moving up, moving down, moving left, and moving right.

[0219] The described everyday scenarios and tasks are constructed by creating virtual scenes, simulating the processed electromyographic signals and recognized gestures into corresponding game actions, and completing daily tasks in the virtual scenes.

[0220] The game actions include camera movement and virtual hand movements.

[0221] The virtual hand gestures include opening a door, closing a door, grabbing, and putting down.

[0222] The daily tasks include opening and closing doors, turning lights on and off, and grabbing items.

[0223] Wrist rolling controls the camera's perspective, and dead zone thresholding helps prevent accidental operation; wrist extension / flexion gestures trigger door opening and closing animations, and fist clenching drives SphereCast to grasp objects. The Rigidbody component manages object dynamics, and the Animator controls interactive animations. Based on the collaboration between highly realistic scenes and a physics engine, patients can enhance their fine motor skills while performing everyday tasks.

[0224] Example 15:

[0225] The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 11 to 14, further, the construction steps of the virtual hand in the life-like scenarios and tasks are as follows:

[0226] S1 creates geometric models of the palm, fingers, and joints based on real hand anatomy.

[0227] S2 utilizes Blender's skeletal animation tools to achieve independent control of each joint, constructing a hierarchical skeletal system that covers the skeletal nodes of the palm and five finger joints.

[0228] S3 uses binding technology to bind the geometric model of the palm, fingers, and joints, as well as a hierarchical skeletal system, to drive the virtual hand to move.

[0229] Example 16:

[0230] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 11 to 15. Furthermore, in everyday scenarios and tasks, the system simulates the mechanical feedback of virtual hand grasping actions based on the collision body dynamic adjustment method and the mass-damping physics engine.

[0231] The radius R of the collider is shown below:

[0232] R = 8 + 2(RMS / 0.5) (1)

[0233] In the formula, RMS is the root mean square value of the electromyographic signal. When the electromyographic signal is greater than a preset threshold, the radius of the collider is increased to assist grasping; when the electromyographic signal is less than the preset threshold, the radius of the collider is decreased for precise grasping. By dynamically optimizing the collision range through the intensity of the electromyographic signal, the influence of real hand muscle exertion on grasping stability is simulated, enhancing the patient's awareness of control over the electromyographic signal.

[0234] The preset threshold can be set within the range of approximately 0.3-0.7, which can better reflect the degree of muscle activation and also help distinguish effective grasping actions. When it is below 0.3, the collision radius is close to the base value of 8 (smaller), reducing false grasps; when it is above 0.7, the collision radius will increase according to the formula, improving the error tolerance and stability of grasping judgment.

[0235] The forces acting on the virtual hand are shown below:

[0236] F = ma + cv (2)

[0237] In the formula, F represents the force on the virtual hand mapped by the electromyographic signal. m is the mass of the virtual hand, c is the damping coefficient of the virtual hand, a is the acceleration of the virtual hand, and v is the velocity of the virtual hand.

[0238] The stronger the electromyographic signal and the larger the value of F, the higher the acceleration of the virtual hand. However, the speed is limited by the combined effect of damping (c) and mass (m). The dynamic range of the virtual hand mass m is set to 200-1000g to simulate the feeling of grasping objects of different weights. The damping coefficient c is set to 0.8Ns / m to control the resistance of the virtual hand movement and avoid shaking or instability during high-speed movement.

[0239] Example 17:

[0240] The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback is described in any one of Examples 11 to 16. Furthermore, the visual feedback includes real-time motion feedback of the user's virtual hand and real-time electromyography intensity radar comparison feedback between the user and a healthy user.

[0241] The visual feedback provides two windows: the first window displays the real-time effect of the virtual hand's fine movements, providing patients with an intuitive dynamic control effect and improving immersion and control precision; the second window displays real-time electromyography (EMG) intensity radar maps of the affected and healthy sides, facilitating guidance for users to strive towards the corresponding EMG level of the healthy side.

[0242] Example 18:

[0243] The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback is described in any one of Examples 11 to 17. Furthermore, the vibration feedback includes feedback on the completion of daily tasks and games, and feedback on the grip strength level.

[0244] The vibration tactile feedback provides two modes of tactile feedback: The first mode is used to provide feedback on task completion, which is conveyed through different vibration durations. Short vibrations are used for low-intensity operation confirmation or failure reminders, medium vibrations enhance the perception of task progress, and long vibrations simulate real touch with high-intensity continuous vibrations to enhance immersion and sense of accomplishment, and increase user sensitivity and engagement. The second mode is used to provide feedback on grip strength, based on the average intensity grading of electromyographic signals of the forearm extensor and flexor muscles. Force information is transmitted through different vibration channels and frequencies. As the intensity of the electromyographic signal increases, the vibration channels are activated sequentially, and the vibration frequency increases, enabling precise perception and adjustment of grip strength.

[0245] The completion status of daily tasks and games is fed back through different vibration durations.

[0246] The grip strength level is fed back through different vibration channels and vibration frequencies.

[0247] The grip strength level is characterized using a ratio-weighted method, as shown below:

[0248]

[0249] In the formula, D represents the grip strength level. MAV 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar flexor carpi radialis muscles in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given, and MVC represents the maximum voluntary contraction of the muscle.

[0250] Example 19:

[0251] The rehabilitation training system for forearm amputees based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 11 to 18, and further, the rehabilitation training data includes electromyographic indicators and rehabilitation training indicators.

[0252] The electromyographic parameters include muscle activity intensity and antagonistic coefficient.

[0253] The rehabilitation training indicators include the game score rate of the digital mesotherapy game and the task success rate of life-like scenarios and tasks.

[0254] The game score rates are shown below:

[0255]

[0256] In the formula, λ represents the game score rate. The average score in the game. This represents the average game time.

[0257] The success rates of the tasks are shown below:

[0258]

[0259] In the formula, γ represents the task success rate. q1 represents the number of times the gesture was successfully completed. q0 represents the total number of gesture attempts.

[0260] Example 20:

[0261] The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback, the main technical contents of which are described in any one of Examples 11 to 19, further wherein the application layer module for planning personalized rehabilitation path includes a muscle strength level adaptation module and an antagonistic muscle synergistic adaptation module, to set up digital large-scale synthetic electromyography games and life-like scenarios and tasks.

[0262] The muscle strength level adaptation module classifies users' muscle activity intensity and, with reference to the strength of different muscles in healthy individuals, into corresponding muscle strength levels. The user's muscle strength level is divided into four levels from weak to strong, and is adapted to the difficulty levels of the digital synthetic electromyography game.

[0263] The users' muscle strength levels are divided into four levels, from weak to strong.

[0264] The digital synthetic electromyography game is divided into four levels from easy to difficult.

[0265] When the root mean square value of the electromyography signal of some muscles is detected to be lower than that of other muscles, the muscle strength level adaptation module performs rehabilitation training on the detected muscles.

[0266] The antagonistic muscle synergistic adaptation module evaluates the coordination of wrist flexion and extension movements by calculating the antagonistic synergistic coefficients of extensor and flexor muscles, and sets training movement types for everyday tasks in life scenarios based on the calculated antagonistic synergistic coefficients.

[0267] The antagonistic synergistic coefficients of the extensor and flexor muscles are shown below:

[0268]

[0269] In the formula, α is the wrist flexion antagonism coefficient, and β is the wrist extension antagonism coefficient. RMS 屈肌 RMS is the sum of the root mean square (RMS) values ​​of the electromyographic signals of the radial and ulnar wrist flexors in patients with forearm amputations. 伸肌 It is the sum of the root mean square values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations.

[0270] Example 21:

[0271] See Figures 1 to 3 The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback includes: hardware layer, data processing layer, application layer and user interface layer.

[0272] The hardware layer is used to collect and transmit electromyographic signals of the forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback.

[0273] The data processing layer is used to preprocess, extract features, and recognize gestures from the collected electromyographic signals of the residual limb in order to generate usable control commands.

[0274] The application layer is used to dynamically map electromyographic signals and simulate them with a physics engine, integrating visual-tactile bimodal feedback and personalized training planning to achieve real-time response of virtual interaction and adaptive optimization of rehabilitation training, thereby improving training efficiency, immersion and patient compliance.

[0275] The user interface layer provides an intuitive interface for electromyography classification and rehabilitation training, facilitating operation and monitoring by patients with forearm amputations and rehabilitation physicians.

[0276] The hardware layer specifically includes:

[0277] The wearable electromyography (EMG) acquisition device collects EMG signals from eight channels of the forearm stump muscles in real time and transmits the data to a computer via Bluetooth. The sites for collecting EMG signals from the stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus.

[0278] A 9-axis inertial measurement unit (IMU) is integrated into the wearable data acquisition device. It includes three sets of sensors—accelerometer, gyroscope, and magnetometer—that work together to capture three-dimensional spatial acceleration, angular velocity, and geomagnetic vector data, respectively. This enables multi-dimensional motion state fusion calculation to monitor the user's upper limb posture and motion state, such as wrist rotation, arm raising, and hand angle adjustment.

[0279] The vibration meter is integrated into the wearable data acquisition device to provide vibration feedback, helping patients to have a more realistic experience in a virtual reality environment.

[0280] The data processing layer specifically includes:

[0281] The electromyography (EMG) signal preprocessing module optimizes EMG signal quality through three levels of real-time processing: First, an adaptive notch filter with a center frequency of 50Hz and a bandwidth of ±2Hz is used to eliminate power frequency interference. Then, a fourth-order Butterworth bandpass filter (20-450Hz) is used to filter out baseline drift and high-frequency noise. Subsequently, the signal amplitude is automatically adjusted based on a dynamic gain control algorithm (gain range 1-100 times). For the 8-channel data (quantization range -128 to 128), the module performs invalid data removal (5 truncations at the beginning) and windowing processing (50 sets / frame). Finally, the preprocessed data is saved as a structured text file according to action category, providing standardized input for subsequent analysis.

[0282] The feature extraction module employs a time-frequency domain fusion feature extraction method, segmenting the output signal from the preprocessing module using a sliding window mechanism (250ms window length, 75% overlap). In the time domain analysis, the mean absolute value (MAV), root mean square (RMS), and zero-crossing rate (ZC) of the electromyographic signal within the window are calculated to characterize muscle activation intensity, signal fluctuation characteristics, and movement periodicity, respectively. In the frequency domain analysis, the energy ratio (ER) of the effective frequency band from 20-450Hz is extracted using Fast Fourier Transform (FFT), and then fused with multi-band energy distribution characteristics for analysis.

[0283] The gesture recognition and classification module is based on a lightweight convolutional neural network (CNN) for gesture classification, employing a combination of offline training and online deployment: an 8-channel electromyography (EMG) signal classification model is trained using PyTorch, converted to ONNX format, and integrated into Unity3D, achieving real-time inference in less than 50ms using the Barracuda engine. A dynamic calibration mechanism establishes a user-personalized feature model through 30 seconds of baseline motion capture, and dynamically optimizes the classification threshold using a sliding window method to ensure stable cross-user recognition accuracy. The classified gestures include, but are not limited to, clenched fist, extended palm, wrist extension, wrist flexion, and relaxed hand.

[0284] The application layer specifically includes:

[0285] The virtual interactive environment is constructed through digitally synthesized electromyography games and lifelike scenarios and tasks. The virtual hand uses collision dynamic adjustment technology and a mass-damping physics engine to accurately simulate grasping mechanical feedback, enhancing the realism of operation. It also integrates a dual-modal feedback system, including visual and vibratory-tactile feedback, to guide patients in real-time movement calibration. Based on the patient's rehabilitation progress, the system can achieve personalized rehabilitation path planning.

[0286] The digital electromyography (EMG) game is based on the classic 2048 rule. Wearable EMG acquisition devices recognize EMG gestures such as fist clenching, palm extension, and wrist extension / flexion, mapping them to virtual block movement and merging operations (up, down, left, right). The game uses a 4×4 grid to dynamically manage block states, generating new blocks through a two-dimensional array and a random algorithm, and implementing movement and merging logic based on row / column detection (e.g., doubling adjacent identical numbers). Unity coroutines control animation delays and smooth transitions, combined with real-time score updates and victory / defeat status feedback, enhancing the patient's hand coordination, EMG signal control accuracy, and reaction speed.

[0287] The simulated everyday scenarios and tasks are achieved by constructing a virtual kitchen scene to simulate routine tasks (opening doors, turning lights on and off, and grasping objects). This is combined with wearable electromyography (EMG) acquisition devices to enable refined interactions: wrist movement controls the camera's view, and dead-zone thresholds prevent accidental operations. Wrist extension / flexion gestures trigger door opening and closing animations, fist clenching drives SphereCast to grasp objects, and Rigidbody components manage object dynamics, while Animators control the interactive animations. Based on the synergy between the highly realistic scenarios and the physics engine, patients can enhance their fine motor skills while completing these everyday tasks.

[0288] Calculation formula:

[0289] R = 8 + 2(RMS / 0.5) gives the radius of the collision body, where RMS is the root mean square value of the real-time electromyography (EMG) signal. When the EMG signal is strong, the collision radius is increased to assist grasping. When the signal is weak, the radius needs to be reduced precisely to improve the patient's EMG sensitivity. By dynamically optimizing the collision range through EMG signal intensity, the influence of real hand muscle exertion on grasping stability is simulated, strengthening the patient's awareness of controlling the EMG signal.

[0290] Calculation formula:

[0291] The force on the virtual hand is calculated as F = ma + cv, where F is the driving force mapped from the electromyographic signal, m is the mass of the virtual hand, c is the damping coefficient of the virtual hand, a is the acceleration of the virtual hand, and v is the velocity of the virtual hand. The stronger the electromyographic signal, the larger F, and the higher the acceleration of the virtual hand, but the velocity will be limited by the combined effect of damping (c) and mass (m). The dynamic range of the virtual hand mass m is set to 200-1000g to simulate the feeling of grasping objects of different weights. The damping coefficient c is set to 0.8 Ns / m to control the resistance of the virtual hand's movement and avoid shaking or instability during high-speed movement.

[0292] The visual feedback provides two windows: the first window displays the real-time effect of the virtual hand's fine motor skills, providing patients with intuitive dynamic control and enhancing immersion and control precision. The second window displays real-time electromyography (EMG) intensity radar maps of the affected and healthy sides, guiding the user to strive towards the corresponding EMG level on the healthy side.

[0293] The vibration-haptic feedback provides two modes of haptic feedback: The first mode is used to provide feedback on task completion, conveyed through different vibration durations. Short vibrations are used for low-intensity operation confirmation or failure reminders, medium vibrations enhance the perception of task progress, and long vibrations simulate real touch with high-intensity continuous vibration to improve immersion and sense of accomplishment, and enhance user's motion sensitivity and engagement. The second mode is used to provide feedback on grip strength. Based on the average intensity grading of electromyographic signals from the forearm extensor and flexor muscles, force information is transmitted through different vibration channels and frequencies. As the intensity of the electromyographic signal increases, the vibration channels are activated sequentially, and the vibration frequency increases, enabling precise perception and adjustment of grip strength.

[0294] The grip strength level is characterized by a ratio weighting method to better reflect the synergistic effect of extensor and flexor muscles.

[0295] Calculation formula:

[0296] The grip strength level, MAV, is obtained. 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar wrist flexors in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given. The maximum voluntary contraction of the muscle is represented by the MAV. The electromyographic signals are standardized to 60% of the maximum voluntary contraction to avoid muscle fatigue.

[0297] The rehabilitation effect was quantitatively analyzed using an "assessment-training-reassessment" method. Specific indicators included electromyographic parameters (root mean square value and antagonism coefficient of electromyography signals) and training performance (game score rate in digital synthetic electromyography games and success rate in performing tasks in everyday scenarios). The root mean square value (RMS) reflects the amplitude change of surface electromyographic signals; when muscles contract under different loads, the RMS value is directly proportional to muscle strength. The antagonism coefficient (AC) reflects the antagonistic relationship between extensor and flexor muscles; this antagonism reflects the patient's ability to isolate and control muscle contraction.

[0298] Calculation formula:

[0299] The game score rate of the digital large-scale synthetic electromyography game was obtained.

[0300] Calculation formula:

[0301] The success rate of operations in the aforementioned everyday scenarios and tasks is obtained.

[0302] The personalized rehabilitation pathway planning consists of two modules: muscle strength level adaptation and antagonistic muscle synergy adaptation. Based on real-time monitoring of task completion data and combined with the patient's rehabilitation progress, the training content and intensity are dynamically adjusted to achieve continuous optimization of the patient's rehabilitation plan.

[0303] The muscle strength level adaptation module classifies the amputee's muscle strength level into four levels: weak, relatively weak, relatively strong, and strong, corresponding to the four difficulty levels of the digital synthetic electromyography game: "one," "two," "three," and "four," respectively. This allows for real-time matching of training intensity with muscle strength level. If the RMS value of some muscles is detected to be significantly higher than that of other muscles, it indicates an imbalance in muscle function recovery. The system will then provide targeted rehabilitation training for the weaker muscles to promote balanced muscle strength development.

[0304] The antagonistic muscle synergy adaptation module assesses the coordination of wrist flexion and extension movements by calculating the antagonistic synergy coefficient (AC value) between extensor and flexor muscles. When the AC value is close to 0, it indicates that the extensor and flexor muscles are relatively balanced in strength, with weak antagonism, making prosthetic control more difficult. When the AC value is close to 1, it indicates a significant difference in strength between the two muscles, with strong antagonism, which is beneficial for wrist movement recognition and prosthetic control. During training, if the AC value is detected to be below 0.5, the system will automatically increase the training ratio of everyday tasks (such as grasping and rotation) to specifically strengthen gesture control ability.

[0305] The force field adaptive adjustment module automatically adjusts the intensity of the virtual force field based on electromyographic signal feedback. When the system detects muscle fatigue or unstable muscle control, it automatically reduces the intensity of the grasping motion to ensure training safety while enhancing the immersion and comfort of the operation.

[0306] The user interface layer specifically includes:

[0307] The interactive interface is developed based on Unity and integrates personalized parameter settings for electromyography (EMG) classification and rehabilitation training. Through real-time EMG signal analysis and VR somatosensory interaction technology, it achieves one-stop management from data collection to personalized rehabilitation training.

[0308] Example 22:

[0309] See Figures 1 to 3 The rehabilitation training system for motor function of patients with forearm limb disabilities based on visual and vibration-tactile feedback includes: hardware layer, data processing layer, application layer and user interface layer.

[0310] The hardware layer is used to collect and transmit electromyographic signals of the forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback.

[0311] The data processing layer is used to preprocess, extract features, and recognize gestures from the collected electromyographic signals of the residual limb in order to generate usable control commands.

[0312] The application layer is used to dynamically map electromyographic signals and simulate them with a physics engine, integrating visual-tactile bimodal feedback and personalized training planning to achieve real-time response of virtual interaction and adaptive optimization of rehabilitation training, thereby improving training efficiency, immersion and patient compliance.

[0313] The user interface layer provides an intuitive interface for electromyography classification and rehabilitation training, facilitating operation and monitoring by patients with forearm amputations and rehabilitation physicians.

[0314] The hardware layer specifically includes:

[0315] The wearable electromyography (EMG) acquisition device collects EMG signals from eight channels of the forearm stump muscles in real time and transmits the data to a computer via Bluetooth. The sites for collecting EMG signals from the stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus.

[0316] A 9-axis inertial measurement unit (IMU) is integrated into the wearable data acquisition device. It includes three sets of sensors—accelerometer, gyroscope, and magnetometer—that work together to capture three-dimensional spatial acceleration, angular velocity, and geomagnetic vector data, respectively. This enables multi-dimensional motion state fusion calculation to monitor the user's upper limb posture and motion state, such as wrist rotation, arm raising, and hand angle adjustment.

[0317] The vibration meter is integrated into the wearable data acquisition device to provide vibration feedback, helping patients to have a more realistic experience in a virtual reality environment.

[0318] The data processing layer specifically includes:

[0319] The electromyography (EMG) signal preprocessing module optimizes EMG signal quality through three levels of real-time processing: First, an adaptive notch filter with a center frequency of 50Hz and a bandwidth of ±2Hz is used to eliminate power frequency interference. Then, a fourth-order Butterworth bandpass filter (20-450Hz) is used to filter out baseline drift and high-frequency noise. Subsequently, the signal amplitude is automatically adjusted based on a dynamic gain control algorithm (gain range 1-100 times). For the 8-channel data (quantization range -128 to 128), the module performs invalid data removal (5 truncations at the beginning) and windowing processing (50 sets / frame). Finally, the preprocessed data is saved as a structured text file according to action category, providing standardized input for subsequent analysis.

[0320] The feature extraction module employs a time-frequency domain fusion feature extraction method, segmenting the output signal from the preprocessing module using a sliding window mechanism (250ms window length, 75% overlap). In the time domain analysis, the mean absolute value (MAV), root mean square value (RMS), and zero-crossing rate (ZC) of the electromyographic signal within the window are calculated to characterize muscle activation intensity, signal fluctuation characteristics, and movement periodicity, respectively. In the frequency domain analysis, the intermediate frequency energy ratio (ER) of the effective frequency band from 20-450Hz is extracted using Fast Fourier Transform (FFT), and fusion analysis is performed combining multi-band energy distribution characteristics.

[0321] The gesture recognition and classification module is based on a lightweight convolutional neural network (CNN) for gesture classification, employing a combination of offline training and online deployment: an 8-channel electromyography (EMG) signal classification model is trained using PyTorch, converted to ONNX format, and integrated into Unity3D, achieving real-time inference in less than 50ms using the Barracuda engine. A dynamic calibration mechanism establishes a user-personalized feature model through 30 seconds of baseline motion capture, and dynamically optimizes the classification threshold using a sliding window method to ensure stable cross-user recognition accuracy. The classified gestures include, but are not limited to, clenched fist, extended palm, wrist extension, wrist flexion, and relaxed hand.

[0322] The application layer specifically includes:

[0323] The virtual interactive environment is constructed through digitally synthesized electromyography games and lifelike scenarios and tasks. The virtual hand uses collision dynamic adjustment technology and a mass-damping physics engine to accurately simulate grasping mechanical feedback, enhancing the realism of operation. It also integrates a dual-modal feedback system, including visual and vibratory-tactile feedback, to guide patients in real-time movement calibration. Based on the patient's rehabilitation progress, the system can achieve personalized rehabilitation path planning.

[0324] The digital electromyography (EMG) game is based on the classic 2048 rule. Wearable EMG acquisition devices recognize EMG gestures such as fist clenching, palm extension, and wrist extension / flexion, mapping them to virtual block movement and merging operations (up, down, left, right). The game uses a 4×4 grid to dynamically manage block states, generating new blocks through a two-dimensional array and a random algorithm, and implementing movement and merging logic based on row / column detection (e.g., doubling adjacent identical numbers). Unity coroutines control animation delays and smooth transitions, combined with real-time score updates and victory / defeat status feedback, enhancing the patient's hand coordination, EMG signal control accuracy, and reaction speed.

[0325] The simulated everyday scenarios and tasks are achieved by constructing a virtual kitchen scene to simulate routine tasks (opening doors, turning lights on and off, and grasping objects). This is combined with wearable electromyography (EMG) acquisition devices to enable refined interactions: wrist movement controls the camera's view, and dead-zone thresholds prevent accidental operations. Wrist extension / flexion gestures trigger door opening and closing animations, fist clenching drives SphereCast to grasp objects, and Rigidbody components manage object dynamics, while Animators control the interactive animations. Based on the synergy between the highly realistic scenarios and the physics engine, patients can enhance their fine motor skills while completing these everyday tasks.

[0326] Calculation formula:

[0327] R = 8 + 2(RMS / 0.5) gives the radius of the collision body, where RMS is the root mean square value of the real-time electromyography (EMG) signal. When the EMG signal is strong, the collision radius is increased to assist grasping. When the signal is weak, the radius needs to be reduced precisely to improve the patient's EMG sensitivity. By dynamically optimizing the collision range through EMG signal intensity, the influence of real hand muscle exertion on grasping stability is simulated, strengthening the patient's awareness of controlling the EMG signal.

[0328] Calculation formula:

[0329] The force on the virtual hand is calculated as F = ma + cv, where F is the driving force mapped from the electromyographic signal, m is the mass of the virtual hand, c is the damping coefficient of the virtual hand, a is the acceleration of the virtual hand, and v is the velocity of the virtual hand. The stronger the electromyographic signal, the larger F, and the higher the acceleration of the virtual hand, but the velocity will be limited by the combined effect of damping (c) and mass (m). The dynamic range of the virtual hand mass m is set to 200-1000g to simulate the feeling of grasping objects of different weights. The damping coefficient c is set to 0.8 Ns / m to control the resistance of the virtual hand's movement and avoid shaking or instability during high-speed movement.

[0330] The visual feedback provides two windows: the first window displays the real-time effect of the virtual hand's fine motor skills, providing patients with intuitive dynamic control and enhancing immersion and control precision. The second window displays real-time electromyography (EMG) intensity radar maps of the affected and healthy sides, guiding the user to strive towards the corresponding EMG level on the healthy side.

[0331] The vibration-haptic feedback provides two modes of haptic feedback: The first mode is used to provide feedback on task completion, conveyed through different vibration durations. Short vibrations are used for low-intensity operation confirmation or failure reminders, medium vibrations enhance the perception of task progress, and long vibrations simulate real touch with high-intensity continuous vibration to improve immersion and sense of accomplishment, and enhance user's motion sensitivity and engagement. The second mode is used to provide feedback on grip strength. Based on the average intensity grading of electromyographic signals from the forearm extensor and flexor muscles, force information is transmitted through different vibration channels and frequencies. As the intensity of the electromyographic signal increases, the vibration channels are activated sequentially, and the vibration frequency increases, enabling precise perception and adjustment of grip strength.

[0332] The grip strength level is characterized by a ratio weighting method to better reflect the synergistic effect of extensor and flexor muscles.

[0333] Calculation formula:

[0334] The grip strength level, MAV, is obtained. 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar flexor carpi radialis muscles in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given. The maximum voluntary contraction of the muscle is represented by the MAV. The electromyographic signals are standardized to 60% of the maximum voluntary contraction to avoid muscle fatigue.

[0335] The rehabilitation effect was quantitatively analyzed using an "assessment-training-reassessment" method. Specific indicators included electromyographic parameters (muscle activity intensity and antagonism coefficient) and training performance (game score rate of digital synthetic electromyography game and success rate of operation in life scenarios and tasks).

[0336] Calculation formula:

[0337] The game score rate of the digital large-scale synthetic electromyography game was obtained.

[0338] Calculation formula:

[0339] The success rate of operations in the aforementioned everyday scenarios and tasks is obtained.

[0340] The personalized rehabilitation pathway planning consists of two modules: muscle strength level adaptation and antagonistic muscle synergy adaptation. Based on real-time monitoring of task completion data and combined with the patient's rehabilitation progress, the training content and intensity are dynamically adjusted to achieve continuous optimization of the patient's rehabilitation plan.

[0341] The muscle strength level adaptation module classifies the amputee's muscle strength level into four levels: weak, relatively weak, relatively strong, and strong, corresponding to the four difficulty levels of the digital synthetic electromyography game: "one," "two," "three," and "four," respectively. This allows for real-time matching of training intensity with muscle strength level. If the RMS value of some muscles is detected to be significantly higher than that of other muscles, it indicates an imbalance in muscle function recovery. The system will then provide targeted rehabilitation training for the weaker muscles to promote balanced muscle strength development.

[0342] The antagonistic muscle synergy adaptation module assesses the coordination of wrist flexion and extension movements by calculating the antagonistic synergy coefficient (AC value) between extensor and flexor muscles. When the AC value is close to 0, it indicates that the extensor and flexor muscles are relatively balanced in strength, with weak antagonism, making prosthetic control more difficult. When the AC value is close to 1, it indicates a significant difference in strength between the two muscles, with strong antagonism, which is beneficial for wrist movement recognition and prosthetic control. During training, if the AC value is detected to be below 0.5, the system will automatically increase the training ratio of everyday tasks (such as grasping and rotation) to specifically strengthen gesture control ability.

[0343] The force field adaptive adjustment module automatically adjusts the intensity of the virtual force field based on electromyographic signal feedback. When the system detects muscle fatigue or unstable muscle control, it automatically reduces the intensity of the grasping motion to ensure training safety while enhancing the immersion and comfort of the operation.

[0344] The user interface layer specifically includes:

[0345] The interactive interface is developed based on Unity and integrates personalized parameter settings for electromyography (EMG) classification and rehabilitation training. Through real-time EMG signal analysis and VR somatosensory interaction technology, it achieves one-stop management from data collection to personalized rehabilitation training.

[0346] The virtual hand model construction scheme in the virtual interactive environment is based on the Blender platform. It uses anatomical structure as its core and achieves precise control of multiple degrees of freedom through three stages: 3D geometric modeling, skeletal system design, and motion rigging. The specific construction process includes:

[0347] (1) Based on the real anatomical features of the hand, a geometric model of the palm, fingers and joints is created to accurately reproduce the three-level joint structure of the finger root, middle and tip.

[0348] (2) Construct a hierarchical skeletal system with 17 skeletal nodes (covering the palm and five finger joints), and use Blender's skeletal animation tools to achieve independent control of each joint, supporting biomimetic simulation of complex movements such as bending and stretching.

[0349] (3) Drive virtual hand movement through skeleton-model binding technology, and combine Unity3D animation controller with real-time mapping of electromyographic signals to generate a library of basic movements such as clenching fist and extending palm, so that patients can complete high-precision rehabilitation training in virtual scene through electromyographic control.

[0350] The control scheme for the virtual hand and virtual camera in the virtual interactive environment is implemented through the IMU (accelerometer, gyroscope, magnetometer) of a wearable data acquisition device, specifically including:

[0351] Electromyographic signals drive the multi-degree-of-freedom movements of the virtual hand. IMU data (Pitch, Yaw, and Roll axes) are synchronously mapped to the virtual hand movements and the camera's viewpoint—the Pitch axis controls hand pitch / vertical movement of the viewpoint, the Yaw axis controls horizontal yaw / horizontal rotation of the viewpoint, and the Roll axis controls hand rotation (camera does not respond). The system parses the data using ArmRotation and CameraLook scripts respectively, sets dead zone thresholds to eliminate minor jitter, and limits the angle range to prevent non-physiological movements (virtual hand Roll / Pitch / Yaw angle constraints, camera X-axis viewpoint limiting). It also supports sensitivity adjustment to balance control precision and response speed, enabling users to synchronously control the virtual hand's grasp and viewpoint adjustment through natural wrist movements, achieving an immersive rehabilitation training experience.

[0352] See Figure 2 As shown, the implementation of the digital synthesis electromyography game is based on GameManager as its core architecture. It dynamically manages the state of grid blocks through a 4×4 two-dimensional array AllTiles, and precisely controls the game flow using GameState enumeration, forming a logical closed loop of "movement direction - node detection - value merging - state backtracking." The specific game flow includes:

[0353] (1) After the game starts, the scene is initialized and two initial number blocks are randomly generated. Then the core loop is entered, and the player's input is continuously listened to to determine the direction of movement.

[0354] (2) If a valid move command is detected, the system will traverse the grid node by node. If the node numbers are equal, the merge logic will be triggered (numerical superposition, clearing the original node and updating the score). Otherwise, the current node will be moved to the edge of the interface and the original position will be cleared.

[0355] (3) After completing the move / merge operation, check whether the remaining nodes in the grid can still be moved (adjacent nodes have the same number or empty space). If they cannot be moved, the game ends and the player is prompted to choose to start over or quit.

[0356] (4) If the player chooses not to restart, the process ends in the "game over" state; otherwise, the grid is reset and the initial blocks are regenerated.

[0357] The interaction between the everyday scenarios and tasks is achieved through the integration of gesture recognition and Unity engine components, specifically including:

[0358] Wearable electromyography (EMG) devices capture specific hand gestures such as wrist extension, wrist flexion, and fist clenching, triggering interaction logic through an event subscription mechanism. State-based tasks such as opening / closing doors and turning lights on / off rely on an Animator state machine, using spatial detection methods to determine interaction conditions and driving animation through Entry / Exit state transitions. Object grasping employs 3D ray detection to precisely locate target objects, applying pulsed force to achieve physical grasping, and simulating muscle compensation effects through a dynamic force field attenuation model (force drops to 0.5f when the distance is <0.2 units). Hierarchical filtering eliminates background interference, and a `heldObjects` list manages multiple object grasping sequences to ensure interaction stability.

[0359] The vibration-haptic feedback employs a hierarchical haptic feedback strategy, combining vibration duration, vibration channel, and frequency to provide precise feedback on task completion effectiveness and grip strength, thereby enhancing the user's operating experience and training results. Specific details include:

[0360] To provide feedback on task completion, the system utilizes the linear resonant actuator (LRA) built into the wearable electromyography (EMG) device, categorizing vibrations into three modes: short vibration (100-500ms), medium vibration (500ms-1s), and long vibration (over 1 second). Short vibrations provide low-intensity, instantaneous cues for confirmation or failure alerts, helping users quickly correct their movements. Medium vibrations reinforce progress at moderate intensity, enhancing the user's perception of training results. Long vibrations employ high-intensity, continuous vibrations to simulate realistic tactile sensations, increasing immersion and a sense of accomplishment during training. This feedback strategy flexibly matches vibration modes to the type of training task and the results, employing a two-way mechanism combining positive rewards and error alerts to effectively promote user accuracy and engagement.

[0361] Regarding grip strength feedback, the system linearly maps muscle contraction (0%-100% maximum voluntary contraction force, MVC) to grip strength based on the electromyographic signal intensity of the forearm extensor and flexor muscles. Considering specific life scenarios, grip strength is divided into three levels: low, medium, and high, corresponding to approximately 30%, 50%, and 70% of MVC, respectively, to meet the needs of grasping different objects. After the grasping action, the system collects 0.5 seconds of electromyographic signals and calculates the average value to assess the actual grip strength. Subsequently, multi-channel vibration and incremental frequency feedback provide information on differences, helping users to accurately perceive and adjust their grip strength.

[0362] See Figure 3 As shown, the user interface adopts a modular design with upper and lower partitions, and its specific contents include:

[0363] The upper part is the electromyography (EMG) acquisition and classification platform, which includes gesture selection (such as clenching a fist, extending a palm, extending the wrist, flexing the wrist, and relaxing), signal acquisition control ("Start Training" and "Start Recognition" buttons), and a real-time recognition result feedback box, forming a complete closed-loop process from action selection, data acquisition (saving as a TXT file), model training (generating an XML classification model) to real-time recognition. The lower part is the "training platform," which is divided into two main modules: a 2048 digital synthesis EMG game and ADL (Activities of Daily Living) scenarios and tasks. The former adapts to the difficulty of the digital synthesis game through a "difficulty selection" menu (such as easy, medium, and difficult), while the latter customizes the complexity of the life scenarios through a "task selection" menu (such as opening a door and grabbing an object). The two modules run independently but share unified EMG assessment data, ensuring clear functional divisions and non-interference in operation, balancing the standardization of scientific research data with the flexibility of training tasks.

[0364] The interface deeply integrates rehabilitation scenarios and individual needs through task types and difficulty levels. For example, in the digital electromyography (EMG) game, the "easy" mode generates low-value blocks at a slow speed, helping patients adapt to basic operations and strengthening hand coordination. In daily life scenarios and task modules, virtual training is mapped to everyday skills (such as grasping and pushing / pulling) through simulations of specific daily actions like opening doors, improving motor transfer. The system dynamically recommends task types based on EMG assessment results (such as antagonism coefficients), while allowing therapists to manually adjust difficulty parameters (such as increasing block speed or improving target precision). This achieves a progressive adaptation from low-level coordination training to high-level daily living skills enhancement, satisfying scenario-based functional goals (such as improving reaction speed in digital games and enhancing practical skills in daily living tasks) while ensuring precise matching between the program and the patient's rehabilitation stage through a dynamic difficulty mechanism, balancing scientific rigor with a personalized experience.

Claims

1. A rehabilitation training system for motor function in patients with forearm amputations based on visual and vibration-tactile feedback, characterized in that, include: Hardware layer, data processing layer, application layer, and user interface layer; The hardware layer is used to collect electromyographic signals of the user's forearm stump in real time, monitor changes in the user's upper limb posture, and provide vibration feedback to the user according to the control instructions of the data processing layer. The data processing layer is used to process the acquired electromyographic signals and recognize gestures, and to generate control commands; The application layer is used to dynamically map and simulate the electromyographic signals and recognized gestures processed by the data processing layer, and to build a rehabilitation training interface. The user interface layer is used to display the gestures and rehabilitation training interface recognized by the data processing layer, providing visual feedback to the user; Users conduct rehabilitation training based on vibration feedback provided by the hardware layer and visual feedback provided by the user interface layer; The application layer plans a personalized rehabilitation path for the user based on the user's rehabilitation training data and the electromyographic signals processed by the data processing layer.

2. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 1, characterized in that, The hardware layer includes a wearable electromyography (EMG) acquisition device, a 9-axis inertial measurement unit, and a vibration meter; The wearable electromyography (EMG) acquisition device is used to acquire the EMG signals of the user's forearm stump in real time. The sites for collecting electromyographic signals of the forearm stump include, but are not limited to, the flexor carpi ulnaris, flexor carpi radialis, extensor carpi ulnaris, and extensor carpi radialis longus. The 9-axis inertial measurement unit and vibration meter are integrated into the wearable electromyography (EMG) acquisition device; The 9-axis inertial measurement unit is used to monitor changes in the user's upper limb posture; The user's upper limb posture changes include wrist rotation, arm raising, and hand angle adjustment; The 9-axis inertial measurement unit includes an accelerometer, a gyroscope, and a magnetometer; The accelerometer is used to capture the acceleration of the user's forearm in three-dimensional space; The gyroscope is used to capture the angular velocity of the user's forearm in three-dimensional space; The magnetometer is used to capture the geomagnetic vector data of the user's forearm in three-dimensional space; The vibration meter provides vibration feedback to the user.

3. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 1, characterized in that, The data processing layer includes an electromyography signal preprocessing module, a feature extraction module, and a gesture recognition and classification module. The electromyography signal preprocessing module includes an adaptive notch filter, a bandpass filter, a dynamic gain control module, an invalid rejection module, and a window processing module; The adaptive notch filter is used to filter the acquired electromyographic signals to obtain electromyographic signals that have eliminated power frequency interference. The bandpass filter is used to filter the electromyographic signal to eliminate power frequency interference, so as to obtain an electromyographic signal with baseline drift and high-frequency noise removed. The dynamic gain control module is used to adjust the amplitude of the electromyographic signal after filtering out baseline drift and high-frequency noise to obtain an amplitude-adjusted electromyographic signal. The invalid data removal module is used to remove invalid data from the amplitude-adjusted electromyographic signals. The window processing module is used to perform windowing processing on the electromyographic signals after removing invalid data to obtain preprocessed data. The feature extraction module is used to extract segmented features from the preprocessed data; The features include time-domain features and frequency-domain features; The time-domain features include the mean absolute value, root mean square value, and zero-crossing rate of the electromyographic signal; The frequency domain characteristics include the proportion of mid-frequency energy; The gesture recognition and classification module performs gesture recognition on the extracted features based on a lightweight convolutional neural network to obtain gesture recognition results. The gesture recognition results include, but are not limited to, clenched fist, extended palm, wrist extension, wrist flexion, and relaxed. The gesture recognition and classification module generates control commands based on the gesture recognition results.

4. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 1, characterized in that, The rehabilitation training interface includes a digital synthetic electromyography game, life-like scenarios, and tasks. The digital large-scale synthetic electromyography game is based on the classic 2048 rule, which maps the recognized gestures to the corresponding game operations, moves and merges virtual blocks, and displays the game score and win / loss status in real time. The classic 2048 rules use a 4×4 grid to dynamically manage the state of the blocks and generate new blocks through a two-dimensional array and a random algorithm; users merge blocks by moving adjacent blocks with the same number, thereby obtaining game scores; The game controls include moving up, moving down, moving left, and moving right; The aforementioned everyday scenarios and tasks are constructed by creating virtual scenes, simulating the processed electromyographic signals and recognized gestures into corresponding game actions, and completing daily tasks in the virtual scenes; The game actions include camera movement and virtual hand movements; The virtual hand gestures include opening a door, closing a door, grabbing, and putting down; The daily tasks include opening and closing doors, turning lights on and off, and grabbing items.

5. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 4, characterized in that, The steps for constructing the virtual hand in the described everyday scenarios and tasks are as follows: S1 creates geometric models of the palm, fingers, and joints based on real hand anatomy. s2 utilizes Blender's skeletal animation tools to achieve independent control of each joint, constructing a hierarchical skeletal system that covers the skeletal nodes of the palm and the five finger joints; S3 uses binding technology to bind the geometric model of the palm, fingers, and joints, as well as a hierarchical skeletal system, to drive the virtual hand to move.

6. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 4, characterized in that, In everyday scenarios and tasks, the mechanical feedback of virtual hand grasping actions is simulated based on the collision body dynamic adjustment method and the mass-damping physics engine. The radius R of the collider is shown below: R = 8 + 2(RMS / 0.5) (1) In the formula, RMS is the root mean square value of the electromyography signal; when the electromyography signal is greater than the preset threshold, the radius of the collider is increased to assist in grasping; when the electromyography signal is less than the preset threshold, the radius of the collider is reduced to perform grasping. The forces acting on the virtual hand are shown below: F = ma + cv (2) In the formula, F is the force on the virtual hand mapped by the electromyographic signal; m is the mass of the virtual hand; c is the damping coefficient of the virtual hand; a is the acceleration of the virtual hand; and v is the velocity of the virtual hand.

7. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 4, characterized in that, The visual feedback includes real-time motion feedback of the user's virtual hand and a real-time electromyography (EMG) intensity radar comparison between the user and a healthy user.

8. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 4, characterized in that, The vibration feedback includes feedback on daily task and game completion status, and grip strength level feedback; The completion status of daily tasks and games is fed back through different vibration durations; The grip strength level is fed back through different vibration channels and vibration frequencies; The grip strength level is characterized using a ratio-weighted method, as shown below: In the formula, D represents the grip strength level; MAV 屈肌 MAV is the sum of the mean absolute values ​​of the electromyographic signals of the radial and ulnar flexor carpi radialis muscles in patients with forearm amputations. 伸肌 The sum of the average absolute values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations is given, and MVC represents the maximum voluntary contraction of the muscle.

9. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 4, characterized in that, The rehabilitation training data includes electromyography indicators and rehabilitation training indicators. The electromyographic parameters include muscle activity intensity and antagonism coefficient; The rehabilitation training indicators include the game score rate of the digital mesotherapy game and the task success rate of life-like scenarios and tasks. The game score rates are shown below: In the formula, λ is the game score rate; The average score of the game; This represents the average game time. The success rates of the tasks are shown below: In the formula, γ is the task success rate; q1 is the number of times the gesture was successfully completed; and q0 is the total number of gesture attempts.

10. The forearm limb amputee rehabilitation training system based on visual and vibration-tactile feedback according to claim 9, characterized in that, The application layer modules used for planning personalized rehabilitation pathways include a muscle strength level adaptation module and an antagonistic muscle synergy adaptation module. The muscle strength level adaptation module classifies the user's muscle activity intensity into corresponding muscle strength levels and matches them with the difficulty level of the corresponding digital electromyography game. The users' muscle strength levels are divided into four levels from weak to strong; The digital electromyography game is divided into four levels from easy to difficult. When the root mean square value of the electromyography signal of some muscles of the user is lower than that of other muscles, the muscle strength level adaptation module performs rehabilitation training on the detected muscles. The antagonistic muscle synergistic adaptation module evaluates the coordination of wrist flexion and extension movements by calculating the antagonistic synergistic coefficients of extensor and flexor muscles, and increases the training ratio of daily tasks in life scenarios and tasks based on the calculated antagonistic synergistic coefficients. The antagonistic synergistic coefficients of the extensor and flexor muscles are shown below: In the formula, α is the wrist flexion antagonism coefficient; β is the wrist extension antagonism coefficient; RMS 屈肌 It is the sum of the root mean square values ​​of the electromyographic signals of the radial and ulnar wrist flexors in patients with forearm amputations. RMS 伸肌 It is the sum of the root mean square values ​​of the electromyographic signals of the radial and ulnar extensor carpi radialis muscles in patients with forearm amputations.

Images