Upper limb rehabilitation robot system with multi-mode perception and self-adaptive regulation and control functions

Abstract

The invention discloses a multi-modal sensing and self-adaptive control upper limb rehabilitation robot system which comprises a user management and scheme configuration module, a magneto-rheological intelligent resistance driving system, a multi-source data acquisition module, a multi-source data fusion processing module, a self-adaptive training parameter adjustment module, a rehabilitation effect evaluation module and a VR scene game training module. The system comprises a rehabilitation excitation feedback module, a multi-mode spasm pre-judgment module, a graded protection execution module and a basic safety guarantee module. The multi-source data acquisition module is used for synchronously acquiring kinematics data, biomechanics data and physiological signals of a patient; using a multi-source data fusion processing module to extract features and generate patient state vectors through a convolutional neural network; based on the vector, the adaptive regulation and control module adopts a mixed algorithm of PID real-time control and reinforcement learning iterative optimization to dynamically adjust parameters such as multi-joint resistance, training speed and the like, so that accurate matching between a training strategy and the real-time state and long-term rehabilitation progress of a patient is realized.

Description

TECHNICAL FIELD

[0001] The application relates to the field of upper limb rehabilitation devices, in particular to an upper limb rehabilitation robot system with multi-modal sensing and adaptive regulation. BACKGROUND

[0002] The Task-Oriented Training (TOT) is explicitly listed as the IA level recommended scheme for upper limb motor function recovery in the "Chinese Stroke Rehabilitation Treatment Guidelines (2023 Edition)", and clinical data shows that high-intensity and high-repetitive training (training time per week > 15 hours, lasting for 8-12 weeks) based on TOT can improve the Fugl-Meyer Assessment (FMA-UE) score of patients by 28%-42%, which is significantly better than traditional passive training. However, the current stroke rehabilitation field is facing three core bottlenecks: imbalance between supply and demand of rehabilitation resources: the number of professional rehabilitation therapists serving per stroke patient worldwide is more than 1:500, which is far lower than the WHO recommended standard of 1:100, and it is difficult to meet the needs of individualized high-intensity training of patients "one-on-one"; Lack of training standardization: manual operation relies on the subjective experience of therapists, and training parameters (such as Range of Motion (ROM) and force loading rate) cannot be quantitatively set, the motion repetition accuracy error is as high as ±12%-18%, and long-time operation is easy to cause muscle fatigue of therapists, which makes it difficult to guarantee the consistency of training; Lack of multi-source data acquisition and evaluation system: lack of integrated acquisition capability of kinematic parameters (such as joint motion trajectory smoothness and motion speed), biomechanical parameters (such as muscle strength and muscle tension) and physiological signals (surface electromyogram sEMG and heart rate variability HRV), which cannot construct a "training-evaluation-adjustment" closed loop, and restricts the individualized optimization of rehabilitation scheme. In order to break through the above bottlenecks, the rehabilitation robot technology (classification in medical engineering field: 1001031 rehabilitation engineering technology) gradually develops in the direction of intelligence.

[0003] Existing upper limb rehabilitation robots, such as the patent with publication number CN106361537B, entitled "A Seven-DOF Upper Limb Rehabilitation Robot Based on Hybrid Drive," achieve seven degrees of freedom (DOF) in the shoulder, elbow, and wrist joints through a motor-reducer drive structure. While controllable training with Freedom of Flight (DOF) improves training repeatability, it still has significant technical limitations: First, the training coverage is incomplete: it only focuses on proximal joints of the upper limb (shoulder, elbow, wrist), without integrating the drive units of distal finger joints (14 degrees of freedom for fingers 2-5), failing to meet the upper limb motor function reconstruction needs of "proximal stability - distal fine dexterity." Second, physiological adaptive regulation is lacking: a closed-loop control mechanism of "multi-source physiological signals - training parameters" has not been constructed, making it impossible to dynamically adjust the amplitude of movement and force loading intensity based on the patient's sEMG amplitude (e.g., >50μV indicating muscle fatigue) and HRV indicators (e.g., SDNN <50ms indicating sympathetic nerve excitation), posing training risks. Third, patient compliance is insufficient: the training mode is mainly based on single repetitive movements, lacking contextualized interactive design based on virtual reality (VR), and clinical follow-up shows that the patient treatment completion rate is <50%, making it difficult to guarantee rehabilitation effects. Magnetorheological technology... The development of MRT (Magnetic Rehabilitation Technology) has provided a new path for compliant actuation of rehabilitation robots. Publication number CN119385806A, entitled "A Magnetorheological Three-Dimensional Force Feedback Upper Limb Active and Passive Rehabilitation Training Device," employs a single electromagnetic excitation magnetorheological damper. It adjusts the damping torque through magnetic field strength, thus solving the impact risk of traditional rigid actuation systems. However, it only achieves force feedback control of the proximal upper limb (shoulder and elbow), does not integrate a distal finger joint actuation module, and lacks a function for quantitative assessment of rehabilitation progress. Publication number CN113827443B, "Magnetic Actuation Finger Rehabilitation Trainer," while achieving passive training of a single finger joint, uses a fixed magnetic field actuation structure, making it unable to form a motion coordination control logic with the proximal upper limb joints, and lacks the ability to monitor the patient's physiological state. In summary, existing upper limb rehabilitation robot technologies, including rigid-drive-based devices and locally improved schemes using magnetorheological technology, still suffer from systemic technical bottlenecks: First, in terms of training range, training of proximal and distal joints is fragmented, failing to address the issues of coordinated multi-degree-of-freedom actuation and integrated functional assessment of the entire upper limb; second, in terms of control strategies, the lack of real-time perception and closed-loop feedback mechanisms based on multi-source physiological information makes it difficult to achieve safe, precise, and individualized adaptive control of training parameters; third, in terms of human-computer interaction, the training modes are simplistic and lack deep integration of motivational methods based on rehabilitation psychology, resulting in low patient compliance and difficulty in ensuring the integrity of the treatment course. Therefore, there is an urgent need in this field for a comprehensive solution that integrates coordinated training of the entire upper limb, intelligent adaptive control, and immersive interactive motivation. Summary of the Invention

[0004] Based on this, the purpose of this invention is to propose an upper limb rehabilitation robot system with multimodal perception and adaptive control, and to build a fully automated and personalized rehabilitation training ecosystem from perception to decision-making to execution to evaluation to optimization.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a multimodal perception and adaptive control upper limb rehabilitation robot system, comprising a user management and scheme configuration module, a magnetorheological intelligent resistance drive system, a multi-source data acquisition module, a multi-source data fusion processing module, an adaptive training parameter adjustment module, a rehabilitation effect evaluation module, a VR scenario game training module, a rehabilitation incentive feedback module, a multimodal spasticity prediction and graded protection execution linkage module, and a basic safety assurance module; each module is connected to a central controller via a CAN bus; the magnetorheological intelligent resistance drive system includes a shoulder damper, an elbow damper, a wrist damper, and a finger magnetorheological damper; the damping forces generated by the shoulder, elbow, wrist, and fingers are transmitted to the patient's limb through a medical polyurethane coupling; The system workflow is as follows: Step 1: The multi-source data acquisition module collects and uploads data. 1) Triggering mechanism: After training starts, the central controller sends a synchronous acquisition command to the multi-source data acquisition module via the CAN bus, using the central controller clock with an accuracy of ±1ms as the benchmark, to ensure that the acquisition timing of the multi-source data acquisition module is consistent. 2) Data preprocessing and encapsulation: The multi-source data acquisition module first preprocesses the raw data, and then encapsulates the data according to the CAN bus data frame format to avoid transmission errors; the CAN bus data frame format includes, but is not limited to, module address, data type, data length, and check bit; 3) Real-time upload: The packaged data is uploaded to the central controller via the CAN bus in a time-division multiplexing manner; Step 2: Central Controller Data Processing and Decision Making After receiving data from the multi-source data acquisition modules transmitted via the CAN bus, the central controller performs data fusion and decision generation; among these, Data fusion: Kinematic, biomechanical, and physiological data collected by the multi-source data acquisition module are sent to the multi-source data fusion processing module, and a 128-dimensional patient state vector is generated by the CNN model; the patient state vector includes, but is not limited to, muscle strength, fatigue, and coordination ability; Decision generation: Based on the patient's state vector, the central controller runs an adaptive training parameter adjustment module, which generates risk level instructions and control instructions through a hybrid PID + reinforcement learning algorithm and an SVM spasticity prediction algorithm, and converts the instructions into CAN bus compatible instruction frames; Step 3: Issuance and Execution Feedback of Control Commands Command issuance: The central controller sends control commands to the magnetorheological intelligent resistance drive system and the graded protection execution module via the CAN bus; specifically, it sends resistance adjustment commands to the magnetorheological intelligent resistance drive system and motion mapping commands to the VR scenario game training module. Execution feedback: After the magnetorheological intelligent resistance drive system and the graded protection execution module complete the instructions, they transmit the execution results back to the central controller via the CAN bus; if it is found that the instruction has not been executed, the adjustment instruction is resent via the CAN bus. Step 4: Key safeguards, including real-time performance and security guarantees. Real-time performance guarantee: The CAN bus transmission delay of ≤10ms enables the central controller to complete the entire process of data acquisition → processing → command issuance → execution feedback within 0.3 seconds, avoiding training risks caused by delays; Security Guarantee: The CAN bus error detection mechanism automatically identifies erroneous data transmission and requests retransmission, ensuring no data loss in the multimodal spasm prediction module and guaranteeing accurate triggering of hierarchical protection measures.

[0006] Preferably, the user management and program configuration module includes user access control, program management, training execution, and data processing; user access control verifies therapist ID and patient identity through user login requests; therapist permissions include comprehensive parameter configuration, report viewing, and data export; patient permissions include training initiation and progress viewing; and users access an encrypted database to ensure data security. The treatment plan management involves configuring and calling the plan. The therapist loads a standard XML template and fine-tunes the parameters in the personalization settings layer according to the patient's specific situation to generate a personalized plan. Training is executed according to a personalized plan, and the data generated during the training process is recorded and stored back in an encrypted database. Data processing involves exporting and archiving the data obtained from training, generating Excel files through the POL library to preserve the raw data for in-depth analysis, or generating PDF reports; the generated Excel files or PDF reports are then exported via USB / network for clinical archiving.

[0007] Preferably, the multi-source data acquisition module includes a kinematic data acquisition module, a biomechanical data acquisition module, and a physiological signal acquisition module; wherein, The kinematic data acquisition module includes an angle sensor and a laser displacement sensor. The angle sensor is installed at the pivot of each joint in the shoulder, elbow, wrist, and fingers to measure the rotation angle of each joint. The laser displacement sensor is fixed on the robot base or main frame to measure the linear displacement and velocity of the end effector. The data collected synchronously by the angle sensor and the laser displacement sensor is fused and processed by the central controller to calculate the joint range of motion (ROM) of each joint, the movement speed of each joint, the smoothness of the movement trajectory of each joint, the action completion rate of each joint, and the time difference of multi-joint coordination. The fused data is sent to the host computer for real-time display and synchronously archived in the database. The physiological signal acquisition module collects the patient's physiological status data through a 4-channel flexible surface electromyography (sEMG) electrode and a photoelectric PPG sensor integrated in the wrist. The physiological status data includes, but is not limited to, sEMG amplitude, HRV time-domain index SDNN, RMSSD, and real-time heart rate. The physiological signal acquisition module is composed of a sensing layer, a processing layer, an analysis layer, and an application layer. The sensing layer simultaneously activates a 4-channel flexible surface electromyography (sEMG) electrode and a photoelectric PPG sensor during patient training; among them, the sEMG electrode uses an Ag / AgCl electrode to pick up weak ionic current signals of 5-500μV from the skin surface. The photoelectric PPG sensor uses 660 / 940nm light to illuminate the skin, and a photodiode receives the transmitted or reflected light signal. In the processing layer, the weak ion current signal is pre-amplified by ×1000 and filtered by a 20-500Hz bandpass filter, and then digitized by a high-precision ADC at 2000Hz sampling with an SNR of 60dB. The optical signal of the PPG sensor is converted into an electrical signal, and after photo-current-voltage conversion, it is sampled at a frequency of 100Hz. The analysis layer performs in-depth processing on the data collected by the sEMG electrodes and PPG sensors. Specifically, the RMS values ​​of the sEMG electrodes are extracted to quantify muscle activity intensity; the parameters collected by the PPG sensors are used to calculate physiological parameters such as HR / HRV / SpO2. The application layer continuously performs state assessment and identification loops on the data processed by the analysis layer to determine whether there are abnormalities in muscle activity intensity, HRV-SDNN, or SpO2; the state assessment sends early warning information to the central controller in real time, and if an abnormality is found, it triggers safety protection; the physiological state data is linked to the biomechanical data acquisition module and the adaptive training parameter adjustment module. The biomechanical data acquisition module works in conjunction with a miniature force sensor and a miniature torque sensor to achieve comprehensive monitoring of training mechanical parameters, including but not limited to resistance values, finger gripping force, joint torque, and force loading rate. Miniature force sensors measure resistance values ​​and finger gripping force; miniature force sensors are installed at the power output end of magnetorheological dampers in the shoulder, elbow, wrist, and fingers to directly measure the real-time resistance value applied to the limbs; miniature force sensors integrated into the hand grip are used to measure finger gripping force. Miniature torque sensors are installed at the drive shafts of shoulder, elbow, and wrist joints to measure joint torque; The force loading rate is calculated by the rate of change of force and joint torque over time.

[0008] Preferably, the multi-source data fusion processing module takes the signals acquired by the kinematic data acquisition module, the biomechanical data acquisition module, and the physiological signal acquisition module as multi-source raw data input for data preprocessing; Data preprocessing involves first removing outliers using the 3σ principle and then performing linear interpolation for imputation. Next, Kalman filtering is used to reduce noise in real time from the angle sensor and sEMG sensor, filtering out high-frequency noise. Finally, a unified clock is used for resampling to synchronize to a 100Hz frequency, and sEMG-RMS muscle activation, muscle fatigue, and multi-joint coordination time difference (SD) are extracted to construct a 32×32 feature matrix. This 32×32 feature matrix is ​​then fed into a 3-layer CNN model. The CNN model uses convolution, pooling, and fully connected operations to generate a 128-dimensional deep fusion vector, i.e., a patient state vector. The patient state vector dimensions include muscle strength level, fatigue level, and coordination ability; muscle strength level is 0-5, fatigue level is 0-10, and coordination ability is 0-10. The 3σ principle is used to remove outliers. The judgment rule is: if a single data point... If the data meets the criteria of Formula 1, it is considered abnormal data and is removed. Formula 1 is as follows: Abnormal data identification: (1) In formula 1, For a single raw data acquisition, the joint angle and sEMG amplitude are represented; μ is the mean of the data in the same batch, and σ is the standard deviation of the data in the same batch. Kalman filter noise reduction formula: $ (2) In formula 2, The filtered data at time k, namely the joint angle and sEMG amplitude; The sensor's original measurement value at time k; A, B, and H are the state transition matrix, control matrix, and observation matrix, respectively. For joint angles, A=1, B=0, and H=1 can be set; Q is the process noise covariance, set to 0.0.1 to match the noise characteristics of the angle sensor; R is the measurement noise covariance, set to 0.1 for sEMG signals and 0.05 for angle signals.

[0009] The adaptive training parameter adjustment module adopts a hybrid algorithm of real-time control by PID algorithm and iterative optimization by reinforcement learning, with RL decision as the core, and constructs a dual optimization mechanism of instantaneous response and long-term adaptation. Real-time PID control uses the deviation between the patient's state vector and the target state as input. An incremental PID algorithm is employed to calculate and output the adjustment amount for resistance / velocity at each joint, as detailed below: Parameters of the incremental PID algorithm When the sEMG amplitude increases by more than 50% from the baseline or HRV-SDNN decreases by less than 50ms, the adjustment is completed within 0.3 seconds. Calculation of the deviation between the patient's state vector and the target state. (3) PID incremental output, i.e., parameter adjustment amount. (4) In Formulas 3 and 4, e(k) represents the deviation at time k. For the patient's target state, This represents the actual state at time k. This represents the adjustment amount of the training parameters at time k, i.e., the adjustment amount of resistance / velocity at each joint; Reinforcement learning is used for iterative optimization, employing the Q-learning algorithm to optimize long-term training strategies. The patient's state vector serves as the state space, while the resistance / speed adjustment / game difficulty switching of each joint forms the action space. Changes in the FMA-UE score and optimization of the patient's state vector are used as the core reward signals, and the action value function is updated iteratively. ; (5) In Formula 5, s is the current patient state vector; Adjust the actions to train the parameters; α is the learning rate, typically set between 0.1 and 0.5. This is the immediate reward, i.e., the increase in FMA-UE score; γ is the discount factor, typically ranging from 0.8 to 0.95. The new state after the action is performed; When a specific dimension in the patient's state vector exceeds a safety threshold, the safety sentinel is triggered, and the system prioritizes the adjustment amount of the PID algorithm; the specific dimension includes, but is not limited to, fatigue level >7 points or sEMG amplitude 50μV; If the safety sentinel is not triggered, the system enters a hybrid decision-making mode, which integrates the real-time fine-tuning of the PID with the long-term policy suggestions from reinforcement learning to generate the final execution instruction. After executing the instruction, the system updates the patient's state vector by acquiring new signals and starts a loop. The game difficulty can be switched through the VR scenario game training module, and the action completion rate is obtained. The VR scenario game training module collects angle data θ1, θ2... of each joint in real time based on angle sensors on each joint. The angle data enters the processing layer for motion mapping and coordinate transformation. The angle data θ1, θ2... is matched with the preset range of standard rehabilitation movements in the system's preset rule base. If the angle data matches the value of the preset range, a game event is triggered to switch the game difficulty. Patients watch virtual images that are completely synchronized with their own movements through a head-mounted display and perform the next action according to the game prompts.

[0010] Preferably, the rehabilitation effect assessment module includes a data quantification layer, a predictive analysis layer, and an output application layer; In the data quantification layer, the data collected by the multi-source data acquisition module is mapped to the FMA-UE mapping score, i.e., the Fugl-Meyer upper limb function score, and the ADL score, i.e., the activities of daily living score. FMA-UE mapping score: A multiple linear regression model is used to integrate joint range of motion, finger grip strength, and movement completion rate. The output is highly correlated with human assessment. 2 =0.92 FMA-UE mapping score; ADL score: ADL score is generated by mapping the patient's game completion time / accuracy in VR games; In the predictive analysis layer, the deviation between the predicted and actual values ​​of the FMA-UE mapping score and ADL score accurately identifies weak links. The actual values ​​of the FMA-UE mapping score and ADL score, the scores truly calculated for the patient during the current training cycle: Upon completion of the current training cycle, the system calculates the current FMA-UE mapping score and current ADL score based on real-time collected multi-source data, according to the formulas clearly stated in the documentation. This provides an objective quantification of the current rehabilitation effect. The calculation is based on: Current FMA-UE mapping score: FMA-UE = 0.2 × ROM compliance rate + 0.3 × grip strength compliance rate + 0.5 × action completion rate, with an error ≤ 3 points; Current ADL score: Based on VR scenario game data mapping, the data comes from real-time records of the VR scenario game training module; The predicted values ​​of FMA-UE mapping score and ADL score are the scores that should be achieved in the current period, calculated based on historical data. Step 1: Collect historical data: Extract the patient's actual FMA-UE score and ADL score for the past N cycles, and record the time variables for each cycle accordingly; Step 2: Establish a linear regression model: With time as the independent variable x and historical actual FMA-UE score and ADL score as the dependent variable y, fit a linear equation y=kx+b, where k is the slope, representing the average daily recovery improvement rate; b is the intercept, representing the initial score. Step 3: Generate FMA-UE score and ADL score prediction values: Substitute the time variable of the current period into the equation, and the calculated y value is the FMA-UE prediction score and ADL prediction score of the current period. At the output application layer, a rehabilitation assessment report is generated based on, but not limited to, raw data curves, FMA-UE mapping scores and ADL score change tables, predictive trend charts, and intervention recommendations for weak links.

[0011] Preferably, the rehabilitation incentive feedback module is based on real-time and historical data input collected by the multi-source data acquisition module, which is continuously monitored by the condition monitoring and judgment module. The achievement badge rule base in the condition monitoring and judgment module has several preset incentive conditions. When the quantitative assessment result or the patient's state vector meets the incentive conditions, the rehabilitation incentive feedback module is configured to generate and present positive feedback information to the patient. Achievement badge rule base, including but not limited to continuous training badges and synergy badges; The incentive conditions are as follows: the continuous training medal requires patients to complete ≥10 minutes for 7 days and achieve a completion rate of ≥80%; the synergy medal requires a score of ≥80 for 30 seconds.

[0012] Preferably, the multimodal spasm prediction and graded protection execution linkage module includes a multimodal spasm prediction module and a graded protection execution module; The multimodal spasticity prediction module obtains a combined feature vector by analyzing the amplitude change rate of sEMG signals, the sudden increase in torque of each joint, and the sudden decrease in the movement speed of each joint. The combined feature vector is then fed into an SVM model classifier for real-time analysis, outputting the risk level of the current state. The risk levels include normal, mild warning, moderate warning, and severe warning. The warning output and the specific risk level are then fed into the graded protection execution module. The graded protection execution module receives the risk level output from the multimodal spasm prediction module and enters the graded judgment module. If the risk is judged to be mild, the magnetorheological dampers in the magnetorheological intelligent resistance drive system reduce the pressure by 20-30% and the corresponding motors slow down. If the risk is judged to be moderate, the current training is stopped and the system is switched to passive relaxation mode. If the risk is judged to be severe, the electromagnetic coil is de-energized within 0.1s using the hybrid excitation structure of each magnetorheological damper. The resistance of each magnetorheological damper instantly drops to the basic safety resistance provided by the permanent magnet and an audible and visual alarm is activated.

[0013] Preferably, the basic safety protection module includes an electrical safety module, an emergency stop and power failure protection module, and an electromagnetic compatibility module; The electrical safety module has an insulation resistance of ≥100MΩ and a grounding resistance of ≤0.1Ω through a safety conductor. It is also equipped with a leakage current protection device to limit the leakage current to ≤100μA. The emergency stop and power failure protection module uses capacitor discharge to maintain the short-term operation of critical circuits and will not automatically start after power is restored. The electromagnetic compatibility module suppresses conducted interference through a common-mode inductor and an X capacitor, and suppresses radiated interference through copper foil shielding.

[0014] Preferably, the FMA-UE mapping score uses a simplified estimation model, and the calculation formula is Formula 1: (6) R represents the pass rate (%) of joint range of motion (ROM), G represents the pass rate (%) of finger grip strength, C represents the completion rate (%), and the scoring error is ≤3 points.

[0015] The beneficial effects of this invention are as follows: 1. Multi-degree-of-freedom coordinated training of the entire upper limb: Through standardized design, the coordinated control of multiple joints of the entire upper limb is realized, which breaks through the limitation of the existing technology that only trains local joints. This enables patients to perform daily coordinated movements of the entire upper limb such as grasping, translation and placement, which significantly improves the conversion efficiency of rehabilitation training to life application.

[0016] 2. Precise control of magnetorheological resistance: By adopting a hybrid excitation structure of electromagnetic coil and permanent magnet and PWM current control technology, it achieves 0.1N-level fine adjustment of independent resistance of multiple joints, breaking through the limitations of fixed mechanical resistance levels and single joint adjustment of ordinary magnetorheological equipment, and ensuring that the resistance of each joint is precisely matched with muscle force.

[0017] 3. Gamified Rehabilitation Training System: VR scenario-based games are designed around the integrated movement of the entire upper limb, which achieves a deep integration of "entertainment and rehabilitation function". Through incentive mechanisms such as achievement badges and progress visualization, the training compliance and training duration are significantly improved.

[0018] 4. Multi-dimensional data fusion and adaptive adjustment: Simultaneously collect three types of data: joint level, overall level and physiological level. Through CNN multimodal fusion and "PID + reinforcement learning hybrid algorithm", full-dimensional quantitative evaluation and personalized parameter dynamic adjustment are realized, which breaks through the limitations of existing technologies such as "only collecting single motion data" and "fixed parameter training".

[0019] 5. Multimodal Spasticity Prediction and Graded Intervention: A three-level safety mechanism of "prediction-protection-intervention" was designed. By integrating multi-source data, the risk of spasticity can be predicted in advance, and graded intervention measures can be taken, filling the technical gap of existing rehabilitation robots that "only provide passive restraint protection" and do not have spasticity prediction and active intervention.

[0020] In summary, the magnetorheological-based intelligent upper limb rehabilitation robot training system of the present invention solves the technical bottlenecks of existing rehabilitation robots in terms of full joint coverage, precise resistance control, training compliance, quantitative assessment and safety protection through multi-dimensional technological innovation, and provides an efficient, safe and personalized intelligent rehabilitation solution for patients with upper limb dysfunction. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a schematic diagram of the magnetorheological intelligent resistance drive system in this invention; Figure 3 This explains the working principle of the user management and scheme configuration module in this invention; Figure 4 This describes the working principle of the kinematic data acquisition module in this invention. Figure 5 This describes the working principle of the biomechanical data acquisition module in this invention. Figure 6 This describes the working principle of the physiological signal acquisition module in this invention; Figure 7 This describes the working principle of the multi-source data fusion processing module in this invention; Figure 8 This explains the working principle of the adaptive training parameter adjustment module in this invention. Figure 9 This explains the working principle of the rehabilitation effect assessment module in this invention. Figure 10 This explains the working principle of the VR scenario game training module in this invention. Figure 11 This describes the working principle of the rehabilitation incentive feedback module in this invention; Figure 12 This describes the working principle of the multimodal spasticity prediction module in this invention. Figure 13 This describes the working principle of the hierarchical protection execution module in this invention; Figure 14 This describes the working principle of the basic security module in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0023] like Figures 1 to 14 As shown, a multimodal perception and adaptive control upper limb rehabilitation robot system includes a user management and program configuration module, a magnetorheological intelligent resistance drive system, a multi-source data acquisition module, a multi-source data fusion processing module, an adaptive training parameter adjustment module, a rehabilitation effect evaluation module, a VR scenario game training module, a rehabilitation incentive feedback module, a multimodal spasticity prediction module, a graded protection execution module, and a basic safety assurance module. Each module is connected to the central controller via a CAN bus. The multi-source data acquisition module includes a kinematic data acquisition module, a biomechanical data acquisition module, and a physiological signal acquisition module. The central controller uses an ARM Cortex-A9 processor (1GHz).

[0024] The user management and protocol configuration module is responsible for the system's access control, protocol management, and data processing. The workflow begins with a user login request. After authentication, the system distinguishes between therapists and patients. Therapists are granted advanced permissions, including comprehensive parameter configuration, report viewing, and data export capabilities. Patients are granted basic permissions, including training initiation and monitoring their own progress. All user operations are securely handled by accessing an encrypted database using AES-256. In the protocol configuration and invocation phase, therapists load a standard XML template (the system can preset multiple standardized protocols) and fine-tune parameters based on the patient's specific situation at the personalization layer, ultimately generating a personalized protocol. At the data interaction layer, an Excel file is generated from the POL library to save the raw data for in-depth analysis; or a PDF report containing data curves, scores, and suggestions is generated for clinical archiving. The Excel file or PDF report can be exported via USB / network. All data generated during training execution is logged and stored back in the encrypted database, forming a complete clinical management loop.

[0025] The magnetorheological intelligent resistance drive system is the core power source of this invention, providing target resistance commands based on the rehabilitation plan or adaptive algorithm. Each magnetorheological damper employs a magnetic circuit structure with a permanent magnet and an electromagnetic coil connected in parallel. The permanent magnet provides a constant basic magnetic field, ensuring a safe initial minimum resistance (approximately 0.5 Nm) at zero current, while the electromagnetic coil generates a variable additional magnetic field through precise PWM current control (accuracy ±0.1 A). The two magnetic fields are superimposed to form a final composite magnetic field. This composite magnetic field acts on the cavity of each magnetorheological damper filled with magnetorheological fluid, triggering a magnetorheological effect: under a strong magnetic field, a huge damping force is output; when the magnetic field weakens, the chain structure of the magnetorheological fluid breaks, the liquid reverts to Newtonian fluid properties, and the damping force drops sharply. By continuously adjusting the magnetic field strength, stepless output of continuously variable resistance is achieved. In this invention, independent damping units are configured for the shoulder, elbow, wrist, and finger joints, enabling independent output from multiple joints. The damping forces generated by each joint are smoothly transmitted to the patient's limbs through a flexible transmission mechanism, such as a medical polyurethane coupling. Simultaneously, a miniature force sensor monitors the output resistance feedback value in real time, comparing it with the target resistance in the signal processing and PWM generation unit. This forms a closed-loop regulation, fine-tuning the PWM duty cycle to ensure precise and stable resistance. This design achieves high-precision, fast-response (≤0.1 seconds) conversion from command to physical resistance, forming the physical basis for the system's personalized rehabilitation training and safety protection.

[0026] The kinematic data acquisition module employs high-precision angle sensors and laser displacement sensors, collecting joint range of motion (ROM), joint motion velocity, trajectory smoothness, motion completion rate, and multi-joint coordination time difference via a CAN bus (transmission delay ≤10ms). The angle sensors and laser displacement sensors synchronously acquire data through independent communication links. Angle sensors installed at each joint collect angle information with a resolution of 0.01°; laser displacement sensors collect joint displacement and velocity information with a velocity accuracy of ±0.1m / s. The acquired information is digitized, encapsulated, and transmitted via the CAN bus, ensuring a delay of ≤10ms. At the receiving end, the data is received and deframed to restore the original data. Next, the data enters the data preprocessing and fusion stage, undergoing filtering, calibration, and time alignment to prepare for subsequent calculations. At the core indicator calculation layer, the system calculates multiple quantitative indicators based on preprocessed data, including joint range of motion (ROM), motion velocity, trajectory smoothness (through jerk analysis), multi-joint coordination time difference, and "action completion rate" (by comparing the actual trajectory with the target trajectory). These final results are transmitted to the host computer for real-time display via Ethernet / serial port through the data output and storage layer, and are simultaneously archived in the database. The system continuously determines whether the task has ended to decide whether to terminate the current data acquisition process.

[0027] The physiological signal acquisition module includes a 4-channel flexible surface electromyography (sEMG) electrode and a wrist-integrated photoelectric PPG sensor (which calculates HRV by deriving from heart rate signals). It acquires sEMG amplitude, HRV temporal indices (SDNN, RMSSD), and real-time heart rate to monitor the patient's muscle activation state and fatigue level, preventing overtraining. The sEMG uses Ag / AgCl electrodes to pick up weak ionic current signals of 5-500μV from the skin surface. These signals are pre-amplified by ×1000 and band-pass filtered at 20-500Hz before being digitized by a high-precision ADC at 2000Hz (SNR=60dB). The PPG sensor uses 660 / 940nm light to illuminate the skin, and a photodiode receives the transmitted or reflected light signals, converting them into electrical signals. After photo-current-voltage conversion, the signals are sampled at 100Hz. The acquired raw data undergoes in-depth processing in the analysis layer: the sEMG signal's RMS value is extracted to quantify muscle activity intensity; the PPG signal is used to calculate key physiological parameters such as HR / HRV / SpO2. The system continuously performs state assessment and identification cycles to determine if there are any abnormalities, such as HRV-SDNN < 50ms or SpO2 anomalies. State assessment involves sending real-time warnings to the central controller to trigger safety protection; simultaneously, physiological state data is linked to the biomechanical module and adaptive training parameter adjustment module, providing core input for achieving true physiological closed-loop adjustment training.

[0028] The biomechanical data acquisition module includes miniature force sensors and miniature torque sensors. The miniature force sensors are integrated into the output terminals of each magnetorheological damper, while the miniature torque sensors are located on the drive shafts of the shoulder, elbow, wrist, and finger joints. These sensors collect real-time resistance values, finger gripping force, joint torque, and force loading rate, providing a mechanical basis for training intensity adaptation. Force signals are acquired through miniature force sensors installed at the output terminals of the magnetorheological dampers or on the handle. The core of these sensors is a strain gauge, which deforms under force, resulting in a change in resistance. This change is converted into a weak differential voltage via a Wheatstone bridge, and then amplified and filtered to become a high-precision analog voltage (range 0-50N, accuracy ±0.1N). Torque signals are acquired through torque sensors installed on the drive shafts of the shoulder, elbow, wrist, and finger joints. The elastic body undergoes torsional deformation under torque, causing a change in resistance in the attached strain gauge. This change is also converted into a standard voltage signal via a bridge and amplification circuit (range 0-50Nm, accuracy ±0.05Nm). All analog voltage signals are acquired in real-time by the central controller and converted into digital quantities. The collected real-time biomechanical data is immediately compared with the "personalized threshold" set by the system for the patient. Based on the comparison result, the system immediately makes a judgment and triggers corresponding actions: if the measured value is less than the threshold, it indicates insufficient training intensity, and the system can increase the resistance through adaptive control logic; if the measured value is greater than the threshold, there is a risk of overtraining, and the system will take measures such as reducing resistance to avoid the risk, thus forming a rapid control closed loop based on biomechanical feedback.

[0029] The multi-source data fusion processing module takes the raw data collected by the kinematic data acquisition module, biomechanical data acquisition module, and physiological signal acquisition module, removes outliers through the 3σ principle, performs Kalman filtering for noise reduction, and performs unified clock resampling, extracts the root mean square (RMS) of sEMG and the motor coordination index (standard deviation of the time difference of each joint movement), and fuses the multimodal data based on the convolutional neural network (CNN) to output a patient state vector containing muscle strength level, fatigue level, and coordination ability.

[0030] The multi-source data fusion processing module first processes the raw data collected by the kinematic data acquisition module, biomechanical data acquisition module, and physiological signal acquisition module into a cleaning and alignment layer for data preprocessing: obvious abnormal data points are removed using the 3σ principle and linear interpolation is performed to fill them, followed by Kalman filtering to suppress high-frequency noise, and finally, all data streams are synchronized to a frequency of 100Hz through unified clock resampling to ensure data consistency in the time dimension. The preprocessed data is constructed into a 32×32 feature matrix, which is fed into a 3-layer CNN model for feature extraction and information refinement. The model automatically learns and extracts features such as sEMG-RMS (muscle activation and fatigue) and multi-joint coordination time difference (SD) (motor coordination) from the data through convolution + pooling + fully connected operations. The output layer of the CNN model generates a 128-dimensional deep fusion vector, i.e., a patient state vector, including muscle strength level (0-5), fatigue level (0-10), and coordination ability (0-10). The patient's state vector is output to the control decision module, serving as a key basis for subsequent decisions such as the adaptive adjustment of training parameters module and the prediction of spasticity risk module, thereby achieving closed-loop adjustment of training and realizing data-driven precision rehabilitation.

[0031] The specific hierarchical structure of the CNN model is as follows: 1. Input layer The system receives a single-channel feature matrix with a dimension of 32×32. This matrix is ​​constructed from kinematic, biomechanical, and physiological features after preprocessing and adaptation, providing the raw input for subsequent feature extraction.

[0032] 2. Convolutional layers (2 layers, core feature extraction layer) Convolutional layer 1: Multiple small-sized convolutional kernels (e.g., 3×3) are used to extract local features from the input feature matrix through a sliding window operation, enhancing the ability to capture subtle features (such as changes in joint motion speed and sEMG-RMS fluctuations). Convolutional Layer 2: Based on the feature map output by Convolutional Layer 1, another convolution operation is performed to further fuse local features into globally related features (such as the collaborative features of "joint torque + muscle activation"), thereby improving the depth of feature representation.

[0033] 3. Pooling layer (2 layers, for dimensionality compression and feature selection) Alternate configurations with convolutional layers, employing a max pooling strategy (a conventional, efficient pooling method): Pooling layer 1: Corresponding to the output of convolutional layer 1, it uses a 2×2 pooling kernel and a stride of 2 to downsample the convolutional feature map, reducing the data dimensionality while retaining key features and avoiding overfitting; Pooling layer 2: Corresponding to the output of convolutional layer 2, it also uses a 2×2 pooling kernel and a stride of 2 to further compress the dimensions and extract the core fusion features.

[0034] 4. Fully connected layer (2 layers, feature integration and mapping) The two-dimensional feature map output from the pooling layer is flattened into a one-dimensional vector and then fed into the vector to achieve non-linear feature integration. Fully connected layer 1: Receives the flattened one-dimensional feature vector, and achieves cross-dimensional feature fusion through full connectivity between neurons, integrating the scattered features of "motion-mechanics-physiology" into a unified feature space; Fully connected layer 2: Further optimizes the output of fully connected layer 1, adjusts the feature dimensions to adapt to the requirements of the output layer, and lays the foundation for the generation of the final state vector.

[0035] 5. Output layer The output is a 128-dimensional deep fusion vector, which directly corresponds to the patient status assessment indicators of "muscle strength level (0-5 grade - fatigue level (0-10 points) - coordination ability (0-10 points)", providing the core decision basis for subsequent adaptive training parameter adjustment.

[0036] 32×32 Feature Matrix Construction Process The construction of the 32×32 feature matrix follows a core process of "multi-source data preprocessing → feature classification and extraction → dimension adaptation → matrix concatenation," and is based entirely on data processing rules and feature types. The specific steps are as follows: 1. Step 1: Preprocessing of multi-source raw data First, the raw data collected in three categories—kinematics, biomechanics, and physiology—were standardized to ensure data quality and time synchronization. Outlier removal: The 3σ principle is used to remove abnormal interference data during the sensor acquisition process (such as sudden false alarms from force sensors). Noise suppression: Reduce data noise (such as power frequency noise in sEMG signals and measurement noise from angle sensors) through Kalman filtering algorithm. Time synchronization: Based on the central controller clock (accuracy ±1ms), the three types of data are timestamped and resampled to 100Hz to ensure data consistency in the time dimension.

[0037] 2. Step 2: Multi-dimensional feature classification and extraction From the preprocessed raw data, key indicators were extracted according to feature type (kinematic features (joint angles, velocities), biomechanical features (resistance, torque), and physiological features (sEMG-RMS, HRV-SDNN)): Kinematic characteristics: Extract core indicators such as range of motion (ROM), movement speed, and trajectory smoothness of each joint (shoulder, elbow, wrist, and fingers) (based on data from angle sensors and laser displacement sensors). Biomechanical characteristics: Extract real-time resistance values ​​(force sensor at the output end of the magnetorheological damper), joint torque (torque sensor of the shoulder and elbow joint drive shaft), force loading rate, and other indicators; Physiological characteristics: sEMG root mean square (RMS, reflecting muscle activation and fatigue) and HRV time-domain index (SDNN, reflecting sympathetic nerve state) were extracted (based on 4-channel sEMG electrodes and wrist PPG sensor data).

[0038] 3. Step 3: Feature Dimension Adaptation Because the original dimensions of the three types of features differ (e.g., kinematic features include multiple indicators for four joints, and physiological features include two core indicators), standardization is required to adapt all types of features to a unified dimension to meet the requirements for constructing a 32×32 matrix. Feature standardization: Normalize each type of feature (e.g., map joint angles to the 0-1 interval, map sEMG-RMS to the 0-10 interval) to eliminate dimensional differences; Dimensional adjustment: Based on the importance and number of features, the three types of features are adjusted to 32-dimensional feature vectors respectively (e.g., kinematic features occupy 12 dimensions, biomechanical features occupy 10 dimensions, and physiological features occupy 10 dimensions, for a total of 32 dimensions), to ensure that the dimension of a single feature adapts to the "column" or "row" requirements of the matrix.

[0039] 4. Step 4: Feature Matrix Concatenation The adapted 32-dimensional kinematic feature vectors, 32-dimensional biomechanical feature vectors, and 32-dimensional physiological feature vectors are then concatenated into a 32×32 single-channel feature matrix according to preset rules (e.g., "row-direction concatenation": each type of feature occupies 10-12 rows out of 32, totaling 32 rows; the column direction is uniformly 32 columns). Matrix structure: The "rows" of the matrix correspond to feature types and sub-indicators, and the "columns" correspond to time series or feature dimensions (time segments based on 100Hz sampling). Final output: The concatenated 32×32 feature matrix is ​​directly used as the input to the CNN model, realizing the integrated input of multimodal features of "motion-mechanics-physiology".

[0040] Methods for constructing the input feature matrix of a CNN model Construction logic of multi-source data into a 32×32 feature matrix To convert heterogeneous multi-source data into a 32×32 single-channel feature matrix suitable for CNN model input, the system adopts a standardized process of "temporal window slicing → feature extraction → dimensionality normalization → spatial reorganization". The specific construction method is as follows: Time window definition and data slicing The system uses a 1-second processing window. Based on a unified sampling frequency of 100Hz after data synchronization, each time window contains data from 100 time points.

[0041] Within this time window, raw data streams from three major categories of sensors—kinematic, biomechanical, and physiological signals—are captured simultaneously.

[0042] Feature extraction and preliminary vectorization For the raw data within each time window, a set of representative feature indicators are calculated by category to form a preliminary feature vector: Kinematic feature vector (12-dimensional): Extracted from the angle and velocity signals of four joints: shoulder, elbow, wrist, and fingers. Three features are calculated for each joint: mean angle, mean angular velocity, and trajectory smoothness (variance of jerk). A total of 4 joints × 3 features = 12 dimensions.

[0043] Biomechanical feature vector (10-dimensional): extracted from force and torque signals. Includes: real-time resistance values ​​of 4 joints (4-dimensional), real-time torque of shoulder and elbow joints (2-dimensional), overall force loading rate (1-dimensional), gripping force (1-dimensional), and torque change rate (2-dimensional). A total of 10 dimensions.

[0044] Physiological feature vector (10-dimensional): Extracted from physiological signals. Includes: root mean square (RMS) value of 4-channel sEMG signal (4-dimensional), heart rate (1-dimensional), SDNN of HRV (1-dimensional) and RMSSD (1-dimensional), blood oxygen saturation (SpO2) (1-dimensional), and the rate of decline of sEMG median frequency (MF) (2-dimensional, used for fatigue assessment). Total 10 dimensions.

[0045] Thus, each 1-second time window is represented as a joint feature vector of 12+10+10=32 dimensions.

[0046] Time series construction and matrix generation To preserve temporal dynamics and meet the two-dimensional input requirements of CNN, the system continuously collects 32 consecutive time windows (i.e., 32 seconds of data).

[0047] Arranging the 32 "32-dimensional joint feature vectors" corresponding to these 32 time windows in chronological order naturally forms a 32 (time series) × 32 (feature dimension) matrix.

[0048] Each row of this matrix represents a specific time point, and each column represents a specific feature dimension across time. This matrix serves as the input to the CNN model.

[0049] The adaptive training parameter adjustment module is the core of intelligent decision-making for personalized and dynamic rehabilitation training. It adopts a hybrid algorithm of PID real-time control and reinforcement learning (Q-learning) iterative optimization to construct a dual optimization mechanism of instantaneous response and long-term adaptation, as detailed below: Real-time PID control: Taking the patient's state vector output from the multi-source data fusion module as input, when the fatigue level dimension (corresponding to sEMG amplitude increasing by >50% from baseline) or the sympathetic excitation dimension (corresponding to HRV-SDNN <50ms) in the patient's state vector triggers a threshold, the PID algorithm precisely reduces the corresponding joint resistance by 10%-20% (0.1Nm-level fine-tuning) and the training speed by 15%-20% within 0.3 seconds, quickly avoiding the risk of overtraining. If safe, the fused final execution instructions are generated. These instructions are executed precisely: 0.1Nm-level resistance adjustment is achieved by controlling each magnetorheological damper through PWM signals, angle adaptation within ±5° is achieved through angle closed-loop control, and automatic switching of game difficulty can be realized. After training execution, the system updates the state vector through new signal acquisition, thereby starting the next intelligent adjustment cycle, forming a continuously optimized adaptive closed loop.

[0050] Reinforcement learning iterative optimization (non-static staging adaptation): Using patient state vectors as the state space (including muscle strength grade 0-5, fatigue level 0-10, and synergy ability 0-10), resistance fine-tuning / speed adjustment / game difficulty switching as the action space, and changes in FMA-UE scores (such as the improvement in score after a single training session, and the cumulative improvement over 7 / 14 / 30 days) and state vector optimization (such as improved synergy ability and controllable fatigue level) as core reward signals, long-term strategy iteration is achieved. In the initial stage, Brunnstrom I-VI was used as the basic adaptation template (such as the initial parameters of "passive dominance + low resistance" preset in I-II), rather than as the basis for final optimization. Each cycle (7 days) updates the action value function (Q(s,a)) based on historical state vector trends and FMA-UE reward feedback: If a certain state-action combination, such as adjusting shoulder resistance by +0.2 Nm under muscle strength grade 3 + fatigue score 4, can improve the FMA-UE score by ≥2 points and improve the patient's coordination ability in the state vector by 1-2 points, then the combination strategy is strengthened; if there is no improvement in FMA-UE or a sudden increase in fatigue, then the strategy is weakened. Dynamically optimize training mode during long-term iteration: For the same Brunnstrom stage IV patient, if the state vector shows that the muscle strength grade increases from grade 3 to grade 4 and the FMA-UE score increases from 30 to 35, reinforcement learning will automatically upgrade the training mode from "active assistance + medium resistance" to "active resistance + high synergy", rather than maintaining fixed stage parameters. Parameter adjustment precision: All parameter adjustments rely on the above dual algorithm output, which can achieve resistance fine-tuning at the 0.1Nm level, training angle adaptation within ±5°, VR game difficulty switching at 3 levels, and single training duration adjustable from 5 to 30 minutes, ensuring that the parameters are accurately matched with the patient's real-time status and long-term rehabilitation progress.

[0051] The rehabilitation effect assessment module established a regression model between collected data and clinical scores. The simplified FMA-UE estimation model is: FMA-UE = 0.2 × ROM target achievement rate + 0.3 × grip strength target achievement rate + 0.5 × movement completion rate, with an error ≤ 3 points. ROM target achievement rate is correlated with clinical wrist and elbow joint movement dimensions; grip strength target achievement rate is correlated with clinical hand movement dimensions; and movement completion rate is correlated with clinical coordination / speed and implicit reflex activity dimensions. The simplified FMA-UE estimation model is fitted based on training data from 120 stroke patients (20 each from Brunnstrom stages I-VI). The correlation coefficient R with the manual FMA-UE score is [missing value]. 2 =0.92, which meets the accuracy requirements for clinical quantitative assessment.

[0052] The ADL scoring model is based on a mapping between game completion time and accuracy (e.g., a virtual kitchen completion time of <2 minutes and 100% accuracy corresponds to an ADL score of 10). Trend prediction uses a linear regression algorithm, with historical scores from 7 / 14 / 30 days as independent variables, to predict rehabilitation progress over the next 1-2 weeks, and residual analysis is used to identify weaknesses. The report generation module uses a PDF template engine to automatically populate the original data curves, score change tables, prediction curves, and intervention recommendations, generating an assessment report.

[0053] The VR scenario game training module is developed based on the Unity engine and uses the ECS architecture to achieve high-performance rendering (frame rate ≥ 60fps), supporting mainstream VR headsets such as Oculus Quest 2. Game content includes simulations of everyday life scenarios such as "virtual kitchen," "clothes organization," and "gardening activities," achieving a 1:1 mapping between mechanical joint movements and virtual arm actions through homogeneous coordinate transformation. The system employs Asynchronous Time Warp (ATW) technology to control VR display latency to ≤ 20ms, preventing user dizziness.

[0054] The module's input layer uses joint angle sensors to collect angle data (θ1, θ2...) from each joint in real time. This angle data is then sent to the processing layer for motion mapping and coordinate transformation. The system's preset rule base matches the collected angle data (θᵢ) of each joint with the preset range of standard rehabilitation movements. Once a match is found, a game event is triggered (e.g., in the "virtual kitchen" scene, reaching a specific angle with the shoulder joint triggers a virtual hand-grabbing action). Simultaneously, the real-time rendering and image generation unit, based on the Unity engine's ECS architecture and GPU acceleration, efficiently renders the game visuals at a frame rate of ≥60fps and utilizes a "physics engine" to handle collision detection and special effects. To ensure immersion and prevent dizziness, the system employs advanced technologies such as "ATW predictive distortion" in the "synchronous display and latency compensation" stage, controlling the total motion-to-photon latency to ≤20ms. Ultimately, patients watch virtual images that are completely synchronized with their own movements through a head-mounted display and perform the next action according to the game prompts, forming a highly coordinated closed loop of "action-perception" that enables rehabilitation training to accurately achieve functional goals in a fun way.

[0055] The rehabilitation motivation and feedback module includes an achievement badge system, a progress visualization system, and a non-competitive feedback system. The achievement badge system includes badges for perfect attendance, coordination mastery, perfect performance, strength growth, homeostasis maintenance, range of motion expansion, grasping precision control, adaptive challenger, spasticity predictor, and overall progress. Progress visualization uses the ECharts library to generate line / bar charts of data such as FMA-UE scores and joint range of motion achievement rates. Non-competitive feedback only displays longitudinal comparisons of the patient's own training data, such as a 10% improvement in grip strength compared to yesterday, avoiding psychological pressure and continuously stimulating and maintaining the patient's training motivation. Real-time and historical data are input into this module and continuously monitored by a conditional listening and judgment unit. This unit calls various preset incentive conditions from the achievement badge rule base, such as the continuous training badge requiring ≥10 minutes of training over 7 days and an achievement rate ≥80%, and the coordination badge requiring a score ≥80 for 30 seconds. Once the conditions are met, the system immediately enters the feedback generation and presentation stage. At the feedback execution layer, the system provides patients with immediate positive feedback in multimedia formats: including pop-up dynamic badge animations, line / bar charts generated using the ECharts library to visually demonstrate progress trends, and non-competitive text such as "Your grip strength has improved by 15% compared to last week." After receiving positive feedback, patients' intrinsic motivation for training is enhanced, making them more willing to engage in the next training session, forming a reinforcing loop that effectively addresses the core pain point of low patient compliance in traditional rehabilitation training.

[0056] The multimodal spasticity prediction module extracts features such as the sEMG amplitude change rate (ΔRMS / Δt > 50% / s), sudden increase in joint torque (ΔT / Δt > 30% / s), and sudden drop in motion velocity (Δv / Δt > 50% / s). These features are then input into a trained SVM model (accuracy ≥ 90%), and the module outputs classification results for normal / mild / moderate / severe risk 0.3-0.8 seconds in advance. First, the system extracts three key indicators in real-time from sEMG signals, joint torque data, and motion trajectory data: the sEMG signal amplitude change rate (ΔRMS / Δt > 50% / s), the sudden increase in joint torque (ΔT / Δt > 30% / s), and the sudden drop in motion velocity (Δv / Δt > 50% / s). The extracted features are recorded and combined into a feature vector, which serves as input to the machine learning model. This feature vector is then fed into the trained SVM model classifier for real-time analysis. The SVM model, through comprehensive calculation of these multimodal features, outputs a classification judgment of the current state, namely the risk level—normal, mild warning, moderate warning, and severe warning. Thanks to the capture of early, subtle features, the model can achieve early warnings 0.5-1 seconds in advance. Finally, the module transmits the warning output and the specific risk level to the graded protection execution module, thus achieving a leap from passive response to proactive prediction.

[0057] The graded protection execution module receives a risk level signal, which enters the graded judgment stage. Based on different risk levels, corresponding intervention measures are initiated: For mild risk: each magnetorheological damper is depressurized by 20-30%, accompanied by a corresponding motor speed reduction. For moderate risk: the current active training is immediately stopped, and the system switches to passive relaxation mode. In this mode, the robot will move the patient's joints at a very slow speed (5° / s) for 10-15 seconds to provide gentle movement. For severe risk: the electromagnetic coil is de-energized within 0.1s, instantly reducing the resistance to the basic safe resistance provided by the permanent magnet (approximately 0.5Nm). Simultaneously, an audible and visual alarm is immediately triggered, prompting intervention and notifying the therapist or medical staff for emergency manual treatment. After all levels of intervention measures are implemented, the system does not terminate its operation but immediately enters a dynamic monitoring cycle, continuously collecting the patient's physiological and motor signals and reassessing safety. If the system determines safety, the process guides the resumption of training, gradually restoring the normal training rhythm. If the risk persists, the system returns to the monitoring stage for continuous observation, and may even re-trigger or maintain the current intervention measures until a safe state is confirmed.

[0058] The basic safety protection modules include electrical safety, emergency stop and power failure protection, and electromagnetic compatibility. Electrical safety is ensured through insulation design with an insulation resistance ≥100MΩ, a safe grounding conductor to ensure a grounding resistance ≤0.1Ω, and a leakage current protection device to limit leakage current to ≤100μA, forming a comprehensive electric shock protection system. Emergency stop and power failure protection: when the emergency stop button is pressed or an unexpected power outage occurs, the system immediately executes a physical power-off procedure, using capacitor discharge to maintain the brief operation of critical circuits. It may also use clutches or other methods to free joints, ensuring the patient's limbs are not injured by the device locking, and it will not automatically restart after power is restored. Electromagnetic compatibility: Conducted interference is suppressed through common-mode inductors and X capacitors, and radiated interference is suppressed through copper foil shielding, ensuring that it does not affect other medical equipment or is unaffected by external interference, thus guaranteeing the reliability of the entire treatment environment.

Claims

1. A multi-modal sensing and adaptive regulation upper limb rehabilitation robot system, characterized in that, The system comprises a user management and scheme configuration module, a magneto-rheological intelligent resistance driving system, a multi-source data acquisition module, a multi-source data fusion processing module, an adaptive training parameter adjustment module, a rehabilitation effect evaluation module, a VR scenario game training module, a rehabilitation incentive feedback module, a multi-modal spasm prediction and grading protection execution linkage module, and a basic safety guarantee module; each module is connected with the central controller through a CAN bus; the magneto-rheological intelligent resistance driving system comprises a shoulder damper, an elbow damper, a wrist damper, and a finger damper; the damping force generated by the shoulder, elbow, wrist, and finger is transmitted to the patient's limbs through a medical polyurethane coupling; The system workflow is as follows: First step: data acquisition and uploading by the multi-source data acquisition module 1) Trigger mechanism: after the training is started, the central controller sends a synchronous acquisition instruction to the multi-source data acquisition module through the CAN bus, unifies the central controller clock with an accuracy of ±1 ms as the benchmark, and ensures that the multi-source data acquisition module acquisition timing is consistent; 2) Data preprocessing and packaging: the multi-source data acquisition module first preprocesses the original data, and then packages the data according to the CAN bus data frame format to avoid transmission errors; The CAN bus data frame format includes but is not limited to module address, data type, data length, and check bit; 3) Real-time uploading: the packaged data is uploaded to the central controller through the CAN bus in a time division multiplexing manner; Second step: central controller data processing and decision making After the central controller receives the multi-source data acquisition module data transmitted by the CAN bus, data fusion and decision making are performed; wherein, Data fusion: the kinematics, biomechanics, and physiological data collected by the multi-source data acquisition module are input into the multi-source data fusion processing module, and a 128-dimensional patient state vector is generated through a CNN model; the patient state vector includes but is not limited to muscle strength, fatigue, and coordination ability; Decision making: based on the patient state vector, the central controller runs the adaptive training parameter adjustment module, generates risk level instructions and control instructions through a PID+reinforcement learning hybrid algorithm and an SVM spasm prediction algorithm, and converts the instructions into CAN bus compatible instruction frames; Third step: control instruction issuing and execution feedback Instruction issuing: the central controller sends the control instructions to the magneto-rheological intelligent resistance driving system and the grading protection execution module through the CAN bus; specifically, resistance adjustment instructions are issued to the magneto-rheological intelligent resistance driving system, and action mapping instructions are issued to the VR scenario game training module; Execution feedback: after the magneto-rheological intelligent resistance driving system and the grading protection execution module complete the instructions, the execution results are returned to the central controller through the CAN bus; if it is found that the instructions have not been executed, the adjustment instructions are reissued through the CAN bus; Fourth step: key safeguards, including real-time and safety guarantees, Real-time guarantee: the CAN bus transmission delay is ≤10 ms, which supports the central controller to complete the data acquisition→processing→instruction issuing→execution feedback whole process within 0.3 seconds, avoiding training risks caused by delays; Security assurance: CAN bus error detection mechanism automatically identifies transmission error data and requires retransmission, ensuring that the multi-modal spasm prediction module has no data loss, and ensuring that the hierarchical protection measures are accurately triggered.

2. The multi-modal sensing and self-adaptive regulating upper limb rehabilitation robot system according to claim 1, wherein, The user management and scheme configuration module includes user permission control, scheme management, training execution, and data processing. The user permission control performs therapist identity verification and patient identity verification through a user login request; therapist permissions include comprehensive parameter configuration, report viewing, and data export; patient permissions include training start and progress viewing; User data security is ensured by accessing an encrypted database; The scheme management loads an XML standard template for the therapist, fine-tunes parameters according to the patient's specific situation at the individualization setting layer, and generates an individualized scheme through scheme configuration and calling; The training execution generates data during the training process, which is recorded and stored back to the encrypted database according to the individualized scheme; The data processing exports and archives the data obtained from the training execution through data export and archiving, and generates Excel files through the POL library to save raw data for in-depth analysis, or generates PDF reports; The generated Excel files or PDF reports are exported through USB / network for clinical archiving.

3. The multi-modal sensing and self-adaptive regulating upper limb rehabilitation robot system according to claim 1, wherein, The multi-source data acquisition module includes a kinematics data acquisition module, a biomechanics data acquisition module, and a physiological signal acquisition module; The kinematics data acquisition module includes angle sensors and laser displacement sensors, the angle sensors are installed on the rotation shafts of the shoulder, elbow, wrist, and finger joints to measure the rotation angles of the joints, and the laser displacement sensors are fixed on the robot base or main frame to measure the linear displacement and speed of the end effector; The data collected by the angle sensors and laser displacement sensors are fused and processed by the central controller to calculate the joint range of motion ROM, joint movement speed, joint movement trajectory smoothness, joint action completion rate, and multi-joint coordination time difference, and the fused and processed data are sent to the upper computer for real-time display and synchronous database archiving; The physiological signal acquisition module acquires physiological state data of the patient through 4-channel flexible surface electromyography sEMG electrodes and photoelectric PPG sensors integrated in the wrist; the physiological state data includes but is not limited to sEMG amplitude, HRV time domain indicators SDNN, RMSSD, and real-time heart rate; the physiological signal acquisition module has a sensing layer, a processing layer, an analysis layer, and an application layer; The sensing layer synchronously starts the 4-channel flexible surface electromyography sEMG electrodes and photoelectric PPG sensors during patient training; the sEMG electrodes use Ag / AgCl electrodes to pick up weak ionic current signals of 5-500 μV from the skin surface; The photoelectric PPG sensor uses 660 / 940 nm light emission to irradiate the skin, and a photodiode receives the transmitted or reflected light signal; The processing layer digitizes the weak ionic current signal by high-precision ADC with 2000 Hz sampling SNR=60 dB after pre-amplification ×1000 and 20-500 Hz band-pass filtering; The light signal of the PPG sensor is converted into an electrical signal, which is sampled at a frequency of 100 Hz after light-electricity-voltage conversion; The analysis layer performs deep processing on the data collected by the sEMG electrode and the PPG sensor; specifically, the sEMG electrode extracts the RMS value to quantify muscle activity intensity; and the parameters collected by the PPG sensor are used to calculate HR / HRV / SpO2 physiological parameters; The application layer continuously performs state evaluation and identification cycles on the processed data from the analysis layer to determine whether the muscle activity intensity, HRV-SDNN, or SpO2 is abnormal; the state evaluation sends real-time warning information to the central controller, and triggers safety protection if there is an abnormality; and the physiological state data are linked to the biomechanics data acquisition module and the adaptive training parameter adjustment module; The biomechanics data acquisition module cooperates with the micro force sensor and the micro torque sensor to comprehensively monitor the training mechanics parameters; the mechanics parameters include but are not limited to resistance value, finger grip force, joint torque, and force loading rate; The micro force sensor measures the resistance value and the finger grip force; the micro force sensor is installed at the power output end of the magneto-rheological damper installed on the shoulder, elbow, wrist, and finger to directly measure the real-time resistance value applied to the limb; and the micro force sensor integrated in the hand grip is used to measure the finger grip force; The micro torque sensor is installed at the driving shaft of the shoulder, elbow, and wrist joints to measure the joint torque; The force loading rate is calculated by the rate of change of force and joint torque over time.

4. The multi-modal sensing and self-adaptive control upper limb rehabilitation robot system according to claim 1, wherein, The multi-source data fusion processing module inputs the signals collected by the kinematics data acquisition module, the biomechanics data acquisition module, and the physiological signal acquisition module as multi-source raw data for data preprocessing; The data preprocessing first removes abnormal data using the 3σ principle and performs linear interpolation filling, then filters the high-frequency noise of the angle sensor and the sEMG sensor in real time through Kalman filtering, and finally resamples the signals to a frequency of 100 Hz through a unified clock, extracts the sEMG-RMS muscle activation and muscle fatigue, and the multi-joint coordination time difference SD, and constructs a 32×32 feature matrix; the 32×32 feature matrix enters a 3-layer CNN model, the CNN model performs convolution + pooling + full connection operations, and the output layer of the CNN model generates a 128-dimensional deep fusion vector, i.e., a patient state vector; the dimensions of the patient state vector include muscle strength grade, fatigue degree, and coordination ability; the muscle strength grade is 0-5, the fatigue degree is 0-10, and the coordination ability is 0-10; 3σ principle eliminates abnormal data, the determination rule is: if a single data satisfies formula 1, it is determined as abnormal data and is eliminated, and formula 1 is as follows Abnormal data determination: (1) In Equation 1, is the single raw acquisition data, joint angle, sEMG amplitude; μ is the mean value of the same batch of data, and σ is the standard deviation of the same batch of data. Kalman filter denoising formula: (2) In formula 2, The filtered data at the kth moment, that is, joint angle, sEMG amplitude; The original measurement value of the sensor at the kth moment; A, B, and H are state transition matrix, control matrix, and observation matrix, and for joint angle, A=1, B=0, and H=1; Q is process noise covariance, which is set to 0.0.1, which is suitable for the noise characteristics of the angle sensor; R is measurement noise covariance, which is set to 0.1 for sEMG signal and 0.05 for angle signal.

5. The multi-modal sensory perception and self-adaptive regulation upper limb rehabilitation robot system according to claim 1, wherein, The adaptive training parameter adjustment module adopts a hybrid algorithm of real-time PID control and reinforcement learning iterative optimization, takes RL decision as the core, and constructs a dual optimization mechanism of instantaneous response and long-term adaptation; specifically, The PID algorithm real-time control takes the deviation between the patient state vector and the target state as input, calculates and outputs the adjustment amount of each joint resistance / speed using the incremental PID algorithm, and is specifically as follows: Parameters of incremental PID algorithm, When the sEMG amplitude is increased by more than 50% from the baseline or the HRV-SDNN is less than 50 ms, the adjustment is completed within 0.3 seconds. patient state vector and target state deviation calculation, (3) PID incremental output, i.e., parameter adjustment amount, （4） In the formula 3 and the formula 4, e(k) is the deviation at the kth moment, is the patient target state, is the actual state at the kth moment, is the training parameter adjustment amount at the kth moment, that is, the adjustment amount of the resistance / speed of each joint. The reinforcement learning iterative optimization adopts a Q-learning algorithm to optimize a long-term training strategy, takes a patient state vector as a state space, takes resistance / speed adjustment of each joint / game difficulty switching as an action space, takes FMA-UE score change and patient state vector optimization as a core reward signal, and iteratively updates an action value function ; （5） s is the current patient state vector in equation 5; is the training parameter adjustment action; a is the learning rate, typically taken as 0.1-0.5; is the immediate reward, i.e. the FMA-UE score improvement; g is the discount factor, typically taken as 0.8-0.95; is the new state after performing the action; When a specific dimension in the patient state vector exceeds a safety threshold, the safety sentinel triggers, and the system prioritizes the adjustment amount of the PID algorithm; the specific dimension includes but is not limited to fatigue degree > 7 points or sEMG amplitude 50 μV; If the safety sentinel is not triggered, the system enters a hybrid decision mode, fusing the real-time fine-tuning amount of the PID with the long-term policy suggestion of reinforcement learning to generate the final execution instruction; After completing the execution instruction, the system updates the patient state vector through new signal acquisition, starting a cycle; The game difficulty switching is completed by the VR scenario game training module, and the action completion rate is obtained; The VR scenario game training module is based on the angle sensors on each joint to collect the angle data θ1, θ2... of each joint in real time, and the angle data enters the processing layer for action mapping and coordinate conversion, matches the angle data θ1, θ2... with the preset range of the standard rehabilitation action in the rule library of the system, and if the values of the angle data and the preset range are consistent, the game event is triggered, and the game difficulty is switched; The patient watches the virtual picture completely synchronized with his own action through the head-mounted display, and makes the next action according to the game prompt.

6. The multi-modal sensory perception and self-adaptive regulation upper limb rehabilitation robot system according to claim 1, wherein, The rehabilitation effect evaluation module includes a data quantification layer, a prediction analysis layer and an output application layer; In the data quantification layer, the data collected by the multi-source data acquisition module is mapped into the FMA-UE mapping score, i.e. the Fugl-Meyer upper limb function score, and the ADL score, i.e. the activity of daily living score; FMA-UE mapping score: a multivariate linear regression model was used to fuse the joint range of motion, grip strength and action completion rate of each joint to output the FMA-UE mapping score with a high correlation with the manual assessment R 2 = 0.

92. ADL score: according to the game completion time / correctness rate of the patient in the VR game, the ADL score is generated; In the prediction analysis layer, the deviation between the predicted value and the actual value of the FMA-UE mapping score and the ADL score accurately marks the weak link; The actual value of the FMA-UE mapping score and the ADL score is the real score calculated by the patient in the current training period: the patient completes the current training period, and the system calculates the current FMA-UE mapping score and the current ADL score based on the real-time collected multi-source data according to the formula in the document, which is an objective quantification of the current rehabilitation effect, and the calculation basis is: Current FMA-UE mapping score: FMA-UE = 0.2 x ROM standard rate + 0.3 x grip strength standard rate + 0.5 x action completion rate, error ≤ 3 points; Current ADL score: based on VR scenario game data mapping, the data comes from the real-time record of the VR scenario game training module; The predicted value of the FMA-UE mapping score and the ADL score is the current period score calculated based on the historical data; Step 1: Collect historical data: extract the historical actual FMA-UE score and ADL score of the patient in the past N periods, and record the time variable of each period; Step 2: Establish a linear regression model: take the time variable as the independent variable x, and the historical actual FMA-UE score and ADL score as the dependent variable y, fit the linear equation y = kx + b, k is the slope, representing the daily rehabilitation improvement rate; b is the intercept, representing the initial score; Step 3: Generate FMA-UE score and ADL score prediction value: replace the time variable of the current period into the equation, and the calculated y value is the FMA-UE prediction score of the current period and the ADL prediction score of the current period; In the output application layer, based on the raw data curve, FMA-UE mapping score and ADL score change table, prediction trend chart and intervention suggestion for weak links, a rehabilitation evaluation report is generated.

7. The multi-modal sensory perception and self-adaptive regulation upper limb rehabilitation robot system according to claim 1, wherein, The rehabilitation incentive feedback module is based on real-time and historical data input collected by the multi-source data acquisition module, which is continuously monitored by the conditional monitoring and judgment module, and a plurality of incentive conditions are preset in the achievement badge rule library in the conditional monitoring and judgment module; when the quantitative evaluation result or the patient state vector meets the incentive condition, the rehabilitation incentive feedback module is configured to generate and present positive feedback information to the patient; The achievement badge rule library includes but is not limited to continuous training badges and cooperation badges. The incentive condition requires that the continuous training badge requires the patient to be 7 days≥10min&achieve rate≥80%; the cooperation badge requires "score≥80 for 30s".

8. The multi-modal sensory perception and self-adaptive regulation upper limb rehabilitation robot system according to claim 1, wherein, The multi-modal spasm prediction and grading protection execution linkage module includes a multi-modal spasm prediction module and a grading protection execution module. The multi-modal spasm prediction module obtains a combined feature vector through the amplitude change rate of the sEMG signal, the sudden increase of the torque of each joint, and the sudden drop rate of the movement speed of each joint, and the combined feature vector is sent to an SVM model classifier for real-time analysis to output the risk level of the current state; the risk level includes normal, mild warning, moderate warning, and severe warning. The warning output and the specific risk level enter the grading protection execution module. The grading protection execution module receives the risk level output by the multi-modal spasm prediction module into the grading judgment module, and if it is judged as a mild risk, each damper in the magneto-rheological intelligent resistance driving system is reduced by 20-30% and the corresponding motor is reduced in speed; if it is judged as a moderate risk, the current training is stopped and switched to a passive relaxation mode; If it is judged as a severe risk, the mixed excitation structure of each damper is used to de-energize the electromagnetic coil within 0.1s, and the resistance of each damper is instantly reduced to the basic safety resistance provided by the permanent magnet and the audible and visual alarm is started.

9. The multi-modal sensing and self-adaptive regulating upper limb rehabilitation robot system according to claim 1, wherein, The basic safety protection module includes an electrical safety module, an emergency stop and power-off protection module, and an electromagnetic compatibility module. The electrical safety module has an insulation resistance of ≥100MΩ, a safety ground resistance of ≤0.1Ω, and a leakage protection device configured to limit the leakage current to ≤100μA; The emergency stop and power-off protection module uses capacitor discharge to maintain the temporary operation of the key circuit, and will not automatically start after power recovery; The electromagnetic compatibility module suppresses conducted interference through common-mode inductance + X capacitor and suppresses radiated interference through copper foil shielding.

10. The multi-modal sensing and self-adaptive regulating upper limb rehabilitation robot system according to claim 6, wherein, The FMA-UE mapping score uses a simplified estimation model, and the calculation formula is formula 1: （6） R is the compliance rate of joint range of motion ROM (%), G is the compliance rate of finger grip force (%), and C is the action completion rate (%), with a score error of ≤3 points.

Images