A virtual-real fusion multi-modal adaptive force feedback control method and system for rehabilitation training

Abstract

The application discloses a kind of virtual-real fusion multi-modal adaptive force feedback control method and system for rehabilitation training, comprising: multi-source data acquisition module real-time synchronous acquisition multiple data;Threshold self-adapting calibration module, in pre-evaluation stage, based on statistical distribution model and fatigue discount factor, self-adapting solution various threshold of current training period;Decoupling determination and classification module carries out double-dimensional feature extraction to the data obtained, solves intention index and performance index, and combines the threshold determination condition calibrated, and the real-time state of patient is decoupled classification;Multi-modal dynamic scheduling module according to decoupling state, trigger corresponding perception pump-in, cognitive degradation or physical correction mechanism, dynamically adjust parameter weight, and output instruction;Virtual-real fusion execution module executes immersive audio-visual rendering update under the premise of not interfering with electroencephalogram acquisition;While in bottom controller based on jacobian matrix and state-dependent impedance control law, drive rehabilitation robot output real-time mechanical intervention.The application realizes the accurate synchronization of virtual reality decision and physical robot execution, can smoothly switch impedance characteristics under different states, ensure the safety, compliance and effectiveness of mechanical intervention.

Description

Technical Field

[0001] This invention relates to the fields of intelligent human-computer interaction, robot control technology, and rehabilitation medicine technology, and in particular to a virtual-real fusion multimodal adaptive force feedback control method and system for rehabilitation training. Background Technology

[0002] Neurological injuries (such as stroke and traumatic brain injury) often lead to severe limb motor dysfunction. Repetitive, high-intensity exercise training based on the principle of neuroplasticity is currently recognized as the most effective rehabilitation method. In recent years, rehabilitation robots have gradually become an important vehicle for clinical rehabilitation due to their ability to provide precise and quantifiable physical interventions. Among various configurations, end-effector traction rehabilitation robots are widely used due to their good ergonomic compatibility and minimal intervention during donning and doffing. Furthermore, if a planar parallel mechanism is introduced into the end-effector rehabilitation platform, it is possible to significantly reduce the inertia of moving parts while achieving extremely high structural stiffness and end-effector positioning accuracy, thereby creating an extremely stable and safe physical interaction foundation for patients.

[0003] Traditional robot admittance or impedance control often relies on end-effector torque sensors to capture the patient's physical exertion. However, when patients are in the early stages of rehabilitation and their affected limbs are extremely weak, traditional controllers may become stuck in a dead zone due to the inability to detect physical movement, failing to provide timely assistance. To address this, brain-computer interface (BCI) technology has been introduced, forming a closed-loop brain-computer collaborative rehabilitation model. This model primarily decodes the patient's cortical motor intentions by analyzing specific rhythmic phenomena in electroencephalogram (EEG) signals, and then triggers robot assistance when the intention exceeds a certain threshold.

[0004] While brain-computer interface (BCI) theoretically opens up motor control pathways from the central nervous system to peripheral execution, the control logic of existing technologies remains rigid in real-world clinical scenarios. Current BCI rehabilitation systems generally use pre-calibrated fixed thresholds as the sole trigger for robot assistance. However, during prolonged and intensive rehabilitation training, brain-injured patients are prone to cognitive fatigue and inattention, and the absolute activation level of their cerebral cortex undergoes an irreversible natural decline over time. If the system rigidly adheres to the initial high trigger threshold, patients often become unable to overcome this threshold in the later stages of training due to brain exhaustion, thus completely losing robot assistance. This not only deviates from the original intention of "on-demand assistance" but also causes patients a strong sense of frustration, severely reducing their training adherence.

[0005] While the triggering mechanism is rigid, existing technologies also have serious misconceptions in assessing patients' motor states, conflating "subjective neural intention" with "objective physical manifestations." In clinical neurorehabilitation, patients often experience ineffective spasms or severe compensatory movements driven by strong motor intentions due to weakness or abnormal muscle tone in the target muscle groups. For example, the intention might be to extend the arm forward, but the actual manifestation is trunk tilting or abnormal shoulder shrugging. Existing systems often present a one-sided logic of "only looking at EEG and ignoring the quality of movement." Once active EEG signals are detected, they blindly trigger adaptive mechanical assistance and virtual reality scene rewards. This intervention, lacking decoupling and discernment, essentially rewards and reinforces the patient's erroneous compensatory exertion. Long-term use can easily lead to the solidification of neural circuits in abnormal motor postures.

[0006] At the end-stage intervention level, existing advanced rehabilitation platforms often fall into the technical trap of "overloading multimodal sensory feedback." In pursuit of so-called immersion, most systems indiscriminately apply high-polygon virtual reality visuals, continuous auditory rhythmic cues, and strong mechanical force feedback to patients simultaneously. For patients with already damaged brain nerves and extremely limited information processing bandwidth, this high-frequency sensory bombardment can easily trigger severe cognitive overload. Especially at critical intervention points where the patient's brain is fatigued or their movements are severely deformed, the continuously superimposed and redundant multimodal feedback not only fails to guide and correct but also exacerbates sensory conflict, becoming a huge cognitive interference.

[0007] Furthermore, existing systems generally suffer from a disconnect between virtual reality scene rendering and underlying physical haptic feedback, lacking a real-time synchronization mechanism across modalities. This invention aims to establish a virtual-real mapping architecture that deeply integrates visual, auditory, and haptic feedback. Summary of the Invention

[0008] Purpose of the invention: This invention provides a virtual-real fusion multimodal adaptive force feedback control method and system for rehabilitation training, which realizes precise synchronization between virtual reality decision-making and physical robot execution, and can smoothly switch impedance characteristics under different states to ensure the safety, compliance and effectiveness of mechanical intervention.

[0009] Technical solution: The present invention provides a virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training, comprising the following steps:

[0010] Step 1: Synchronous acquisition of multi-source heterogeneous data; real-time acquisition of continuous EEG signals collected by EEG sensors, and kinematic and dynamic state data collected by the end force / torque sensor and active joint encoder of the rehabilitation robot;

[0011] Step 2: Personalized dynamic threshold adaptive calibration; In the pre-assessment phase, by guiding patients to perform resting state and unassisted active exploratory tasks, baseline EEG and distal kinematics data are collected. Using statistical distribution models and fatigue discount factors, the intention intensity threshold and performance qualification threshold for the current training cycle are adaptively calculated.

[0012] Step 3: Intent-Performance Dual-Dimensional Feature Perception and Decoupling; Feature extraction is performed on the multi-source heterogeneous data during the training process, and real-time intent indicators representing the intensity of the patient's subjective motor intention and cognitive state are calculated, as well as real-time performance indicators representing the accuracy of objective motor execution and the degree of compensatory force exertion. Based on the threshold judgment conditions generated in Step 2, the patient's current real-time state is decoupled and classified into collaborative states of different intervention levels.

[0013] Step 4: Multimodal collaboration and adaptive mapping scheduling based on cognitive bandwidth constraints; Based on the decoupling state and real-time cognitive load index output in Step 3, construct a collaborative allocation mechanism under cognitive bandwidth constraints that characterizes the limit of patient information processing, and generate adaptive variable impedance control commands.

[0014] Step 5: Virtual-real fusion underlying execution system control; the adaptive variable impedance control command is mapped to the expected interaction force vector of the rehabilitation robot end effector in Cartesian space. Based on the real-time collected actual interaction force and robot kinematic model, the driving torque of each active joint is generated after joint space dynamics compensation, driving the mechanical body of the rehabilitation robot to perform adaptive mechanical intervention synchronized with the virtual reality scene in physical space.

[0015] Furthermore, in step 2, the adaptive calculation of the intent intensity threshold and performance pass threshold for the current training cycle specifically includes the following steps:

[0016] Step 21, Intent Intensity Threshold Calibration: Duration in Resting State Inside, energy sequences of specific frequency bands of brainwaves are collected and the mean is calculated. with standard deviation The mean baseline cognitive load was calculated by combining the current energy ratio of EEG bands in the frontal and occipital regions. Calculate the intention intensity threshold scalar with fatigue discount characteristics. :

[0017]

[0018] in, This is the confidence level adjustment coefficient. As a fatigue discount factor, The factor is a preset attenuation coefficient; as the patient's cognitive load baseline increases, the factor decreases, automatically lowering the intent trigger threshold.

[0019] Step 22, Performance Acceptance Threshold Calibration: In the unassisted active exploratory task, calculate the average baseline position deviation between the patient's actual end-effector trajectory and the reference trajectory. and average baseline normal compensatory force Construct a performance qualification threshold scalar :

[0020]

[0021] in, In order to improve the margin coefficient, and This is the corresponding error penalty coefficient;

[0022] Step 23, Calibration of Compensatory Safety Threshold: Guide the patient to apply maximum voluntary isometric contraction in the non-task-constrained direction and collect the maximum undesired normal force amplitude. Set a compensation safety threshold ,in This is for the safety factor.

[0023] Furthermore, in step 3, calculating the real-time intention index that characterizes the intensity of the patient's subjective motor intention and cognitive state specifically includes the following steps:

[0024] Step 31, Time-Frequency Domain Signal Analysis: Bandpass filtering and independent component analysis are performed on the acquired continuous EEG signals to eliminate artifacts from electrooculography (EOG) and electromyography (EMG); simultaneously, the actual planar interaction force sequence of the end effector force / torque sensor in Cartesian coordinates is extracted. With the actual position sequence of the end ;

[0025] Step 32: Motor Intent Calculation Based on Brain Load Compensation: Calculate the event-related desynchronization features of specific frequency bands in the sensorimotor cortex as the basic intent index, and integrate the energy ratio of EEG frequency bands in the frontal and occipital regions for cognitive load compensation. Finally, comprehensively calculate the scalar intent index for the current moment. :

[0026]

[0027] in, For the present Real-time task power at any given moment For the area Wave power, For the pillow area Wave power, and These are the normalized weighting coefficients. For the integration time window, For integration variables;

[0028] Step 33, Orthogonal Decomposition and Performance Evaluation of Undesirable Compensatory Force: Let the unit vector of the tangential direction of the desired trajectory be... The actual interaction force vector Perform orthogonal decomposition along the task direction to extract the normal compensating force vector. :

[0029]

[0030] Combined with actual location sequence With reference trajectory position sequence Positional deviation vector between Calculate objective scalar performance indicators :

[0031] .

[0032] Furthermore, in step 3, the patient's current real-time state is decoupled and classified into collaborative states at different intervention levels, specifically as follows:

[0033] when and When the terminal velocity obtained by taking the derivative of the terminal position with respect to time approaches zero, it is determined to be state A, which is characterized by high intention but weak performance.

[0034] when and Furthermore, when the cognitive load integral term in step 32 indicates high cognitive load, it is determined to be state B with low intention and poor performance.

[0035] when And the amplitude of the normal compensatory force Exceeding the compensation safety threshold When this occurs, it is judged as state C, where the intention is high but the performance is wrong;

[0036] when and When this occurs, it is determined to be a normal collaborative state.

[0037] Furthermore, in step 4, constructing a collaborative allocation mechanism under the cognitive bandwidth constraint that characterizes the patient's information processing limit, and generating adaptive variable impedance control commands specifically includes the following steps:

[0038] Step 41: Construct a collaborative allocation mechanism under cognitive bandwidth constraints:

[0039] Extracting real-time cognitive load indicators from step 32 That is, the forehead area Wave and Pillow Area The ratio of wave power, combined with the baseline cognitive load calibrated in step 21. Construct a cooperative suppression coefficient for coordinating the intensity of multimodal feedback. The calculation method is as follows:

[0040]

[0041] in, The preset cognitive overload penalty factor is used as a global constraint variable, with a value range of (0,1]. This factor is applied when the real-time cognitive load... Below or equal to baseline level hour, Taking the maximum value of 1 indicates sufficient cognitive bandwidth, allowing all multimodal feedback resources to be allocated; when cognitive load is overloaded, As the overload level decreases exponentially, it dynamically suppresses visual information density and auditory stimulus intensity at the global level, thereby achieving the first layer of cognitive protection at the system architecture level.

[0042] Step 42, responding to the sensor pumping and spatial excitation mapping of state A:

[0043] When in a state of high intent but weak performance, the system determines that central activation needs to be enhanced;

[0044] Auditory channel: Extracts the tangential velocity of the desired motion trajectory to generate the basic beat and introduces continuous tone modulation; the sound envelope amplitude of the auditory command. With real-time intent metrics There is a positive correlation mapping, fundamental frequency Using the chord frequency band, the calculation formula is:

[0045]

[0046] This achieves auditory stimulation that "the stronger the intention, the fuller the induced melody";

[0047] Visual Channel: Constructing a visually guiding vector field that leads to a target trajectory in AR / VR scenarios. It is generated by fusing the target tangential traction component and the current position normal correction component:

[0048]

[0049] in and As the directional weighting coefficient, it is used to render along the patient's field of vision. The arrangement of flowing particle streams or arrows, the rendering density and transparency of the particle streams are determined by a co-inhibition coefficient. With real-time performance metrics In combined modulation, the worse the performance, the more prominent the visual cues.

[0050] Step 43: Addressing cognitive degradation and information entropy reduction in state B:

[0051] When the individual is in a state of low intent and poor performance and cognitive load exceeds the limit, the overload prevention mechanism is forcibly implemented.

[0052] Cooperative inhibition coefficient Force zeroing;

[0053] Visual channel: Stripping away all scene-level rendering and guided particle flow, polygon culling rate and It exhibits a direct proportional mapping, retaining only the minimalist basic trajectory segments;

[0054] Auditory channel: Apply global silence and cut off all rhythmic induction feedback;

[0055] Force perception channel: Sets the desired impedance field to zero, degrades the system to a purely transparent and safe bottom-line mode, and provides a window for cognitive recovery;

[0056] Step 44: Addressing the auditory and physical orientation correction of errors in state C:

[0057] When in a compensatory state of high intention-misbehavior, sensory reset and physical constraints are performed;

[0058] Auditory channel: Triggers dissonant tone alerts that map to compensatory errors, extracting the amplitude of the normal compensatory force from the actual interaction force. Map it to auditory frequency offset :

[0059]

[0060] in As the baseline warning frequency, For frequency offset gain, the greater the normal compensation force, the sharper the sound frequency, forming an intuitive auditory negative feedback;

[0061] Visual and force synergy: Visual reward animations are truncated while simultaneously stimulating underlying force control. By introducing a directional projection matrix, the virtual stiffness matrix parallel to... The normal stiffness component of the direction forms a physical spatial barrier, completing the synchronous correction of "auditory warning - physical restriction".

[0062] Furthermore, in step 5, the adaptive variable impedance control command is mapped to the desired interaction force vector of the rehabilitation robot's end effector in Cartesian space, specifically including the following steps:

[0063] Step 51, Kinematic Position Modeling and Jacobian Matrix Mapping: Assume the rehabilitation robot has... There are 1 active joint, and the joint angle vector obtained by the encoder is: A nonlinear mapping from joint space to distal Cartesian task space is established using a forward kinematic model. Differentiate the position mapping equation with respect to time to establish the joint angular velocity vector. Cartesian velocity vector at the end mapping relationship ,in For the robot's real-time Jacobian matrix;

[0064] Step 52: Constructing the state-dependent Cartesian space impedance model: Calculating the desired force field vector of the control end effector. :

[0065]

[0066] in, , , The end-point reference trajectory position, velocity, and acceleration vectors are set for the virtual reality target, respectively; , , These are the calculated actual end position, velocity, and acceleration vectors, respectively. The virtual inertia matrix is ​​a preset constant. and These are the virtual damping matrix and virtual stiffness matrix, which are dynamically updated according to the control commands for states A, B, C, or normal states.

[0067] Furthermore, in step 5, the generation of the driving torque for each active joint after joint space dynamics compensation specifically involves: performing low-level dynamics compensation and torque generation based on actual human-computer interaction forces; and combining the solved Jacobian matrix. The error between the expected force in Cartesian space and the actual interaction force is mapped to the joint space, and the expected driving torque vector of each active joint is calculated. :

[0068]

[0069] in, This is the transpose of the real-time Jacobian matrix. This refers to the actual interaction force vector measured in step 31; For the compensation terms of the Coriolis force and centrifugal force of the robot, For gravity compensation, This is the compensation term for the viscosity and Coulomb friction between the motor and the mechanical structure.

[0070] A system for a virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training includes: a multi-source data acquisition module that synchronously acquires the patient's electroencephalogram (EEG) signals in real time, as well as the end-effector kinematics state and human-computer interaction force data of the rehabilitation robot;

[0071] The threshold adaptive calibration module, during the pre-evaluation phase, adaptively calculates the intent intensity threshold, performance qualification threshold, and compensation safety threshold for the current training cycle based on the statistical distribution model and fatigue discount factor.

[0072] The decoupling judgment and classification module performs two-dimensional feature extraction on the acquired data, calculates the intent index and performance index, and decouples and classifies the patient's current real-time status in combination with the calibrated threshold judgment conditions.

[0073] The multimodal dynamic scheduling module, based on the decoupling state, triggers the corresponding sensory pumping, cognitive degradation, or physical correction mechanism to dynamically adjust the parameter weights of the visual, auditory, and physical multimodal feedback channels and outputs instructions.

[0074] The virtual-real fusion execution module includes an EEG-compatible visual presentation unit and a non-interference auditory projection unit, which performs immersive audiovisual rendering updates without interfering with EEG acquisition; at the same time, the underlying controller drives the rehabilitation robot to output real-time mechanical intervention based on the Jacobian matrix and state-dependent impedance control law.

[0075] Furthermore, the rehabilitation robot includes a mechanical body with multiple active degrees of freedom; each active joint of the mechanical body is equipped with a motor that provides driving torque and a high-precision encoder that acquires the joint angle state; its end effector is rigidly connected to a six-axis force / torque sensor, which is used to directly collect the force vector of human-machine interaction in the whole space and serve as a direct feedback source for the underlying variable impedance control; the configuration of the mechanical body includes a parallel linkage mechanism or a series multi-joint robotic arm.

[0076] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) Dynamic adaptive mechanism of intent threshold: By introducing a cognitive load discount factor in the threshold calibration, the intent trigger threshold is automatically reduced as the patient's fatigue level increases, which solves the core problem that the traditional fixed threshold causes the patient to be unable to trigger assistance due to brain exhaustion during long-term training, and ensures that the patient can receive on-demand assistance throughout the training process; (2) Dual-dimensional decoupling assessment of intent-performance: By constructing intent indicators and performance indicators respectively, and combining the orthogonal decomposition extraction of normal compensatory force, the subjective motor intent and objective execution quality are separated and cross-judged, which can accurately distinguish between essentially different clinical scenarios such as "wanting to move but unable to move", "cognitive fatigue" and "false compensation", and avoid the transmission of the original intent from the root. (3) Restrained dynamic scheduling of multimodal feedback: abandoning the traditional paradigm of indiscriminate stacking of multimodal feedback, a differentiated scheduling strategy of sensory pumping, cognitive degradation, and physical correction was established - maximizing multisensory input when induction is needed, actively reducing cognitive load when fatigued, and implementing physical constraints when compensating, thus realizing the intelligent allocation of feedback resources and effectively avoiding sensory conflict and cognitive overload; (4) Safe execution mechanism of virtual-real integration: by constructing a state-dependent variable impedance control model and a dynamic compensation method based on force error, the precise synchronization of virtual reality decision-making and physical robot execution was realized, and the impedance characteristics can be smoothly switched in different states to ensure the safety, compliance and effectiveness of mechanical intervention. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the system structure of the present invention.

[0078] Figure 2 This is a diagram illustrating the personalized dynamic threshold adaptive calibration framework of the present invention.

[0079] Figure 3 This is a flowchart illustrating the intent of the present invention—a two-dimensional feature perception and decoupling classification process.

[0080] Figure 4 This is a flowchart of the multimodal adaptive feedback process of the present invention. Detailed Implementation

[0081] A virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training includes the following steps:

[0082] Step 1: Synchronous acquisition of multi-source heterogeneous data;

[0083] Real-time acquisition of multimodal physiological and physical data of patients during rehabilitation training. Physiological data includes continuous EEG signals acquired via an electroencephalogram (EEG) sensor. EEG signal acquisition employs the international 10-20 standard lead layout, focusing on acquiring EEG activity in the sensorimotor cortex regions (leads C3, C4, and Cz), while simultaneously acquiring EEG signals from the frontal region (leads Fp1, Fp2, and Fz) and the occipital region (leads O1 and O2) for cognitive load assessment. The sampling rate of the EEG signals is no less than 256 Hz, and after pre-amplification and analog-to-digital conversion, the signals are input into the host computer for processing.

[0084] The physical data includes kinematic and dynamic state data acquired through the end effector force / torque sensors and active joint encoders of the rehabilitation robot. The end effector of the rehabilitation robot is rigidly connected to a six-axis force / torque sensor for real-time acquisition of the interactive force components in three directions of Cartesian space. Interaction torque components in three directions Each active joint is equipped with a high-precision incremental encoder to acquire joint angle vectors in real time. With joint angular velocity vector ,in This refers to the number of active joints of the robot. In a specific embodiment of the present invention, the rehabilitation robot is a desktop two-degree-of-freedom parallel rehabilitation robot, therefore the number of active joints... Task space dimension .

[0085] Step 2: Personalized dynamic threshold adaptive calibration;

[0086] This step aims to address the problem that traditional fixed thresholds cannot adapt to changes in patients' cognitive fatigue. The specific process is as follows: Figure 2 As shown. In the pre-assessment phase before formal training, patients are guided to perform resting-state and unassisted active exploratory tasks, and baseline EEG and distal kinematics data are collected. Using a statistical distribution model and a fatigue discount factor, the intent intensity threshold and performance qualification threshold for the current training cycle are adaptively calculated.

[0087] Step 21: Intent intensity threshold calibration;

[0088] Duration in the resting state Within a short period (usually set to 60-120 seconds), specific frequency bands (typically 100-120 seconds) are collected from the sensorimotor cortex. Rhythm, 8~12Hz, and The energy sequence of the rhythm (13~30Hz). Let the average energy of this frequency band in the resting state be... Standard deviation is The basic intent threshold can then be expressed as: ,in This is the confidence level adjustment coefficient (generally taken as 1.5~3.0).

[0089] Furthermore, considering the impact of dynamic changes in cognitive load on EEG characteristics in brain-injured patients during prolonged training, this invention introduces a fatigue discount factor. Specifically, it calculates the frontal region... Wave power (4~8Hz) With the pillow area Wave power (8~13Hz) The ratio of the average baseline cognitive load to the mean baseline cognitive load :

[0090]

[0091] This ratio has been confirmed by neuroergonomic studies as an effective indicator for assessing cognitive load—frontal cortex. Wave enhancement and pillow region Wave suppression reflects an increase in cognitive resource input. Based on this, an intention intensity threshold scalar with fatigue discounting characteristics... The calculation is as follows:

[0092]

[0093] in The preset attenuation coefficient (within the range of 0.05 to 0.20, determined through clinical pre-experiments based on the characteristics of the patient population) is used. This is the fatigue discount factor. When the patient's cognitive load is at baseline... When increased, the factor automatically decreases, thereby lowering the threshold for triggering intent.

[0094] Step 22: Performance qualification threshold calibration;

[0095] In unassisted active exploratory tasks, patients are required to make their best effort to follow a pre-set reference trajectory. Movement, recording the actual trajectory at the end. Actual interaction force at the end Calculate the mean baseline position deviation between the patient's actual trajectory and the reference trajectory. and average baseline normal compensatory force :

[0096]

[0097] in To actively attempt the duration of the task, Let be the tangential unit direction vector of the reference trajectory (i.e., along the desired motion direction). Construct a performance-qualified threshold scalar. :

[0098]

[0099] in To improve the margin factor (usually taken as 0.5~0.8), and The corresponding error penalty coefficients (with dimensions of the reciprocal of length and the reciprocal of force, respectively, calibrated through preliminary experiments).

[0100] Step 23: Calibration of the compensation safety threshold;

[0101] Guide the patient to apply maximum voluntary isometric contraction in the non-task-constrained direction (i.e., exert maximum force in the normal direction without joint movement), and collect the maximum undesired normal force amplitude. Set a compensation safety threshold:

[0102]

[0103] in The safety factor (generally 0.3~0.5, determined according to clinical safety guidelines) is used when the normal compensatory force amplitude... Exceed At that time, the system determined that there was a dangerous compensatory force exertion behavior.

[0104] Step 3: Intent-Representation Two-Dimensional Feature Perception and Decoupling;

[0105] This step aims to achieve a separate assessment of "subjective intent" and "objective performance," and its specific process is as follows: Figure 3 As shown, features are extracted from the multi-source heterogeneous data during the training process, and real-time intention indicators representing the intensity of the patient's subjective motor intention and cognitive state, as well as real-time performance indicators representing the accuracy of objective motor execution and the degree of compensatory force exertion, are calculated. Based on the threshold judgment conditions generated in step 2, the patient's current real-time state is decoupled and classified into collaborative states of different intervention levels.

[0106] Step 31: Time-frequency domain signal analysis;

[0107] Preprocessing was performed on the acquired continuous EEG signals: first, a 4th-order Butterworth bandpass filter was used for filtering from 1 to 45 Hz; then, Independent Component Analysis (ICA) was used to eliminate artifacts in electrooculography (EOG) and electromyography (EMG). Let the preprocessed EEG signals from the sensorimotor cortex be... A short-time Fourier transform (STFT) was performed on it, using a Hamming window as the window function, with a window length of 1 second and an overlap rate of 50%, to obtain the time spectrum. .calculate Rhythm (8~12Hz) and Real-time task power of rhythm (13~30Hz) :

[0108]

[0109] Simultaneously, the actual planar interaction force sequence in the Cartesian coordinate system is obtained through an end force / torque sensor. The actual position sequence of the end effector is obtained through an encoder and a kinematic model. .

[0110] Step 32: Calculation of motor intent based on brain load compensation;

[0111] Sensorimotor cortex Rhythm and Event-related desynchronization (ERD) of rhythms has been widely confirmed as a reliable neural marker of motor intention. Calculation of baseline intention indices:

[0112]

[0113] in This is the reference power for the sensorimotor cortex region in the resting state (taken from the resting state data in step 21). A positive value indicates the presence of a movement intention, and a larger value indicates a stronger intention.

[0114] However, relying solely on ERD indicators has limitations: cognitive fatigue in brain-injured patients leads to a decrease in overall cortical activation levels. The decrease may stem from both exercise intention and fatigue. Therefore, this invention introduces a cognitive load compensation term. The frontal lobe is defined. Wave power With the pillow area Wave power The ratio is the real-time cognitive load index. :

[0115]

[0116] Real-time cognitive load relative to baseline The increment (taken from step 21) is used as a penalty term to comprehensively calculate the scalar intention index at the current moment. :

[0117]

[0118] in and Normalized weighting coefficients (satisfying) The specific value was determined through preliminary experiments and optimization. The integration time window (usually 5-10 seconds) is used. It is the integral variable.

[0119] Step 33: Orthogonal decomposition and performance evaluation of undesirable compensatory power;

[0120] Let the unit vector of the tangential direction of the desired trajectory be... (from reference trajectory) (derived by taking the time derivative and normalizing), the actual interaction force vector Perform orthogonal decomposition along the task direction to extract the normal compensating force vector. :

[0121]

[0122] The geometric meaning of this decomposition is: The projection scalar of the interaction force in the tangential direction of the task, multiplied by Then the tangential component is obtained; from Subtracting the tangential component from the tangential component yields the normal component perpendicular to the mission direction. .

[0123] Combined with actual location sequence With reference trajectory position sequence Positional deviation vector between Calculate objective scalar performance indicators :

[0124]

[0125] in , Consistent with the penalty coefficient in step 22.

[0126] Step 34: Decoupling state classification based on dual thresholds;

[0127] Based on the threshold generated in step 2 , and The patient's current real-time state is decoupled and classified into one of the following four cooperative states:

[0128] when and And the derivative of the end position with respect to time The obtained terminal velocity approaches zero (i.e.) ,in When the preset speed threshold is set (generally 0.01~0.02 m / s), it is judged as state A (high intent - zero / weak performance).

[0129] when and And the cognitive load integral term in step 32 Exceeding the preset cognitive fatigue threshold (Generally, a baseline is taken) When the percentage is 20% to 30%, it is classified as State B (low intention - poor performance).

[0130] when And the amplitude of the normal compensatory force Exceeding the compensation safety threshold When this occurs, the state is determined to be C (high intent - wrong performance).

[0131] when and When this occurs, it is determined to be a normal collaborative state.

[0132] Step 4: Multimodal collaboration and adaptive mapping scheduling based on cognitive bandwidth constraints;

[0133] This step aims to achieve "restrained and intelligent" feedback resource allocation, avoiding sensory conflict and cognitive overload. Based on the decoupling state output from step 3, the system's host computer dynamically adjusts the parameter weights of the visual, auditory, and force multimodal feedback channels at the output end; multimodal collaborative control for different states, such as... Figure 4 As shown, by differentially enabling, weakening or disabling specific feedback modes, corresponding virtual reality visual rendering instructions, auditory rhythm instructions and state-dependent adaptive variable impedance control instructions are generated.

[0134] To address the interference issue between traditional multimedia devices and high-density EEG caps in the physical space of the head, and to maximize the patient's immersive experience during rehabilitation, this invention employs an EEG-compatible, non-interference design for the audiovisual feedback hardware. In the visual channel, this embodiment uses lightweight AR glasses, utilizing their nose pads and ear hooks to avoid physical pressure on core EEG electrodes in the parietal and occipital regions of the cerebral cortex. In the auditory channel, bone conduction headphones or external active speakers placed on both sides of the rehabilitation platform are used. The host computer outputs the synthesized digital audio signal after digital-to-analog conversion and power amplification via an audio interface.

[0135] Step 41: Construct a collaborative allocation mechanism under cognitive bandwidth constraints;

[0136] The host computer control program extracts the real-time cognitive load index updated at a specific sampling rate in step 32. Construct a cooperative suppression coefficient for coordinating the intensity of multimodal feedback. :

[0137]

[0138] in, The average baseline cognitive load pre-stored in system memory in step 21; This is a cognitive overload penalty factor. In this embodiment, The value of is not a fixed constant, but is automatically generated by the system during the pre-assessment phase by fitting the variance of the patient's EEG rhythm energy fluctuations, to characterize the physiological sensitivity of that specific patient to information overload. The calculated value... As a global software constraint variable, it is sent to the visual rendering engine and audio synthesis module in real time through inter-process communication.

[0139] Step 42: Responding to the sensory pumping and spatial excitation mapping of state A;

[0140] In this state, the patient "want to move but can't," and the system invokes a strategy to maximize sensory stimulation.

[0141] In terms of the auditory channel, the host computer's audio synthesis module abandons fixed-frequency cues, extracts the tangential velocity of the reference trajectory to generate the basic beat, and introduces a continuous tone modulation algorithm. The amplitude of the sound envelope of the audio output... Calculated using the following mapping:

[0142]

[0143] in, This refers to the maximum safe sound pressure level threshold set in the configuration file based on clinical audiological standards. This represents the historical peak values ​​of intent metrics recorded by the system's sliding window. This mapping ensures that "the stronger the intent, the louder the sound," and the overall output is affected by... constraint.

[0144] In terms of the visual channel, the rendering engine in AR glasses constructs a visually guiding vector field that tends towards the target trajectory. :

[0145]

[0146] in, and The weighting coefficients are calculated and provided in real time by the system trajectory planning module and the forward kinematics module, respectively. and The ratio is dynamically allocated by the local curvature of the trajectory. The rendering engine relies on... The direction of particle flow or dynamic arrows is generated, and its rendering generation rate (i.e., visual density in the image) is related to... It exhibits a linear positive correlation mapping.

[0147] Regarding the force sensing channel, the host computer sends an impedance modification command to the lower-level controller, adjusting the damping parameters. Reduced to 50%~70% of the nominal value, and superimposed with the micro-vibration excitation force vector generated by the underlying controller. .

[0148] Step 43: Address the cognitive degradation and information entropy reduction in state B (low intent - poor performance).

[0149] In this state, the patient's cognitive resources are exhausted. The system's main control program forcibly changes the global variables. Set to zero.

[0150] In the visual pass, the rendering engine relies on... The excessive force calculation determines the polygon culling rate, disables the particle flow system, culls scene textures, and degrades to a minimalist wireframe display mode. In the auditory channel, the audio interface triggers a hardware-level mute interrupt, cutting off all acoustic induction. In the force channel, the lower-level controller will input the desired force field. Setting it to zero allows the underlying mechanical body to enter a zero-force field transparent support mode.

[0151] Step 44: Address the auditory and physical orientation correction of errors in state C;

[0152] In this state, there is a dangerous compensation mechanism, and the system must perform multimodal correction within milliseconds.

[0153] In the auditory channel, the error sonification algorithm is triggered. The system extracts the normal compensating force amplitude resolved from the torque sensor. The alarm frequency is mapped using the voltage-controlled oscillator (VCO) principle. :

[0154]

[0155] Frequency offset gain Ensure that the compensatory capacity approaches the safety threshold At this point, the audio sharply shifts to a high-frequency, jarring region, creating negative feedback. Simultaneously, the visual channel truncates the reward animation and flashes a red warning border.

[0156] In the force sensing channel, the underlying real-time controller extracts the unit vector of the normal compensating force direction. Instantly reconstruct the Cartesian space virtual stiffness matrix within one control cycle (typically ≤1ms). :

[0157]

[0158] in, The stiffness surge scalar value set for the system. This orthogonal projection matrix. The system ensures that it generates extremely high impedance stiffness only in the normal direction of the patient's incorrect force application, producing a strong resistive driving torque; while maintaining its original compliant control characteristics in the tangential direction of the desired movement. At the same time, the underlying controller drives the motor to generate a tactile vibration warning of 2~5Hz by superimposing a high-frequency disturbance current.

[0159] Step 5: Control of the underlying execution system for virtual-real integration;

[0160] This step aims to map decisions made in virtual reality to precise, compliant mechanical interventions in the physical world, as illustrated in the control block diagram below. Figure 4 As shown.

[0161] Step 51: Kinematic position modeling and Jacobian matrix mapping;

[0162] Assume the rehabilitation robot has One active joint, in this specific embodiment A five-bar parallel mechanism is adopted. Let the joint angle vector obtained by the encoder be... A nonlinear mapping from joint space to distal Cartesian task space is established using a forward kinematic model. .

[0163] For a five-bar linkage, its forward kinematics can be analytically expressed as: Let the coordinates of the two active joints be... and ,in The base link length is given, and the input angles of the two active joints are respectively... and The two active linkages are of equal length ( The two driven links are of equal length. ), then the end position It can be given in the following closed form:

[0164]

[0165] Among auxiliary variables and Defined as:

[0166]

[0167] Differentiate the position mapping equation with respect to time to establish the joint angular velocity vector. Cartesian velocity vector at the end Mapping relationship:

[0168]

[0169] in The real-time Jacobian matrix for the robot.

[0170] Step 52: Construction of the state-dependent Cartesian space impedance model;

[0171] To achieve compliant assistance and constraint of patient movement, this invention employs an impedance control paradigm based on position error to calculate the desired force field vector of the control end effector. :

[0172]

[0173] in , , The end-point reference trajectory position, velocity, and acceleration vectors are set for the virtual reality target, respectively; , , These are the calculated actual end position, velocity, and acceleration vectors, respectively. The virtual inertia matrix is ​​a preset constant (usually set as a diagonal matrix, with diagonal elements ranging from 0.5 to 2.0 kg). and These are the virtual damping matrix and virtual stiffness matrix, which are dynamically updated based on the different state control commands in step 4. The update rules are as follows:

[0174] Under normal collaborative conditions , The system provides nominal compliance assistance.

[0175] In state A (sensory pumping), This reduces motion resistance and simultaneously feeds forward the micro-vibration excitation force into the total desired force field. .

[0176] In state B (cognitive degradation), to achieve a safe and transparent chassis mode, the reference expectation force field set by the upper layer is... Set directly to zero.

[0177] In state C (physical correction), the stiffness matrix of the orientation correction strategy in step 44 is updated as follows: .

[0178] Step 53: Joint spatial dynamics compensation and torque generation based on actual interactive forces;

[0179] First, a dynamic model of the rehabilitation robot is constructed. For a five-bar parallel mechanism, its dynamic equation can be expressed as:

[0180]

[0181] in The inertia matrix, Let the Coriolis force and centrifugal force be vectors. The gravity vector (in this embodiment, the robot only moves in the horizontal plane, therefore...) ), This is the active joint driving torque vector. The equivalent torque is the end effector force mapped to the joint space.

[0182] Then, combine the Jacobian matrix obtained in step 51. The error between the expected force and the actual interaction force in Cartesian space is mapped to the joint space, and dynamic feedforward compensation is introduced to calculate the expected driving torque vector of each active joint. :

[0183]

[0184] in This represents the actual interactive force vector measured by the end force / torque sensor. The viscous and Coulomb friction compensation terms between the motor and the mechanical structure are modeled as follows:

[0185]

[0186] in This is the viscous friction coefficient matrix. The two vectors are the Coulomb friction amplitude vectors and can be pre-calibrated through system identification experiments.

[0187] Step 54: Bottom-level motor drive and closed-loop control;

[0188] The desired driving torque vector calculated in step 53 As the input command for the current loop, it is mapped to the desired current of each joint motor through the following relationship:

[0189]

[0190] in This refers to the reduction ratio of the motor reducer. This represents the torque constant of the motor. The underlying servo driver adopts a three-loop control architecture, with the current loop sampling frequency not less than 10kHz and the speed and position loop sampling frequencies at 1kHz. Simultaneously, the encoder signal is fed back to the controller at a frequency not less than 1kHz, forming a fully closed-loop control to ensure that the end effector performs adaptive mechanical intervention synchronized with the virtual reality scene in physical space.

[0191] like Figure 1 As shown, a virtual-real fusion multimodal adaptive force feedback control system for rehabilitation training includes: a multi-source data acquisition module, a threshold adaptive calibration module, a decoupling judgment and classification module, a multimodal dynamic scheduling module, and a virtual-real fusion execution module. In a specific embodiment, the underlying rehabilitation robot platform of the system adopts a desktop two-degree-of-freedom five-bar parallel mechanism as the physical actuator.

[0192] The multi-source data acquisition module is used to collect the patient's electroencephalogram (EEG) signals in real time, as well as the end-effector kinematics and human-machine interaction force data of the rehabilitation robot.

[0193] The threshold adaptive calibration module is used to adaptively calculate the intent intensity threshold, performance qualification threshold, and compensation safety threshold for the current training cycle based on the statistical distribution model and fatigue discount factor during the pre-evaluation stage.

[0194] The decoupling determination and classification module is used to extract two-dimensional features from the acquired data, calculate intent indicators and performance indicators, and decouple and classify the patient's current real-time status in combination with the calibrated threshold determination conditions.

[0195] The multimodal dynamic scheduling module is used to trigger corresponding sensory pumping, cognitive degradation or physical correction mechanisms according to the decoupling state, dynamically adjust the parameter weights of the visual, auditory and force multimodal feedback channels, and output instructions.

[0196] The virtual-real fusion execution module is used to perform audiovisual rendering updates in the virtual reality host computer, and at the same time, in the underlying controller, it drives the rehabilitation robot to output real-time mechanical intervention based on the Jacobian matrix and the state-dependent impedance control law.

[0197] This invention solves the core problems mentioned in the background art, such as threshold rigidity, confusion of intention expression, and redundancy of multimodal feedback, and realizes precise, restrained and intelligent virtual-real fusion multimodal adaptive feedback rehabilitation training.

Claims

1. A virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training, characterized in that, Includes the following steps: Step 1: Synchronous acquisition of multi-source heterogeneous data; real-time acquisition of continuous EEG signals collected by EEG sensors, and kinematic and dynamic state data collected by the end force / torque sensor and active joint encoder of the rehabilitation robot; Step 2: Personalized dynamic threshold adaptive calibration; During the pre-assessment phase, by guiding patients to perform resting and unassisted active tackling tasks, baseline EEG and distal kinematics data are collected. Using statistical distribution models and fatigue discount factors, the intent intensity threshold and performance qualification threshold for the current training cycle are adaptively calculated. Step 3: Intent-Performance Dual-Dimensional Feature Perception and Decoupling; Feature extraction is performed on the multi-source heterogeneous data during the training process, and real-time intent indicators representing the intensity of the patient's subjective motor intention and cognitive state are calculated, as well as real-time performance indicators representing the accuracy of objective motor execution and the degree of compensatory force exertion. Based on the threshold judgment conditions generated in Step 2, the patient's current real-time state is decoupled and classified into collaborative states of different intervention levels. Step 4: Multimodal collaboration and adaptive mapping scheduling based on cognitive bandwidth constraints; Based on the decoupling state and real-time cognitive load index output in Step 3, construct a collaborative allocation mechanism under cognitive bandwidth constraints that characterizes the limit of patient information processing, and generate adaptive variable impedance control commands. Step 5: Virtual-real fusion underlying execution system control; the adaptive variable impedance control command is mapped to the expected interaction force vector of the rehabilitation robot end effector in Cartesian space. Based on the real-time collected actual interaction force and robot kinematic model, the driving torque of each active joint is generated after joint space dynamics compensation, driving the mechanical body of the rehabilitation robot to perform adaptive mechanical intervention synchronized with the virtual reality scene in physical space.

2. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, Step 2, the adaptive calculation of the intent intensity threshold and performance pass threshold for the current training cycle specifically includes the following steps: Step 21, Intent Intensity Threshold Calibration: Duration in Resting State Inside, energy sequences of specific frequency bands of brainwaves are collected and the mean is calculated. with standard deviation The mean baseline cognitive load was calculated by combining the current energy ratio of EEG bands in the frontal and occipital regions. Calculate the intention intensity threshold scalar with fatigue discount characteristics. : in, This is the confidence level adjustment coefficient. As a fatigue discount factor, The factor is a preset attenuation coefficient; as the patient's cognitive load baseline increases, the factor decreases, automatically lowering the intent trigger threshold. Step 22, Performance Acceptance Threshold Calibration: In the unassisted active exploratory task, calculate the average baseline position deviation between the patient's actual end-effector trajectory and the reference trajectory. and average baseline normal compensatory force Construct a performance qualification threshold scalar : in, In order to improve the margin coefficient, and This is the corresponding error penalty coefficient; Step 23, Calibration of Compensatory Safety Threshold: Guide the patient to apply maximum voluntary isometric contraction in the non-task-constrained direction and collect the maximum undesired normal force amplitude. Set a compensation safety threshold ,in This is for the safety factor.

3. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, Step 3, calculating the real-time intention index that characterizes the intensity of the patient's subjective motor intention and cognitive state, specifically includes the following steps: Step 31, Time-Frequency Domain Signal Analysis: Bandpass filtering and independent component analysis are performed on the acquired continuous EEG signals to eliminate artifacts from electrooculography (EOG) and electromyography (EMG); simultaneously, the actual planar interaction force sequence of the end effector force / torque sensor in Cartesian coordinates is extracted. With the actual position sequence of the end ; Step 32: Motor Intent Calculation Based on Brain Load Compensation: Calculate the event-related desynchronization features of specific frequency bands in the sensorimotor cortex as the basic intent index, and integrate the energy ratio of EEG frequency bands in the frontal and occipital regions for cognitive load compensation. Finally, comprehensively calculate the scalar intent index at the current moment. : in, For the present Real-time task power at any given moment For the area Wave power, For the pillow area Wave power, and These are the normalized weighting coefficients. For the integration time window, For integration variables; Step 33, Orthogonal Decomposition and Performance Evaluation of Undesirable Compensatory Force: Let the unit vector of the tangential direction of the desired trajectory be... The actual interaction force vector Perform orthogonal decomposition along the task direction to extract the normal compensating force vector. : Combined with actual location sequence With reference trajectory position sequence Positional deviation vector between Calculate objective scalar performance indicators : 。 4. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 3, characterized in that, In step 3, the patient's current real-time state is decoupled and classified into collaborative states at different intervention levels, specifically as follows: when and When the terminal velocity obtained by taking the derivative of the terminal position with respect to time approaches zero, it is determined to be state A, which is characterized by high intention but weak performance. when and Furthermore, when the cognitive load integral term in step 32 indicates high cognitive load, it is determined to be state B with low intention and poor performance. when And the amplitude of normal compensatory force Exceeding the compensation safety threshold When this occurs, it is judged as state C, where the intention is high but the performance is wrong; when and When this occurs, it is determined to be a normal collaborative state.

5. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, In step 4, constructing a collaborative allocation mechanism under the cognitive bandwidth constraint that characterizes the patient's information processing limit, and generating adaptive variable impedance control commands specifically includes the following steps: Step 41: Construct a collaborative allocation mechanism under cognitive bandwidth constraints: Extracting real-time cognitive load indicators Construct a cooperative suppression coefficient for coordinating the intensity of multimodal feedback. : in, As a preset cognitive overload penalty factor, As the baseline cognitive load, this coefficient serves as a global constraint, dynamically adjusting the density of visual information and the intensity of auditory stimuli; Step 42, responding to the sensor pumping and spatial excitation mapping of state A: When in a state of high intent but weak performance, the system determines that central activation needs to be enhanced; Auditory channel: Extracts the tangential velocity of the desired motion trajectory to generate the basic beat and introduces continuous tone modulation; the sound envelope amplitude of the auditory command. With real-time intent metrics There is a positive correlation mapping, fundamental frequency Using the chord frequency band, the calculation formula is: This achieves auditory stimulation that "the stronger the intention, the fuller the induced melody"; Visual Channel: Constructing a visually guiding vector field that leads to a target trajectory in AR / VR scenarios. It is generated by fusing the target tangential traction component and the current position normal correction component: in and As the directional weighting coefficient, it is used to render along the patient's field of vision. The arrangement of flowing particle streams or arrows, the rendering density and transparency of the particle streams are determined by a co-inhibition coefficient. With real-time performance metrics In combined modulation, the worse the performance, the more prominent the visual cues. Step 43: Addressing cognitive degradation and information entropy reduction in state B: When the individual is in a state of low intent and poor performance and cognitive load exceeds the limit, the overload prevention mechanism is forcibly implemented. Cooperative inhibition coefficient Force zeroing; Visual channel: Stripping away all scene-level rendering and guided particle flow, polygon culling rate and It exhibits a direct proportional mapping, retaining only the minimalist basic trajectory segments; Auditory channel: Apply global silence and cut off all rhythmic induction feedback; Force perception channel: Sets the desired impedance field to zero, degrades the system to a purely transparent and safe bottom-line mode, and provides a window for cognitive recovery; Step 44: Addressing the auditory and physical orientation correction of errors in state C: When in a compensatory state of high intention-misbehavior, sensory reset and physical constraints are performed; Auditory channel: Triggers dissonant tone alerts that map to compensatory errors, extracting the amplitude of the normal compensatory force from the actual interaction force. Map it to auditory frequency offset : in As the baseline warning frequency, For frequency offset gain, the greater the normal compensation force, the sharper the sound frequency, forming an intuitive auditory negative feedback; Visual and force synergy: Visual reward animations are truncated while simultaneously stimulating underlying force control. By introducing a directional projection matrix, the virtual stiffness matrix parallel to... The normal stiffness component in the direction forms a physical spatial barrier, completing the synchronous correction of "auditory warning - physical restriction".

6. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, In step 5, the adaptive variable impedance control command is mapped to the desired interaction force vector of the rehabilitation robot's end effector in Cartesian space, specifically including the following steps: Step 51, Kinematic Position Modeling and Jacobian Matrix Mapping: Assume the rehabilitation robot has... There are 1 active joint, and the joint angle vector obtained by the encoder is: A nonlinear mapping from joint space to distal Cartesian task space is established using a forward kinematic model. Differentiate the position mapping equation with respect to time to establish the joint angular velocity vector. Cartesian velocity vector at the end mapping relationship ,in For the robot's real-time Jacobian matrix; Step 52: Constructing the state-dependent Cartesian space impedance model: Calculating the desired force field vector of the control end effector. : in, , , The end-point reference trajectory position, velocity, and acceleration vectors are set for the virtual reality target, respectively; , , These are the calculated actual end position, velocity, and acceleration vectors, respectively. The virtual inertia matrix is ​​a preset constant. and These are the virtual damping matrix and virtual stiffness matrix, which are dynamically updated according to the control commands for states A, B, C, or normal states.

7. The virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, In step 5, the driving torque of each active joint is generated after joint space dynamics compensation as follows: Based on the actual human-computer interaction forces, low-level dynamics compensation and torque generation are performed, combined with the solved Jacobian matrix. The error between the expected force in Cartesian space and the actual interaction force is mapped to the joint space, and the expected driving torque vector of each active joint is calculated. : in, This is the transpose of the real-time Jacobian matrix. This refers to the actual interaction force vector measured in step 31; For the compensation terms of the Coriolis force and centrifugal force of the robot, For gravity compensation, This is the compensation term for the viscosity and Coulomb friction between the motor and the mechanical structure.

8. A system for implementing the virtual-real fusion multimodal adaptive force feedback control method for rehabilitation training as described in claim 1, characterized in that, include: The multi-source data acquisition module collects the patient's electroencephalogram (EEG) signals in real time, as well as the end-effector kinematics and human-computer interaction force data of the rehabilitation robot. The threshold adaptive calibration module, during the pre-evaluation phase, adaptively calculates the intent intensity threshold, performance qualification threshold, and compensation safety threshold for the current training cycle based on the statistical distribution model and fatigue discount factor. The decoupling judgment and classification module performs two-dimensional feature extraction on the acquired data, calculates the intent index and performance index, and decouples and classifies the patient's current real-time status in combination with the calibrated threshold judgment conditions. The multimodal dynamic scheduling module, based on the decoupling state, triggers the corresponding sensory pumping, cognitive degradation, or physical correction mechanism to dynamically adjust the parameter weights of the visual, auditory, and physical multimodal feedback channels and outputs instructions. The virtual-real fusion execution module includes an EEG-compatible visual presentation unit and a non-interference auditory projection unit, which performs immersive audiovisual rendering updates without interfering with EEG acquisition; at the same time, the underlying controller drives the rehabilitation robot to output real-time mechanical intervention based on the Jacobian matrix and state-dependent impedance control law.

9. The virtual-real fusion multimodal adaptive force feedback control system for rehabilitation training as described in claim 8, characterized in that, The rehabilitation robot includes a mechanical body with multiple active degrees of freedom; each active joint of the mechanical body is equipped with a motor that provides driving torque and a high-precision encoder that acquires the joint angle state. Its end effector is rigidly connected to a six-axis force / torque sensor, which is used to directly acquire the force vector of human-machine interaction in the whole space and serve as a direct feedback source for the underlying variable impedance control; the configuration of the mechanical body includes a parallel linkage mechanism or a series multi-joint robotic arm.

Images