Rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction

Abstract

This application relates to the field of neurorehabilitation engineering and discloses a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction. The system's training subsystem collects EEG signals from the affected side, peripheral electromyography (EMG) signals from the healthy side, and peripheral kinematic signals, constructing a "fusion-type teacher signal" of peripheral EMG and peripheral kinematic signals. A deep learning model is then trained to learn the mapping from EEG signals to this fusion signal. The system's application subsystem, based on real-time EEG signals, simultaneously predicts coordinated EMG patterns and coordinated movement trajectories using the model. The coordinated EMG patterns are used to generate feedforward instructions for functional electrical stimulation to reproduce physiological coordination patterns; the coordinated movement trajectory serves as a dynamic target trajectory, compared with the actual kinematic signals from the affected side, and the trajectory error is calculated to generate feedback correction instructions for functional electrical stimulation. This application achieves natural, precise, and personalized motor assistance.

Description

Technical Field

[0001] This application relates to the fields of neurorehabilitation engineering, brain-computer interface (BCI) and artificial intelligence, and in particular to an intelligent rehabilitation system based on functional electrical stimulation (FES). Background Technology

[0002] Brain-computer interface-functional electrical stimulation (BCI-FES) technology is an important direction in the field of neurorehabilitation. Its basic idea is to decode the patient's brain's motor intentions through signals such as electroencephalography (EEG), triggering functional electrical stimulation (FES) to stimulate the relevant muscles on the affected side to assist or compensate for motor tasks such as grasping and walking. This type of technology has significant application potential in the recovery of upper and lower limb function in patients with stroke-induced hemiplegia and spinal cord injuries.

[0003] However, existing BCI-FES systems still face several technical bottlenecks in practical applications:

[0004] First, the teacher signal is incomplete. During the training phase, some systems do not use the teacher signal and directly adopt a preset fixed stimulus program; others only use a single modality of physiological signal (such as electromyography of the unaffected side only, or only the movement trajectory) as the learning target. A single modality of teacher signal cannot simultaneously provide two layers of information: "how the muscle is activated" and "what kind of movement trajectory is generated after activation." This results in insufficient mapping relationships learned by the model, and the generated FES control commands are prone to problems such as disordered muscle activation timing, unreasonable relative intensity, and mismatch with the expected movement trajectory.

[0005] Secondly, there is a lack of real-time feedback correction mechanisms. Existing systems mostly employ a pure feedforward control structure, directly generating FES commands based on the intention signal and applying them to the affected limb, with little real-time monitoring and closed-loop adjustment of the actual movement effects after stimulation. When muscle fatigue, changes in electrode-skin contact impedance, or decoding deviations caused by fluctuations in attention occur, the system still outputs fixed or nearly fixed stimulation parameters, unable to dynamically adjust according to actual movement deviations. This leads to problems such as insufficient grip strength and unstable joint mobility, affecting the stability and success rate of rehabilitation training.

[0006] Furthermore, predicted motion trajectories have not been effectively used for closed-loop control. Some studies have attempted to learn electromyography and motion trajectories simultaneously during the training phase; however, in application, the prediction results are often used separately or only as auxiliary assessments. The predicted trajectory is not used as a dynamic reference target that changes over time, nor is it compared in real-time with the actual motion trajectory of the affected side to construct a trajectory error feedback loop. The lack of closed-loop control targeting the predicted trajectory makes it difficult for the system to achieve a synergistic effect of "feedforward drive + trajectory tracking feedback," thus limiting the naturalness, accuracy, and robustness of the movements.

[0007] Furthermore, individualized adaptation is insufficient. Many existing BCI-FES systems still rely on universal, standardized stimulation procedures, using similar parameters and activation modes for different patients. This makes it difficult to fully consider the differences among patients in muscle strength, joint range of motion, muscle distribution, movement habits, and injury severity. The lack of personalized adaptation can easily lead to stimulation that is too weak or too strong, and muscle synergy patterns that do not conform to the patient's natural movement patterns. This not only reduces rehabilitation efficiency but may also reinforce abnormal movement patterns, affecting patient training adherence.

[0008] In summary, the existing BCI-FES rehabilitation system has significant shortcomings in teacher signal design, real-time feedback correction, closed-loop utilization of prediction results, and individualized adaptation. These shortcomings are easily manifested as rigid control, unnatural movements, low personalization, and poor robustness, and new technical solutions are urgently needed to improve it. Summary of the Invention

[0009] The purpose of this application is to provide a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction to solve the problems mentioned in the background art.

[0010] This application discloses a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction, comprising:

[0011] The training subsystem includes:

[0012] The training signal acquisition module is used to simultaneously acquire the central motor intention signal (EEG) of the cerebral hemisphere corresponding to the affected limb, the peripheral electromyographic signal (sEMG) of the healthy limb, and the peripheral kinematic signal (Kinematics) of the healthy limb.

[0013] The model training module is used to train a deep learning model based on the EEG, sEMG and Kinematics acquired by the training signal acquisition module, wherein the sEMG and Kinematics are used as fusion teacher signals, so that the deep learning model can simultaneously predict the corresponding coordinated electromyographic pattern P-sEMG and coordinated motion trajectory P-Kinematics based on the input EEG.

[0014] The application subsystem includes:

[0015] The application signal acquisition module is used to acquire the EEG of the affected side of the brain and the actual kinematic signals of the affected side of the limbs in real time (A-Kinematics).

[0016] The real-time decoding module is used to input the EEG acquired by the application signal acquisition module into the deep learning model trained by the model training module, and decode it in real time to obtain the P-sEMG and the P-Kinematics.

[0017] The feedforward control module is used to generate feedforward control commands for multi-channel functional electrical stimulation (FES) based on the P-sEMG obtained by the real-time decoding module.

[0018] The error calculation module is used to compare the P-Kinematics obtained by the real-time decoding module as the target trajectory that changes over time with the A-Kinematics acquired by the application signal acquisition module, and calculate the trajectory error.

[0019] The feedback control module is used to generate the feedback correction command of the FES based on the trajectory error calculated by the error calculation module.

[0020] The instruction fusion module is used to fuse the feedforward control instruction generated by the feedforward control module and the feedback correction instruction generated by the feedback control module to generate the final FES instruction;

[0021] The stimulus execution module is used to output the final FES instruction generated by the instruction fusion module.

[0022] In a preferred embodiment, the model training module uses the sEMG and the kinematics as fused teacher signals, wherein the sEMG reflects the timing and intensity of muscle activation, and the kinematics reflects the actual spatiotemporal trajectory of limb movement, thereby providing the deep learning model with a physiological blueprint that includes muscle drive patterns and movement execution results.

[0023] In a preferred embodiment, the deep learning model includes a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a Transformer network, for processing the temporal mapping relationship from the EEG to the P-sEMG and the P-Kinematics.

[0024] In a preferred embodiment, the deep learning model is trained using a hybrid loss function in the model training module. :

[0025]

[0026] in, This represents the mean square error function. and These are the weighting coefficients. The coordinated electromyographic patterns predicted by the deep learning model. The sEMG acquired by the training signal acquisition module. The coordinated motion trajectory predicted by the deep learning model. The Kinematics acquired by the training signal acquisition module.

[0027] In a preferred embodiment, the training subsystem further includes a preprocessing module for preprocessing the acquired signals:

[0028] Bandpass filtering, notch filtering, and independent component analysis (ICA) were performed on the EEG to remove artifacts;

[0029] The sEMG is filtered, rectified, and low-pass filtered to extract the envelope signal;

[0030] The attitude calculation of the Kinematics is performed to obtain Euler angles or quaternions.

[0031] In a preferred embodiment, the model training module is used to align the preprocessed EEG, the sEMG envelope signal, and the Euler angles or quaternions by timestamps and divide them into time windows, each time window having a length of 100 milliseconds to 500 milliseconds.

[0032] In a preferred embodiment, the feedforward control module is used to modulate the pulse width or amplitude of the feedforward control command according to the amplitude of the P-sEMG, wherein the larger the amplitude of the P-sEMG, the greater the stimulation intensity of the corresponding FES channel.

[0033] In a preferred embodiment, the feedback control module includes a PID controller, an adaptive controller, or a model predictive controller for generating the feedback correction command based on the trajectory error, wherein the PID controller includes a proportional (P), an integral (I), and a derivative (D) gain to achieve tracking of the P-Kinematics.

[0034] In a preferred embodiment, the instruction fusion module is used to fuse the feedforward control instruction and the feedback correction instruction through additive fusion or gain modulation to generate the final FES instruction.

[0035] Specifically, the additive fusion is as follows:

[0036] For the Each FES channel, at time... The final instruction is:

[0037]

[0038] in, For the first The final FES command for each channel, The feedforward control command generated based on the P-sEMG, The feedback correction command is generated based on the trajectory error.

[0039] This application, through the above-mentioned technical solution, has achieved technological advancements in many aspects compared to existing technologies.

[0040] First, during the training phase, the deep learning model is trained by simultaneously collecting EEG signals of central motor intent from the cerebral hemisphere corresponding to the affected limb, sEMG signals of peripheral electromyography (EMG) from the healthy limb, and kinematic signals of the healthy limb. The sEMG and kinematic signals are then used as a fused teacher signal to train the model, enabling it to learn a more complete mapping relationship from central intent to peripheral physiological patterns. This fused teacher signal approach overcomes the limitations of existing technologies that use only a single modality signal or no teacher signal at all. It allows the deep learning model to learn not only "how muscles should be activated" (via sEMG) but also "what kind of movement trajectory this activation will produce" (via kinematics). Since sEMG reflects the timing and intensity of muscle activation (i.e., muscle drive patterns), while kinematics reflects the actual spatiotemporal trajectory of limb movement (i.e., movement execution results), their fusion provides the model with a more complete physiological blueprint encompassing both "cause" and "effect," thereby enhancing the model's understanding of human coordinated movement patterns. This enhanced understanding helps improve the quality of predicted outputs during the application phase. Specifically, the predicted coordinated electromyography (P-sEMG) patterns and the predicted coordinated motion trajectories (P-Kinematics) can more accurately reflect the physiological state of the unaffected limb when performing the target action.

[0041] Secondly, employing deep learning models capable of handling temporal mapping relationships, such as Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), Gated Recurrent Units (GRUs), or Transformer networks, enables the system to effectively capture the inherent long-term temporal dependencies in human motion signals. Human motion intention (EEG), muscle activation (sEMG), and limb movement (Kinematics) are all highly temporally correlated dynamic processes; the current state often depends on historical states over a period of time. In particular, LSTMs or GRUs, through gating mechanisms, can selectively memorize and forget information, thus considering the progressive nature of motion intention, the sequential nature of muscle activation, and the continuity of movement trajectories when learning the mapping from EEG to P-sEMG and P-Kinematics. This temporal modeling capability helps improve the consistency between the predicted output and the actual physiological process in terms of instantaneous values ​​and temporal evolution patterns, resulting in smoother and more natural motion during the application phase.

[0042] Furthermore, by employing a hybrid loss function to jointly optimize the prediction errors of sEMG and Kinematics, the deep learning model is required to simultaneously reduce the prediction errors of both modalities during training. This joint optimization mechanism enables the model's internal representation to simultaneously encode muscle activation patterns and motion trajectory information, rather than treating them as independent learning tasks. Since muscle activation is a physiological driver of motion trajectory, there is an inherent causal relationship between the two. The design of the hybrid loss function allows the model to maintain consistency and coordination between the two output branches when learning this causal relationship. This consistency and coordination ensures that, in the application phase, when the model simultaneously predicts P-sEMG and P-Kinematics based on the affected side's EEG, the two maintain a physiological match similar to that of the unaffected limb during actual movement, thus providing high-quality input for subsequent feedforward-feedback dual-loop control.

[0043] Furthermore, in the application phase, the acquired EEG signals undergo the same preprocessing as in the training phase (including bandpass filtering, notch filtering, and independent component analysis (ICA) to remove artifacts), and the IMU data is used for pose calculation to obtain Euler angles or quaternions. This improves the consistency between the real-time signal and the training data in the feature space. This consistency helps the deep learning model to decode accurately in the application phase. Simultaneously, by dividing the preprocessed signal into time windows of the same length as during training, the time scale of the input data matches the scale during model training, enabling the model to fully utilize its learned temporal dependencies for prediction.

[0044] In terms of feedback control, the predicted coordinated motion trajectory P-Kinematics is used as the target trajectory that changes over time. This trajectory is compared with the real-time acquired actual kinematic signals A-Kinematics of the affected limb to calculate the trajectory error. Based on this error, feedback correction commands for the FES (Feedforward Electrification System) are generated, constructing a real-time trajectory tracking closed loop. This closed-loop control mechanism endows the system with a certain degree of anti-interference capability. During rehabilitation training, the muscle state of the affected limb changes over time due to fatigue and strength variations. The electrode-skin contact impedance of the FES system changes due to sweating and electrode displacement. Furthermore, the brain's intention signal may deviate due to fluctuations in attention. These factors can all cause the actual response of the affected limb to the feedforward FES commands to deviate from expectations. Traditional pure feedforward control systems have limited ability to cope with these interference factors, mainly relying on the accuracy of the feedforward commands and the accuracy of system modeling. This application introduces a feedback correction loop based on P-Kinematics, enabling the system to continuously monitor the deviation between the actual movement trajectory of the affected limb and the target P-Kinematic trajectory. Once a significant deviation (i.e., trajectory error E(t)) is detected, the feedback controller (such as a PID controller, adaptive controller, or model predictive controller) immediately generates a correction command, dynamically adjusting the intensity of the FES stimulus to encourage the affected limb to move closer to the target trajectory. This real-time correction mechanism significantly improves the system's robustness and movement accuracy, allowing the affected limb to still track the target trajectory well even in the presence of various interference factors, thus improving the quality and stability of the movement.

[0045] In particular, when using an adaptive PID controller as a feedback controller, the proportional (P), integral (I), and derivative (D) gains can be adjusted online to adapt to the time-varying characteristics of the affected limb's muscle state. At different stages of rehabilitation training, the patient's muscle strength, fatigue level, and neural response speed all change, making it difficult for a controller with fixed parameters to maintain good performance throughout the entire rehabilitation process. Adaptive PID, by identifying changes in the parameters of the controlled object (the dynamic response characteristics of the affected limb under FES) online, dynamically adjusts the control gain, thereby helping the feedback control maintain a better working state. This adaptive capability improves the system's adaptability to slow time-varying factors such as muscle fatigue and strength changes, and also enhances its ability to suppress fast time-varying factors such as electrode shifting and impedance abrupt changes.

[0046] In terms of command fusion, the feedforward control command and feedback correction command are fused using additive fusion or gain modulation to generate the final FES command, achieving an organic combination of "physiological pattern driving" and "error feedback correction." The key to this fusion strategy is that the feedforward control command provides the "basic pattern" of FES stimulation, determining which muscles are activated at what time and how the relative activation intensity is distributed, which contributes to the coordination of the movement; while the feedback correction command adjusts the stimulation intensity according to the deviation between the actual movement and the target movement, which contributes to the accuracy and robustness of the movement. By fusing the two within each control cycle, the system can achieve a more natural and precise control effect. Specifically, when the actual movement trajectory of the affected limb is close to the target trajectory (i.e., the trajectory error is small), the contribution of the feedback correction command is small, and the FES is mainly driven by the feedforward command. At this time, the system can better reproduce the physiological coordination pattern of the healthy side. When a large deviation occurs (such as muscle fatigue leading to a weakened response), the contribution of the feedback correction command increases, correcting the deviation by increasing or decreasing the stimulation intensity of certain muscle channels, prompting the actual trajectory to move closer to the target trajectory. This dynamic trade-off allows the system to maintain a certain level of physiological coordination while also possessing a high-precision trajectory tracking capability.

[0047] In summary, this application, through the dual mapping capability established during the training phase using "fusion-type teacher signals" (from central intention to peripheral electromyographic patterns, and from central intention to motor trajectory), and the synergistic cooperation between physiological pattern reproduction and real-time trajectory correction achieved during the application phase through "feedforward-feedback dual closed-loop control," helps to improve the problems of rigid control patterns, unnatural movements, lack of personalization, and insufficient robustness in existing BCI-FES systems. Regarding the naturalness of movement, the movements generated by the system are closer to the natural movements of the healthy limb in terms of multi-joint coordination, muscle activation timing, and trajectory smoothness. Regarding movement accuracy, the tracking error of the affected limb on the target trajectory is reduced, and relatively stable tracking performance is maintained even in the presence of external interference or changes in internal state. Regarding personalization, because the teacher signals come from the patient's own healthy limb, the trained model is more adapted to the individual patient's physiological characteristics and movement habits. Regarding the neurorehabilitation mechanism, the synchronicity of motor intention and muscle activation, as well as the coordination of movement provided by the system, provides induction conditions for neural plasticity, which helps promote the recovery of the patient's central nervous system function. The realization of these technical effects is the synergistic effect of multiple technical features in a specific combination, which reflects the overall advantages of the technical solution of this application.

[0048] The specification of this application contains numerous technical features distributed across various technical solutions. Listing all possible combinations of these technical features (i.e., technical solutions) would make the specification excessively lengthy. To avoid this problem, the various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which are considered to have been described in this specification), unless such a combination of technical features is technically infeasible. For example, one example discloses feature A+B+C, and another example discloses feature A+B+D+E. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; they cannot be used simultaneously. Feature E can technically be combined with feature C. Therefore, the solution A+B+C+D should not be considered as described because it is technically infeasible, while the solution A+B+C+E should be considered as described. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the workflow of a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction according to an embodiment of this application.

[0050] Figure 2 This is a schematic diagram of the structure of a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction according to an embodiment of this application. Detailed Implementation

[0051] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0052] Explanation of some concepts:

[0053] Brain-Computer Interface (BCI): A technological system that establishes a direct communication pathway between the brain and external devices. BCI collects and analyzes brain neural activity signals (such as EEG, ECoG, or functional near-infrared spectroscopy fNIRS) to decode the user's motor intentions, cognitive states, or other mental activities, and converts this information into control commands to drive external devices (such as prostheses, wheelchairs, computers, or functional electrical stimulation systems). In the field of rehabilitation, BCI is mainly used to detect patients' motor intentions, providing "intention-driven" control signals for rehabilitation training.

[0054] Functional Electrical Stimulation (FES) is a technique that induces or assists movement by applying low-intensity electrical pulses to muscles or motor nerves to induce muscle contraction. FES delivers electrical stimulation pulses to target muscles via surface electrodes or implanted electrodes, enabling muscles that have lost or partially lost their motor function to activate according to specific patterns, producing functionally meaningful movements. FES stimulation parameters include pulse frequency (typically 20-50 Hz), pulse width (typically 50-500 μs), and stimulation amplitude (current intensity, typically 0-100 mA), which determine the strength and duration of muscle contraction.

[0055] Surface electromyography (sEMG) signals are electrical activity signals of muscles acquired through electrodes attached to the skin surface. sEMG signals reflect the potential changes generated during muscle fiber depolarization, and their amplitude and frequency characteristics are closely related to muscle activation, contractile force, and fatigue state. In this application, sEMG is used to record muscle activation patterns in the unaffected limb during coordinated movements, serving as one of the teacher signals for training a deep learning model. The envelope signal of the sEMG (extracted through rectification and low-pass filtering) smoothly reflects the time-varying trend of muscle activation intensity.

[0056] Kinematics refers to physical quantities that describe the motion state of an object (specifically, a human limb in this application), including position, displacement, velocity, acceleration, angle, angular velocity, angular acceleration, etc., without involving the forces causing the motion. In rehabilitation engineering, kinematics are commonly used to quantify and evaluate characteristics such as limb movement trajectory, joint range of motion, and motion smoothness. In this application, kinematics are primarily acquired through inertial measurement units (IMUs) or optical motion capture systems, representing the limb's posture in three-dimensional space and its temporal evolution in the form of Euler angles or quaternions.

[0057] An Inertial Measurement Unit (IMU) is a sensor module that integrates an accelerometer, gyroscope, and (in some cases) magnetometer to measure the linear acceleration, angular velocity, and magnetic field direction of an object. By fusing the triaxial acceleration and triaxial angular velocity data acquired by the IMU (e.g., using Kalman filtering or complementary filtering), the object's attitude (usually represented by Euler angles or quaternions) and trajectory can be calculated in real time. In this application, the IMU is worn at a specific location on the limb (e.g., the back of the hand, forearm) to acquire kinematic signals.

[0058] Deep learning models are machine learning models based on multi-layer neural networks. They can automatically extract high-level feature representations by learning from large amounts of data and establish complex nonlinear mapping relationships between inputs and outputs. In this application, deep learning models specifically refer to model architectures capable of processing temporal data, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), gated recurrent units (GRUs), or Transformer networks. These models, through their internal recurrent connections or attention mechanisms, can capture the temporal dependencies in the input sequence, making them particularly suitable for processing data with significant temporal characteristics, such as human motion signals.

[0059] Fusion Teacher Signal: In the context of this application, this specifically refers to a technique that uses surface electromyography (sEMG) and kinematics signals from the unaffected limb as joint supervision targets (i.e., "teacher signals") for two parallel output branches of a deep learning model to train the model. sEMG and kinematics are regressed and predicted by their respective independent output branches, while the two output branches share a low-level encoder. The prediction errors of the two branches are calculated separately using a hybrid loss function and then weighted for joint optimization. Unlike traditional single-modal teacher signals, fusion teacher signals simultaneously contain physiological information at two levels: "muscle-driven patterns" (sEMG) and "motor execution results" (kinematics), enabling the model to learn a more complete and physiologically meaningful mapping of movement patterns. Feedforward Control: A control strategy in which the controller directly calculates control commands and applies them to the controlled object based on the system's input (or reference signals) and prior knowledge of the system's dynamic characteristics, without relying on the measurement and feedback of the actual output results. In this application, feedforward control refers to generating basic stimulation instructions for the FES based on predicted coordinated electromyographic patterns (P-sEMG), which provide a driving blueprint for "how to activate muscles" based on physiological patterns learned from the healthy side.

[0060] Feedback control is a control strategy in which the controller continuously measures the actual output of the controlled object, compares it with the desired target value (reference signal or setpoint), and generates control commands based on the error between the two to reduce or eliminate the error. In this application, feedback control refers to generating a correction command for the field of motion (FES) based on the error between the actual motion trajectory (A-Kinematics) and the predicted target trajectory (P-Kinematics) of the affected limb through a controller (such as a PID controller), to ensure that the actual motion can accurately track the target trajectory.

[0061] A PID controller (Proportional-Integral-Derivative Controller) is a classic feedback controller whose output consists of three terms: a proportional term (P, proportional to the current error), an integral term (I, accumulating historical errors to eliminate steady-state errors), and a derivative term (D, predicting error trends to suppress overshoot). The PID controller adjusts three gain parameters (P, I, I, D, D, D) to achieve this. , , This technology enables effective control of various dynamic systems. In this application, a PID controller (particularly an adaptive PID controller) is used to generate feedback correction commands for the FES based on the trajectory error, thereby achieving accurate tracking of the predicted target trajectory. The adaptive PID controller can adjust the gain parameters online to adapt to the time-varying characteristics of the controlled object (affected limb).

[0062] The following is a brief summary of some of the innovative aspects of this application:

[0063] In summary, the technical problem this application aims to solve stems from a deep-seated contradiction in existing brain-computer interface-functional electrical stimulation (BCI-FES) rehabilitation systems: the rigidity of control patterns and insufficient robustness of execution. Existing technologies typically employ discrete "intention-instruction" mapping logic, directly triggering pre-defined, fixed stimulation programs with decoded motor intentions. While this unidirectional, open-loop control paradigm is feasible for basic motor assistance, it fundamentally severs the continuity, dynamism, and feedback regulation inherent in the relationship between central motor intentions and peripheral physiological execution. The technical concept of this application lies in fundamentally reconstructing the information processing and control execution paths of the BCI-FES system by constructing a cross-limb, cross-modal "fusion physiological blueprint" learning mechanism during the training phase and establishing a two-layer closed-loop control architecture coupling "predictive feedforward drive" and "real-time trajectory tracking correction" during the application phase.

[0064] Specifically, the first layer of inventiveness of this application lies in the following: In step 100 of the training phase, the system does not merely collect peripheral signals from the unaffected limb in a single modality as the supervision target, but simultaneously collects surface electromyography (sEMG) signals and kinematics signals from the unaffected limb, and inputs both as an organically integrated "fusion teacher signal" into the training process of the deep learning model. The uniqueness of this fusion teacher signal lies in the fact that sEMG carries the temporal pattern and intensity distribution of muscle activation, essentially reflecting the "driving intention" of the neuromuscular system; while kinematics carries the actual movement trajectory of the limb in three-dimensional space and its time-varying characteristics, essentially reflecting the "execution result" of the aforementioned driving intention under biomechanical constraints. By simultaneously incorporating the "cause" (electromyographic driving pattern) and the "effect" (movement trajectory result) into the model's learning objective, the deep learning model, when completing the mapping learning from the affected side's brain intention signal (EEG) to the unaffected side's peripheral pattern, not only establishes a neuromuscular mapping relationship between central intention and muscle activation, but also further establishes an intention-behavior mapping relationship between central intention and the final movement trajectory. The establishment of this dual mapping is not a simple information superposition, but rather a joint optimization of the sEMG prediction error and the Kinematics prediction error in the hybrid loss function L (Formula 1). This forces the model's internal representation to simultaneously encode physiological knowledge at two levels: "how to activate muscles" and "what kind of movement will be produced after activation." This enables the model to have a deeper understanding of human coordinated movement patterns and a stronger generalization ability.

[0065] The second layer of inventiveness of this application lies in the following: In the application phase, the deep learning model simultaneously outputs predicted coordinated electromyographic patterns (P-sEMG) and predicted coordinated kinematic trajectories (P-Kinematics) based on real-time acquired EEG data from the affected side. These two predicted outputs are not used passively and separately in parallel, but are organically integrated into a "feedforward-feedback" dual closed-loop control architecture. In the feedforward loop (step 500), P-sEMG is converted into feedforward control instructions for multi-channel FES. These instructions, based on the physiological coordination patterns learned from the healthy side, determine the timing, relative intensity, and synergistic relationship of activation of each muscle channel, thereby providing a "physiologically correct" driving blueprint for the affected limb. However, feedforward drive alone is insufficient because the affected limb, as a dysfunctional, highly nonlinear, and time-varying actuator, may deviate from expectations in its actual response to FES instructions due to various factors such as muscle fatigue, changes in electrode impedance, and fluctuations in brain signals. The originality of this application lies in giving P-Kinematics, which was originally only used as a model training target, a completely new functional role in the application stage—as a dynamically changing "reference trajectory target" (Setpoint). In the feedback loop (steps 600-800), the system acquires the actual kinematic signal A-Kinematics of the affected limb in real time through the IMU, compares it with P-Kinematics in real time, calculates the trajectory error E(t) (Equation 3), and converts this error into a feedback correction command for FES through a PID controller. Finally, in the command fusion unit in step 900, the feedforward command based on the physiological coordination mode and the feedback correction command based on the actual error are fused (Equation 2) to jointly drive the multi-channel FES stimulator.

[0066] The inherent logical connection and synergistic cooperation of this "feedforward-feedback" dual closed-loop control architecture can be described as follows: The feedforward loop undertakes the function of "physiological pattern reproduction," which, based on P-sEMG, ensures the coordination and naturalness of FES stimulation in spatiotemporal patterns, solving the problem of "how the movement is natural." The feedback loop undertakes the function of "trajectory tracking correction," which, based on P-Kinematics as a dynamic target, continuously monitors the deviation between A-Kinematics and the target and applies correction, solving the problems of "how the movement is precise" and "how to combat interference." The two loops are not independent of each other, but complement each other functionally through the "homogeneous but heterogeneous" predictive outputs of P-sEMG and P-Kinematics (which originate from the decoding of the same EEG input by the same deep learning model, but represent two different dimensions of physiological information: electromyographic patterns and movement trajectories, respectively). The feedforward loop provides the "skeleton" of control (i.e., the basic muscle activation patterns), while the feedback loop injects the "soul" (i.e., the ability to accurately track the target trajectory) into this "skeleton" through real-time correction. This technical solution, which deeply couples "predictive physiological blueprint-driven" and "closed-loop error feedback correction" within the same control cycle, enables the system to maintain a high-fidelity reproduction of the natural movement pattern of the healthy side (because the feedforward instructions are derived from the healthy side pattern learned during training), and to actively adapt to the actual state changes and external disturbances of the affected limb (because the feedback correction continuously monitors and corrects the actual execution results).

[0067] It is particularly important to emphasize that the role of P-Kinematics as a "dynamic reference target" in this application is not an obvious technical arrangement, but rather a creative functional extension and repositioning of the deep learning model output. In conventional machine learning applications, the model's predicted output is usually directly used for decision-making or control. However, in this application, the predicted output of P-Kinematics is creatively used as an "ideal state reference" for the feedback control system. This essentially involves a deep fusion of two previously relatively independent control theory paradigms—"feedforward prediction" and "feedback control"—through the multi-output structure of the deep learning model. The technical effect of this fusion is not a simple superposition of the two control methods, but a synergistic enhancement resulting from inherent coupling: the feedforward loop provides the feedback loop with a "physiologically reasonable" target trajectory (rather than a fixed target artificially set by engineers), while the feedback loop provides the feedforward loop with an "executionally guaranteed" correction mechanism (rather than simply relying on the accuracy of the feedforward command). It is this dual mapping capability established during the training phase through "fusion teacher signals" and the dual closed-loop control architecture constructed during the application phase through "differentiated function assignment of predicted output" that together constitute the inventive contribution of this application that is difficult for a person skilled in the art to obviously conceive of based on the prior art.

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0069] Before describing the implementation methods of this application in detail, the specific application scenarios targeted by this application and the specific technical problems faced in these scenarios will be explained first, so as to better understand the innovation and advantages of the technical solution of this application.

[0070] Through long-term and in-depth research, the inventors of this application discovered that, taking stroke-induced hemiplegia as an example, patients typically still exhibit strong subjective motor intentions when performing daily living skills training (such as grasping a cup, picking up utensils, and buttoning clothes). However, their active contraction ability in the affected hand is limited, making it difficult to complete complete and coordinated grasping and relaxing movements. Existing BCI-FES systems mostly decode the patient's grasping intentions and trigger preset electrical stimulation programs to drive the contraction of flexor and extensor muscles to assist in completing grasping movements. However, in actual training, the following situations often occur: First, the intention-stimulus mapping relationship established by the system based on incomplete teacher signals cannot simultaneously reflect the muscle synergistic activation pattern and the expected movement trajectory, resulting in inconsistency between the stimulation pattern and the physiological synergistic pattern, and stiff and unnatural movement performance; Second, as the training time increases, factors such as muscle fatigue on the affected side, changes in electrode contact status, and fluctuations in EEG signals can cause the actual grasping trajectory to deviate from the ideal trajectory. However, the traditional pure feedforward control structure lacks real-time feedback correction based on actual movement deviations, and cannot adjust the stimulation intensity and distribution in time, resulting in insufficient grasping force, incomplete finger closure, or insufficient release; Third, different patients have significant individual differences in muscle strength, joint range of motion, and grasping habits. Universal standardized stimulation programs are difficult to adapt to the physiological and functional state of each patient in a timely manner, affecting the effectiveness and comfort of rehabilitation training.

[0071] Therefore, in the specific application scenarios mentioned above, there is an urgent need for a BCI-FES rehabilitation system that can fully utilize central motor intention signals and healthy peripheral multimodal physiological signals, reproduce electromyographic activation that conforms to physiological coordination patterns, use predicted movement trajectories as dynamic reference targets, introduce real-time trajectory error feedback correction, and achieve personalized movement assistance based on individual patient characteristics, so as to improve the naturalness, accuracy, and robustness of movements.

[0072] See Figure 1 and Figure 2 This embodiment provides a rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction. This system is mainly used for the recovery of motor function of the affected upper or lower limbs in stroke patients or spinal cord injury patients.

[0073] This rehabilitation system, based on central-peripheral fusion decoding and trajectory closed-loop correction, includes:

[0074] The training subsystem includes:

[0075] The training signal acquisition module is used to simultaneously acquire the central motor intention signal (EEG) of the cerebral hemisphere corresponding to the affected limb, the peripheral electromyographic signal (sEMG) of the healthy limb, and the peripheral kinematic signal (Kinematics) of the healthy limb.

[0076] The model training module is used to train a deep learning model based on the EEG, sEMG and Kinematics acquired by the training signal acquisition module, wherein the sEMG and Kinematics are used as fusion teacher signals, so that the deep learning model can simultaneously predict the corresponding coordinated electromyographic pattern P-sEMG and coordinated motion trajectory P-Kinematics based on the input EEG.

[0077] The application subsystem includes:

[0078] An application signal acquisition module is used to acquire the EEG of the affected side of the brain and the actual kinematic signals (A-Kinematics) of the affected side of the limbs in real time.

[0079] The real-time decoding module is used to input the EEG acquired by the application signal acquisition module into the deep learning model trained by the model training module, and decode it in real time to obtain the P-sEMG and the P-Kinematics.

[0080] The feedforward control module is used to generate feedforward control commands for multi-channel functional electrical stimulation (FES) based on the P-sEMG obtained by the real-time decoding module.

[0081] The error calculation module is used to compare the P-Kinematics obtained by the real-time decoding module as the target trajectory that changes over time with the A-Kinematics acquired by the application signal acquisition module, and calculate the trajectory error.

[0082] The feedback control module is used to generate the feedback correction command of the FES based on the trajectory error calculated by the error calculation module.

[0083] The instruction fusion module is used to fuse the feedforward control instruction generated by the feedforward control module and the feedback correction instruction generated by the feedback control module to generate the final FES instruction;

[0084] The stimulus execution module is used to apply the final FES command generated by the instruction fusion module to the affected limb.

[0085] Furthermore, the core of this system lies in the following: During the training phase, the system simultaneously collects the central motor intention signal (EEG) from the cerebral hemisphere corresponding to the affected limb, the peripheral electromyographic (sEMG) signal from the healthy limb, and the peripheral kinematics signal from the healthy limb, using the latter two as "fusion-type teacher signals" to train the deep learning model. During the application phase, a "feedforward-feedback" dual closed-loop control structure is used. This structure utilizes the predicted coordinated electromyographic pattern (P-sEMG) to generate feedforward control commands for functional electrical stimulation (FES) to reproduce the physiological coordination pattern. It also utilizes the predicted coordinated movement trajectory (P-Kinematics) as a dynamic target, combined with the actual kinematics signal (A-Kinematics) of the affected limb for closed-loop correction, thereby achieving natural, precise, and robust motor control.

[0086] The following describes the detailed implementation method for the training phase.

[0087] The goal of the training phase is to build a deep learning model that can map the motor intentions of the affected side of the brain to the fused motor patterns of the healthy side. Specifically, this phase includes four main steps: signal acquisition, signal preprocessing, data alignment and segmentation, and model training.

[0088] (a) Synchronous signal acquisition

[0089] like Figure 1 As shown, in step 100 of the training phase, three physiological signals need to be collected simultaneously. Specifically, the patient, upon prompting, intends to drive the affected limb to perform a target action (e.g., a grasping action), while the unaffected limb actually performs the target action. It should be noted that during this process, the system simultaneously collects the following three signals:

[0090] First, the central motor intention signal (EEG) of the cerebral hemisphere corresponding to the affected limb. This EEG signal is acquired using a multi-lead EEG acquisition device worn on the patient's head, preferably acquiring leads covering the motor cortex region, such as C3, C4, and C2. These signals reflect the neural electrical activity of the cerebral cortex when the patient generates a motor intention. It should be noted that, in alternative embodiments, the central motor intention signal (EEG) can also be replaced with functional near-infrared spectroscopy (fNIRS) signals or cortical electroencephalography (ECoG) signals to adapt to different clinical scenarios and signal quality requirements.

[0091] Second, peripheral electromyography (sEMG) signals from the unaffected limb. These sEMG signals are acquired using a multi-channel surface electromyography (SEMG) acquisition array, which covers the major synergistic muscle groups of the unaffected limb. For example, for hand grasping movements, the muscle groups to be acquired include, but are not limited to, the flexor digitorum superficialis, flexor digitorum profundus, extensor digitorum commonis, and flexor pollicis brevis. These muscle groups are illustrative examples; in practice, equivalent synergistic muscle groups can be selected based on the task and are not considered limiting. These sEMG signals reflect the timing and intensity of muscle activation in the unaffected limb during coordinated movements, i.e., the "muscle drive pattern."

[0092] Third, peripheral kinematics signals of the unaffected limb. These kinematics signals are acquired using an inertial measurement unit (IMU) or an optical motion capture system. In this embodiment, an IMU is preferably used for acquisition, and the data acquired by the IMU includes triaxial acceleration and triaxial angular velocity. Specifically, the IMU installed on the unaffected limb is located on the back of the hand and / or forearm to capture the postural changes of the hand and forearm during movement. The above installation location is an exemplary scheme, and equivalent locations and numbers can be selected according to specific rehabilitation tasks and wearing conditions, and does not constitute a limitation. These kinematics signals reflect the actual spatiotemporal trajectory of limb movement, i.e., the "motion execution result."

[0093] It is important to note that during signal acquisition, the three signals must be strictly synchronized, meaning that the instantaneous values ​​of EEG, sEMG, and Kinematics must be recorded at the same timestamp. This synchronization is fundamental for establishing accurate mapping relationships later. A unified acquisition trigger signal or hardware synchronization mechanism is typically used to ensure accurate time alignment.

[0094] (ii) Signal preprocessing

[0095] After step 100 and before step 200, the acquired raw signal is preprocessed to remove noise, extract effective features, and convert the signal into a format suitable for model training.

[0096] For EEG signals, bandpass filtering is first performed, typically in the 0.5 Hz to 50 Hz band, to preserve EEG rhythms related to motor intent (such as μ and β rhythms). Notch filtering is then applied to remove power line interference at 50 Hz or 60 Hz. Furthermore, to remove artifacts such as electrooculography (EOG) and electromyography (EMG), independent component analysis (ICA) is used to decompose the EEG signal, identifying and removing independent components representing artifacts, thereby obtaining a cleaner EEG signal.

[0097] For sEMG signals, bandpass filtering (e.g., 20 Hz to 500 Hz) is first applied to remove motion artifacts and high-frequency noise. Then, the filtered signal is rectified, i.e., the absolute value is taken, converting the bipolar signal to a unipolar signal. Next, low-pass filtering (with a cutoff frequency typically between 5 Hz and 10 Hz) is performed to extract the envelope signal of the sEMG. It should be noted that this envelope signal smoothly reflects the change in muscle activation intensity over time and is a key feature used to guide FES intensity modulation.

[0098] For kinematics signals, attitude calculations are required on the raw acceleration and angular velocity data acquired by the IMU. Common methods include Kalman filtering or complementary filtering algorithms, which fuse triaxial acceleration and triaxial angular velocity to obtain limb attitude represented by Euler angles or quaternions. More specifically, Euler angles (such as pitch, roll, and yaw) or quaternions can intuitively describe the orientation and trajectory of limbs in three-dimensional space.

[0099] Through the above preprocessing steps, clean and well-defined EEG signals, sEMG envelope signals, and Euler angles or quaternions are obtained, thus laying the data foundation for subsequent model training.

[0100] (III) Data Alignment and Partitioning

[0101] In step 200, the preprocessed EEG, sEMG envelope signal, and Euler angles or quaternions are aligned according to timestamps. Specifically, since the three signals may undergo different filtering and calculation steps during preprocessing, they need to be precisely aligned again based on the original synchronization timestamps to ensure that the EEG sample at each time point corresponds to the sEMG envelope and Kinematics value at the same time point.

[0102] Furthermore, the aligned data is divided into time windows, each ranging in length from 100 ms to 500 ms. It should be noted that the choice of time window requires a trade-off between temporal resolution and feature stability: shorter windows (e.g., 100 ms) can capture rapidly changing motion details but may introduce more noise; longer windows (e.g., 500 ms) provide more stable features but may lose information about instantaneous changes. In this embodiment, a 200 ms time window is preferred, and some overlap (e.g., 50% overlap) between adjacent windows is allowed to increase the number of training samples and improve the model's ability to learn temporal continuity.

[0103] More specifically, each time window constitutes a training sample. For each sample, the input is the EEG features within that window (which can be the original EEG signal, extracted frequency domain features, or time-frequency features), and the output is the sEMG envelope signal and kinematics signal (Euler angles or quaternions) within that window. For example, the sampling rate of the EEG signal can be 250 Hz or 500 Hz. As a non-limiting example of "frequency domain features," after performing the preprocessing described above, the band power1 of the μ rhythm (e.g., 8-13 Hz) and β rhythm (e.g., 14-30 Hz) within each time window can be calculated and used as the input feature vector. Alternatively, as mentioned earlier, the preprocessed original signal sequence can be directly used as the input to the deep learning model, which will then extract the temporal features itself. In training step 200, the sEMG and kinematics are used as a fused teacher signal. The sEMG reflects the timing and intensity of muscle activation, while the kinematics reflects the actual spatiotemporal trajectory of limb movement, thereby providing the deep learning model with a physiological blueprint that includes muscle drive patterns and movement execution results. It is important to emphasize that this fused teacher signal design is one of the core innovations of this application. It enables the model to not only learn "how to activate muscles" but also "what kind of movement results this activation will produce," thus establishing a more complete and physiologically meaningful mapping relationship.

[0104] (iv) Deep learning model training

[0105] The deep learning model includes a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a Transformer network, used to process the temporal mapping relationship from the EEG to the P-sEMG and P-Kinematics. In this embodiment, an LSTM network is preferred because the EEG, sEMG, and Kinematics signals related to human motion all have significant temporal dependencies, and LSTM can effectively capture long-term temporal dependencies, avoiding the gradient vanishing problem of traditional RNNs.

[0106] Specifically, the LSTM comprises 64 to 256 units, with 2 to 4 network layers. For example, a network structure containing 2 layers of LSTM, with 128 LSTM units per layer, can be used. Further, the network's input layer receives a preprocessed EEG feature sequence, and the output layer is divided into two parallel branches: one branch is used for regressing and predicting the multi-channel sEMG envelope sequence (i.e., P-sEMG), and the other branch is used for regressing and predicting the kinematics sequence (i.e., P-kinematics). This multiple-input multiple-output (MIMO) structure allows the model to simultaneously learn the outputs of two different modalities, while the two output branches share the underlying LSTM encoder, thereby achieving joint representation and learning of features.

[0107] During model training, a hybrid loss function L is used to optimize the network parameters:

[0108]

[0109] Here, MSE represents the mean squared error function, used to measure the difference between the predicted and the actual values. Specifically, The coordinated electromyographic patterns predicted by the deep learning model. The sEMG envelope signal acquired and preprocessed in step 100; The coordinated motion trajectory predicted by the deep learning model. The Kinematics (Eulerian angles or quaternions) collected and preprocessed in step 100. Weighting coefficients. and This is used to balance the contributions of the two loss terms, and its value can be adjusted according to the scale and importance of the actual data. For example, it can be set to... , This indicates that equal importance is given to sEMG prediction and kinematics prediction; however, these two weights can be adjusted according to specific task requirements. For example, if more emphasis is placed on the accuracy of motion trajectories, the weights can be increased. The value of .

[0110] It should be noted that by using the aforementioned hybrid loss function, the model must minimize not only the prediction error of sEMG but also the prediction error of Kinematics during training. This joint optimization mechanism enables the model's learned internal representation to simultaneously encode muscle activation patterns and motion trajectory information, thereby giving the model stronger generalization ability and the ability to understand physiological movement patterns.

[0111] Furthermore, the training process employs standard backpropagation, such as using the Adam optimizer; the data is divided into training / validation / test sets, and an early stopping strategy is optionally employed to stop training when the loss on the validation set no longer decreases, in order to prevent overfitting.

[0112] After training, the resulting deep learning model can simultaneously output predicted coordinated electromyographic patterns (P-sEMG) and coordinated motor trajectories (P-Kinematics) based on the input EEG signals from the affected side of the brain. This model serves as a bridge between the training and application phases and is a core component of the entire rehabilitation system.

[0113] The following describes the detailed implementation method in the application phase.

[0114] The goal of the application phase is to use a trained deep learning model to decode the patient's motor intentions in real time and drive the affected limb to produce natural and precise coordinated movements through a "feedforward-feedback" dual closed-loop control structure. Specifically, this phase includes six main steps: real-time signal acquisition, model decoding, feedforward control command generation, feedback control command generation, command fusion, and FES stimulus application.

[0115] (I) Real-time EEG acquisition and model decoding

[0116] In step 300, the EEG of the affected side of the brain is acquired in real time. At this time, the patient only needs to generate a motor intention (e.g., an intention to grasp an object), without the cooperation of the unaffected limbs. The acquired EEG signal undergoes the same preprocessing procedure as in the training phase (bandpass filtering, notch filtering, ICA artifact removal) and is divided into time windows of the same length as during training (e.g., 200 ms).

[0117] In step 400, the EEG collected in step 300 is input into the deep learning model trained in step 200, and the P-sEMG and P-Kinematics are decoded in real time. It should be noted that this decoding process is performed in real time; that is, after each new EEG time window arrives, the model immediately performs forward propagation calculations and outputs the corresponding P-sEMG and P-Kinematics. Due to the use of an efficient LSTM network structure, which can run on a GPU or embedded AI chip, the entire decoding latency can be controlled within a short time (e.g., 100-200ms), thus ensuring the real-time performance of the system.

[0118] More specifically, the P-sEMG is a multi-channel temporal signal with the same number of channels as the sEMG used during training (e.g., 8 channels, corresponding to 8 major synergistic muscles). The P-sEMG value of each channel represents the activation intensity that the corresponding muscle "should" produce under the current movement intention. The P-Kinematics is a signal describing limb posture, such as the pitch, roll, and yaw angles of the wrist joint expressed in Euler angles. These two predictive outputs will be used for subsequent feedforward control and feedback control, respectively.

[0119] (ii) Generation of feedforward control commands

[0120] In step 500, feedforward control commands for multi-channel functional electrical stimulation (FES) are generated based on the P-sEMG obtained in step 400. It is important to emphasize that this process embodies the innovative concept of "physiological pattern reproduction," meaning that instead of simply triggering a preset stimulation program, stimulation commands are dynamically generated based on the electromyographic activation pattern predicted by the model and consistent with the healthy side.

[0121] Specifically, the pulse width or amplitude of the feedforward control command is modulated according to the amplitude of the P-sEMG, wherein the larger the amplitude of the P-sEMG, the greater the stimulation intensity of the corresponding FES channel. For example, a linear or nonlinear mapping function can be used to convert the envelope amplitude of the P-sEMG (whose value is typically normalized and ranges from 0 to 1) into the pulse width (e.g., from 50 μs to 400 μs) or stimulation amplitude (e.g., from 0 mA to 100 mA) of the FES. A simple linear mapping method is as follows:

[0122]

[0123] in, Let P-sEMG amplitude (normalized value) be the i-th channel at time t. and These are the minimum and maximum pulse widths, respectively (e.g., 50 μs and 400 μs).

[0124] In this embodiment, the stimulation frequency of the FES remains constant at 30 Hz to 50 Hz. This frequency range produces smooth muscle tetanic contractions, avoiding rapid fatigue caused by excessively high frequencies or twitching contractions caused by excessively low frequencies. Through the above conversion, the feedforward control command for each FES channel... It has been determined that the instruction includes parameters such as stimulation frequency and pulse width (or amplitude).

[0125] It should be noted that the advantage of this P-sEMG-based feedforward control method lies in its ability to reproduce the natural muscle activation patterns of the unaffected limb during coordinated movements, including the temporal coordination and relative intensity relationships of multiple muscles. This results in movements that are no longer mechanical, pre-programmed actions, but rather physiologically coordinated and natural movements.

[0126] (III) Acquisition of actual kinematic signals and calculation of trajectory error

[0127] In application step 600, the actual kinematic signals (A-Kinematics) of the affected limb are acquired in real time. Similar to the training phase, the A-Kinematics are acquired in real time by an inertial measurement unit (IMU) installed on the affected limb. The data acquired by the IMU includes triaxial acceleration and triaxial angular velocity. The actual posture (Eulerian angles or quaternions) of the affected limb is calculated in real time using the same attitude calculation algorithm (such as Kalman filtering) as in the training phase.

[0128] In step 700, the P-Kinematics obtained in step 400 are used as the target trajectory that changes over time and compared with the A-Kinematics collected in step 600 to calculate the trajectory error. Specifically, the trajectory error is calculated in real time according to the following formula:

[0129]

[0130] in, This represents the trajectory error at time t. This represents the P-Kinematics obtained at time t in step 400. This represents the A-Kinematics acquired at time t in step 600. Further, if both P-Kinematics and A-Kinematics are represented in Euler angles (e.g., three angles: pitch, roll, yaw), then... It is also a three-dimensional vector, with each component representing the error of the corresponding angle.

[0131] It should be noted that the trajectory error includes the angular error of key joints. For example, for upper limb grasping movements, key joints may include the wrist joint, metacarpophalangeal joints, etc.; for lower limb walking movements, key joints may include the ankle joint, knee joint, hip joint, etc. In practical applications, the most important joints can be selected for error calculation and control based on the specific rehabilitation task.

[0132] It is important to emphasize that this step embodies the core idea of ​​"trajectory closed-loop correction" in this application. P-Kinematics is not merely a predictive output; it is given the role of a "dynamic reference target." By comparing it with the actual motion trajectory A-Kinematics in real time, the system can continuously monitor whether the affected limb moves along an "ideal, coordinated" trajectory. Once a deviation is detected, it can be corrected through feedback control.

[0133] (iv) Generation of feedback correction instructions

[0134] In application step 800, a PID controller, an adaptive controller, or a model predictive controller is used to generate a feedback correction instruction for the FES based on the trajectory error, wherein the PID controller includes a proportional P, an integral I, and a derivative D gain to achieve tracking of the P-Kinematics.

[0135] In this embodiment, an adaptive PID controller is preferably used. Specifically, when a PID controller is used, it is an adaptive PID controller, capable of adjusting the proportional gain (P), integral gain (I), and derivative gain (D) online to adapt to the time-varying characteristics of the muscle state of the affected limb. It should be noted that this is because during rehabilitation, the patient's muscle strength, fatigue state, and neural response characteristics may change over time. A PID controller with fixed parameters is unlikely to maintain optimal performance, while an adaptive PID controller can automatically adjust the control gain according to the changing trend of the error, thereby improving robustness.

[0136] More specifically, the output of the PID controller (i.e., the feedback correction command) Calculate according to the following standard PID control law:

[0137]

[0138] in, Let be the error of the i-th control channel (corresponding to a certain joint angle) at time t; , , These are the proportional, integral, and differential gains, respectively. Furthermore, the proportional term... Provides instantaneous correction proportional to the current error; integral term Used to eliminate steady-state errors and ensure long-term tracking accuracy; differential term It is used to predict error change trends and suppress overshoot and oscillation.

[0139] Furthermore, adaptive PID dynamically adjusts the parameters of the controlled object (i.e., the dynamic response characteristics of the affected limb under FES) by identifying parameter changes online. , , The value. For example, this online identification and adjustment can be implemented based on the principle of Model Reference Adaptive Control (MRAC), where the controller parameters are adjusted to make the response of the controlled object track a preset reference model; or, it can employ the principle of Self-Tuning Regulation (STR), which recursively estimates the model parameters of the controlled object online and calculates and adjusts the PID gain in real time based on this. For example, when muscle fatigue is detected as causing a weakened response, the value can be appropriately increased. To enhance control strength; when the error oscillates, it can be increased To improve damping.

[0140] It should be noted that the generated feedback correction command This indicates the additional stimulus modulation amount applied on top of the feedforward FES command in order to ensure that the actual movement trajectory of the affected limb follows the target trajectory. For example, if the actual flexion angle of the affected wrist is less than the target angle (i.e., under-flexion), then... If positive, the PID output is... A positive result means that the stimulation intensity of the FES channel controlling the flexor muscles needs to be increased to encourage further wrist flexion.

[0141] (v) Command fusion and FES stimulus application

[0142] In application step 900, the feedforward control command and the feedback correction command are fused using either additive fusion or gain modulation to generate the final FES command. In this embodiment, additive fusion is preferred, specifically:

[0143] For the i-th FES channel, its final instruction at time t is:

[0144]

[0145] in, This is the final FES instruction for the i-th channel. The feedforward control command generated based on the P-sEMG, The feedback correction command is generated based on the trajectory error.

[0146] It is important to emphasize that the physical meaning of this additive fusion strategy is: feedforward instructions. It provides a "basic stimulation pattern" based on a physiological blueprint, which determines which muscles are activated at what time and how the relative activation intensity is distributed, ensuring motor coordination; while feedback correction instructions The intensity of the stimulus is dynamically fine-tuned based on the deviation between the actual movement and the target movement, which ensures the precision of the movement. The combination of the two achieves a control effect that is both natural and precise.

[0147] It should be noted that in some cases, gain modulation can also be used, that is:

[0148]

[0149] in, It is a gain modulation factor calculated based on trajectory error. The advantage of this method is that it can keep the relative mode of the feedforward command (i.e., the relative intensity relationship between channels) unchanged, and only scale the absolute intensity of the whole or each channel.

[0150] In application step 1000, the final FES command generated in step 900 is applied to the affected limb. Specifically, the stimulation parameters of the final FES command include: a stimulation frequency of 30 Hz to 50 Hz; a pulse width of 50 μs to 400 μs, or a stimulation amplitude of 0 mA to 100 mA; wherein the pulse width or the stimulation amplitude is dynamically modulated by the final FES command. More specifically, the FES stimulator is based on... The value of modulates the stimulation pulse parameters applied to the electrode of the i-th target muscle in the affected limb. For example, if If the value is large, the pulse width is set to a larger value (e.g., 300 μs), thereby generating a stronger muscle contraction force; if When the value is small, the pulse width is set to a smaller value (e.g., 100 μs), resulting in a weaker contraction force.

[0151] It is important to emphasize that through this multi-channel, time-varying FES stimulation, the affected limb, driven by the brain's motor intention, produces coordinated movements similar to those of the healthy side. Moreover, due to the introduction of a real-time trajectory tracking closed loop based on P-Kinematics, even in the presence of interference factors such as muscle fatigue, slight electrode displacement, or minor fluctuations in brain signals, the system can continuously adjust the FES intensity through a feedback correction mechanism to ensure that the actual movement trajectory of the affected limb faithfully tracks the ideal target trajectory, thereby achieving precise and robust motor control.

[0152] In this embodiment, firstly, regarding the naturalness of movement, due to the adoption of a feedforward control strategy combining "fusion-type teacher signals" (a combination of sEMG and kinematics) and "physiological pattern reproduction," the system can learn and reproduce the natural coordination patterns of the patient's own unaffected limbs. Furthermore, this pattern encompasses complex coordination relationships between multiple muscles in time, space, and intensity; therefore, the resulting movements are no longer rigid, robotic, pre-programmed actions, but rather smooth, coordinated, and physiologically characteristic natural movements. Through kinematic analysis (such as calculating the smoothness of joint angle trajectories and the coordination coefficients between multiple joints), it can be quantitatively verified that the movements generated by this system are highly similar in coordination to the natural movements of the unaffected limbs.

[0153] Secondly, regarding motion accuracy and robustness, the system possesses the ability to actively combat interference due to the introduction of a real-time closed-loop correction mechanism based on "predicted trajectories" (P-Kinematics). It's important to note that traditional pure feedforward BCI-FES systems, once a stimulus command is issued, cannot adjust based on the actual effect. Therefore, they are highly sensitive to factors such as fluctuations in brain signals, muscle fatigue, and changes in electrode impedance, easily leading to motion execution failure or distortion. This system, however, continuously monitors the actual motion trajectory of the affected limb (A-Kinematics) and compares it in real-time with the ideal target trajectory (P-Kinematics). This allows it to promptly detect deviations and generate correction commands via a PID controller. For example, when muscle fatigue weakens the response to feedforward commands, the feedback controller detects that the actual trajectory lags behind the target trajectory, automatically increasing the stimulus intensity to force the affected limb to "catch up" with the target trajectory. This dual closed-loop structure of "feedforward + feedback" significantly improves the system's robustness. In experimental verification, compared with the pure feedforward system, the root mean square error (RMSE) of the affected limb tracking the target trajectory was significantly reduced under the conditions of external disturbance or simulated muscle fatigue, indicating that it has higher anti-interference ability and motion accuracy.

[0154] Furthermore, this system is highly personalized. Specifically, all teacher signals (sEMG, kinetics) collected during the training phase come from the patient's own unaffected limbs, rather than from someone else's "standard movement template" or a fixed program designed by an engineer. Therefore, the trained model is perfectly adapted to the patient's own physiological characteristics, movement habits, and neuromuscular properties. This personalization not only improves the accuracy of control but also makes the system feel more natural and comfortable for the patient, making it easier for them to accept and adhere to rehabilitation training.

[0155] Finally, from the perspective of the mechanism of neurorehabilitation, this system provides an afferent / efferent neural impulse pattern that is highly synchronized with the patient's motor intentions and physiologically coordinated. It should be noted that, according to Hebb's Law ("neurons that fire together, wire together"), when the motor intention neurons in the central nervous system and the actual activation of peripheral muscles occur synchronously and repeatedly, the neural pathway connections between them can be strengthened, promoting the remodeling and recovery of damaged nerve function. Therefore, this system is not only a tool to assist movement, but also a rehabilitation training method that may promote long-term neurological function recovery.

[0156] In summary, the rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction proposed in this application, by innovatively combining "fusion-type teacher signals" and "prediction-correction dual closed-loop control," achieves natural, precise, robust, and personalized motor assistance and rehabilitation training for the affected limbs of stroke or spinal cord injury patients. Compared with existing technologies, it has significant technological progress and application value.

[0157] It should be noted that, optionally, in the hybrid loss function shown in formula (1), the weighting coefficients... and This is used to balance the contributions of sEMG prediction error and Kinematics prediction error. In one specific embodiment of this application, and The value range is from 0.1 to 0.9, and The specific value can be determined according to the following principles:

[0158] (1) When the data scales (amplitude ranges) of the two signals are similar, the following settings can be configured: This indicates that both are given equal importance;

[0159] (2) When more attention is paid to the accuracy of motion trajectory tracking, the accuracy can be increased. The value (e.g.) , );

[0160] (3) When more attention is paid to the accuracy of muscle activation patterns, the intensity can be increased. The value (e.g.) , );

[0161] (4) In practical applications, the optimal result can be obtained by performing a grid search on the validation set or by making adjustments based on experience. and combination.

[0162] In addition, optionally, to ensure the accuracy of trajectory error calculation, the system needs to perform coordinate system calibration during the application phase startup. Specifically, since the training phase collects data from the healthy limb, while the application phase controls the affected limb, the system defaults to a mirror mapping strategy. This is done when calculating trajectory error. Previously, the IMU data (Euler angles or quaternions) actually collected from the affected limb needed to be converted to the same world coordinate system as the training data from the healthy limb. If Euler angles were used, the roll and yaw angles needed to be mirrored (e.g., inverted or offset) to eliminate the anatomical symmetry differences between the left and right limbs and ensure... and Compare under the same spatial reference.

[0163] This embodiment is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0164] The above embodiments have the following technical effects:

[0165] First, during the training phase, the deep learning model is trained by simultaneously collecting EEG signals of central motor intent from the cerebral hemisphere corresponding to the affected limb, sEMG signals of peripheral electromyography (EMG) from the healthy limb, and kinematic signals of the healthy limb. The sEMG and kinematic signals are then used as a fused teacher signal to train the model, enabling it to learn a more complete mapping relationship from central intent to peripheral physiological patterns. This fused teacher signal approach overcomes the limitations of existing technologies that use only a single modality signal or no teacher signal at all. It allows the deep learning model to learn not only "how muscles should be activated" (via sEMG) but also "what kind of movement trajectory this activation will produce" (via kinematics). Since sEMG reflects the timing and intensity of muscle activation (i.e., muscle drive patterns), while kinematics reflects the actual spatiotemporal trajectory of limb movement (i.e., movement execution results), their fusion provides the model with a more complete physiological blueprint encompassing both "cause" and "effect," thereby enhancing the model's understanding of human coordinated movement patterns. This enhanced understanding helps improve the quality of predicted outputs during the application phase. Specifically, the predicted coordinated electromyography (P-sEMG) patterns and the predicted coordinated motion trajectories (P-Kinematics) can more accurately reflect the physiological state of the unaffected limb when performing the target action.

[0166] Secondly, employing deep learning models capable of handling temporal mapping relationships, such as Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), Gated Recurrent Units (GRUs), or Transformer networks, enables the system to effectively capture the inherent long-term temporal dependencies in human motion signals. Human motion intention (EEG), muscle activation (sEMG), and limb movement (Kinematics) are all highly temporally correlated dynamic processes; the current state often depends on historical states over a period of time. In particular, LSTMs or GRUs, through gating mechanisms, can selectively memorize and forget information, thus considering the progressive nature of motion intention, the sequential nature of muscle activation, and the continuity of movement trajectories when learning the mapping from EEG to P-sEMG and P-Kinematics. This temporal modeling capability helps improve the consistency between the predicted output and the actual physiological process in terms of instantaneous values ​​and temporal evolution patterns, resulting in smoother and more natural motion during the application phase.

[0167] Furthermore, by employing a hybrid loss function to jointly optimize the prediction errors of sEMG and Kinematics, the deep learning model is required to simultaneously reduce the prediction errors of both modalities during training. This joint optimization mechanism enables the model's internal representation to simultaneously encode muscle activation patterns and motion trajectory information, rather than treating them as independent learning tasks. Since muscle activation is a physiological driver of motion trajectory, there is an inherent causal relationship between the two. The design of the hybrid loss function allows the model to maintain consistency and coordination between the two output branches when learning this causal relationship. This consistency and coordination ensures that, in the application phase, when the model simultaneously predicts P-sEMG and P-Kinematics based on the affected side's EEG, the two maintain a physiological match similar to that of the unaffected limb during actual movement, thus providing high-quality input for subsequent feedforward-feedback dual-loop control.

[0168] Furthermore, in the application phase, the acquired EEG signals undergo the same preprocessing as in the training phase (including bandpass filtering, notch filtering, and independent component analysis (ICA) to remove artifacts), and the IMU data is used for pose calculation to obtain Euler angles or quaternions. This improves the consistency between the real-time signal and the training data in the feature space. This consistency helps the deep learning model to decode accurately in the application phase. Simultaneously, by dividing the preprocessed signal into time windows of the same length as during training, the time scale of the input data matches the scale during model training, enabling the model to fully utilize its learned temporal dependencies for prediction.

[0169] In terms of feedback control, the predicted coordinated motion trajectory P-Kinematics is used as the target trajectory that changes over time. This trajectory is compared with the real-time acquired actual kinematic signals A-Kinematics of the affected limb to calculate the trajectory error. Based on this error, feedback correction commands for the FES (Feedforward Electrification System) are generated, constructing a real-time trajectory tracking closed loop. This closed-loop control mechanism endows the system with a certain degree of anti-interference capability. During rehabilitation training, the muscle state of the affected limb changes over time due to fatigue and strength variations. The electrode-skin contact impedance of the FES system changes due to sweating and electrode displacement. Furthermore, the brain's intention signal may deviate due to fluctuations in attention. These factors can all cause the actual response of the affected limb to the feedforward FES commands to deviate from expectations. Traditional pure feedforward control systems have limited ability to cope with these interference factors, mainly relying on the accuracy of the feedforward commands and the accuracy of system modeling. The above embodiment introduces a feedback correction loop based on P-Kinematics, enabling the system to continuously monitor the deviation between the actual movement trajectory of the affected limb and the target trajectory (P-Kinematics). Once a significant deviation is detected (i.e., trajectory error E(t)), the feedback controller (such as a PID controller, adaptive controller, or model predictive controller) immediately generates a correction command, dynamically adjusting the intensity of the FES stimulus to encourage the affected limb to move closer to the target trajectory. This real-time correction mechanism significantly improves the system's robustness and movement accuracy, allowing the affected limb to still track the target trajectory well even in the presence of various interference factors, thus improving the quality and stability of the movement.

[0170] In particular, when using an adaptive PID controller as a feedback controller, the proportional (P), integral (I), and derivative (D) gains can be adjusted online to adapt to the time-varying characteristics of the affected limb's muscle state. At different stages of rehabilitation training, the patient's muscle strength, fatigue level, and neural response speed all change, making it difficult for a controller with fixed parameters to maintain good performance throughout the entire rehabilitation process. Adaptive PID, by identifying changes in the parameters of the controlled object (the dynamic response characteristics of the affected limb under FES) online, dynamically adjusts the control gain, thereby helping the feedback control maintain a better working state. This adaptive capability improves the system's adaptability to slow time-varying factors such as muscle fatigue and strength changes, and also enhances its ability to suppress fast time-varying factors such as electrode shifting and impedance abrupt changes.

[0171] In terms of command fusion, the feedforward control command and feedback correction command are fused using additive fusion or gain modulation to generate the final FES command, achieving an organic combination of "physiological pattern driving" and "error feedback correction." The key to this fusion strategy is that the feedforward control command provides the "basic pattern" of FES stimulation, determining which muscles are activated at what time and how the relative activation intensity is distributed, which contributes to the coordination of the movement; while the feedback correction command adjusts the stimulation intensity according to the deviation between the actual movement and the target movement, which contributes to the accuracy and robustness of the movement. By fusing the two within each control cycle, the system can achieve a more natural and precise control effect. Specifically, when the actual movement trajectory of the affected limb is close to the target trajectory (i.e., the trajectory error is small), the contribution of the feedback correction command is small, and the FES is mainly driven by the feedforward command. At this time, the system can better reproduce the physiological coordination pattern of the healthy side. When a large deviation occurs (such as muscle fatigue leading to a weakened response), the contribution of the feedback correction command increases, correcting the deviation by increasing or decreasing the stimulation intensity of certain muscle channels, prompting the actual trajectory to move closer to the target trajectory. This dynamic trade-off allows the system to maintain a certain level of physiological coordination while also possessing a high-precision trajectory tracking capability.

[0172] In summary, the above embodiments, through the dual mapping capability established during the training phase using "fusion-type teacher signals" (from central intention to peripheral electromyographic patterns, and from central intention to motor trajectory), and through the synergistic cooperation of physiological pattern reproduction and real-time trajectory correction achieved during the application phase using "feedforward-feedback dual closed-loop control," help improve the problems of rigid control patterns, unnatural movements, lack of personalization, and insufficient robustness in existing BCI-FES systems. Regarding the naturalness of movement, the movements generated by the system are closer to the natural movements of the healthy limb in terms of multi-joint coordination, muscle activation timing, and trajectory smoothness. Regarding movement accuracy, the tracking error of the affected limb on the target trajectory is reduced, and it maintains relatively stable tracking performance even in the presence of external interference or changes in internal state. Regarding personalization, because the teacher signals come from the patient's own healthy limb, the trained model is more adapted to the individual patient's physiological characteristics and movement habits. Regarding the neurorehabilitation mechanism, the synchronicity of movement intention and muscle activation, as well as the coordination of movement provided by the system, provide induction conditions for neural plasticity, which helps promote the recovery of the patient's central nervous system function. The realization of these technical effects is the synergistic effect of multiple technical features in a specific combination, which reflects the overall advantages of the technical solutions in the above embodiments.

[0173] All references to this application are considered to be incorporated in their entirety into the disclosure of this application so that they can serve as a basis for modifications if necessary. Furthermore, it should be understood that after reading the foregoing disclosure of this application, those skilled in the art can make various alterations or modifications to this application, and these equivalent forms also fall within the scope of protection claimed in this application.

Claims

1. A rehabilitation system based on central-peripheral fusion decoding and trajectory closed-loop correction, characterized in that, include: The training subsystem includes: The training signal acquisition module is used to simultaneously acquire the central motor intention signal (EEG) of the cerebral hemisphere corresponding to the affected limb, the peripheral electromyographic signal (sEMG) of the healthy limb, and the peripheral kinematic signal (Kinematics) of the healthy limb. The model training module is used to train a deep learning model based on the EEG, sEMG and Kinematics acquired by the training signal acquisition module, wherein the sEMG and Kinematics are used as fusion teacher signals, so that the deep learning model can simultaneously predict the corresponding coordinated electromyographic pattern P-sEMG and coordinated motion trajectory P-Kinematics based on the input EEG. The application subsystem includes: The application signal acquisition module is used to acquire the EEG of the affected side of the brain and the actual kinematic signals of the affected side of the limbs in real time (A-Kinematics). The real-time decoding module is used to input the EEG acquired by the application signal acquisition module into the deep learning model trained by the model training module, and decode it in real time to obtain the P-sEMG and the P-Kinematics. The feedforward control module is used to generate feedforward control commands for multi-channel functional electrical stimulation (FES) based on the P-sEMG obtained by the real-time decoding module. The error calculation module is used to compare the P-Kinematics obtained by the real-time decoding module as the target trajectory that changes over time with the A-Kinematics acquired by the application signal acquisition module, and calculate the trajectory error. The feedback control module is used to generate the feedback correction command of the FES based on the trajectory error calculated by the error calculation module. The instruction fusion module is used to fuse the feedforward control instruction generated by the feedforward control module and the feedback correction instruction generated by the feedback control module to generate the final FES instruction; The stimulus execution module is used to output the final FES instruction generated by the instruction fusion module.

2. The system as described in claim 1, characterized in that, In the model training module, the sEMG and the kinematics are used as fused teacher signals, wherein the sEMG reflects the timing and intensity of muscle activation, and the kinematics reflects the actual spatiotemporal trajectory of limb movement, thereby providing the deep learning model with a physiological blueprint that includes muscle drive patterns and movement execution results.

3. The system as described in claim 1, characterized in that, The deep learning model includes a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a Transformer network, used to process the temporal mapping relationship from the EEG to the P-sEMG and the P-Kinematics.

4. The system as described in claim 1, characterized in that, In the model training module, the deep learning model is trained using a hybrid loss function. : in, This represents the mean square error function. and These are the weighting coefficients. The coordinated electromyographic patterns predicted by the deep learning model. The sEMG acquired by the training signal acquisition module. The coordinated motion trajectory predicted by the deep learning model. The Kinematics acquired by the training signal acquisition module.

5. The system as described in claim 1, characterized in that, The training subsystem also includes a preprocessing module for preprocessing the acquired signals: Bandpass filtering, notch filtering, and independent component analysis (ICA) were performed on the EEG to remove artifacts; The sEMG is filtered, rectified, and low-pass filtered to extract the envelope signal; The attitude calculation of the Kinematics is performed to obtain Euler angles or quaternions.

6. The system as described in claim 5, characterized in that, The model training module is used to align the preprocessed EEG, sEMG envelope signal and Euler angles or quaternions according to timestamps and divide them into time windows, each with a length of 100 milliseconds to 500 milliseconds.

7. The system as described in claim 1, characterized in that, The feedforward control module is used to modulate the pulse width or amplitude of the feedforward control command according to the amplitude of the P-sEMG, wherein the larger the amplitude of the P-sEMG, the greater the stimulation intensity of the corresponding FES channel.

8. The system as described in claim 1, characterized in that, The feedback control module includes a PID controller, an adaptive controller, or a model prediction controller, used to generate the feedback correction command based on the trajectory error. The PID controller includes a proportional gain (P), an integral gain (I), and a derivative gain (D) to track the P-Kinematics.

9. The system as described in claim 1, characterized in that, The instruction fusion module is used to fuse the feedforward control instruction and the feedback correction instruction through additive fusion or gain modulation to generate the final FES instruction. Specifically, the additive fusion is as follows: For the Each FES channel, at time... The final instruction is: in, For the first The final FES command for each channel, The feedforward control command generated based on the P-sEMG, The feedback correction command is generated based on the trajectory error.

10. The system as described in claim 5, characterized in that, The attitude representation of the Kinematics uses quaternions.

Images