Multimodal bci rehabilitation system for patients with intracerebral hemorrhage

Abstract

The embodiment of the application provides a kind of multi-modal BCI rehabilitation system for cerebral hemorrhage patient, can solve the problem that existing BCI system lacks safety monitoring and adaptive training adjustment. It includes: signal acquisition module;Multi-modal signal processing and motor intention classification module is used to extract electroencephalogram modal feature and near-infrared modal feature from electroencephalogram signal and functional near-infrared spectroscopy signal respectively, the motor intention of patient is classified, and motor intention classification result and classification confidence are obtained;Safety monitoring module is used to determine the oxygenated hemoglobin concentration rate of change of the patient according to functional near-infrared spectroscopy signal, compare the oxygenated hemoglobin concentration rate of change with the preset safety threshold, and carry out safety monitoring;Online incremental learning module is used to obtain the training data generated by patient in current motor imagination training task, and the training data is used to update classifier parameters;Rehabilitation scheme grading module is used to determine the training difficulty level corresponding to patient.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a multimodal BCI rehabilitation system for patients with cerebral hemorrhage. Background Technology

[0002] Intracerebral hemorrhage (ICH) is a high-mortality subtype of stroke, and survivors often suffer from severe motor dysfunction. Brain-computer interface (MI-BCI) based on motor imagery can drive external devices by decoding the patient's motor intentions, promoting the recovery of neuroplasticity. However, the application of BCI to ICH patients faces specific challenges: (1) Brain injury in patients leads to abnormal neural signal patterns, which differ significantly from healthy subjects, limiting the effectiveness of cross-subject transfer learning; (2) Existing BCI systems lack safety monitoring mechanisms and cannot assess the patient's cerebral hemodynamic status in real time during training, posing a risk of overtraining; (3) The system's training program is fixed and cannot be adaptively adjusted according to the patient's real-time performance, resulting in insufficient personalization. Summary of the Invention

[0003] This application provides a multimodal BCI rehabilitation system for patients with cerebral hemorrhage, which can solve the problems of existing BCI systems lacking safety monitoring and adaptive training adjustment.

[0004] This application provides a multimodal BCI rehabilitation system for patients with cerebral hemorrhage, comprising: The signal acquisition module is used to simultaneously acquire the patient's electroencephalogram (EEG) signals and functional near-infrared spectral signals during the patient's motor imagery training task. The multimodal signal processing and motion intention classification module is used to extract EEG modal features and near-infrared modal features from the EEG signal and the functional near-infrared spectral signal, respectively, and classify the patient's motion intention based on the EEG modal features and the near-infrared modal features to obtain the motion intention classification result and classification confidence. The safety monitoring module is used to determine the rate of change of the patient's oxyhemoglobin concentration based on the functional near-infrared spectral signal, compare the rate of change of the oxyhemoglobin concentration with a preset safety threshold, and output a control command to pause the current motor imagery training task when the rate of change of the oxyhemoglobin concentration exceeds the preset safety threshold. An online incremental learning module is used to acquire training data generated by the patient in the current motor imagery training task, and update the classifier parameters using the training data while keeping the feature extractor parameters in the multimodal signal processing and motor intention classification module unchanged; and The rehabilitation program grading module is used to determine the training difficulty level corresponding to the patient based on the classification confidence level and hemodynamic indicators characterizing the changes in oxyhemoglobin concentration.

[0005] In summary, the multimodal BCI rehabilitation system for patients with cerebral hemorrhage provided in this application simultaneously acquires EEG signals and functional near-infrared spectroscopy signals, enabling motor intention classification to obtain both neural electrical activity information and hemodynamic information. This reduces classification instability caused by noise, individual patient differences, or lesion differences affecting single-modal signals. The functional near-infrared spectroscopy signal serves two different functions in this scheme: its extracted near-infrared modal features participate in motor intention classification, and its oxyhemoglobin concentration change rate is also used for training safety assessment. Therefore, the same acquired signal serves both motor intention recognition and training risk control, rather than merely as an additional classification feature of the EEG signal. The safety monitoring module and the rehabilitation program grading module are controlled using different time scales. The safety monitoring module continuously judges the oxyhemoglobin concentration change rate during the current training process and promptly pauses the task when it exceeds a safety threshold; the rehabilitation program grading module integrates the classification confidence and hemodynamic indicators of the current round of training to adjust the parameters for the next round of training. Therefore, the system can handle safety situations requiring immediate response during training and can also adjust the subsequent training load based on the overall performance over a training period. Furthermore, the online incremental learning module enables the classifier to be updated as patient data changes, while the rehabilitation program grading module continues to adjust the training difficulty based on the classification confidence provided by the updated classifier. This reduces the judgment bias caused by a classifier that has been fixed for a long time. For example, if the patient's signal has changed but the classifier has not been updated, a decrease in classification confidence may be mistakenly interpreted as a decline in the patient's training ability; updating the classifier using the current patient data can, to some extent, distinguish between model mismatch and a decline in the patient's actual performance.

[0006] Correspondingly, the electronic system and computer-readable storage medium provided in the embodiments of the present invention also have the above-mentioned technical effects. Attached Figure Description

[0007] Figure 1 A schematic structural block diagram of a possible multimodal BCI rehabilitation system for patients with cerebral hemorrhage, provided in an embodiment of this application; Figure 2 A schematic diagram of the hardware structure of a possible multimodal BCI rehabilitation system for patients with cerebral hemorrhage, provided in an embodiment of this application; Figure 3 This is a schematic structural block diagram of a possible electronic system provided for an embodiment of this application. Detailed Implementation

[0008] This application provides a multimodal BCI rehabilitation system for patients with cerebral hemorrhage, which can solve the problems of existing BCI systems lacking safety monitoring and adaptive training adjustment.

[0009] To resolve the above issues, please refer to [link / reference]. Figure 1 This is a schematic structural block diagram of a multimodal BCI rehabilitation system for patients with cerebral hemorrhage, provided in an embodiment of this application. Specifically, it may include: The signal acquisition module 11 is used to simultaneously acquire the patient's electroencephalogram (EEG) signals and functional near-infrared spectral signals during the patient's motor imagery training task. The multimodal signal processing and motion intention classification module 12 is used to extract EEG modal features and near-infrared modal features from the EEG signal and the functional near-infrared spectral signal, respectively, and classify the patient's motion intention based on the EEG modal features and the near-infrared modal features to obtain the motion intention classification result and classification confidence. Safety monitoring module 13 is used to determine the rate of change of the patient's oxyhemoglobin concentration based on the functional near-infrared spectral signal, compare the rate of change of the oxyhemoglobin concentration with a preset safety threshold, and output a control command to pause the current motor imagery training task when the rate of change of the oxyhemoglobin concentration exceeds the preset safety threshold. Online incremental learning module 14 is used to acquire training data generated by the patient in the current motor imagery training task, and update the classifier parameters using the training data while keeping the feature extractor parameters in the multimodal signal processing and motor intention classification module unchanged; and The rehabilitation program grading module 15 is used to determine the training difficulty level corresponding to the patient based on the classification confidence level and hemodynamic indicators characterizing the changes in the oxygenated hemoglobin concentration.

[0010] It is understandable that when patients with cerebral hemorrhage perform motor imagery training tasks, their brains simultaneously undergo changes in electrophysiological activity and hemodynamics. Electroencephalography (EEG) signals can reflect the electrical activity of neuronal populations with high temporal resolution, especially the rapid changes in energy at specific frequency bands in the sensorimotor cortex after the initiation of motor imagery. Functional near-infrared spectroscopy (FIR) signals can reflect changes in the concentrations of oxyhemoglobin and deoxyhemoglobin in brain tissue, characterizing the cortical activation state of the patient during motor imagery tasks from the perspective of blood oxygen supply and cerebral blood flow response. The physiological processes represented by these two signals are not the same. EEG signals respond quickly to neural activity but are easily affected by electrooculography (EOG), electromyography (EMG), and environmental noise; FIR signals change relatively slowly but can provide information related to local cerebral blood flow and oxygen metabolism. Therefore, this approach does not rely solely on one signal to determine whether the patient has generated motor intention, but rather simultaneously collects both signals, extracts EEG modal features and FIR modal features separately, and combines the two types of features to classify motor intention. After obtaining the motor intention classification results, the system not only uses the classification results to drive the rehabilitation equipment, but also uses classification confidence to evaluate whether the patient's current motor imagery state is easily identifiable, and uses the rate of change in oxyhemoglobin concentration to evaluate whether the patient's current cerebral hemodynamic changes are within acceptable limits. Classification confidence primarily answers whether the patient can stably generate identifiable motor imagery patterns, while the rate of change in oxyhemoglobin concentration primarily answers whether continuing the current training load might cause excessively rapid changes in cerebral hemodynamics. When the rate of change in oxyhemoglobin concentration exceeds a preset safety threshold, the system pauses the current training task, thus avoiding further increasing the training load simply because the classification result is good. Current training data can also be used to update the classifier, gradually adapting the classification boundaries to the patient's own signal distribution. The system further determines the training difficulty based on classification confidence and hemodynamic indicators, and adjusts the task duration, number of training trials, or rest interval for the next round of training. Thus, the system forms a continuous iterative rehabilitation adjustment process: during training, neural electrical activity and cerebral hemodynamics information are acquired simultaneously, the movement intention is identified and the safety status is judged based on the two types of information, and then the classifier is updated and subsequent training parameters are adjusted based on the patient's actual performance.

[0011] For example, EEG electrodes and a near-infrared (NIR) probe can be placed on the patient's head. The EEG electrodes cover at least the sensorimotor cortex region related to motor imagery, and the NIR probe is placed in the scalp region corresponding to or adjacent to the EEG electrodes, so that the two acquisition methods can observe the activity of the same or similar brain regions as much as possible. Before training begins, the system displays motor imagery task prompts to the patient. For example, prompts such as imagining the left hand making a fist, imagining the right hand making a fist, or remaining at rest are displayed. When the task prompt appears, the task control device generates an event marker and sends the event marker to both the EEG acquisition device and the functional near-infrared spectroscopy acquisition device. The event marker can include at least the task type, task start time, task end time, and training trial number. Since the sampling rate of EEG signals is usually higher than that of functional near-infrared spectroscopy signals, the system can retain the original sampling sequences of both signals separately and extract data segments corresponding to the same task based on the event markers. If necessary, the data segments can be time-corrected according to a unified time reference so that both EEG data points and NIR data points can be associated with the time position relative to the task start time. For example, an EEG acquisition device can use a sampling rate of 256 Hz, while a functional near-infrared spectroscopy acquisition device can use a sampling rate of 11 Hz. When a motor imagery cue appears at the tenth second, both acquisition devices record the event marker corresponding to the cue. During subsequent feature extraction, the system can extract baseline data before the cue appears and task data after the cue appears, rather than simply extracting data based on the start times of the files from each device. In another implementation, if there is a fixed transmission delay between the two devices, the fixed delay amount can be determined through synchronization testing before training, and the corresponding timestamps can be compensated before the data enters the multimodal signal processing and motor intention classification module.

[0012] For example, for EEG signals, bandpass filtering can be performed first to suppress baseline drift, high-frequency noise, and some irrelevant frequency components. Subsequently, short-time Fourier transform can be used to perform time-frequency analysis on the EEG signals, extracting power variations, event-related desynchronization amplitude, event-related desynchronization duration, or characteristic differences between different brain regions in the 8 Hz to 12 Hz and 12 Hz to 27 Hz frequency bands. For instance, an EEG signal before the start of the task can be used as a baseline to calculate the proportion of frequency band power changes during the task execution phase relative to the baseline phase. When a patient imagines right-hand movement, if a significant decrease in frequency band power occurs in the corresponding channel of the left sensorimotor cortex, this change can be considered one of the EEG modal characteristics of right-hand movement imagination. For functional near-infrared spectral signals, light intensity changes at different wavelengths can be converted into changes in oxyhemoglobin concentration and deoxyhemoglobin concentration according to the Beer-Lambert law. Subsequently, the concentration signals are subjected to low-frequency bandpass filtering suitable for hemodynamic signals, and baseline correction is performed using the signal mean before the start of the task. Near-infrared modal features can include the increase in oxyhemoglobin concentration relative to baseline, the decrease in deoxyhemoglobin concentration relative to baseline, the slope of concentration change, the time of peak occurrence, the average concentration change during the task, and the differences in concentration changes between different brain regions. After feature extraction for both modalities, the EEG modal features and near-infrared modal features can be normalized separately to reduce the influence of different feature dimensions and value ranges on the classification results. Subsequently, the two types of features can be input into a motor intention classifier. The classifier can output the probability corresponding to each candidate motor intention category, and determine the category with the highest probability as the motor intention classification result, and determine the corresponding probability or the calibrated category score as the classification confidence. For example, candidate categories include left-hand motor imagery, right-hand motor imagery, and resting state. If the classifier outputs a probability of 85% for left-hand motor imagery, 10% for right-hand motor imagery, and 5% for resting state, then left-hand motor imagery can be determined as the classification result, and 85% can be used as the classification confidence. To avoid drastic fluctuations in confidence caused by a single outlier data point, statistics can be collected on multiple consecutive classification results in the same training trial, and the average confidence or median confidence within the continuous time window can be used as the classification confidence for that training trial.

[0013] For example, functional near-infrared spectroscopy signals not only participate in motor intention classification but also serve a training safety monitoring function. Motor imagery training may cause changes in the concentration of oxyhemoglobin in relevant brain regions. Moderate changes can reflect brain region participation in the task, but if the concentration of oxyhemoglobin changes too rapidly in a short period of time, it may indicate that the patient's current hemodynamic response is strong, or that the patient is affected by fatigue, stress, respiratory changes, or other factors. Compared to simply judging whether the concentration of oxyhemoglobin has reached a certain absolute value, monitoring the rate of change of oxyhemoglobin concentration can identify rapid changes in a short period of time earlier. The rate of change of oxyhemoglobin concentration can be expressed as:

[0014] in, This represents the rate of change in oxyhemoglobin concentration at the current moment. This indicates the current concentration of oxyhemoglobin. This indicates the time window used for calculating the rate of change. The preset safety threshold should use the same dimension as the rate of change in oxyhemoglobin concentration. This threshold can be determined based on the patient's baseline data before training, the patient's individual condition, and pre-set safety rules. When the calculated rate of change exceeds the preset safety threshold, the system will not wait for the current training session to end naturally, but will instead output a pause control command. Before training begins, functional near-infrared spectral signals can be collected from the patient in a resting state to determine the baseline of oxyhemoglobin concentration and its natural fluctuation range. Subsequently, during motor imagery training, the safety monitoring module continuously receives the pre-processed oxyhemoglobin concentration signal. The rate of change can be calculated continuously according to a sliding time window. For example, the difference in oxyhemoglobin concentration between the start and end points of the window can be calculated every two seconds, and then this difference can be divided by two seconds to obtain the rate of change in oxyhemoglobin concentration for the current window. To make the comparison results meaningful, the preset safety threshold can be set as a rate of change threshold determined based on the current patient's baseline state, rather than directly comparing the concentration value with the rate of change. In implementation, thresholds can be set separately for excessively rapid increases and decreases in oxyhemoglobin concentration to accommodate different abnormal change directions. When the rate of change exceeds the safety threshold, the safety monitoring module outputs a pause control command to the motor imagery training task execution device. The task execution device can stop presenting new motor imagery prompts, stop the rehabilitation robot, functional electrical stimulation device, or virtual training scenario from continuing to execute the current action, and display a rest prompt to the patient. In one embodiment, to reduce false pauses caused by individual noise points, the oxyhemoglobin concentration signal can be filtered first, and the state exceeding the threshold can be required to persist for a preset confirmation time before outputting the pause control command. However, the confirmation time should not be set too long to avoid weakening the timeliness of safety monitoring.

[0015] For example, when a patient performs a motor imagery training task, the system already knows the target category corresponding to the current task prompt. For instance, if the system presents the patient with a right-hand motor imagery prompt, then this training session can have a right-hand motor imagery label. The signal acquisition module obtains the EEG signal and functional near-infrared spectral signal of this training session, the multimodal signal processing and motor intention classification module obtains the corresponding features, and the category corresponding to the task prompt can be used as a supervision label. After collecting several training sessions, the online incremental learning module constructs a small-scale incremental training dataset from these training sessions. During incremental updates, the parameters of the feature extractor are set to a non-updateable state, and the backpropagation process only calculates and updates the classifier parameters. For example, the feature extractor outputs a feature vector representing the multimodal brain signal of the current training session, and the classifier outputs the category probabilities corresponding to left-hand motor imagery, right-hand motor imagery, and resting state based on this feature vector. Online updates only adjust the parameters between the feature vector and the category probabilities, allowing the classifier to gradually adapt to the patient's feature distribution. In one specific implementation, incremental updates can be performed every 5 to 20 training sessions, using a learning rate lower than that of the pre-training phase. A smaller learning rate can prevent classifier parameters from changing drastically due to a small amount of abnormal training data. After the update, the new classifier can be used for the next round of motion imagery training. If the current training data is interrupted due to a safety warning, or the corresponding signal quality does not meet the requirements, the training trial can be excluded from parameter updates to avoid abnormal data affecting the classifier.

[0016] For example, the rehabilitation program grading module can statistically analyze the classification confidence level and hemodynamic parameters of each training round after its completion. Classification confidence level can be characterized by the average, median, or percentage of low-confidence trials across multiple effective training trials. Hemodynamic parameters can be characterized by the maximum, average, fluctuation range, or proportion of the rate of change in oxyhemoglobin concentration relative to a preset safety threshold. Training difficulty can be categorized into advanced, intermediate, and beginner levels. When classification confidence is high and oxyhemoglobin concentration changes remain within a safe range, an advanced training program can be used, appropriately extending the duration of a single motor imagery task or increasing the number of training trials. For example, the duration of a single motor imagery task can be set to fifteen seconds, and the number of training trials within a training period can be set to thirty. When classification confidence is in the moderate range, or when oxyhemoglobin concentration fluctuates slightly but has not yet reached the safety limit, an intermediate training program can be used. For example, the duration of a single motor imagery task can be set to ten seconds, and the number of training trials within a training period can be set to twenty. When classification confidence is low, or the rate of change in oxyhemoglobin concentration is close to the preset safety threshold, a basic training program can be used. For example, the duration of a single motor imagery task can be shortened to five seconds, the number of training trials in a training session can be reduced to ten, and the rest interval between adjacent tasks can be appropriately extended. When classification confidence and hemodynamic parameters show different adjustment tendencies, the safety status represented by the hemodynamic parameters can be prioritized. For example, if classification confidence is high but the rate of change in oxyhemoglobin concentration is close to the safety threshold, the training difficulty should not be increased; if classification confidence is low but the hemodynamic status is stable, the task difficulty can be maintained or reduced first, and online incremental learning can be used to determine whether the low confidence is caused by the classifier not yet adapting to the patient. Adjusting the training difficulty does not require changing all training parameters simultaneously. The system can adjust only the task duration, only the number of training trials or the rest interval, or adjust two or three of them according to the patient's condition.

[0017] In summary, this approach improves the stability of motor intention recognition in patients with cerebral hemorrhage and its adaptability to individual differences. By monitoring the rate of change in oxyhemoglobin concentration, it promptly suspends unsuitable training tasks, enhancing the safety of the training process. It allows for rapid adjustment of the classifier even with limited patient data, reducing reliance on complete offline retraining. The training difficulty is determined based on the patient's motor intention recognition level and cerebral hemodynamic status, reducing the possibility of training programs being too difficult, too easy, or mismatched with the patient's condition. Furthermore, the training parameters can be gradually adjusted according to the patient's actual condition at different training stages, making it more suitable for applications where there are significant individual differences in cerebral hemorrhage patients and their rehabilitation status is constantly changing.

[0018] In one embodiment, the signal acquisition module includes an electroencephalogram (EEG) acquisition device, a functional near-infrared spectroscopy (NIRS) acquisition device, and an event tagging unit; The EEG acquisition device has no less than 30 EEG acquisition channels and a sampling rate of no less than 200 Hz. The functional near-infrared spectral acquisition device has no less than 36 near-infrared acquisition channels, uses a dual-wavelength near-infrared light source of 760 nm and 850 nm, and has a sampling rate of no less than 10 Hz. The event tagging unit is used to write event tags corresponding to the same motor imagery training event into the EEG signal and the functional near-infrared spectral signal, respectively, so that the two signals are aligned on a unified time axis.

[0019] For example, before training, EEG electrodes, near-infrared (NIIR) light source probes, and detection probes are placed on the patient's head. The EEG electrodes can primarily cover C3, C4, Cz, and their adjacent areas, or additional channels can be added to the motor-related areas on both the affected and healthy sides depending on the location of the cerebral hemorrhage lesion. The NIIR light source probes and detection probes can cover the left and right motor cortexes, creating a spatial association between the brain regions corresponding to the functional near-infrared spectral signals and the brain regions corresponding to the EEG signals. The EEG acquisition device can be configured with 32 EEG acquisition channels and a sampling rate of 256 Hz. The functional near-infrared spectral acquisition device can be configured with 36 near-infrared acquisition channels, using 760 nm and 850 nm near-infrared light sources and a sampling rate of 11 Hz. The task control device displays prompts to the patient, such as imagining clenching their left fist, imagining clenching their right fist, or remaining at rest. Each time a task prompt appears, the event tagging unit generates an event tag containing the training trial number, task category, and task start time, and writes it to the EEG data stream and functional near-infrared spectral data stream, respectively. At the end of the task, a corresponding end event tag can also be written. In subsequent processing, the system locates the two types of signals based on the same training trial number. For example, the event marker numbered twelfth training trial corresponds to one sampling point in the EEG signal and another sampling point in the functional near-infrared spectroscopy signal. The system sets both sampling points as relative time zero points to extract EEG signal segments and functional near-infrared spectroscopy signal segments before and after the same task prompt. If there is a fixed transmission delay between the two acquisition devices, the delay can be measured using a synchronization trigger signal before training and compensated for when writing event markers or aligning data. If the EEG acquisition device receives the event marker two milliseconds later than the near-infrared acquisition device, the EEG event time can be corrected forward by two milliseconds during alignment.

[0020] In one embodiment, the multimodal signal processing and motion intent classification module includes a preprocessing and time-frequency analysis submodule, which is used for: The EEG signal is bandpass filtered from 0.5 Hz to 50 Hz, and the filtered EEG signal is subjected to short-time Fourier transform to extract event-related desynchronization features in the 8 Hz to 12 Hz frequency band and the 12 Hz to 27 Hz frequency band. According to Beer-Lambert's law, the functional near-infrared spectral signal is converted into oxyhemoglobin concentration signals and deoxyhemoglobin concentration signals. These signals are then bandpass filtered from 0.01 Hz to 0.1 Hz, and cerebral hemodynamic response characteristics are extracted. Based on the mean value of the functional near-infrared spectral signal within a preset time period before the start of the motor imagery training task, the cerebral hemodynamic response characteristics are baseline corrected.

[0021] For example, for EEG signals, bandpass filtering of 0.5 Hz to 50 Hz can be applied to each EEG channel. If necessary, channels with significant distortion can be identified and removed, or time periods with abnormal instantaneous amplitudes can be marked, without changing the bandpass range. When performing a short-time Fourier transform on the filtered EEG signal, a preset analysis window can be selected, with partial overlap between adjacent windows. For each EEG channel, the average power in the 8 Hz to 12 Hz band and the 12 Hz to 27 Hz band is calculated during the pre-task baseline phase and the task execution phase, respectively. The degree of event-related desynchronization can be determined by the ratio of the change in band power during the task phase to the change in band power during the baseline phase. For example, the following can be used:

[0022] in, Indicates the frequency band power during the mission phase. This represents the frequency band power during the baseline phase before the task. A negative result indicates that the frequency band power during the task phase is lower than the frequency band power during the baseline phase. The larger the absolute value, the more pronounced the event-related desynchronization in the corresponding frequency band. For functional near-infrared spectral signals, the optical density change is first determined based on the incident and received light intensities at two wavelengths. Then, the relative changes in the concentrations of oxyhemoglobin and deoxyhemoglobin at the two wavelengths are calculated using their extinction coefficients. Subsequently, the oxyhemoglobin and deoxyhemoglobin concentration signals are bandpass filtered from 0.01 Hz to 0.1 Hz. Using the three seconds before the start of the motion visualization task as the baseline time period, the average concentrations of oxyhemoglobin and deoxyhemoglobin during this time period are calculated. The concentration values ​​at each sampling point during the task phase are subtracted from the corresponding baseline average value to obtain the baseline-corrected concentration change signal. Brain hemodynamic response characteristics can include the average change in oxyhemoglobin concentration, the maximum change in concentration, the slope of the concentration change, the time required to reach the peak, the change in deoxyhemoglobin concentration, and the differences in changes between different brain regions during the task phase. For example, for right-hand motor imagery training, the increase in oxyhemoglobin concentration in the corresponding near-infrared channel of the left motor cortex can be extracted, and the response differences between the left and right motor cortices can be extracted.

[0023] In one embodiment, the multimodal signal processing and motion intent classification module further includes a source domain selection submodule, which is used for: Obtain source domain EEG samples and source domain functional near-infrared spectral samples corresponding to multiple candidate source domain subjects, as well as target domain EEG samples and target domain functional near-infrared spectral samples corresponding to the patient. Calculate the first Wasserstein distance between the source domain EEG samples of each candidate source domain subject and the target domain EEG samples, and the second Wasserstein distance between the source domain functional near-infrared spectral samples of each candidate source domain subject and the target domain functional near-infrared spectral samples. The source domain subjects for EEG modality transfer are determined based on the first Wasserstein distance, and the source domain subjects for functional near-infrared spectroscopy modality transfer are determined based on the second Wasserstein distance, so that the EEG modality and the functional near-infrared spectroscopy modality select their respective corresponding source domain subjects.

[0024] For example, a consistent preprocessing method is used for data from all candidate source domain subjects and target patients. For EEG modalities, features within the same channel range, frequency band, and task time period are extracted. For functional near-infrared spectroscopy modalities, features within the same brain region, hemoglobin type, and time range are extracted. Let the EEG feature distribution of the i-th candidate source domain subject be... The distribution of EEG characteristics of the target patients is as follows Then the first Wasserstein distance can be expressed as:

[0025] Let the functional near-infrared spectral feature distribution of the subject in the iii-th candidate source domain be... The functional near-infrared spectral characteristics of the target patient are as follows: Then the second Wasserstein distance can be expressed as:

[0026] In actual calculations, either an optimal transfer calculation method based on empirical samples or a Sinkhorn distance calculation method with entropy regularization can be used to reduce the computational load under conditions of a large number of samples. For EEG modalities, the candidate source domain subject with the smallest first Wasserstein distance can be selected as the EEG source domain subject, or multiple candidate source domain subjects with small distances can be selected to form an EEG source domain subject set. For functional near-infrared spectroscopy modalities, independent selection can be performed according to the second Wasserstein distance. The finally selected EEG source domain subjects and near-infrared source domain subjects can be the same or different. For example, the candidate source domain subjects include subject A, subject B, and subject C. If the EEG distance comparison results show that subject A is closest to the target patient, and the near-infrared distance comparison results show that subject C is closest to the target patient, then the system selects subject A's EEG data to assist in EEG modal transfer and selects subject C's functional near-infrared spectroscopy data to assist in near-infrared modal transfer.

[0027] In one embodiment, the source domain selection submodule is used to select the source domain subjects based on an adaptive Wasserstein trust score, which is determined according to an adaptive distance threshold, a temporal trust score, and a multi-granularity distance. The adaptive distance threshold is updated based on the Wasserstein distance statistics between the target domain sample and multiple candidate source domain samples, as well as the threshold adjustment coefficient that changes with the training process. The temporal trust score is used to characterize the stability of distance changes among subjects in the same candidate source domain across multiple adjacent training phases. The multi-granularity distance is obtained by fusing Wasserstein distances at different time scales or different feature levels.

[0028] For example, for the t-th training phase, the Wasserstein distances between all candidate source domains and the target domain are calculated. The base distance scale is determined based on the statistical centers of these distances. Then, the threshold adjustment coefficient is determined based on the number of samples in the target domain, the degree of distance dispersion, or the training stage. Thus we get:

[0029] in, This represents the adaptive distance threshold for the t-th training phase. In the early stages of training, when there are fewer samples in the target domain, the threshold adjustment coefficient can be appropriately increased to retain more candidate source domains. As the number of samples in the target domain increases, the threshold adjustment coefficient can be gradually decreased to improve the stringency of source domain selection. The temporal trust score can be recursively determined based on the distance in the current training phase and the trust score in the previous training phase. For example, it can be calculated as follows:

[0030] in, This represents the temporal trust score of the i-th candidate source domain at the t-th training stage. This indicates its current Wasserstein distance. This represents the retention coefficient of historical trust scores. When a candidate source domain has small distances across multiple consecutive training phases, its temporal trust score remains high. If a candidate source domain only shows small distances occasionally in a single training phase, while the distances in other phases are larger, its trust score will not immediately rise to a high level. Multi-granularity distances can be calculated from different time scales. For example, the distances can be calculated separately for the entire training trial, the early time period after the task begins, and the time period when the task response is more obvious. They can also be calculated from different feature levels, such as calculating the distances for preprocessed base features, convolutional intermediate layer features, and classifier input features. Let the distance corresponding to the g-th granularity be... Then the multi-granularity distance can be expressed as:

[0031] Where, ω g This represents the weight of the g-th granularity, with the sum of all granularity weights being one. Ultimately, the adaptive Wasserstein trust score is determined by combining whether the candidate source domains meet the adaptive distance threshold, the temporal trust score, and the multi-granularity distance ranking results, prioritizing candidate source domains with higher trust scores. The above calculations are performed separately for the EEG modality and the functional near-infrared spectroscopy modality, rather than directly mixing the raw distances from the two modalities.

[0032] In one embodiment, the multimodal signal processing and motion intent classification module includes a progressive dual-domain alignment submodule, which is used for: In the previous training phase of the transfer training process, the second-order statistical difference between the source domain features corresponding to the source domain subjects and the target domain features corresponding to the patients is constrained by deep correlation alignment loss in order to perform initial alignment of the source domain features and the target domain features. In the later training stage of the transfer learning process, a deep canonical correlation analysis loss is introduced on top of the deep correlation alignment loss to align the source domain features and the target domain features based on their relevance; and according to The weights of the domain alignment loss are attenuated, where, Indicates the first The domain alignment loss weights corresponding to each training epoch This represents the initial domain alignment loss weights. This represents the total number of training rounds in the transfer training process.

[0033] For example, ...

[0034] In one embodiment, the multimodal signal processing and motion intent classification module includes a channel attention-enhanced depth-separable convolution classification submodule; The depthwise separable convolutional classification submodule includes a one-dimensional convolutional layer, a depthwise separable one-dimensional convolutional layer, a compressed excitation channel attention unit, a multi-scale feature fusion unit, and an adaptive depth selection unit. The compressed excitation channel attention unit is set after the depth-separable one-dimensional convolutional layer and is used to generate channel weights based on the contribution of different feature channels to the classification of motion intent. The multi-scale feature fusion unit is used to fuse features obtained from different receptive fields, and the adaptive depth selection unit is used to select network feature layers that participate in the motion intention classification based on the patient's multimodal signal features.

[0035] For example, after transfer training begins, the training process is divided into a previous training phase and a subsequent training phase. For instance, if the total number of training rounds is 100, the first 40 rounds can be designated as the previous training phase, and the last 60 rounds as the subsequent training phase. In the previous training phase, source domain samples and target domain samples are input into a shared feature extractor to obtain source domain feature matrices and target domain feature matrices, respectively. The covariance matrices of the two feature matrices are calculated separately, and the difference between the covariance matrices is used to construct a depth-related alignment loss.

[0036] This loss can be expressed as:

[0037] in, Represents the source domain characteristic covariance matrix. Represents the feature covariance matrix of the target domain. Represents the feature dimension. Let Frobenius norm be denoted. In the next training phase, in addition to continuing to calculate the deep correlation alignment loss, deep canonical correlation analysis loss is also introduced. Deep canonical correlation analysis can establish a correspondence between feature dimensions with similar variation patterns in the two domains by maximizing the correlation between the source domain feature projections and the target domain feature projections. In the t-th training epoch, the domain alignment loss weights are determined according to the following formula:

[0038] When training just started close to the initial weight Domain alignment has a significant impact on parameter updates. As training progresses, Gradually reduce the weights to allow the classification loss to play a greater role in later training stages. The model optimization objective in the previous training phase can include both the classification loss and the weighted deep correlation alignment loss. The model optimization objective in the later training phase can further include the deep canonical correlation analysis loss. Each loss should be calculated separately and combined according to preset weights, rather than treating the two alignment methods as a single constraint. EEG modalities and functional near-infrared spectroscopy modalities can be aligned separately, or the fused features can be aligned after initial alignment of each modality.

[0039] In some examples, the safety monitoring module includes an oxyhemoglobin change rate monitoring submodule and a threshold warning and pause submodule; The oxyhemoglobin change rate monitoring submodule is used to calculate the oxyhemoglobin concentration change rate according to a two-second sliding time window; The threshold warning and pause submodule is used to trigger a safety warning, pause the current motor imagery training task and output a rest prompt when the rate of change of the oxyhemoglobin concentration exceeds the preset safety threshold. After pausing the current motor imagery training task, the threshold warning and pause submodule continuously monitors the oxyhemoglobin concentration. When the oxyhemoglobin concentration returns to within 10 percent of the baseline concentration and remains stable for at least five seconds, it outputs a control command to resume the motor imagery training task.

[0040] For example, EEG modal features and near-infrared modal features, after time alignment and dimensionality adjustment, are input into a one-dimensional convolutional layer. The one-dimensional convolutional layer can initially extract local temporal variations and map the input to a preset number of feature channels. The depthwise separable one-dimensional convolutional layer first performs convolution along the time direction on each input channel, and then uses pointwise convolution to combine the information from different channels. This can both extract the temporal pattern of each channel and learn the correlation between different channels. The features output by the depthwise separable one-dimensional convolutional layer can be represented as feature tensors with channel and time dimensions. The compressed excitation channel attention unit first performs global statistics on each channel along the time dimension to obtain the descriptive value of each channel; then, it obtains the weights corresponding to each channel through a channel weight generation network; finally, it multiplies the features of each channel with the corresponding weights. For example, when a patient imagines right-hand movement, the left central EEG channel and the adjacent infrared channel have relatively stable responses, while other channels are greatly affected by noise. The channel attention unit can increase the weights of the features corresponding to the aforementioned effective channels and decrease the weights of the noisy channels. The multi-scale feature fusion unit can set feature extraction paths with different convolutional kernel lengths. Shorter convolutional kernels are used to extract localized, rapid changes, while longer kernels are used to extract changes over longer periods. The outputs of each path are fused after being adjusted for the time dimension. An adaptive depth selection unit receives features from different network layers and generates selection weights for each layer's features based on the patient's current data. If shallow local features are more discriminative for the current patient, their weights are increased; if deeper, abstract features are more stable, their weights are increased. The fused features are input to a fully connected classification layer, which outputs the probability and classification confidence score for each motion intent category.

[0041] In some examples, the online incremental learning module is used for: The data generated when the patient first uses the multimodal BCI rehabilitation system, or the data generated by the patient during a new training period, are identified as incremental training data. Freeze the parameters of the one-dimensional convolutional layer, the depthwise separable one-dimensional convolutional layer, and the compressed excitation channel attention unit in the feature extractor; Using incremental training data corresponding to 5 to 20 training trials, the gradient of the fully connected layer of the classifier is updated, wherein the learning rate used for gradient update is 10% to 20% of the pre-training learning rate.

[0042] For example, before training begins, the oxyhemoglobin concentration signal of the patient at rest is collected to determine the baseline concentration. During motor imagery training, calculations are performed using a two-second sliding window:

[0043] in, This indicates the current concentration of oxyhemoglobin. This indicates the concentration of oxyhemoglobin two seconds ago.

[0044] The preset safety threshold should be in the same unit as the rate of change, and can be determined based on the distribution of the rate of change corresponding to the patient's natural fluctuations before training, the patient's individual safety requirements, and clinical configuration.

[0045] when When the preset safety threshold is exceeded, the threshold warning and pause submodule outputs a safety warning, stops presenting new motor imagery tasks, pauses the current rehabilitation equipment operation, and displays a rest prompt.

[0046] During the pause, the system continues to acquire functional near-infrared spectral signals, but no new training tasks are assigned to the patient. The system determines whether the current oxyhemoglobin concentration meets the following requirements:

[0047] When the concentration remains within this range for five consecutive seconds, a recovery control command is output. If the concentration exceeds this range at any point during the five-second period, the continuous stabilization time is recalculated. For example, if the patient enters the baseline allowable range three seconds after a pause, but exceeds the range again in the fifth second, training cannot resume in the eighth second; instead, the patient must wait for the concentration to re-enter the allowable range and stabilize continuously for another five seconds.

[0048] For example, when a patient uses the system for the first time, 5 to 20 training trials with explicit task prompts can be performed before formal training. The motor imagery category corresponding to the task prompts serves as the training label, and the collected EEG and functional near-infrared spectroscopy signals serve as incremental training data. At the start of a new training period for the same patient, the initial few training trials of that period can also be used as incremental training data to adapt to changes in electrode position, probe contact, fatigue status, and signal distribution changes caused by rehabilitation progress. Before incremental updates, one-dimensional convolutional layers, depthwise separable one-dimensional convolutional layers, and compressed excitation channel attention units are set to an untrainable state. The incremental training data, after passing through a frozen feature extractor, forms a feature vector, and backpropagation and parameter updates are performed only on fully connected layers. If the learning rate during the pre-training phase is 0.001, the incremental update learning rate can be set to 0.0001 to 0.0002. Incremental updates can perform a small number of training rounds, and the classifier parameters corresponding to the current patient and the current training period are saved after the update. A reliable correspondence should be established between the incremental data and the task prompt labels. Training trials interrupted due to safety warnings, severe signal quality abnormalities, or failure of patients to complete tasks may not be used for incremental updates.

[0049] In some examples, the rehabilitation program grading module is used to classify the training difficulty level into advanced, intermediate, and beginner levels, wherein: when the classification confidence level is not less than 80% and the change in oxyhemoglobin concentration is within a safe range, the training difficulty level is determined to be advanced, the duration of the motor imagery task is adjusted to 15 seconds, and the number of training trials in a single training session is adjusted to 30; when the classification confidence level is not less than 60% but less than 80%, or when the oxyhemoglobin concentration fluctuates below the preset safety threshold, the training difficulty level is determined to be intermediate, the duration of the motor imagery task is adjusted to 10 seconds, and the number of training trials in a single training session is adjusted to 20; when the classification confidence level is less than 60%, or when the rate of change in oxyhemoglobin concentration reaches 80% of the preset safety threshold, the training difficulty level is determined to be beginner, the duration of the motor imagery task is adjusted to 5 seconds, the number of training trials in a single training session is adjusted to 10, and the rest interval between adjacent motor imagery tasks is adjusted to 20 seconds.

[0050] For example, to avoid overlapping conditions across different training levels, the decision can be made based on a safety priority order. First, if the rate of change in oxyhemoglobin concentration exceeds a preset safety threshold, the safety monitoring module pauses the current training and does not proceed to the normal training level. Second, if the classification confidence level is below 60%, or the rate of change in oxyhemoglobin concentration has reached 80% of the preset safety threshold, it is determined to be in the beginner level. In the beginner level, the duration of the motor imagery task is set to five seconds, the number of training trials within a single training session is set to ten, and the rest interval between adjacent tasks is set to twenty seconds. If the beginner level conditions are not met, but the classification confidence level is not lower than 60% but lower than 80%, or if there are slight fluctuations in oxyhemoglobin concentration that do not reach the preset safety threshold of 80%, it is determined to be in the intermediate level. In the intermediate level, the task duration is set to ten seconds, and the number of training trials is set to twenty. When the classification confidence level is not lower than 80%, and the change in oxyhemoglobin concentration is stable within a safe range, it is determined to be in the advanced level. In the advanced level, the task duration is set to fifteen seconds, and the number of training trials is set to thirty. Classification confidence can be achieved by using the average confidence of multiple valid training trials in the current round, or by using median confidence to reduce the impact of a single outlier trial. Whether oxyhemoglobin concentration is stable can be determined based on the maximum rate of change, the range of fluctuation in the rate of change, and the duration of proximity to the safe threshold.

[0051] In some examples, a rehabilitation training process control module and a report output module are also included; The rehabilitation training process control module is used to configure initial motor imagery training parameters based on the patient's motor function baseline, control the patient to perform motor imagery training tasks, and after completing the current round of motor imagery training tasks, control the rehabilitation program grading module to determine the training difficulty level and corresponding training parameters of the next round of motor imagery training tasks. The report output module is used to generate a training report and training suggestions for the next round based on the classification results of the motion intention, classification confidence, hemodynamic indicators, safety warning results, classifier parameter update results, and training parameters for the next round of motion imagery training tasks corresponding to the current round of motion imagery training tasks.

[0052] For example, before training begins, a baseline for motor function can be determined based on the patient's affected side, motor function score, sustained attention time, and previous training status. Based on this baseline, the initial task category, task duration, number of training trials, and rest interval are set. During training, the process control module presents motor imagery prompts according to the initial parameters, while the control signal acquisition module simultaneously acquires both types of signals, and ensures that motor intention classification, safety monitoring, and incremental learning run according to the corresponding training sequence. After the current round of training, the process control module obtains the classification results and classification confidence scores for each training trial, statistically analyzes the rate of change in oxyhemoglobin concentration, the number of safety warnings, and task pauses, and obtains the classifier parameter versions before and after online incremental learning. The rehabilitation program grading module determines the difficulty and parameters for the next round of training based on the current round's indicators. The process control module saves these parameters as the task configuration for the next round. The report output module generates a training report based on this. Training reports can include patient identification, training date, current training period, type of motor imagery task, number of effective training trials, classification results for each category, average classification confidence, rate of change in oxyhemoglobin concentration, safety warning results, pause duration, classifier parameter update status, next training level, duration of the next task, number of training trials, and rest interval. To ensure comparability between different training reports, version identifiers can be set for classifier parameters, allowing healthcare professionals to determine which version of the classifier generated the classification results in each report.

[0053] In some cases, existing multimodal classification systems typically assume a relatively stable correspondence between EEG activity and oxyhemoglobin response—that is, when a patient generates strong motor imagery EEG features, the relevant brain region should show a corresponding change in oxyhemoglobin concentration after a certain delay. However, patients with cerebral hemorrhage may experience decreased vascular reactivity, local perfusion impairment, or abnormal neurovascular coupling around the lesion. The patient may have already generated significant event-related desynchronization features, but the change in oxyhemoglobin concentration in the corresponding brain region may be weak or significantly delayed. In such cases, directly fusing the two modalities may invalidate the genuine motor intent and misinterpret the lack of hemodynamic response as the patient not performing the task. Therefore, in one embodiment, a neurovascular coupling state recognition submodule can be set up to determine the EEG activation start time, EEG activation intensity, oxyhemoglobin response start time, and oxyhemoglobin response intensity in each training trial. The hemodynamic response delay is determined based on the time difference between the onset of EEG activation and the onset of oxyhemoglobin response; the neurovascular coupling coefficient is determined based on the correspondence between EEG activation intensity and oxyhemoglobin response intensity. When multiple consecutive training trials show that EEG activation meets the effective conditions, but the near-infrared modal response is consistently absent, significantly delayed, or mismatched with the EEG activation intensity, the corresponding brain region is marked as a neurovascular decoupling brain region. For the near-infrared channels marked as neurovascular decoupling brain regions, their data are not directly deleted, but are used for two purposes: first, to reduce the fusion weight of the corresponding near-infrared features in motor intention classification, avoiding the rejection of effective EEG features; second, to continue to retain their oxyhemoglobin concentration change rate for safety monitoring, to prevent the error of equating low classification contribution with no need for safety monitoring.

[0054] In some cases, conventional functional near-infrared spectroscopy (FIR) analysis typically uses a fixed-task time window to extract hemodynamic response features. However, the location of the lesion, local vascular status, and recovery stage vary among patients with cerebral hemorrhage, and the delay of the oxyhemoglobin response relative to the EEG response may differ significantly. If all patients are treated with the same time window, the actual hemodynamic peak of the target patient may lie outside the feature extraction window. In this case, the system may incorrectly assume that the target patient's near-infrared response is weak and select the wrong source domain or incorrectly reduce the training level. Based on this, in one embodiment, an individualized hemodynamic response window determination submodule is set. First, the neural activity reference time is determined based on the desynchronization start time of EEG events within the same training trial. Then, within a preset wide search time range, the inflection point, the position of the maximum upward slope, or the response peak of the oxyhemoglobin concentration curve is searched. Based on the time difference between the EEG reference time and the near-infrared response position in multiple effective training trials, the individual hemodynamic delay corresponding to the patient's current training period is determined. The system does not directly move the raw functional near-infrared spectral signal. Instead, it adjusts the near-infrared feature extraction window based on the individual's hemodynamic delay, ensuring that EEG and near-infrared features are extracted within their respective timeframes that conform to their physiological response patterns. When the patient's individual hemodynamic delay undergoes continuous changes at different training periods, the feature extraction window is updated, but historical delay records are retained to determine whether the change is a one-time anomaly or a gradual development during the rehabilitation process.

[0055] In some cases, writing the same event marker to both EEG and functional near-infrared spectroscopy (FIR) signals only proves that a correspondence was established between the two at the moment the event was written; it does not guarantee that the two acquisition devices will maintain the same timing rate throughout the entire training period. When the training duration is long, the internal clocks of the two devices may develop slight frequency deviations. Events at the beginning of training can be accurately aligned, but deviations of tens of milliseconds or even larger may gradually develop in the later stages of training. This deviation is difficult to detect when checking the first event marker but will affect subsequent cross-modal feature correspondences. Based on this, in one embodiment, synchronization anchor events are written at the beginning, several intermediate, and end positions of the training period, and a segmented time mapping relationship is established based on the timestamp differences of the synchronization anchors in the two signals. In addition to device event markers, common physiological rhythms can be extracted from the two signals as auxiliary verification information. For example, EEG electrodes may record ECG or pulse-related components, and pulse fluctuations may also exist in the FIR signal. The system determines whether time drift still exists after event marker alignment by comparing the periodic positions of the common physiological rhythms. When the synchronization anchor indicates that the time deviation gradually increases with the training duration, the system does not use a fixed time offset, but instead calculates a time scale correction coefficient to map the time axis of one device to the time axis of the other device. If a device restart, data caching pause, or sudden time jump occurs during training, the training period is divided into multiple time periods, and time mapping relationships are established for each period to avoid a single sudden deviation affecting the entire training period.

[0056] In some cases, a small Wasserstein distance does not necessarily indicate that the candidate source domain and the target patient have similar neurophysiological patterns. They may appear similar in distribution due to shared device noise, identical probe pressure, similar head movement artifacts, or sensory responses evoked by the same task cue. If the system selects source domains solely based on the minimum distance principle, it may choose candidate source domains with similar artifacts rather than those with similar physiological responses to motor intent. Since the overall distribution is indeed similar, this problem is difficult to detect through ordinary distance ranking. Therefore, in one embodiment, a source domain similarity attribution testing submodule is set up to compare the distances of candidate source domains before and after removing interfering components. First, candidate interfering components not directly related to motor intent are identified, including electrooculography-related components, high-frequency electromyography components, whole-brain synchronous near-infrared fluctuations, superficial blood flow components, and instantaneously evoked components of task cues. Then, the original Wasserstein distance with candidate interfering components retained and the physiological characteristic Wasserstein distance after suppressing candidate interfering components are calculated respectively. If a candidate source domain is close to the target patient in the original feature space, but the distance increases significantly after suppressing interfering components, then the similarity of the candidate source domain is determined to be mainly contributed by the interfering components, and its confidence score is reduced. Conversely, if the candidate source domain still maintains a small distance from the target patient after suppressing interfering components, it indicates that its similarity is more likely derived from physiological characteristics related to motor imagery. Counterfactual tests can also be performed on task time periods. For example, the category conditional distance can be recalculated after randomly permuting the task labels of the candidate source domains. If the similarity remains high after label permutation, it indicates that the similarity may not originate from the motor intention category structure.

[0057] In some cases, traditional motor imagery classification typically distinguishes between left and right limb motor imagery based on the activation patterns of the contralateral sensorimotor cortex. However, patients with cerebral hemorrhage may exhibit compensatory lateral hemisphere involvement, bilateral co-recruitment, or reorganization of residual areas on the affected side. In such cases, when performing right-hand motor imagery, the patient may not necessarily exhibit typical left-sided dominance, but rather right-sided dominance or bilateral distribution. Conventional transfer learning might interpret this change in lateralization as an inconsistency between the source and target domains and attempt to eliminate the difference through domain alignment, potentially weakening the patient's true compensatory characteristics. Therefore, in one embodiment, a lesion-constrained homologous brain region mapping submodule can be implemented. A correspondence between homologous brain regions of the left and right hemispheres is established based on the patient's lesion side, the side with motor impairment, and the spatial locations of EEG electrodes and near-infrared channels. For each motor imagery task, the affected-side dominance index, the healthy-side dominance index, and the bilateral synergy index are calculated. When the target patient's principal discriminant features stably appear in the hemisphere opposite to the conventional source domain, and this reversal persists across multiple training trials, it is identified as a candidate functional reorganization pattern, rather than being directly rejected as an abnormal sample. Before source domain transfer, source domain features can be spatially rearranged according to the mapping relationship of homologous brain regions, so that the contralateral activation patterns in the source domain are mapped to the brain regions where the target patient actually performs the functions. Spatial rearrangement only changes the correspondence between brain regions and does not modify the task category labels. When the patient shows bilateral joint recruitment, bilateral features can be preserved and the alignment constraint requiring unilateral dominance can be reduced to avoid forcing the target patient to revert to the unilateral distribution of healthy source domains.

[0058] In some cases, a high confidence score for the classifier does not necessarily mean it has identified motor imagery. The system may learn spurious features generated by task prompt sounds, screen brightness changes, prompt key electrical signals, or fixed task sequences. These problems are particularly difficult to notice when the training and test sets use the same task sequence. Classification accuracy may be high, but the classifier is actually providing results after the task prompt appears, before the patient has had time to generate motor imagery. Based on this, in one embodiment, a physiologically reachable time verification submodule can be set up. For each training trial, classification confidence is calculated for the time period before the task prompt appears, the early time period after the task prompt appears, and the time period during which the patient may generate an effective neural response. If the classifier has consistently output high confidence results consistent with the upcoming task label before the task prompt appears, or obtains high confidence at a time point that does not conform to the physiological response sequence of EEG and hemodynamics, the corresponding feature is marked as a prompt leakage candidate feature. The system further performs permutation tests on the task sequence or changes the presentation of the prompt. If a high-confidence feature changes with the cuing device, cuing sequence, or screen, but not with the patient's motor imagery state, the classification weight of that feature or the corresponding channel is reduced. For functional near-infrared spectroscopy modalities, so-called hemodynamic responses occurring prematurely after the task cue are examined. If the response time is earlier than the reasonable range allowed by neurovascular coupling, it is not considered a valid motor intention feature.

[0059] In some cases, existing recovery conditions require the oxyhemoglobin concentration to return to within 10% of the baseline for five seconds. However, some patients may experience a brief dip followed by a rebound after pausing training. If the system only determines the timing of recovery based on whether the concentration is within the allowable range, it might resume training during the brief dip, coinciding with the subsequent rebound and triggering a safety warning again. Therefore, in one embodiment, trend stability and rebound risk prediction are added to the pause-and-recovery decision. After the oxyhemoglobin concentration enters the baseline allowable range, the system not only records the duration but also calculates the rate of change and its trend. A recovery control command is only output when the concentration is within the allowable range, the absolute value of the rate of change is below the recovery threshold, and the rate of change does not show a resurgence trend. The system can also predict the oxyhemoglobin concentration within a preset time range based on the observed recovery curve after pausing. If the prediction indicates that the concentration may approach the safety threshold again after resuming training, the pause status is maintained. For patients experiencing multiple rebounds, the rebound time after pausing can be recorded, and the observation duration for the next recovery can be adjusted to be no shorter than the historical rebound time.

[0060] In some cases, online incremental learning typically uses task cues as training labels. However, training trials paused midway by the safety monitoring module are not fully executed. The first half may contain valid motor imagery signals, while the second half may contain abnormal hemodynamic changes, fatigue responses, or attentional shifts caused by pause cues. If the entire paused training trial is used to update the classifier with the original task label, the classifier may learn abnormal states as corresponding motor intention features. Deleting all paused trials would result in the loss of important information close to the safety boundary. Therefore, in one embodiment, a segmented usage allocation mechanism for safety-interrupted training trials can be set. Based on the task start time, the first time the safety threshold is reached, and the pause command output time, the interrupted training trial is divided into a safety pre-segment, a critical transition segment, and a pause post-segment. The safety pre-segment is only used for incremental classifier updates if the minimum effective task duration is met and the signal quality is acceptable. The critical transition segment is not used for motor intention classifier updates but is used to estimate the hemodynamic patterns of the patient when approaching the safety threshold. The pause post-segment is used to analyze the recovery process and is not used as supervised training data for the original motor intention category. You can also set classification update weights for each training trial based on the proportion of the security front end to the total task time. Higher weights are assigned when the completion rate is high and the classification confidence is stable; if the task is paused shortly after it starts, it should not be used for classifier updates.

[0061] In some cases, the rehabilitation program grading module uses 60% and 80% as confidence thresholds. However, after online incremental learning updates the fully connected layers, the calibration relationship of the classifier output probabilities may change. The 80% before and after the update may not represent the same level of reliability. A parameter update might make the overall classifier output sharper, even if the classification accuracy doesn't improve, the confidence level might generally increase. If the updated original confidence level is directly used to adjust the training difficulty, the system might incorrectly increase the level. Therefore, in one embodiment, a confidence calibration submodule across classifier versions can be set up. Before each online update, a set of anchored training trials that do not participate in the current parameter update is reserved. After the update, the same set of anchored training trials is processed using both the pre-update and post-update classifiers, and the original confidence distribution, classification accuracy, and class intervals are compared. Based on the correspondence on the anchored training trials, the original confidence level output by the updated classifier is converted into a calibrated confidence level comparable to the pre-update version. The rehabilitation program grading module uses the calibrated confidence level instead of directly using the original probabilities output by the fully connected layer. If the original confidence of the classifier increases after an update, but the classification accuracy or class margin stability on the anchored training trials does not improve, then the training difficulty should be limited. For multiple consecutive classifier versions, calibration parameters can be saved for each version, allowing the confidence in the training report to be compared across different training periods.

[0062] In some cases, rehabilitation programs typically control training load based on the number of completed training trials. However, training trials paused by the safety monitoring module, while not fully completed, still involve the patient performing a portion of the motor imagery task, generating corresponding neural activity and hemodynamic load. If the system only counts complete training trials, the same number of tasks might be added after the pause, causing the patient's actual total training load to exceed the planned value. Conversely, if all interrupted trials are counted as a single complete trial, the actual amount of training the patient can complete might be underestimated. Therefore, in one embodiment, a training load measurement mechanism based on effective exposure is implemented. For each training trial, the effective training exposure is calculated based on the actual task duration, the effective classification duration of the motor intention, the classification confidence level, and changes in oxyhemoglobin concentration. A fully completed training trial with stable classification confidence can be recorded as one standard exposure unit. Training trials paused midway are converted into partial exposure units based on the ratio of the actual duration to the planned duration, the effective motor imagery time, and the hemodynamic load. When a particular interruption trial, though short in duration, has a rate of change in oxyhemoglobin concentration close to the safety threshold, its physiological load calculation value can be higher than the value obtained simply based on the proportion of duration. The rehabilitation program grading module no longer determines whether to continue training solely based on the number of completed training trials, but rather determines the remaining training volume based on the cumulative effective exposure and cumulative hemodynamic load during the current training period.

[0063] The following is a detailed description of the multimodal BCI rehabilitation system for patients with cerebral hemorrhage in this application from the perspective of hardware processing. Please refer to [link to relevant documentation]. Figure 2 One embodiment of the multimodal BCI rehabilitation hardware system 300 for patients with cerebral hemorrhage in this application includes: The system includes an input device 301, an output device 302, a processor 303, and a memory 304, wherein the number of processors 303 can be one or more. Figure 2 Taking a processor 303 as an example. In some embodiments of this application, the input device 301, output device 302, processor 303, and memory 304 can be connected via a bus or other means, wherein... Figure 2 Taking the example of a connection between China and Israel via a bus.

[0064] Please see Figure 3 , Figure 3 A schematic diagram of an embodiment of the electronic system provided in this application.

[0065] like Figure 3As shown, this application provides an electronic system including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, it can achieve... Figure 1 Any of the corresponding implementation methods in the embodiments.

[0066] Since the electronic system described in this embodiment is a device used to implement a multimodal BCI rehabilitation system for patients with cerebral hemorrhage according to the embodiments of this application, those skilled in the art can understand the specific implementation methods and various variations of the electronic system in this embodiment based on the rehabilitation system described in the embodiments of this application. Therefore, how the electronic system implements the rehabilitation system in the embodiments of this application will not be described in detail here. As long as those skilled in the art implement the device used in the rehabilitation system in the embodiments of this application, it falls within the scope of protection of this application.

Claims

1. A multimodal BCI rehabilitation system for patients with cerebral hemorrhage, characterized in that, include: The signal acquisition module is used to simultaneously acquire the patient's electroencephalogram (EEG) signals and functional near-infrared spectral signals during the patient's motor imagery training task. The multimodal signal processing and motion intention classification module is used to extract EEG modal features and near-infrared modal features from the EEG signal and the functional near-infrared spectral signal, respectively, and classify the patient's motion intention based on the EEG modal features and the near-infrared modal features to obtain the motion intention classification result and classification confidence. The safety monitoring module is used to determine the rate of change of the patient's oxyhemoglobin concentration based on the functional near-infrared spectral signal, compare the rate of change of the oxyhemoglobin concentration with a preset safety threshold, and output a control command to pause the current motor imagery training task when the rate of change of the oxyhemoglobin concentration exceeds the preset safety threshold. An online incremental learning module is used to acquire training data generated by the patient in the current motor imagery training task, and update the classifier parameters using the training data while keeping the feature extractor parameters in the multimodal signal processing and motor intention classification module unchanged. as well as The rehabilitation program grading module is used to determine the training difficulty level corresponding to the patient based on the classification confidence level and hemodynamic indicators characterizing the changes in oxyhemoglobin concentration.

2. The rehabilitation system according to claim 1, characterized in that: The signal acquisition module includes an electroencephalogram (EEG) acquisition device, a functional near-infrared spectroscopy acquisition device, and an event labeling unit; The EEG acquisition device has no less than 30 EEG acquisition channels and a sampling rate of no less than 200 Hz. The functional near-infrared spectral acquisition device has no less than 36 near-infrared acquisition channels, uses a dual-wavelength near-infrared light source of 760 nm and 850 nm, and has a sampling rate of no less than 10 Hz. The event tagging unit is used to write event tags corresponding to the same motor imagery training event into the EEG signal and the functional near-infrared spectral signal, respectively, so that the two signals are aligned on a unified time axis.

3. The rehabilitation system according to claim 1, characterized in that, The multimodal signal processing and motion intent classification module includes a preprocessing and time-frequency analysis submodule, which is used for: The EEG signal is bandpass filtered from 0.5 Hz to 50 Hz, and the filtered EEG signal is subjected to short-time Fourier transform to extract event-related desynchronization features in the 8 Hz to 12 Hz frequency band and the 12 Hz to 27 Hz frequency band. According to Beer-Lambert's law, the functional near-infrared spectral signal is converted into oxyhemoglobin concentration signals and deoxyhemoglobin concentration signals. These signals are then bandpass filtered from 0.01 Hz to 0.1 Hz, and cerebral hemodynamic response characteristics are extracted. Based on the mean value of the functional near-infrared spectral signal within a preset time period before the start of the motor imagery training task, the cerebral hemodynamic response characteristics are baseline corrected.

4. The rehabilitation system according to claim 1, characterized in that, The multimodal signal processing and motion intent classification module further includes a source domain selection submodule, which is used for: Obtain source domain EEG samples and source domain functional near-infrared spectral samples corresponding to multiple candidate source domain subjects, as well as target domain EEG samples and target domain functional near-infrared spectral samples corresponding to the patient. Calculate the first Wasserstein distance between the source domain EEG samples of each candidate source domain subject and the target domain EEG samples, and the second Wasserstein distance between the source domain functional near-infrared spectral samples of each candidate source domain subject and the target domain functional near-infrared spectral samples. The source domain subjects for EEG modality transfer are determined based on the first Wasserstein distance, and the source domain subjects for functional near-infrared spectroscopy modality transfer are determined based on the second Wasserstein distance, so that the EEG modality and the functional near-infrared spectroscopy modality select their respective corresponding source domain subjects.

5. The rehabilitation system according to claim 4, characterized in that, The source domain selection submodule is used to select the source domain subjects based on the adaptive Wasserstein trust score, which is determined according to the adaptive distance threshold, the temporal trust score and the multi-granularity distance. The adaptive distance threshold is updated based on the Wasserstein distance statistics between the target domain sample and multiple candidate source domain samples, as well as the threshold adjustment coefficient that changes with the training process. The temporal trust score is used to characterize the stability of distance changes among subjects in the same candidate source domain across multiple adjacent training phases. The multi-granularity distance is obtained by fusing Wasserstein distances at different time scales or different feature levels.

6. The rehabilitation system according to claim 4, characterized in that, The multimodal signal processing and motion intent classification module includes a progressive dual-domain alignment submodule, which is used for: In the previous training phase of the transfer training process, the second-order statistical difference between the source domain features corresponding to the source domain subjects and the target domain features corresponding to the patients is constrained by deep correlation alignment loss in order to perform initial alignment of the source domain features and the target domain features. In the later training stage of the transfer training process, a deep canonical correlation analysis loss is introduced on the basis of the deep correlation alignment loss to perform correlation alignment between the source domain features and the target domain features. as well as according to The weights of the domain alignment loss are attenuated, where, Indicates the first The domain alignment loss weights corresponding to each training epoch This represents the initial domain alignment loss weights. This represents the total number of training rounds in the transfer training process.

7. The rehabilitation system according to claim 6, characterized in that, The multimodal signal processing and motion intent classification module includes a channel attention-enhanced depth-separable convolution classification submodule; The depthwise separable convolutional classification submodule includes a one-dimensional convolutional layer, a depthwise separable one-dimensional convolutional layer, a compressed excitation channel attention unit, a multi-scale feature fusion unit, and an adaptive depth selection unit. The compressed excitation channel attention unit is set after the depth-separable one-dimensional convolutional layer and is used to generate channel weights based on the contribution of different feature channels to the classification of motion intent. The multi-scale feature fusion unit is used to fuse features obtained from different receptive fields, and the adaptive depth selection unit is used to select network feature layers that participate in the motion intention classification based on the patient's multimodal signal features.

8. The rehabilitation system according to claim 1, characterized in that, The safety monitoring module includes an oxygenated hemoglobin change rate monitoring submodule and a threshold warning and pause submodule; The oxyhemoglobin change rate monitoring submodule is used to calculate the oxyhemoglobin concentration change rate according to a two-second sliding time window; The threshold warning and pause submodule is used to trigger a safety warning, pause the current motor imagery training task and output a rest prompt when the rate of change of the oxyhemoglobin concentration exceeds the preset safety threshold. After pausing the current motor imagery training task, the threshold warning and pause submodule continuously monitors the oxyhemoglobin concentration. When the oxyhemoglobin concentration returns to within 10 percent of the baseline concentration and remains stable for at least five seconds, it outputs a control command to resume the motor imagery training task.

9. The rehabilitation system according to claim 1, characterized in that, The online incremental learning module is used for: The data generated when the patient first uses the multimodal BCI rehabilitation system, or the data generated by the patient during a new training period, are identified as incremental training data. Freeze the parameters of the one-dimensional convolutional layer, the depthwise separable one-dimensional convolutional layer, and the compressed excitation channel attention unit in the feature extractor; Using incremental training data corresponding to 5 to 20 training trials, the gradient of the fully connected layer of the classifier is updated, wherein the learning rate used for gradient update is 10% to 20% of the pre-training learning rate.

10. The rehabilitation system according to claim 1, characterized in that, It also includes a rehabilitation training process control module and a report output module; The rehabilitation training process control module is used to configure initial motor imagery training parameters based on the patient's motor function baseline, control the patient to perform motor imagery training tasks, and after completing the current round of motor imagery training tasks, control the rehabilitation program grading module to determine the training difficulty level and corresponding training parameters of the next round of motor imagery training tasks. The report output module is used to generate a training report and training suggestions for the next round based on the classification results of the motion intention, classification confidence, hemodynamic indicators, safety warning results, classifier parameter update results, and training parameters for the next round of motion imagery training tasks corresponding to the current round of motion imagery training tasks.

Images