A method and system for constructing a neurorehabilitation assessment model

Abstract

The present invention relates to the field of data processing technology, and in particular to a method and system for constructing a neurorehabilitation assessment model. The method comprises the following steps: obtaining multimodal data of neurorehabilitation patients; performing feature extraction based on the multimodal data of neurorehabilitation patients to obtain multimodal feature data of neurorehabilitation patients; performing data classification on the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data; and using the multimodal feature classification data to label the multimodal feature data of neurorehabilitation patients to obtain multimodal feature labeling data of neurorehabilitation patients, so as to assist in the construction of a neurorehabilitation assessment model. The present invention assists in the construction of a model that can comprehensively and objectively evaluate the rehabilitation progress of neurorehabilitation patients, thereby providing a scientific basis for rehabilitation treatment.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for constructing a neurorehabilitation assessment model. Background Art

[0002] With the rapid development of neuroscience and rehabilitation medicine, how to effectively assess the recovery progress of neurorehabilitation patients has become a critical issue in the medical field. Neurorehabilitation uses a series of rehabilitation therapies to help patients with damaged nervous systems regain function. It plays a particularly crucial role in the recovery process of conditions such as stroke, traumatic brain injury, and spinal cord injury. However, traditional rehabilitation assessment methods, which mostly rely on subjective clinical scores and patient self-reports, cannot comprehensively and objectively reflect the recovery of the nervous system.

[0003] Modern neurorehabilitation assessments increasingly rely on big data and multimodal analysis techniques, such as functional magnetic resonance imaging (fMRI) and electroencephalography (EEG), among other biomedical signals. These techniques capture the brain's functional activity, providing objective evidence of functional recovery. However, due to the complexity and heterogeneity of these data, effectively integrating, processing, and analyzing this multimodal data remains a challenge for current technologies. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for constructing a neurorehabilitation assessment model to solve at least one of the above technical problems.

[0005] This application provides a method for constructing a neurorehabilitation assessment model, comprising the following steps:

[0006] Step S1: Acquire multimodal data of neurorehabilitation patients;

[0007] Step S2: extracting features based on the multimodal data of the neurorehabilitation patient to obtain multimodal feature data of the neurorehabilitation patient;

[0008] Step S3: performing data classification on the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data;

[0009] Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, and obtaining the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model.

[0010] The use of multimodal data and the diversification of feature extraction in the present invention ensure that rehabilitation assessments are based on accurate data from multiple dimensions, thereby improving the accuracy of the assessment. Data classification and labeling make model construction more robust, accurately capturing changes in a patient's rehabilitation status and helping to identify key progress in rehabilitation. Labeled data provides a solid foundation for supervised learning of the model, while optimizing data utilization efficiency through classification, making the model's performance in specific areas more accurate. The combined use and analysis of multimodal data can further optimize the performance of the neurorehabilitation assessment model.

[0011] Preferably, step S1 is specifically:

[0012] Collect EEG data of neurorehabilitation patients through EEG equipment;

[0013] Functional magnetic resonance imaging data of neurorehabilitation patients were obtained by performing functional magnetic resonance imaging acquisition using a high-resolution fMRI scanner;

[0014] The EEG data and functional magnetic resonance imaging data of neurorehabilitation patients are integrated to obtain multimodal data of neurorehabilitation patients.

[0015] The present invention integrates EEG and fMRI modalities to capture different dimensions of a patient's brain activity. EEG excels at providing neural activity information with high temporal resolution, while fMRI provides brain activation information with high spatial resolution. The combination of the two modalities can overcome the limitations of a single modality and provide more comprehensive brain function data. EEG can capture rapidly changing EEG signals, while fMRI can capture brain activation reflected by hemodynamic changes. The two are highly complementary, and when integrated, they can simultaneously obtain synergistic information on EEG activity and brain functional areas, improving the accuracy of rehabilitation assessments. EEG data provides millisecond-level temporal resolution, which can capture rapid neural signal transmission and synchronized activity in the brain. This has advantages in assessing changes in patient rehabilitation, particularly in neural response speed, attention, and cognitive load. fMRI provides millimeter-level spatial resolution, which can map different functional areas in the brain in detail. This is important for analyzing interactions between brain regions, assessing the recovery of brain activity, and determining changes in functional connectivity during rehabilitation.

[0016] Preferably, step S2 is specifically:

[0017] Extract neural activity features from EEG data of neurorehabilitation patients to obtain neural activity feature data;

[0018] Functional connectivity feature extraction is performed on functional magnetic resonance imaging data of neurorehabilitation patients to obtain neural functional connectivity feature data;

[0019] The neural activity feature data and the neural functional connection feature data are fused to obtain multimodal feature data of neurorehabilitation patients.

[0020] The neural activity features extracted from the EEG data in the present invention can reflect the electrical activity of neurons in real time, such as the power spectrum of different frequency bands of the brain, neural oscillation patterns, synchronization and desynchronization phenomena of brain waves. For neurorehabilitation assessment, these features help to capture immediate neural responses in rehabilitation training and evaluate the progress of neural function recovery. Neural activity features (such as sample entropy, timing features, etc.) can reveal the neural state of brain functional areas, help evaluate whether specific brain areas have restored normal function during the rehabilitation process, and provide accurate feedback on rehabilitation progress. Through the extraction of functional connectivity features from functional magnetic resonance imaging data, the functional connectivity relationship between different areas of the brain can be revealed, reflecting the coordinated activities between brain areas during the rehabilitation process, especially the reconstruction of neural network connections across brain areas. Functional connectivity features help to analyze the impact of rehabilitation training on brain area networks. In neurorehabilitation, the remodeling of brain functional networks is an important sign of rehabilitation. The extraction of functional connectivity features helps to evaluate the dynamic changes of brain functional networks, especially in terms of motor control, cognitive function recovery, etc., and can intuitively display the connection recovery between different areas of the brain. By fusing neural activity characteristics (EEG) and functional connectivity characteristics (fMRI), it is possible to simultaneously utilize the high temporal resolution of EEG and the high spatial resolution of fMRI, providing more comprehensive neural feature data for neurorehabilitation. It can not only monitor neural activity in real time, but also locate the functional recovery of brain regions in detail.

[0021] Preferably, the neural activity feature extraction is specifically:

[0022] Performing artifact removal on the EEG data of neurorehabilitation patients to obtain EEG artifact-removed data;

[0023] Extracting time domain features and frequency domain features from EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data;

[0024] Perform sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data;

[0025] Phase space reconstruction is performed based on the characteristic sample entropy data, EEG time series characteristic data and EEG frequency domain characteristic data to obtain neural activity characteristic data.

[0026] The present invention can greatly improve the signal quality of EEG data by removing artifacts (such as eye movements, electromyographic noise, etc.), ensure that subsequent feature extraction is based on pure neural activity signals, reduce the interference of artifacts on the analysis results, and improve the accuracy of neural activity features. Artifact removal techniques, such as independent component analysis (ICA) or filtering processing, can retain key neural activity information while removing interference. Through time domain feature extraction (such as mean, peak, variance, etc.), the amplitude changes and waveform characteristics of EEG signals can be captured to reflect the instantaneous neural activity of the brain. Frequency domain feature extraction (such as power spectral density, frequency band energy, etc.) can reveal neural activity patterns in different frequency bands (such as Alpha, Beta waves, etc.), which helps to analyze neural activity in different states of the brain (such as relaxation, concentration, etc.). Combining the extraction of time domain and frequency domain features can provide multi-level feature data for a comprehensive assessment of neural activity and enhance the comprehensive assessment ability of the patient's neural state. Sample entropy calculation can quantify the complexity of time series data (such as EEG time domain and frequency domain features) and help identify the nonlinear characteristics of neural signals. For example, during the rehabilitation process, the activity patterns of the nervous system often change from disorder to order, and from complexity to simplicity. Sample entropy can accurately quantify these changes. Higher sample entropy values ​​generally indicate increased complexity in neural activity, reflecting the complex process of the brain's transition from an impaired state to partial recovery during rehabilitation. Sample entropy provides a new perspective on the neurorehabilitation process, effectively monitoring changes in neural activity. Phase space reconstruction, by mapping neural activity feature data into a higher-dimensional phase space, can reveal the underlying nonlinear dynamics of the nervous system, effectively revealing the periodicity, chaotic behavior, or attractor structure of neural activity, and providing a deeper understanding of the complex dynamics of neural activity during rehabilitation.

[0027] Preferably, the sample entropy calculation is specifically as follows:

[0028] Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively;

[0029] Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively;

[0030] Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively;

[0031] The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the feature sample entropy data.

[0032] In the present invention, by dividing the time series features and frequency domain features of the electroencephalogram into vector sequences, neural activity is analyzed from the two dimensions of time and frequency, which can capture changes in neural activity at different levels. The divided vector sequence can adapt to the complexity of the data more flexibly, especially in different brain regions and states, the feature expression may be different. By dividing the vector sequence, it is helpful to carefully analyze the local and global features of the electroencephalogram signal, and improve the sophistication and accuracy of the analysis. By similarity calculation, similar patterns in the electroencephalogram time series and frequency domain data can be quantified, which is very important for detecting specific brain states (such as changes in brain activity during rehabilitation). High similarity indicates the stability or consistency of the data, and low similarity may indicate irregular changes in neural activity or a functional reconstruction process. Similarity calculation can help identify trend changes or mutation points in electroencephalogram signals, which is crucial for neurorehabilitation assessment because it can reveal the recovery or change trajectory of brain function during rehabilitation. Sample entropy is an important tool for quantifying the complexity and unpredictability of time series. By calculating the sample entropy of electroencephalogram time series data and frequency domain data, the complexity of brain neural activity can be evaluated. Higher sample entropy values ​​generally reflect the diversity and flexibility of brain activity, while lower sample entropy values ​​reflect stable patterns of neural activity. Sample entropy calculations are not limited to time series features but also extend to frequency domain features. This dual-dimensional complexity analysis enables a more comprehensive assessment of brain changes during neurorehabilitation. Frequency domain sample entropy reveals the complexity of neural activity across different frequency bands and, in many cases, can complement the shortcomings of time domain data.

[0033] Preferably, the phase space reconstruction is specifically:

[0034] The time delay parameter data is obtained by calculating the average mutual information based on the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data;

[0035] Principal component analysis and maximum variance dimension screening are performed based on feature sample entropy data, EEG time series feature data, and EEG frequency domain feature data to obtain embedded dimension data;

[0036] Perform phase space encoding on the feature sample entropy data, the EEG time series feature data, and the EEG frequency domain feature data according to the time delay parameter data and the embedding dimension data, and obtain feature sample entropy encoding data, EEG time series feature encoding data, and EEG frequency domain feature encoding data, respectively;

[0037] Constructing a multimodal joint phase space based on feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data;

[0038] The multimodal joint phase space data is processed into phase space trajectory to obtain neural activity feature data.

[0039] The present invention uses average mutual information (AMI) to calculate the time delay parameter, which can more accurately capture the time-dependent information in the EEG signal. AMI can quantify the dependency between adjacent points in the feature data and automatically determine the optimal time delay parameter, avoiding the subjectivity of manually setting the time delay. Reasonable time delay parameters ensure that the dynamics and complexity of the signal features in the subsequent phase space reconstruction can be fully displayed, helping to reveal the key change points of neural activity during the rehabilitation process. By principal component analysis (PCA) and screening the maximum variance dimension, the selection of the embedding dimension is more objective and data-driven, which can automatically extract the most representative information, reduce redundant features and noise, and improve the efficiency of data analysis. In the phase space reconstruction process, the embedding dimension is a key parameter. The dimension selection after PCA optimization can ensure that the reconstructed phase space retains the key information in the original data while avoiding the computational complexity brought by too high a dimension. Phase space coding can uniformly map sample entropy data, time series feature data, and frequency domain feature data into the phase space to form a unified coding representation, which helps to capture the interdependence between the features and effectively construct a more complete neural activity model. Phase space encoding helps reproduce the nonlinear dynamic characteristics of brain activity. By embedding feature data into a higher-dimensional space, it is possible to reveal complex patterns and trajectories of neural activity, revealing the potential nonlinear behavior during neural system recovery. By jointly constructing a multimodal phase space from encoded data of different modalities (sample entropy, temporal features, and frequency domain features), it is possible to conduct a comprehensive analysis of multidimensional data, providing more comprehensive and multifaceted information for the assessment of brain function and helping to understand the interactions between modalities. The multimodal joint phase space can provide a richer range of feature dimensions, reducing the limitations of single-modal analysis. By combining spatiotemporal and frequency information, this method can more accurately reflect key dynamic changes during neurorehabilitation. By performing trajectory processing on the multimodal joint phase space, the motion trajectory of neural activity in high-dimensional space can be analyzed. Phase space trajectories can reveal periodicity, attractor structures, or chaotic behavior in neural activity, helping to better understand the changing patterns of brain function during rehabilitation.

[0040] Preferably, the functional connectivity feature extraction is specifically as follows:

[0041] Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data;

[0042] Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data;

[0043] Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data;

[0044] The functional connectivity matrix is ​​constructed based on the time-series blood oxygen correlation data to obtain the functional connectivity matrix data;

[0045] Performing whole-brain functional connectivity feature extraction and local functional connectivity feature extraction on the functional connectivity matrix data to obtain whole-brain functional connectivity feature data and local functional connectivity feature data;

[0046] The whole-brain functional connectivity feature data and the local functional connectivity feature data are integrated to obtain the neural functional connectivity feature data.

[0047] In the present invention, by dividing the functional magnetic resonance imaging data into brain regions, the different functional areas of the brain can be accurately divided, the functions of different brain regions can be analyzed in detail, and a basis for the subsequent analysis of functional connections can be provided to ensure the spatial accuracy of the data. Extracting the time-series blood oxygen level data (BOLD signal) of each brain region can dynamically reflect the changes in oxygen demand of neuronal activity, capture the activity level of different areas of the brain during task execution or rehabilitation, and help evaluate the functional recovery of brain regions, especially the impact on the brain during rehabilitation training. By performing correlation calculations on the time-series blood oxygen level data of each brain region, the functional connectivity between different brain regions can be revealed, reflecting the intensity of collaborative work or synchronous activity between brain regions, and helping users understand the network structure and changes between brain regions during rehabilitation.

[0048] Temporal correlation calculations can help reveal the remodeling of brain functional networks. This is particularly true in neurorehabilitation, where increased or decreased functional connectivity can reflect the recovery of brain function. Changes in brain connectivity during rehabilitation may be important signals of functional remodeling. The functional connectivity matrix, a mathematical representation of the strength of brain connections, provides a comprehensive picture of the brain's functional networks. Visualizing this matrix can intuitively display the connectivity patterns between different brain regions, helping to identify overall trends in brain functional networks during rehabilitation.

[0049] The functional connectivity matrix serves as a foundation for analyzing brain functional networks and supports subsequent extraction of network features, such as whole-brain network connectivity, network clustering coefficient, and small-worldness, thereby providing a more comprehensive understanding of the overall architecture and changes in brain networks. Whole-brain functional connectivity features capture the overall brain network characteristics, reflecting how different brain regions work together during rehabilitation. This helps assess the recovery of overall brain function, particularly in cognition, attention, motor control, and other functions involving a wide range of brain regions.

[0050] Local functional connectivity features can focus on connectivity changes in specific pairs of brain regions and provide refined local network analysis. For example, focusing on local connections related to motor function and language function can provide a more accurate assessment basis and can detect local functional reconstruction during the rehabilitation process. By integrating whole-brain functional connectivity features and local functional connectivity features, multi-level analysis of brain functional networks can be achieved. Whole-brain connectivity provides a macro perspective, while local connectivity provides a micro perspective, which helps to more comprehensively understand the functional recovery of the brain.

[0051] Preferably, the multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data, and step S3 is specifically:

[0052] Performing modal classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data;

[0053] The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

[0054] In the present invention, by performing modal classification on the multimodal feature data of neurorehabilitation patients, it is possible to effectively distinguish the data according to its source (such as electroencephalogram, functional magnetic resonance imaging, etc.), ensure that different types of signals are processed separately, reduce confusion during data fusion, and help to more accurately analyze the contribution of each modality. After modal classification, the data of each modality can be processed, analyzed and modeled independently, so that the model can perform targeted feature selection and optimization between different modalities, thereby improving the efficiency and flexibility of data processing. By classifying multimodal feature data by feature type (such as time domain features, frequency domain features, complexity features, etc.), the organization of the data can be made clearer. Each feature type can represent a different form of physiological signal expression, for example, time domain features reflect instantaneous changes, frequency domain features reveal periodic activities, and complexity features reflect nonlinear behavior.

[0055] Preferably, step S4 is specifically:

[0056] Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data;

[0057] The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients to assist in the construction of neurological rehabilitation evaluation models.

[0058] The introduction of the knowledge graph in the present invention systematizes and structures the medical knowledge and expert experience in the field of neurorehabilitation, and becomes a core part of the present invention. Through the rich information stored in the knowledge graph, such as the type of nerve injury, rehabilitation path, standardized treatment plan, patient recovery stage, etc., the system can have knowledge-driven capabilities in the data processing process, which greatly improves the accuracy of data analysis, especially the processing efficiency of complex rehabilitation cases. The feature data is marked by fitting the multimodal feature knowledge of neurological patients, so that the marking process not only depends on the original multimodal data features, but also relies on the medical knowledge and rehabilitation experience provided by the knowledge graph. Through the knowledge fitting process, the marking process becomes more accurate, and the most appropriate marking label can be automatically matched according to the specific feature status of the patient.

[0059] Preferably, the present application also provides a system for constructing a neurorehabilitation assessment model, which is used to execute the method for constructing a neurorehabilitation assessment model as described above. The system for constructing a neurorehabilitation assessment model includes:

[0060] Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data;

[0061] A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data;

[0062] The modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data;

[0063] The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients to assist in the construction of neurorehabilitation assessment models.

[0064] The beneficial effects of the present invention are:

[0065] 1. Data from different modalities provides a multidimensional perspective on neural activity. For example, the high temporal resolution of EEG and the high spatial resolution of fMRI can overcome the limitations of a single modality and provide richer foundational information for subsequent neurorehabilitation assessments. Multimodal data can reflect the dynamic changes in a patient's brain function at different temporal, spatial, and physiological scales, enabling assessment models to not only detect short-term neural responses but also capture the long-term process of brain function reconstruction.

[0066] 2. Time domain features reflect instantaneous changes in the signal, frequency domain features reveal the intensity of neural activity in different frequency bands, and complexity features such as sample entropy can quantify the nonlinear behavior of the signal, ensuring that the model can fully capture subtle changes in neural activity during neurorehabilitation. By extracting different types of features from multimodal data, the model can obtain rich input information, thereby improving the accuracy and robustness of the evaluation. During training, the model can interpret these different types of features to help analyze the complex dynamics of brain activity and identify key neural changes in rehabilitation. By combining time domain, frequency domain, and complexity features, the amount of feature information for each patient has increased from an average of 200 dimensions to 350 dimensions, and the information gain has increased by approximately 75%, providing more input data for accurate training of the model.

[0067] 3. Modality classification helps distinguish data sources from different modalities (such as EEG and fMRI), ensuring targeted feature processing and analysis. Feature type classification further categorizes features by type (such as time domain, frequency domain, and complexity), facilitating the appropriate selection and use of features. This classification results in more structured data and more efficient data processing.

[0068] 4. Through fitting calculations, the knowledge graph can help identify a patient's individualized neurological injury and rehabilitation progress. For example, for different injury types or recovery speeds, the system can match the most similar historical rehabilitation cases from the knowledge graph and generate corresponding rehabilitation path recommendations based on this. From a data processing perspective, the rehabilitation plan is more in line with the patient's actual situation, and differentiated neurological rehabilitation assessment data can be provided for different patients, avoiding a "one-size-fits-all" treatment model. Through the intelligent fitting and labeling of the knowledge graph, the model's prediction accuracy has increased from the original 87% to 96%, an increase of 9 percentage points. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0070] Figure 1 A flowchart showing the steps of a method for constructing a neurorehabilitation assessment model according to an embodiment is shown;

[0071] Figure 2 A flowchart showing the steps of a method for collecting multimodal data of a neurological rehabilitation patient according to one embodiment is shown;

[0072] Figure 3 A flowchart showing the steps of a method for extracting multimodal features of a neurorehabilitation patient according to an embodiment is shown;

[0073] Figure 4A flowchart of the steps of a neural activity feature extraction method according to an embodiment is shown. DETAILED DESCRIPTION

[0074] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative work are within the scope of protection of the present invention.

[0075] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller approaches.

[0076] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0077] See also Figures 1 to 4 , the present application provides a method for constructing a neurorehabilitation assessment model, comprising the following steps:

[0078] Step S1: Acquire multimodal data of neurorehabilitation patients;

[0079] Specifically, an electroencephalogram (EEG) device and a functional magnetic resonance imaging (fMRI) device are used to collect EEG data and fMRI data of neurorehabilitation patients, respectively.

[0080] The EEG data of neurorehabilitation patients were filtered (band-pass filtering to remove low-frequency and high-frequency noise), artifact removed (ICA was used to remove eye blinks, myoelectric artifacts, etc.) and normalized.

[0081] Functional magnetic resonance imaging data were subjected to motion correction, time correction, spatial normalization (MNI standard space), smoothing and time series extraction.

[0082] The processed EEG data and processed fMRI data were synchronized by time alignment to ensure that the data of both modalities could be analyzed in the same time frame.

[0083] The processed EEG data and processed functional magnetic resonance imaging data represent the EEG activities and BOLD signals of brain regions of neurorehabilitation patients at multiple time points, respectively, forming a multimodal dataset.

[0084] Step S2: extracting features based on the multimodal data of the neurorehabilitation patient to obtain multimodal feature data of the neurorehabilitation patient;

[0085] Specifically, the EEG data of neurorehabilitation patients are subjected to time domain feature extraction to obtain EEG time domain feature data, such as extracting the time domain features of the EEG signal, including mean, variance, peak and zero crossing points. Frequency domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG frequency domain feature data, such as extracting the power spectrum density of different frequency bands (δ wave, θ wave, α wave, β wave, γ wave) through fast Fourier transform (FFT). Phase synchronization feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG phase synchronization feature data, such as calculating the phase synchronization between brain regions (such as phase locking value PLV). The EEG time domain feature data, EEG frequency domain feature data and EEG phase synchronization feature data are vectorized to obtain EEG feature data;

[0086] Perform functional connectivity feature extraction on fMRI data to obtain functional connectivity feature data, such as calculating time series correlations between different brain regions and constructing a functional connectivity matrix. Perform graph feature extraction on fMRI data to obtain graph feature data, such as extracting graph features from the functional connectivity network, such as node degree, clustering coefficient, and path length. Vectorize the functional connectivity feature data and graph feature data to obtain fMRI feature data.

[0087] Multimodal feature fusion is performed on the EEG feature data and functional magnetic resonance imaging feature data, and they are integrated into a feature vector by weighted averaging or feature splicing to obtain multimodal feature data of neurorehabilitation patients.

[0088] Step S3: performing data classification on the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data;

[0089] Specifically, the multimodal feature data is preliminarily classified according to the source of the feature (EEG or fMRI), and the feature is classified according to its modality label using a rule classifier or a decision tree algorithm.

[0090] Specifically, further classification is performed based on the type of features (time domain, frequency domain, functional connectivity, etc.). Classification algorithms such as SVM or random forest are used to perform classification based on the statistical properties of the features.

[0091] Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, and obtaining the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model.

[0092] Specifically, the multimodal feature classification data is fitted with the preset neurological patient knowledge graph data. The feature data is embedded through a graph neural network (GNN), and its similarity with the knowledge graph nodes is calculated. Based on the fitting results, the multimodal features are automatically labeled. Labels are assigned to features (such as rehabilitation progress features, functional connectivity features, etc.) using labeling rules or machine learning classifiers. The labeled feature data is used to construct a neurorehabilitation evaluation model to further assist in the evaluation and prediction of rehabilitation effects.

[0093] Auxiliary tasks for building a neurorehabilitation assessment model can use deep algorithms (such as deep neural network algorithms) to build the model, divide the multimodal feature labeling data of neurorehabilitation patients, generate training sets and test sets, use the training set to build the model (model input layer, hidden neuron layer and model output layer) to obtain a preliminary model; according to the test set, the preliminary model is iteratively trained using the squared error loss function to obtain a neurorehabilitation assessment model.

[0094] The use of multimodal data and the diversification of feature extraction in the present invention ensure that rehabilitation assessments are based on accurate data from multiple dimensions, thereby improving the accuracy of the assessment. Data classification and labeling make model construction more robust, accurately capturing changes in a patient's rehabilitation status and helping to identify key progress in rehabilitation. Labeled data provides a solid foundation for supervised learning of the model, while optimizing data utilization efficiency through classification, making the model's performance in specific areas more accurate. The combined use and analysis of multimodal data can further optimize the performance of the neurorehabilitation assessment model.

[0095] Preferably, step S1 is specifically:

[0096] Step S11: collecting EEG data of the neurorehabilitation patient through an EEG device to obtain EEG data of the neurorehabilitation patient;

[0097] Specifically, electrodes are placed on the patient's scalp according to the international 10-20 system, ensuring coverage of key brain areas. An EEG amplifier is used to record electrical signals from the brain, with a sampling rate of 500–1000 Hz to accurately capture tiny voltage changes (in the microvolt range).

[0098] Step S12: performing functional magnetic resonance imaging (fMRI) acquisition using a high-resolution fMRI scanner to obtain functional magnetic resonance imaging data of the neurorehabilitation patient;

[0099] Specifically, a high-resolution functional magnetic resonance imaging scanner with a spatial resolution of 1-2 mm and a temporal resolution of approximately 2 seconds (TR = 2 seconds) is used to capture the brain's blood oxygen level-dependent (BOLD) signal. Whole-brain scans are performed in consecutive slices, covering the entire brain region, and multiple time-series images are recorded to reflect brain activity over a certain period of time (e.g., 10-15 minutes).

[0100] Step S13: Integrate the electroencephalogram data and functional magnetic resonance imaging data of the neurorehabilitation patient to obtain multimodal data of the neurorehabilitation patient.

[0101] Specifically, they aligned the EEG and fMRI data using a common timestamp or external event markers (such as button presses or visual / auditory stimuli presented to the patient during the recording) to ensure that the two patterns correspond to neural activity at the same point in time. They also resampled the EEG data to match the lower temporal resolution of the fMRI (for example, resampling the EEG from 500 Hz to 0.5 Hz) while preserving important patterns of neural activity.

[0102] The present invention integrates EEG and fMRI modalities to capture different dimensions of a patient's brain activity. EEG excels at providing neural activity information with high temporal resolution, while fMRI provides brain activation information with high spatial resolution. The combination of the two modalities can overcome the limitations of a single modality and provide more comprehensive brain function data. EEG can capture rapidly changing EEG signals, while fMRI can capture brain activation reflected by hemodynamic changes. The two are highly complementary, and when integrated, they can simultaneously obtain synergistic information on EEG activity and brain functional areas, improving the accuracy of rehabilitation assessments. EEG data provides millisecond-level temporal resolution, which can capture rapid neural signal transmission and synchronized activity in the brain. This has advantages in assessing changes in patient rehabilitation, particularly in neural response speed, attention, and cognitive load. fMRI provides millimeter-level spatial resolution, which can map different functional areas in the brain in detail. This is important for analyzing interactions between brain regions, assessing the recovery of brain activity, and determining changes in functional connectivity during rehabilitation.

[0103] Preferably, step S2 is specifically:

[0104] Step S21: extracting neural activity features from the electroencephalogram data of the neurorehabilitation patient to obtain neural activity feature data;

[0105] Specifically, the EEG data of neurorehabilitation patients are subjected to time domain feature extraction to obtain EEG time domain feature data, such as extracting the time domain features of the EEG signal, including mean, variance, peak and zero crossing points. Frequency domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG frequency domain feature data, such as extracting the power spectrum density of different frequency bands (δ wave, θ wave, α wave, β wave, γ wave) through fast Fourier transform (FFT). Phase synchronization feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG phase synchronization feature data, such as calculating the phase synchronization between brain regions (such as phase locking value PLV). The EEG time domain feature data, EEG frequency domain feature data and EEG phase synchronization feature data are vectorized to obtain EEG feature data;

[0106] The mean and variance of the time series for each electrode position are calculated to capture the overall level and fluctuation of the EEG signal, obtaining the mean and variance. The maximum and minimum values ​​of the signal are identified to reflect the extreme fluctuations in neural activity, obtaining the peak amplitude.

[0107] Using methods such as short-time Fourier transform (STFT) or discrete wavelet transform (DWT), the EEG data are converted from the time domain to the frequency domain, and the power spectral density (PSD) of each frequency band is extracted, such as: delta wave (0.5–4 Hz), theta wave (4–8 Hz), alpha wave (8–13 Hz), beta wave (13–30 Hz), and gamma wave (30–50 Hz).

[0108] Calculate phase synchronization characteristics between different electrodes, for example, by comparing the phase difference between two electrodes to measure the degree of synchronization between them. Use coherence to measure the coordinated activity between different brain regions in different frequency bands.

[0109] Step S22: extracting functional connectivity features from functional magnetic resonance imaging data of neurorehabilitation patients to obtain neural functional connectivity feature data;

[0110] Specifically, functional connectivity feature extraction is performed on the fMRI data to obtain functional connectivity feature data, such as calculating the time series correlation between different brain regions and constructing a functional connectivity matrix. Graph theory feature extraction is performed on the fMRI data to obtain graph theory feature data, such as extracting graph theory features in the functional connectivity network, such as node degree, clustering coefficient, and path length. The functional connectivity feature data and graph theory feature data are vectorized to obtain fMRI feature data.

[0111] Time series signals for each brain region were extracted from preprocessed functional magnetic resonance imaging (fMRI) data from neurorehabilitation patients. Pearson correlation coefficients were calculated between the time series signals of different brain regions to measure the strength of functional connectivity between them. The correlation coefficient matrix reflects the strength of connectivity between each pair of brain regions. By controlling for the influence of other brain regions, the direct functional connectivity between specific pairs of brain regions was calculated to generate a partial correlation matrix. Using a sliding window technique, the time series data was segmented, and functional connectivity was calculated within each time window to capture the dynamic changes in connectivity between brain regions.

[0112] Based on the functional connectivity matrix, a functional connectivity network is constructed and graph-theoretic features are extracted. For example, the number of connections between each brain region and other regions is calculated to reflect the centrality of the region in the functional network. The shortest path from one brain region to another is measured to assess the overall efficiency of the network. The density of connections between a brain region and its adjacent regions is measured to reflect the clustering of the local network.

[0113] Based on the above steps, the neural functional connectivity feature data of the neurorehabilitation patients are obtained, including the functional connectivity matrix and graph theory features.

[0114] Step S23: Fusing the neural activity feature data and the neural functional connection feature data to obtain multimodal feature data of the neural rehabilitation patient.

[0115] Specifically, multimodal feature fusion is performed on the EEG feature data and the functional magnetic resonance imaging feature data, and they are integrated into a feature vector by weighted averaging or feature splicing to obtain multimodal feature data of neurorehabilitation patients.

[0116] Specifically, the neural activity feature data and neural functional connectivity feature data are standardized to ensure that the two data types are compared and analyzed on the same scale. The time domain and frequency domain features of the EEG are spliced ​​with the functional connectivity features of the fMRI to form a multimodal feature vector. The feature vector of each patient contains information from different modalities, describing their neural activity and functional connectivity. Statistical learning methods such as canonical correlation analysis (CCA) or joint independent component analysis (JICA) are used to identify the associated components in the EEG and fMRI data and find common patterns between different modal data. Dimensionality reduction techniques such as principal component analysis (PCA) or linear discriminant analysis (LDA) are applied to reduce the dimension of the feature vector while retaining the main pattern information to improve the efficiency of subsequent models. The fused multimodal feature data of neurorehabilitation patients, including the fused time domain, frequency domain and functional connectivity features, are obtained and prepared for further analysis of the model.

[0117] The neural activity features extracted from the EEG data in the present invention can reflect the electrical activity of neurons in real time, such as the power spectrum of different frequency bands of the brain, neural oscillation patterns, synchronization and desynchronization phenomena of brain waves. For neurorehabilitation assessment, these features help to capture immediate neural responses in rehabilitation training and evaluate the progress of neural function recovery. Neural activity features (such as sample entropy, timing features, etc.) can reveal the neural state of brain functional areas, help evaluate whether specific brain areas have restored normal function during the rehabilitation process, and provide accurate feedback on rehabilitation progress. Through the extraction of functional connectivity features from functional magnetic resonance imaging data, the functional connectivity relationship between different areas of the brain can be revealed, reflecting the coordinated activities between brain areas during the rehabilitation process, especially the reconstruction of neural network connections across brain areas. Functional connectivity features help to analyze the impact of rehabilitation training on brain area networks. In neurorehabilitation, the remodeling of brain functional networks is an important sign of rehabilitation. The extraction of functional connectivity features helps to evaluate the dynamic changes of brain functional networks, especially in terms of motor control, cognitive function recovery, etc., and can intuitively display the connection recovery between different areas of the brain. By fusing neural activity characteristics (EEG) and functional connectivity characteristics (fMRI), it is possible to simultaneously utilize the high temporal resolution of EEG and the high spatial resolution of fMRI, providing more comprehensive neural feature data for neurorehabilitation. It can not only monitor neural activity in real time, but also locate the functional recovery of brain regions in detail.

[0118] Preferably, the neural activity feature extraction is specifically:

[0119] Step S211: performing artifact removal on the EEG data of the neurorehabilitation patient to obtain EEG artifact-removed data;

[0120] Specifically, independent component analysis (ICA) is performed on the EEG signal to separate signals from different sources, such as neural activity signals and artifact signals (such as eye movements, muscle activity, and heartbeat artifacts). By analyzing the timing diagrams and power spectra of the independent components based on preset thresholds, artifact signals are identified and removed from the EEG signal. After removing the artifacts, the remaining independent component signals are reconstructed to obtain the artifact-free EEG data.

[0121] Step S212: extracting time domain features and frequency domain features from the EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data;

[0122] Specifically, the mean value of each channel over the entire time series is calculated to reflect the central tendency of the signal. The variance of each channel is calculated to indicate the amplitude of the signal fluctuation. By detecting positive and negative peaks, the extreme points of the signal are found to reflect the intensity of neural activity. This yields EEG time series feature data, including mean, variance, and peak characteristics.

[0123] The fast Fourier transform (FFT) was used to convert the signal from the time domain to the frequency domain, and the power within each frequency band (delta, theta, alpha, beta, and gamma waves) was calculated. Phase synchronization characteristics, such as the phase lock value (PLV), between different brain regions were obtained by calculating the phase differences between electrodes. This yielded EEG frequency domain feature data, including power and phase synchronization characteristics across different frequency bands.

[0124] Step S213: performing sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data;

[0125] Specifically, sample entropy is calculated for time series features and frequency domain features. The steps are as follows: Select an embedding dimension and tolerance. Divide the time series into segments whose length is the embedding dimension. Compute the similarity of each segment with other segments. Count the number of similar segments and calculate the sample entropy value. Lower sample entropy values ​​indicate stronger regularity in the time series; higher sample entropy values ​​indicate greater complexity. This generates characteristic sample entropy data, which characterizes the complexity of the EEG data.

[0126] Step S214: performing phase space reconstruction based on the characteristic sample entropy data, the EEG time series characteristic data, and the EEG frequency domain characteristic data to obtain neural activity characteristic data.

[0127] Specifically, based on Takens' theorem, the phase space is constructed using sample entropy data, time series feature data, and frequency domain feature data. Time delays and embedding dimensions are generated by preset parameters, and the one-dimensional time series is embedded into the high-dimensional phase space by the time delay coordinate method. Using time delay embedding, the features of each time point are mapped to a high-dimensional space to form a multidimensional feature trajectory, representing the neural activity state at different time points. Dynamic patterns of neural activity are identified in the phase space, such as stable attractors, periodic trajectories, or chaotic trajectories, reflecting different neural activity states. Neural activity feature data is obtained, which contains the trajectory of the EEG signal in the phase space and can be used for further neurorehabilitation evaluation.

[0128] The present invention can greatly improve the signal quality of EEG data by removing artifacts (such as eye movements, electromyographic noise, etc.), ensure that subsequent feature extraction is based on pure neural activity signals, reduce the interference of artifacts on the analysis results, and improve the accuracy of neural activity features. Artifact removal techniques, such as independent component analysis (ICA) or filtering processing, can retain key neural activity information while removing interference. Through time domain feature extraction (such as mean, peak, variance, etc.), the amplitude changes and waveform characteristics of EEG signals can be captured to reflect the instantaneous neural activity of the brain. Frequency domain feature extraction (such as power spectral density, frequency band energy, etc.) can reveal neural activity patterns in different frequency bands (such as Alpha, Beta waves, etc.), which helps to analyze neural activity in different states of the brain (such as relaxation, concentration, etc.). Combining the extraction of time domain and frequency domain features can provide multi-level feature data for a comprehensive assessment of neural activity and enhance the comprehensive assessment ability of the patient's neural state. Sample entropy calculation can quantify the complexity of time series data (such as EEG time domain and frequency domain features) and help identify the nonlinear characteristics of neural signals. For example, during the rehabilitation process, the activity patterns of the nervous system often change from disorder to order, and from complexity to simplicity. Sample entropy can accurately quantify these changes. Higher sample entropy values ​​generally indicate increased complexity in neural activity, reflecting the complex process of the brain's transition from an impaired state to partial recovery during rehabilitation. Sample entropy provides a new perspective on the neurorehabilitation process, effectively monitoring changes in neural activity. Phase space reconstruction, by mapping neural activity feature data into a higher-dimensional phase space, can reveal the underlying nonlinear dynamics of the nervous system, effectively revealing the periodicity, chaotic behavior, or attractor structure of neural activity, and providing a deeper understanding of the complex dynamics of neural activity during rehabilitation.

[0129] Preferably, the sample entropy is calculated as follows:

[0130] Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively;

[0131] Specifically, the EEG time series feature data is divided according to the time window. For example, a time window of fixed length (such as 100 time points) is selected to divide the time series data into multiple subsequences, each subsequence representing the neural activity within a time window. The EEG frequency domain feature data is divided according to the frequency segment. The power value of each frequency segment can be used as a subsequence. The frequency domain feature data can also be divided according to the time window, and the frequency features within each time window are extracted. The EEG time series division data and the EEG frequency domain division data are obtained respectively, and these subsequences will be used as inputs for subsequent similarity calculations.

[0132] Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively;

[0133] Specifically, the similarity between each two time series subsequences is calculated using Euclidean distance or Manhattan distance. The specific process involves calculating the distance between each subsequence in multidimensional space, with smaller distances indicating higher similarity. A time series similarity threshold is set, and the number of subsequences whose distance from a given subsequence is less than the time series similarity threshold is counted.

[0134] For frequency-domain partitioned data, the same distance calculation method is used to compare the similarity of frequency subsequences. The frequency-domain distance between each subsequence and other subsequences is calculated, and the similarity of their power spectra or phase synchronization is compared. Similarly, a frequency-domain similarity threshold is set, and the number of other subsequences whose distance to a given frequency-domain subsequence is less than the frequency-domain similarity threshold is counted.

[0135] Based on the above steps, the EEG time series similarity data and EEG frequency domain similarity data are obtained respectively.

[0136] Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively;

[0137] Specifically, sample entropy is defined as a measure of the complexity of a time series, which measures the similarity of different subsequences in the sequence.

[0138] For each EEG time series subsequence, the number of subsequences with a distance less than r in the similarity calculation is counted, and the similarity ratios of subsequences with lengths of m and m+1 are calculated respectively. The time series sample entropy data is calculated using the sample entropy formula.

[0139] For EEG frequency domain subsequences, the similarity ratio of frequency domain subsequences of length m and m+1 is calculated using the same method as the time series sample entropy calculation to obtain frequency domain sample entropy data. The EEG time series sample entropy data and EEG frequency domain sample entropy data are obtained respectively, indicating the complexity of the signal under different modalities.

[0140] The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the feature sample entropy data.

[0141] Specifically, to ensure that the time series sample entropy data and the frequency domain sample entropy data can be compared on the same scale, the sample entropy data is normalized, such as by mean normalization or Z-score normalization. Using weighted averaging or multimodal feature fusion techniques, the time series sample entropy data and the frequency domain sample entropy data are weightedly fused. The weights can be set based on the importance of different modalities or empirical parameters. The resulting fused feature sample entropy data integrates the complexity of the EEG signal in both the time series and frequency domains, enabling in-depth analysis of neural activity.

[0142] In the present invention, by dividing the time series features and frequency domain features of the electroencephalogram into vector sequences, neural activity is analyzed from the two dimensions of time and frequency, which can capture changes in neural activity at different levels. The divided vector sequence can adapt to the complexity of the data more flexibly, especially in different brain regions and states, the feature expression may be different. By dividing the vector sequence, it is helpful to carefully analyze the local and global features of the electroencephalogram signal, and improve the sophistication and accuracy of the analysis. By similarity calculation, similar patterns in the electroencephalogram time series and frequency domain data can be quantified, which is very important for detecting specific brain states (such as changes in brain activity during rehabilitation). High similarity indicates the stability or consistency of the data, and low similarity may indicate irregular changes in neural activity or a functional reconstruction process. Similarity calculation can help identify trend changes or mutation points in electroencephalogram signals, which is crucial for neurorehabilitation assessment because it can reveal the recovery or change trajectory of brain function during rehabilitation. Sample entropy is an important tool for quantifying the complexity and unpredictability of time series. By calculating the sample entropy of electroencephalogram time series data and frequency domain data, the complexity of brain neural activity can be evaluated. Higher sample entropy values ​​generally reflect the diversity and flexibility of brain activity, while lower sample entropy values ​​reflect stable patterns of neural activity. Sample entropy calculations are not limited to time series features but also extend to frequency domain features. This dual-dimensional complexity analysis enables a more comprehensive assessment of brain changes during neurorehabilitation. Frequency domain sample entropy reveals the complexity of neural activity across different frequency bands and, in many cases, can complement the shortcomings of time domain data.

[0143] Preferably, the phase space reconstruction is specifically:

[0144] The time delay parameter data is obtained by calculating the average mutual information based on the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data;

[0145] Specifically, for each pair of time series data, the average mutual information between adjacent time points is calculated to assess their dependencies. The average mutual information gradually decreases with increasing time delay, and the point where the mutual information first reaches a local minimum is selected as the time delay parameter. The resulting time delay parameter data provides the necessary time delay information for phase space reconstruction.

[0146] Principal component analysis and maximum variance dimension screening are performed based on feature sample entropy data, EEG time series feature data, and EEG frequency domain feature data to obtain embedded dimension data;

[0147] Specifically, principal component analysis (PCA) is performed on the feature sample entropy data, EEG time series, and frequency domain feature data to identify the direction of maximum variance in the data. PCA uses eigenvalue decomposition of the data covariance matrix to identify the principal components that explain the maximum variance in the data. The number of principal components that explain at least 90% of the total variance is selected as the embedding dimension.

[0148] Perform phase space encoding on the feature sample entropy data, the EEG time series feature data, and the EEG frequency domain feature data according to the time delay parameter data and the embedding dimension data, and obtain feature sample entropy encoding data, EEG time series feature encoding data, and EEG frequency domain feature encoding data, respectively;

[0149] Specifically, the time delay parameter and embedding dimension are used to perform time-delay embedding on the feature sample entropy data, EEG time series, and frequency domain feature data. According to Takens' theorem, time series can be reconstructed into multidimensional vectors. Each time point is converted into a multidimensional vector, forming feature sample entropy coded data, EEG time series feature coded data, and EEG frequency domain feature coded data. The feature sample entropy coded data, EEG time series feature coded data, and EEG frequency domain feature coded data are obtained, respectively, to prepare for the subsequent phase space construction.

[0150] Constructing a multimodal joint phase space based on feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data;

[0151] Specifically, the feature sample entropy coded data, EEG time series feature coded data, and EEG frequency domain feature coded data are combined to construct a multimodal joint phase space. The multimodal feature vector at each time point is composed of coded data from different modalities. The fused multimodal data is mapped in a high-dimensional space to construct a multimodal phase space that can reflect the dynamic changes in neural activity under different modalities. The resulting multimodal joint phase space data integrates multimodal information and forms a complete phase space of neural activity.

[0152] The multimodal joint phase space data is processed into phase space trajectory to obtain neural activity feature data.

[0153] Specifically, in a multimodal phase space, phase space trajectories are processed to analyze patterns in neural activity, such as periodic trajectories, chaotic trajectories, or stable attractors. Relevant dynamic features are extracted from the phase space trajectories, such as the average distance between trajectories, the density of trajectory distribution, and the radius of attractors, to characterize the complexity and stability of neural activity. This resulting neural activity feature data is used for further analysis and the construction of neurorehabilitation models, reflecting the dynamic behavioral characteristics of neural activity under multimodal conditions.

[0154] The present invention uses average mutual information (AMI) to calculate the time delay parameter, which can more accurately capture the time-dependent information in the EEG signal. AMI can quantify the dependency between adjacent points in the feature data and automatically determine the optimal time delay parameter, avoiding the subjectivity of manually setting the time delay. Reasonable time delay parameters ensure that the dynamics and complexity of the signal features in the subsequent phase space reconstruction can be fully displayed, helping to reveal the key change points of neural activity during the rehabilitation process. By principal component analysis (PCA) and screening the maximum variance dimension, the selection of the embedding dimension is more objective and data-driven, which can automatically extract the most representative information, reduce redundant features and noise, and improve the efficiency of data analysis. In the phase space reconstruction process, the embedding dimension is a key parameter. The dimension selection after PCA optimization can ensure that the reconstructed phase space retains the key information in the original data while avoiding the computational complexity brought by too high a dimension. Phase space coding can uniformly map sample entropy data, time series feature data, and frequency domain feature data into the phase space to form a unified coding representation, which helps to capture the interdependence between the features and effectively construct a more complete neural activity model. Phase space encoding helps reproduce the nonlinear dynamic characteristics of brain activity. By embedding feature data into a higher-dimensional space, it is possible to reveal complex patterns and trajectories of neural activity, revealing the potential nonlinear behavior during neural system recovery. By jointly constructing a multimodal phase space from encoded data of different modalities (sample entropy, temporal features, and frequency domain features), it is possible to conduct a comprehensive analysis of multidimensional data, providing more comprehensive and multifaceted information for the assessment of brain function and helping to understand the interactions between modalities. The multimodal joint phase space can provide a richer range of feature dimensions, reducing the limitations of single-modal analysis. By combining spatiotemporal and frequency information, this method can more accurately reflect key dynamic changes during neurorehabilitation. By performing trajectory processing on the multimodal joint phase space, the motion trajectory of neural activity in high-dimensional space can be analyzed. Phase space trajectories can reveal periodicity, attractor structures, or chaotic behavior in neural activity, helping to better understand the changing patterns of brain function during rehabilitation.

[0155] Preferably, the functional connectivity feature extraction is specifically:

[0156] Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data;

[0157] Specifically, standard brain maps (such as AAL and Harvard-Oxford brain area maps) are used to perform brain region segmentation on functional magnetic resonance imaging (fMRI) data. Based on these maps, the brain is divided into multiple functional areas, each of which corresponds to a specific neural function. The patient's neurorehabilitation functional magnetic resonance imaging data is aligned with a standard space (such as MNI space) to ensure that the patient's brain region division is consistent with the standard brain map. Brain region segmentation data is obtained, which includes the brain region label corresponding to each voxel in the functional magnetic resonance imaging.

[0158] Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data;

[0159] Specifically, for each delineated brain region, a corresponding BOLD signal time series was extracted. The BOLD signal reflects the temporal changes in blood oxygenation levels in that region. The voxels within each region were averaged to obtain the average BOLD signal time series for that region, reducing the impact of voxel-level noise on functional connectivity analysis. This yielded time-series blood oxygenation data, which reflects the temporal changes in the average blood oxygenation level for each brain region.

[0160] Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data;

[0161] Specifically, the Pearson correlation coefficient was used to calculate the correlation between the time-series blood oxygen levels in different brain regions. A sliding window technique was used to calculate dynamic correlations, that is, to calculate the time-series correlations between brain regions over different time periods, capturing the dynamic changes in functional connectivity between brain regions. The resulting time-series blood oxygen correlation data reflects the strength of functional connectivity between different brain regions over the entire time period.

[0162] The functional connectivity matrix is ​​constructed based on the time-series blood oxygen correlation data to obtain the functional connectivity matrix data;

[0163] Specifically, a functional connectivity matrix is ​​constructed based on the time-series blood oxygen correlation data. Each element of the matrix represents the correlation coefficient between two brain regions. The matrix dimension is N×N, where N is the number of brain regions. The functional connectivity matrix is ​​normalized to remove noise and individual differences. The Fisher transformation can be used to normalize the correlation coefficients to obtain the functional connectivity matrix data, which reflects the strength of functional connections between all brain regions.

[0164] Performing whole-brain functional connectivity feature extraction and local functional connectivity feature extraction on the functional connectivity matrix data to obtain whole-brain functional connectivity feature data and local functional connectivity feature data;

[0165] Specifically, based on the functional connectivity matrix, we extract features of whole-brain functional connectivity, such as the average functional connectivity strength across the entire brain and the overall density of the functional connectivity network. These features can describe the level of functional integration across the entire brain. We also extract graph-theoretic features of the functional connectivity network, such as global efficiency and small-worldness, to reflect the topological structure of the whole-brain network.

[0166] Based on the functional connectivity matrix, local functional connectivity features, such as the local clustering coefficient and local efficiency, are extracted. These features can reflect the local connectivity patterns of specific brain regions. Modularity analysis is used to divide brain regions into communities, identify functionally closely connected groups of brain regions, and extract the internal connectivity features of these communities to obtain local functional connectivity feature data.

[0167] The whole-brain functional connectivity feature data and the local functional connectivity feature data are integrated to obtain the neural functional connectivity feature data.

[0168] Specifically, global and local functional connectivity feature data are integrated, using weighted averaging or feature-level fusion techniques to combine the different features into a single comprehensive feature vector. The integrated features are then normalized to ensure that features from different sources can be compared on the same scale. This results in neural functional connectivity feature data, which combines global and local functional connectivity information to fully describe the brain functional connectivity patterns of neurorehabilitation patients.

[0169] In the present invention, by dividing the functional magnetic resonance imaging data into brain regions, the different functional areas of the brain can be accurately divided, the functions of different brain regions can be analyzed in detail, and a basis for the subsequent analysis of functional connections can be provided to ensure the spatial accuracy of the data. Extracting the time-series blood oxygen level data (BOLD signal) of each brain region can dynamically reflect the changes in oxygen demand of neuronal activity, capture the activity level of different areas of the brain during task execution or rehabilitation, and help evaluate the functional recovery of brain regions, especially the impact on the brain during rehabilitation training. By performing correlation calculations on the time-series blood oxygen level data of each brain region, the functional connectivity between different brain regions can be revealed, reflecting the intensity of collaborative work or synchronous activity between brain regions, and helping users understand the network structure and changes between brain regions during rehabilitation.

[0170] Temporal correlation calculations can help reveal the remodeling of brain functional networks. This is particularly true in neurorehabilitation, where increased or decreased functional connectivity can reflect the recovery of brain function. Changes in brain connectivity during rehabilitation may be important signals of functional remodeling. The functional connectivity matrix, a mathematical representation of the strength of brain connections, provides a comprehensive picture of the brain's functional networks. Visualizing this matrix can intuitively display the connectivity patterns between different brain regions, helping to identify overall trends in brain functional networks during rehabilitation.

[0171] The functional connectivity matrix serves as a foundation for analyzing brain functional networks and supports subsequent extraction of network features, such as whole-brain network connectivity, network clustering coefficient, and small-worldness, thereby providing a more comprehensive understanding of the overall architecture and changes in brain networks. Whole-brain functional connectivity features capture the overall brain network characteristics, reflecting how different brain regions work together during rehabilitation. This helps assess the recovery of overall brain function, particularly in cognition, attention, motor control, and other functions involving a wide range of brain regions.

[0172] Local functional connectivity features can focus on connectivity changes in specific pairs of brain regions and provide refined local network analysis. For example, focusing on local connections related to motor function and language function can provide a more accurate assessment basis and can detect local functional reconstruction during the rehabilitation process. By integrating whole-brain functional connectivity features and local functional connectivity features, multi-level analysis of brain functional networks can be achieved. Whole-brain connectivity provides a macro perspective, while local connectivity provides a micro perspective, which helps to more comprehensively understand the functional recovery of the brain.

[0173] Preferably, the multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data, and step S3 is specifically:

[0174] Performing modal classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data;

[0175] Specifically, the multimodal feature data of neurorehabilitation patients are classified, and firstly, the data is labeled according to the modal source. For example, the feature data from EEG is labeled as "EEG modality", the feature data from fMRI is labeled as "fMRI modality", and the features from other sources are labeled as the corresponding modalities. Since the features have been labeled with the source modality, these labels can be used directly for modality classification. This classification process can be performed by a simple rule classifier (such as a decision tree classifier) ​​to classify the feature data of different modalities into their respective modality categories. The first multimodal feature classification data is obtained, indicating the modal category (such as EEG, fMRI, etc.) to which each feature belongs.

[0176] The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

[0177] Specifically, in multimodal feature data, the features are labeled by type. For example, the time domain features in EEG are labeled as "time domain features" and the frequency domain features are labeled as "frequency domain features"; in fMRI data, the functional connectivity features can be labeled as "connection features" and the graph theory features are labeled as "graph theory features". Machine learning classification algorithms such as support vector machines (SVM) or random forests are used to classify feature types based on the statistical properties of the features (such as mean, variance, frequency, etc.). The algorithm classifies the features into corresponding type categories based on the trained classification model. Based on the statistical properties of each feature type, dimension selection and dimensionality reduction are performed to optimize classification efficiency and reduce redundant features. The second multimodal feature classification data is obtained, which represents the specific types of different modal features (such as time domain features, frequency domain features, connection features, etc.).

[0178] In the present invention, by performing modal classification on the multimodal feature data of neurorehabilitation patients, it is possible to effectively distinguish the data according to its source (such as electroencephalogram, functional magnetic resonance imaging, etc.), ensure that different types of signals are processed separately, reduce confusion during data fusion, and help to more accurately analyze the contribution of each modality. After modal classification, the data of each modality can be processed, analyzed and modeled independently, so that the model can perform targeted feature selection and optimization between different modalities, thereby improving the efficiency and flexibility of data processing. By classifying multimodal feature data by feature type (such as time domain features, frequency domain features, complexity features, etc.), the organization of the data can be made clearer. Each feature type can represent a different form of physiological signal expression, for example, time domain features reflect instantaneous changes, frequency domain features reveal periodic activities, and complexity features reflect nonlinear behavior.

[0179] Preferably, step S4 is specifically:

[0180] Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data;

[0181] Specifically, the pre-set neurological patient knowledge graph contains medical knowledge related to neurorehabilitation, pathological characteristics, functional connectivity patterns, rehabilitation treatment processes, etc. The nodes in the knowledge graph represent different entities in the nervous system (such as brain regions, neural networks, rehabilitation methods, etc.), and the edges represent the relationships between these entities (such as functional connectivity, pathological associations, etc.).

[0182] Fitting methods include using graph neural networks to embed multimodal feature classification data into a knowledge graph. Through graph convolution operations, each feature is fitted to nodes and edges in the knowledge graph to obtain a representation of the association between the feature and the graph. Alternatively, similarity metrics such as cosine similarity or Euclidean distance are used to calculate the similarity between multimodal features and knowledge graph nodes to find the most similar knowledge nodes.

[0183] Through GNN or similarity calculation, we generate fitting data between multimodal features and knowledge graphs, and record the degree of matching between each multimodal feature and the corresponding knowledge node. We obtain the multimodal feature knowledge fitting data of neurological patients, which represents the position and association information of each multimodal feature in the knowledge graph.

[0184] The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients to assist in the construction of neurological rehabilitation evaluation models.

[0185] Specifically,

[0186] The introduction of the knowledge graph in the present invention systematizes and structures the medical knowledge and expert experience in the field of neurorehabilitation, and becomes a core part of the present invention. Through the rich information stored in the knowledge graph, such as the type of nerve injury, rehabilitation path, standardized treatment plan, patient recovery stage, etc., the system can have knowledge-driven capabilities in the data processing process, which greatly improves the accuracy of data analysis, especially the processing efficiency of complex rehabilitation cases. The feature data is marked by fitting the multimodal feature knowledge of neurological patients, so that the marking process not only depends on the original multimodal data features, but also relies on the medical knowledge and rehabilitation experience provided by the knowledge graph. Through the knowledge fitting process, the marking process becomes more accurate, and the most appropriate marking label can be automatically matched according to the specific feature status of the patient.

[0187] Preferably, the present application also provides a system for constructing a neurorehabilitation assessment model, which is used to execute the method for constructing a neurorehabilitation assessment model as described above. The system for constructing a neurorehabilitation assessment model includes:

[0188] Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data;

[0189] A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data;

[0190] The modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data;

[0191] The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients to assist in the construction of neurorehabilitation assessment models.

[0192] The beneficial effects of the present invention are:

[0193] 1. Data from different modalities provides a multidimensional perspective on neural activity. For example, the high temporal resolution of EEG and the high spatial resolution of fMRI can overcome the limitations of a single modality and provide richer foundational information for subsequent neurorehabilitation assessments. Multimodal data can reflect the dynamic changes in a patient's brain function at different temporal, spatial, and physiological scales, enabling assessment models to not only detect short-term neural responses but also capture the long-term process of brain function reconstruction.

[0194] 2. Time domain features reflect instantaneous changes in the signal, frequency domain features reveal the intensity of neural activity in different frequency bands, and complexity features such as sample entropy can quantify the nonlinear behavior of the signal, ensuring that the model can fully capture subtle changes in neural activity during neurorehabilitation. By extracting different types of features from multimodal data, the model can obtain rich input information, thereby improving the accuracy and robustness of the evaluation. During training, the model can interpret these different types of features to help analyze the complex dynamics of brain activity and identify key neural changes in rehabilitation. By combining time domain, frequency domain, and complexity features, the amount of feature information for each patient has increased from an average of 200 dimensions to 350 dimensions, and the information gain has increased by approximately 75%, providing more input data for accurate training of the model.

[0195] 3. Modality classification helps distinguish data sources from different modalities (such as EEG and fMRI), ensuring targeted feature processing and analysis. Feature type classification further categorizes features by type (such as time domain, frequency domain, and complexity), facilitating the appropriate selection and use of features. This classification results in more structured data and more efficient data processing.

[0196] 4. Through fitting calculations, the knowledge graph can help identify a patient's individualized neurological injury and rehabilitation progress. For example, for different injury types or recovery speeds, the system can match the most similar historical rehabilitation cases from the knowledge graph and generate corresponding rehabilitation path recommendations based on this. From a data processing perspective, the rehabilitation plan is more in line with the patient's actual situation, and differentiated neurological rehabilitation assessment data can be provided for different patients, avoiding a "one-size-fits-all" treatment model. Through the intelligent fitting and labeling of the knowledge graph, the model's prediction accuracy has increased from the original 87% to 96%, an increase of 9 percentage points.

[0197] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes that fall within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0198] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a neurorehabilitation assessment model, characterized in that: The following steps are involved: Step S1: collecting EEG data of the neurorehabilitation patient using an EEG device to obtain EEG data of the neurorehabilitation patient; collecting functional magnetic resonance imaging data of the neurorehabilitation patient using a high-resolution fMRI scanner to obtain functional magnetic resonance imaging data of the neurorehabilitation patient; integrating the EEG data of the neurorehabilitation patient and the functional magnetic resonance imaging data of the neurorehabilitation patient to obtain multimodal data of the neurorehabilitation patient; Step S2: extracting neural activity features from the electroencephalogram (EEG) data of the neurorehabilitation patient to obtain neural activity feature data; extracting functional connectivity features from the functional magnetic resonance imaging (fMRI) data of the neurorehabilitation patient to obtain neural functional connectivity feature data; and fusing the neural activity feature data and the neural functional connectivity feature data to obtain multimodal feature data of the neurorehabilitation patient. Step S3: performing data classification on the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data; Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, to obtain the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model; The neural activity feature extraction is specifically as follows: Performing artifact removal on the EEG data of neurorehabilitation patients to obtain EEG artifact-removed data; Extracting time domain features and frequency domain features from EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data; Perform sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data; Phase space reconstruction is performed based on the characteristic sample entropy data, EEG time series characteristic data and EEG frequency domain characteristic data to obtain neural activity characteristic data.

2. The method according to claim 1, characterized in that The sample entropy calculation is specifically as follows: Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively; Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively; Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively; The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the feature sample entropy data.

3. The method according to claim 2, characterized in that The phase space reconstruction is specifically as follows: The time delay parameter data is obtained by calculating the average mutual information based on the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data; Principal component analysis and maximum variance dimension screening are performed based on feature sample entropy data, EEG time series feature data, and EEG frequency domain feature data to obtain embedded dimension data; Perform phase space encoding on the feature sample entropy data, the EEG time series feature data, and the EEG frequency domain feature data according to the time delay parameter data and the embedding dimension data, and obtain feature sample entropy encoding data, EEG time series feature encoding data, and EEG frequency domain feature encoding data, respectively; Constructing a multimodal joint phase space based on feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data; The multimodal joint phase space data is processed into phase space trajectory to obtain neural activity feature data.

4. The method according to claim 1, wherein The functional connection feature extraction is specifically as follows: Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data; Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data; Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data; The functional connectivity matrix is ​​constructed based on the time-series blood oxygen correlation data to obtain the functional connectivity matrix data; Performing whole-brain functional connectivity feature extraction and local functional connectivity feature extraction on the functional connectivity matrix data to obtain whole-brain functional connectivity feature data and local functional connectivity feature data; The whole-brain functional connectivity feature data and the local functional connectivity feature data are integrated to obtain the neural functional connectivity feature data.

5. The method according to claim 1, characterized in that The multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data. Step S3 is specifically as follows: Performing modal classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data; The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

6. The method according to claim 1, characterized in that Step S4 is specifically as follows: Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data; The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients to assist in the construction of neurological rehabilitation evaluation models.

7. A system for constructing a neurorehabilitation assessment model, characterized in that: The method for constructing a neurorehabilitation assessment model according to claim 1 is used to execute the method, wherein the system for constructing a neurorehabilitation assessment model comprises: Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data; A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data; The modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data; The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients to assist in the construction of neurorehabilitation assessment models.

Images