Construction method and system for neural rehabilitation evaluation model

Abstract

The invention relates to the technical field of data processing, in particular to a construction method and system for a neural rehabilitation evaluation model. The method comprises the following steps: acquiring multi-modal data of a neurological rehabilitation patient; performing feature extraction according to the multi-modal data of the neurological rehabilitation patient to obtain multi-modal feature data of the neurological rehabilitation patient; performing data classification on the multi-modal feature data of the neurological rehabilitation patient to obtain multi-modal feature classification data; and marking the multi-modal feature data of the neurological rehabilitation patient by using the multi-modal feature classification data to obtain multi-modal feature marking data of the neurological rehabilitation patient so as to carry out neurological rehabilitation evaluation model construction auxiliary work. According to the method, the model capable of comprehensively and objectively evaluating the rehabilitation progress of the neurological rehabilitation patient is constructed in an auxiliary manner, so that a scientific basis is provided for rehabilitation treatment.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for constructing a neurorehabilitation assessment model. Background Art

[0002] With the rapid development of neuroscience and rehabilitation medicine, how to effectively evaluate the rehabilitation process of neurorehabilitation patients has become an important topic in the medical field. Neurorehabilitation is a series of rehabilitation treatments to help patients with damaged nervous systems restore their functions, especially in the rehabilitation process of stroke, brain trauma, spinal cord injury and other diseases. However, traditional rehabilitation assessment methods mostly rely on subjective clinical scores and patient self-reports, which cannot fully and objectively reflect the recovery of the nervous system.

[0003] Modern neurorehabilitation assessment increasingly relies on big data and multimodal analysis techniques, such as biomedical signals such as functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG). These technologies provide objective evidence of brain function recovery by capturing the functional activities of the brain. However, due to the complexity and heterogeneity of the data, how to effectively integrate, process and analyze these multimodal data remains a difficulty of current technology. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for constructing a neurorehabilitation assessment model to solve at least one of the above technical problems.

[0005] The present application provides a method for constructing a neurorehabilitation assessment model, comprising the following steps:

[0006] Step S1: Acquire multimodal data of neurorehabilitation patients;

[0007] Step S2: extracting features based on the multimodal data of the neurorehabilitation patients to obtain multimodal feature data of the neurorehabilitation patients;

[0008] Step S3: classifying the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data;

[0009] Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, and obtaining the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model.

[0010] The use of multimodal data and the diversification of feature extraction in the present invention ensure that the rehabilitation assessment is based on accurate data in multiple dimensions, thereby improving the accuracy of the assessment. Data classification and labeling make the model construction more robust, and can accurately capture the changes in the patient's rehabilitation status and help identify key progress in rehabilitation. Labeled data provides a solid foundation for the supervised learning of the model, while optimizing the utilization efficiency of the data through classification, making the model more accurate in specific areas. Through the joint use and analysis of multimodal data, the performance of the neurorehabilitation assessment model can be further optimized.

[0011] Preferably, step S1 specifically includes:

[0012] Collect EEG data of neurorehabilitation patients through EEG equipment;

[0013] Functional magnetic resonance imaging data of neurorehabilitation patients were obtained by acquiring functional magnetic resonance imaging through a high-resolution fMRI scanner;

[0014] The EEG data and functional magnetic resonance imaging data of neurorehabilitation patients are integrated to obtain multimodal data of neurorehabilitation patients.

[0015] The present invention can capture different dimensions of the patient's brain activity by integrating the two modalities of electroencephalography and functional magnetic resonance imaging. Electroencephalography is good at providing neural activity information with high temporal resolution, while functional magnetic resonance imaging provides brain area activation information with high spatial resolution. The combination of the two can make up for the limitations of a single modality and provide more comprehensive brain function data. Electroencephalography can capture rapidly changing EEG signals, while functional magnetic resonance imaging can capture brain area activation reflected by hemodynamic changes. The two are highly complementary. After integration, the coordinated information of EEG activity and brain functional areas can be obtained at the same time, thereby improving the accuracy of rehabilitation assessment. Electroencephalogram data provides millisecond-level temporal resolution, which can capture rapid neural signal transmission and synchronized activities in the brain, and has advantages in evaluating the patient's rehabilitation process, especially in changes in neural reaction speed, attention and cognitive load. Functional magnetic resonance imaging provides millimeter-level spatial resolution, which can map out different functional areas in the brain in detail, which is of great significance for analyzing the interaction between brain areas, evaluating the recovery of brain area activity, and determining the changes in brain area functional connectivity during rehabilitation.

[0016] Preferably, step S2 specifically includes:

[0017] Extract neural activity features from the EEG data of neurorehabilitation patients to obtain neural activity feature data;

[0018] Extract functional connectivity features from functional magnetic resonance imaging data of neurorehabilitation patients to obtain neural functional connectivity feature data;

[0019] The neural activity feature data and the neural functional connection feature data are fused to obtain the multimodal feature data of neurorehabilitation patients.

[0020] The neural activity features extracted from the EEG data in the present invention can reflect the electrical activity of neurons in real time, such as the power spectrum of different frequency bands of the brain, neural oscillation patterns, synchronization and desynchronization of brain waves. For neurorehabilitation assessment, these features help capture immediate neural responses in rehabilitation training and evaluate the recovery progress of neural functions. Neural activity features (such as sample entropy, timing features, etc.) can reveal the neural state of brain functional areas, help evaluate whether specific brain areas have restored normal functions during the rehabilitation process, and provide accurate feedback on rehabilitation progress. Through the extraction of functional connectivity features of functional magnetic resonance imaging data, the functional connection relationship between different brain regions can be revealed, reflecting the coordinated activities between brain regions during the rehabilitation process, especially the reconstruction of neural network connections across brain regions. Functional connectivity features help analyze the impact of rehabilitation training on brain area networks. In neurorehabilitation, the remodeling of brain functional networks is an important sign of rehabilitation. The extraction of functional connectivity features helps to evaluate the dynamic changes of brain functional networks, especially in terms of motor control and cognitive function recovery, and can intuitively display the connection recovery between different brain regions. By fusing neural activity characteristics (EEG) and functional connectivity characteristics (fMRI), it is possible to simultaneously utilize the high temporal resolution of EEG and the high spatial resolution of fMRI, providing more comprehensive neural feature data for neurorehabilitation. It can not only monitor neural activity in real time, but also locate the functional recovery of brain regions in detail.

[0021] Preferably, the neural activity feature extraction is specifically:

[0022] Performing artifact removal on the EEG data of neurorehabilitation patients to obtain EEG artifact-removed data;

[0023] Extracting time domain features and frequency domain features of the EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data;

[0024] Perform sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data;

[0025] Phase space reconstruction is performed based on characteristic sample entropy data, EEG time series characteristic data and EEG frequency domain characteristic data to obtain neural activity characteristic data.

[0026] By removing artifacts (such as eye movement, electromyographic noise, etc.), the signal quality of EEG data can be greatly improved in the present invention, ensuring that subsequent feature extraction is based on pure neural activity signals, reducing the interference of artifacts on the analysis results, and improving the accuracy of neural activity features. Artifact removal techniques, such as independent component analysis (ICA) or filtering processing, can retain key neural activity information while removing interference. Through time domain feature extraction (such as mean, peak, variance, etc.), the amplitude changes and waveform characteristics of EEG signals can be captured to reflect the instantaneous neural activity of the brain. Frequency domain feature extraction (such as power spectral density, band energy, etc.) can reveal neural activity patterns in different frequency bands (such as Alpha, Beta waves, etc.), which helps to analyze neural activity in different states of the brain (such as relaxation, concentration, etc.). Combining the extraction of time domain and frequency domain features, it can provide multi-level feature data for the comprehensive evaluation of neural activity and enhance the comprehensive evaluation ability of the patient's neural state. Sample entropy calculation can quantify the complexity of time series data (such as EEG time domain and frequency domain features) and help identify the nonlinear characteristics of neural signals. For example, during the rehabilitation process, the activity patterns of the nervous system often change from disorder to order, and from complex to simple, and these changes can be accurately quantified through sample entropy. Higher sample entropy values ​​usually indicate an increase in the complexity of neural activity, reflecting the complex process of the brain from an impaired state to partial recovery during rehabilitation. Sample entropy provides a new perspective for the neurorehabilitation process and can effectively monitor changes in neural activity. Phase space reconstruction can reveal the potential nonlinear dynamic behavior of the nervous system by mapping neural activity feature data into a higher-dimensional phase space, effectively revealing the periodicity, chaotic behavior or attractor structure of neural activity, and helping to more deeply understand the complex dynamic process of neural activity during rehabilitation.

[0027] Preferably, the sample entropy calculation is specifically as follows:

[0028] Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively;

[0029] Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively;

[0030] Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively;

[0031] The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the characteristic sample entropy data.

[0032] In the present invention, by dividing the time series features and frequency domain features of the electroencephalogram into vector sequences, the neural activity is analyzed from the two dimensions of time and frequency, and the changes in neural activity at different levels can be captured. The divided vector sequence can adapt to the complexity of the data more flexibly, especially in different brain regions and states, the feature performance may be different. By dividing the vector sequence, it is helpful to carefully analyze the local and global features of the electroencephalogram signal, and improve the delicacy and accuracy of the analysis. Through similarity calculation, similar patterns in the electroencephalogram time series and frequency domain data can be quantified, which is very important for detecting specific brain states (such as changes in brain activity during rehabilitation). High similarity indicates the stability or consistency of the data, and low similarity may indicate irregular changes in neural activity or a functional reconstruction process. Similarity calculation can help identify trend changes or mutation points in electroencephalogram signals, which is crucial for neurorehabilitation assessment because it can reveal the recovery or change trajectory of brain function during rehabilitation. Sample entropy is an important tool for quantifying the complexity and unpredictability of time series. By calculating the sample entropy of electroencephalogram time series data and frequency domain data, the complexity of brain neural activity can be evaluated. Higher sample entropy values ​​generally reflect the diversity and flexibility of brain activity, while lower sample entropy values ​​reflect stable neural activity patterns. Sample entropy calculation is not limited to time series features, but also extends to frequency domain features. Through this dual-dimensional complexity analysis, it is possible to more comprehensively evaluate the changes in the brain during neurorehabilitation. Frequency domain sample entropy reveals the complexity of neural activity in different frequency bands, and in many cases can supplement the deficiencies of time domain data.

[0033] Preferably, the phase space reconstruction is specifically:

[0034] The average mutual information is calculated based on the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data to obtain the time delay parameter data;

[0035] According to the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data, principal component analysis and maximum variance dimension screening are performed to obtain the embedded dimension data;

[0036] According to the time delay parameter data and the embedding dimension data, the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data are phase-space encoded to obtain feature sample entropy encoded data, EEG time series feature encoded data and EEG frequency domain feature encoded data respectively;

[0037] Constructing a multimodal joint phase space according to feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data;

[0038] The multimodal joint phase space data is processed by phase space trajectory to obtain neural activity feature data.

[0039] In the present invention, by using average mutual information (AMI) to calculate the time delay parameter, the information of time dependence in the electroencephalogram signal can be captured more accurately. AMI can quantify the dependency of adjacent points in the feature data and automatically determine the optimal time delay parameter, avoiding the subjectivity of manually setting the time delay. Reasonable time delay parameters ensure that the dynamics and complexity of the signal features in the subsequent phase space reconstruction can be fully displayed, helping to reveal the key change points of neural activity in the rehabilitation process. Through principal component analysis (PCA) and screening the maximum variance dimension, the selection of the embedding dimension is more objective and data-driven, which can automatically extract the most representative information, reduce redundant features and noise, and improve the efficiency of data analysis. In the phase space reconstruction process, the embedding dimension is a key parameter. The dimension selection after PCA optimization can ensure that the reconstructed phase space retains the key information in the original data and avoids the computational complexity caused by too high dimensions. Phase space coding can uniformly map sample entropy data, time series feature data and frequency domain feature data into the phase space to form a unified coding representation, which helps to capture the interdependence between the features and effectively build a more complete neural activity model. Phase space encoding helps reproduce the nonlinear dynamic characteristics of brain activity. By embedding feature data into a higher-dimensional space, complex neural activity patterns and trajectories can be revealed, revealing potential nonlinear behaviors in the recovery process of the nervous system. By jointly constructing a multimodal phase space with encoded data of different modalities (sample entropy, time series features, frequency domain features), a comprehensive analysis of multidimensional data can be achieved, providing more comprehensive and multi-angle information for the evaluation of brain function, and helping to understand the interaction between the modalities. The multimodal joint phase space can provide richer feature dimensions and reduce the limitations of single-modal analysis. By combining spatiotemporal and frequency information, this method can more accurately reflect the key dynamic changes in the process of neurorehabilitation. By performing trajectory processing on the multimodal joint phase space, the motion trajectory of neural activity in high-dimensional space can be analyzed. Phase space trajectories can reveal the periodicity, attractor structure or chaotic behavior of neural activity, helping to better understand the changing patterns of brain function during rehabilitation.

[0040] Preferably, the functional connectivity feature extraction is specifically:

[0041] Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data;

[0042] Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data;

[0043] Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data;

[0044] The functional connection matrix is ​​constructed according to the time-series blood oxygen correlation data to obtain the functional connection matrix data;

[0045] Extracting whole-brain functional connection features and local functional connection features from the functional connection matrix data to obtain whole-brain functional connection feature data and local functional connection feature data;

[0046] The whole-brain functional connection feature data and the local functional connection feature data are integrated to obtain the neural functional connection feature data.

[0047] In the present invention, by dividing the functional magnetic resonance imaging data into brain regions, the different functional areas of the brain can be accurately divided, the functions of different brain regions can be analyzed in detail, and the basis for the subsequent analysis of functional connections can be provided to ensure the spatial accuracy of the data. Extracting the time-series blood oxygen level data (BOLD signal) of each brain region can dynamically reflect the changes in oxygen demand of neuronal activity, capture the activity level of different regions of the brain during task execution or rehabilitation, and help evaluate the functional recovery of brain regions, especially the impact on the brain during rehabilitation training. By performing correlation calculations on the time-series blood oxygen level data of each brain region, the functional connectivity between different brain regions can be revealed, reflecting the intensity of collaborative work or synchronous activity between brain regions, and helping users understand the network structure and changes between brain regions during rehabilitation.

[0048] Temporal correlation calculations can help discover the remodeling process of the brain's functional network. Especially in neurorehabilitation, the increase or decrease of functional connectivity can reflect the recovery of brain function. Changes in brain region connectivity during rehabilitation may be an important signal for the reconstruction of brain function. The functional connectivity matrix is ​​a mathematical representation of the strength of brain region connections, which can provide a full picture of the brain's functional network. Through the visualization of the matrix, the connection patterns between different brain regions can be intuitively displayed, helping to discover the overall change trend of the brain's functional network during rehabilitation.

[0049] The functional connectivity matrix can serve as the basis for analyzing brain functional networks and support subsequent network feature extraction, such as the connectivity of the whole brain network, the network clustering coefficient, and the small-worldness, so as to more comprehensively understand the overall architecture of the brain network and its changes. The whole-brain functional connectivity features can capture the overall network characteristics of the brain, reflect how different brain regions work together during the rehabilitation process, and help evaluate the recovery of the overall brain function, especially in cognition, attention, motor control and other functions involving a wide range of brain regions.

[0050] Local functional connectivity features can focus on the connectivity changes of specific brain regions and provide refined local network analysis. For example, focusing on local connections related to motor function and language function can provide a more accurate basis for evaluation and can detect local functional reconstruction during rehabilitation. By integrating whole-brain functional connectivity features and local functional connectivity features, multi-level analysis of brain functional networks can be achieved. Whole-brain connectivity provides a macro perspective, while local connectivity provides a micro perspective, which helps to more comprehensively understand the functional recovery of the brain.

[0051] Preferably, the multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data, and step S3 is specifically:

[0052] Performing modality classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data;

[0053] The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

[0054] In the present invention, by performing modal classification on the multimodal feature data of neurorehabilitation patients, it is possible to effectively distinguish the data according to its source (such as electroencephalogram, functional magnetic resonance imaging, etc.), ensure that different types of signals are processed separately, reduce confusion during data fusion, and help to more accurately analyze the contribution of each modality. After modal classification, the data of each modality can be processed, analyzed and modeled independently, so that the model can perform targeted feature selection and optimization between different modalities, thereby improving the efficiency and flexibility of data processing. By classifying multimodal feature data by feature type (such as time domain features, frequency domain features, complexity features, etc.), the organization of the data can be made clearer. Each feature type can represent different physiological signal manifestations, for example, time domain features reflect instantaneous changes, frequency domain features reveal periodic activities, and complexity features reflect nonlinear behavior.

[0055] Preferably, step S4 is specifically:

[0056] Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data;

[0057] The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

[0058] The introduction of the knowledge graph in the present invention presents the medical knowledge and expert experience in the field of neurorehabilitation in a systematic and structured manner, becoming a core part of the present invention. Through the rich information stored in the knowledge graph, such as the type of nerve injury, rehabilitation pathway, standardized treatment plan, patient recovery stage, etc., the system can have knowledge-driven capabilities in the data processing process, greatly improving the accuracy of data analysis, especially the processing efficiency of complex rehabilitation cases. The feature data is labeled by fitting the multimodal feature knowledge of neurological patients, so that the labeling process not only depends on the original multimodal data features, but also relies on the medical knowledge and rehabilitation experience provided by the knowledge graph. Through the knowledge fitting process, the labeling process becomes more accurate, and the most appropriate label tag can be automatically matched according to the specific feature status of the patient.

[0059] Preferably, the present application also provides a system for constructing a neurorehabilitation assessment model, which is used to execute the method for constructing a neurorehabilitation assessment model as described above. The system for constructing a neurorehabilitation assessment model includes:

[0060] Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data;

[0061] A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data;

[0062] A modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data;

[0063] The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

[0064] The beneficial effects of the present invention are:

[0065] 1. Data from different modalities provide a multidimensional perspective of neural activity, such as the high temporal resolution of EEG and the high spatial resolution of functional magnetic resonance imaging, which can make up for the limitations of a single modality and provide richer basic information for subsequent neurorehabilitation assessment. Multimodal data can reflect the dynamic changes in the patient's brain function at different time scales, spatial scales, and physiological scales, allowing the evaluation model to not only detect short-term neural responses, but also capture the long-term brain function reconstruction process.

[0066] 2. Time domain features reflect instantaneous changes in signals, frequency domain features reveal the intensity of neural activity in different frequency bands, and complexity features such as sample entropy can quantify the nonlinear behavior of signals, ensuring that the model can fully capture subtle changes in neural activity during neurorehabilitation. By extracting different types of features from multimodal data, the model can obtain rich input information, thereby improving the accuracy and robustness of the evaluation. During the training process, the model can be interpreted based on these different types of features to help analyze the complex dynamics of brain activity and identify key neural changes in rehabilitation. By combining time domain, frequency domain, and complexity features, the amount of feature information for each patient has increased from an average of 200 dimensions to 350 dimensions, and the information gain has increased by about 75%, providing more input data for accurate training of the model.

[0067] 3. Modality classification helps to distinguish data sources of different modalities (such as EEG and fMRI), ensuring the pertinence of feature processing and analysis. Feature type classification further classifies features by their types (such as time domain, frequency domain, complexity, etc.) to facilitate the rational selection and use of features. The classified data is more structured and data processing is more efficient.

[0068] 4. Through fitting calculations, the knowledge graph can help identify the patient's individualized neurological injury and rehabilitation progress. For example, for different injury types or rehabilitation speeds, the system can match the most similar historical rehabilitation cases from the knowledge graph and generate corresponding rehabilitation path recommendations based on this. From the perspective of data processing, the rehabilitation plan is more in line with the patient's actual situation, and can provide differentiated neurological rehabilitation assessment data for different patients, avoiding a "one-size-fits-all" treatment model. Through the intelligent fitting and labeling of the knowledge graph, the prediction accuracy of the model has increased from the original 87% to 96%, an increase of 9 percentage points. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0070] Figure 1 A flowchart showing a method for constructing a neurorehabilitation assessment model according to an embodiment of the present invention is provided;

[0071] Figure 2 A flowchart showing a method for collecting multimodal data of a neurorehabilitation patient according to an embodiment of the present invention is provided;

[0072] Figure 3 A flowchart showing a method for extracting multimodal features of a neurorehabilitation patient according to an embodiment of the present invention is shown;

[0073] Figure 4A flowchart of the steps of a neural activity feature extraction method according to an embodiment is shown. DETAILED DESCRIPTION

[0074] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0076] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0077] See also Figures 1 to 4 , the present application provides a method for constructing a neurorehabilitation assessment model, comprising the following steps:

[0078] Step S1: Acquire multimodal data of neurorehabilitation patients;

[0079] Specifically, an electroencephalogram (EEG) device and a functional magnetic resonance imaging (fMRI) device are used to collect EEG data and functional magnetic resonance imaging data of neurorehabilitation patients, respectively.

[0080] The EEG data of neurorehabilitation patients were filtered (bandpass filtering to remove low-frequency and high-frequency noise), artifact removed (ICA was used to remove eye blinks, electromyographic artifacts, etc.), and standardized.

[0081] The functional magnetic resonance imaging data were subjected to motion correction, time correction, spatial normalization (MNI standard space), smoothing and time series extraction.

[0082] The processed EEG data and processed fMRI data were synchronized by time alignment to ensure that the data from both modalities could be analyzed in the same time frame.

[0083] The processed EEG data and processed fMRI data represent the EEG activities and BOLD signals of brain regions of neurorehabilitation patients at multiple time points, respectively, forming a multimodal dataset.

[0084] Step S2: extracting features based on the multimodal data of the neurorehabilitation patients to obtain multimodal feature data of the neurorehabilitation patients;

[0085] Specifically, time domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG time domain feature data, such as extracting time domain features of EEG signals, including mean, variance, peak and zero crossing points. Frequency domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG frequency domain feature data, such as extracting power spectrum density of different frequency bands (δ wave, θ wave, α wave, β wave, γ wave) by fast Fourier transform (FFT). Phase synchronization feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG phase synchronization feature data, such as calculating phase synchronization between brain regions (such as phase locking value PLV). The EEG time domain feature data, the EEG frequency domain feature data and the EEG phase synchronization feature data are vectorized to obtain EEG feature data;

[0086] Perform functional connectivity feature extraction on functional magnetic resonance imaging data to obtain functional connectivity feature data, such as calculating the time series correlation between different brain regions and constructing a functional connectivity matrix. Perform graph theory feature extraction on functional magnetic resonance imaging data to obtain graph theory feature data, such as extracting graph theory features in the functional connectivity network, such as node degree, clustering coefficient, and path length. Perform vectorization on the functional connectivity feature data and graph theory feature data to obtain functional magnetic resonance imaging feature data;

[0087] Multimodal feature fusion is performed on the EEG feature data and functional magnetic resonance imaging feature data, and they are integrated into a feature vector by weighted averaging or feature splicing to obtain multimodal feature data of neurorehabilitation patients.

[0088] Step S3: classifying the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data;

[0089] Specifically, the multimodal feature data is preliminarily classified according to the source of the feature (EEG or functional magnetic resonance imaging), and the feature is classified according to its modality label using a rule classifier or a decision tree algorithm.

[0090] Specifically, further classification is performed based on the type of feature (time domain, frequency domain, functional connectivity, etc.). Classification algorithms such as SVM or random forest are used to perform classification based on the statistical properties of the features.

[0091] Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, and obtaining the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model.

[0092] Specifically, the multimodal feature classification data is fitted with the preset neurological patient knowledge graph data. The feature data is embedded through a graph neural network (GNN), and its similarity with the knowledge graph nodes is calculated. Based on the fitting results, the multimodal features are automatically labeled. Labels are assigned to features (such as rehabilitation progress features, functional connectivity features, etc.) using labeling rules or machine learning classifiers. The labeled feature data is used to build a neurorehabilitation evaluation model to further assist in the evaluation and prediction of rehabilitation effects.

[0093] Auxiliary tasks for building a neurorehabilitation assessment model can use deep algorithms (such as deep neural network algorithms) to build models, divide the multimodal feature labeling data of neurorehabilitation patients, generate training sets and test sets, use the training sets to build models (model input layer, hidden neuron layer, and model output layer) to obtain a preliminary model; according to the test set, the preliminary model is iteratively trained through the squared error loss function to obtain a neurorehabilitation assessment model.

[0094] The use of multimodal data and the diversification of feature extraction in the present invention ensure that the rehabilitation assessment is based on accurate data in multiple dimensions, thereby improving the accuracy of the assessment. Data classification and labeling make the model construction more robust, and can accurately capture the changes in the patient's rehabilitation status and help identify key progress in rehabilitation. Labeled data provides a solid foundation for the supervised learning of the model, while optimizing the utilization efficiency of the data through classification, making the model more accurate in specific areas. Through the joint use and analysis of multimodal data, the performance of the neurorehabilitation assessment model can be further optimized.

[0095] Preferably, step S1 specifically includes:

[0096] Step S11: collecting EEG data of a neurorehabilitation patient through an EEG device to obtain EEG data of the neurorehabilitation patient;

[0097] Specifically, electrodes are placed on the patient's scalp according to the international 10-20 system to ensure coverage of key brain areas. An EEG amplifier is used to record the brain's electrical signals with a sampling rate of 500–1000 Hz to ensure accurate capture of tiny voltage changes (microvolt level).

[0098] Step S12: acquiring functional magnetic resonance imaging data of the neurorehabilitation patient by using a high-resolution fMRI scanner;

[0099] Specifically, a high-resolution functional magnetic resonance imaging scanner is used with a spatial resolution of 1-2 mm and a temporal resolution of about 2 seconds (TR = 2 seconds) to capture the blood oxygen level dependent (BOLD) signal of the brain. Whole-brain scans are performed in consecutive slices, covering the entire brain area, and multiple time series images are recorded to reflect brain activity within a certain period of time (e.g., 10-15 minutes).

[0100] Step S13: Integrate the electroencephalogram data of the neurorehabilitation patient and the functional magnetic resonance imaging data of the neurorehabilitation patient to obtain multimodal data of the neurorehabilitation patient.

[0101] Specifically, the EEG and fMRI data are aligned using a common timestamp or external event markers (such as button presses or visual / auditory stimuli presented to the patient during recording) to ensure that the two patterns correspond to neural activity at the same point in time. The EEG data is resampled to match the lower temporal resolution of the fMRI (for example, resampling the EEG from 500 Hz to 0.5 Hz) while preserving important patterns of neural activity.

[0102] The present invention can capture different dimensions of the patient's brain activity by integrating the two modalities of electroencephalography and functional magnetic resonance imaging. Electroencephalography is good at providing neural activity information with high temporal resolution, while functional magnetic resonance imaging provides brain area activation information with high spatial resolution. The combination of the two can make up for the limitations of a single modality and provide more comprehensive brain function data. Electroencephalography can capture rapidly changing EEG signals, while functional magnetic resonance imaging can capture brain area activation reflected by hemodynamic changes. The two are highly complementary. After integration, the coordinated information of EEG activity and brain functional areas can be obtained at the same time, thereby improving the accuracy of rehabilitation assessment. Electroencephalogram data provides millisecond-level temporal resolution, which can capture rapid neural signal transmission and synchronized activities in the brain, and has advantages in evaluating the patient's rehabilitation process, especially in changes in neural reaction speed, attention and cognitive load. Functional magnetic resonance imaging provides millimeter-level spatial resolution, which can map out different functional areas in the brain in detail, which is of great significance for analyzing the interaction between brain areas, evaluating the recovery of brain area activity, and determining the changes in brain area functional connectivity during rehabilitation.

[0103] Preferably, step S2 specifically includes:

[0104] Step S21: extracting neural activity features from the electroencephalogram data of the neurorehabilitation patient to obtain neural activity feature data;

[0105] Specifically, time domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG time domain feature data, such as extracting time domain features of EEG signals, including mean, variance, peak and zero crossing points. Frequency domain feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG frequency domain feature data, such as extracting power spectrum density of different frequency bands (δ wave, θ wave, α wave, β wave, γ wave) by fast Fourier transform (FFT). Phase synchronization feature extraction is performed on the EEG data of neurorehabilitation patients to obtain EEG phase synchronization feature data, such as calculating phase synchronization between brain regions (such as phase locking value PLV). The EEG time domain feature data, the EEG frequency domain feature data and the EEG phase synchronization feature data are vectorized to obtain EEG feature data;

[0106] The mean and variance of the time series of each electrode position are calculated to capture the overall level and fluctuation of the EEG signal, and the mean and variance are obtained. The maximum and minimum values ​​of the signal are identified to reflect the extreme fluctuations of neural activity, and the peak amplitude is obtained.

[0107] Using methods such as short-time Fourier transform (STFT) or discrete wavelet transform (DWT), the EEG data is converted from the time domain to the frequency domain, and the power spectral density (PSD) of each frequency band is extracted, such as: delta wave (0.5–4 Hz), theta wave (4–8 Hz), alpha wave (8–13 Hz), beta wave (13–30 Hz), and gamma wave (30–50 Hz).

[0108] Calculate phase synchronization features between different electrodes, such as comparing the phase difference between two electrodes to measure the degree of synchronization between them. Use coherence to measure the coordinated activities between different brain regions in different frequency bands.

[0109] Step S22: extracting functional connectivity features from functional magnetic resonance imaging data of neurorehabilitation patients to obtain neural functional connectivity feature data;

[0110] Specifically, functional connectivity feature extraction is performed on functional magnetic resonance imaging data to obtain functional connectivity feature data, such as calculating the time series correlation between different brain regions and constructing a functional connectivity matrix. Graph theory feature extraction is performed on functional magnetic resonance imaging data to obtain graph theory feature data, such as extracting graph theory features in the functional connectivity network, such as node degree, clustering coefficient and path length. Functional connectivity feature data and graph theory feature data are vectorized to obtain functional magnetic resonance imaging feature data;

[0111] The time series signals of each brain region were extracted from the preprocessed functional magnetic resonance imaging data of neurorehabilitation patients. The Pearson correlation coefficients of the time series signals between different brain regions were calculated to measure the strength of the functional connections between them. The correlation coefficient matrix will reflect the strength of the connection between each pair of brain regions. By controlling the influence of other brain regions, the direct functional connections between specific pairs of brain regions were calculated to generate a partial correlation matrix. Using the sliding window technique, the time series data were segmented, and the functional connections within each time window were calculated to capture the dynamic changes of the connections between brain regions.

[0112] Based on the functional connectivity matrix, a functional connectivity network is constructed to extract graph theory features, such as calculating the number of connections between each brain region and other brain regions to reflect the centrality of the brain region in the functional network. Measuring the shortest path from one brain region to another is used to evaluate the overall efficiency of the network. Measuring the density of connections between a brain region and its adjacent brain regions reflects the aggregation of the local network.

[0113] Based on the above steps, the neural functional connection characteristic data of the neurorehabilitation patients are obtained, including the functional connection matrix and graph theory characteristics.

[0114] Step S23: Fusing the neural activity feature data and the neural function connection feature data to obtain multimodal feature data of the neural rehabilitation patient.

[0115] Specifically, multimodal feature fusion is performed on the EEG feature data and the functional magnetic resonance imaging feature data, and they are integrated into a feature vector by weighted averaging or feature concatenation to obtain multimodal feature data of neurorehabilitation patients.

[0116] Specifically, the neural activity feature data and neural functional connectivity feature data are standardized to ensure that the two data types are compared and analyzed on the same scale. The time domain and frequency domain features of the EEG are spliced ​​with the functional connectivity features of the fMRI to form a multimodal feature vector. The feature vector of each patient contains information from different modalities, describing its neural activity and functional connectivity. Statistical learning methods such as canonical correlation analysis (CCA) or joint independent component analysis (JICA) are used to identify the associated components in the EEG and fMRI data and find the common patterns between different modal data. Dimensionality reduction techniques such as principal component analysis (PCA) or linear discriminant analysis (LDA) are applied to reduce the dimension of the feature vector while retaining the main pattern information to improve the efficiency of subsequent models. The fused multimodal feature data of the neurorehabilitation patient, including the fused time domain, frequency domain and functional connectivity features, are obtained and prepared for further analysis of the model.

[0117] The neural activity features extracted from the EEG data in the present invention can reflect the electrical activity of neurons in real time, such as the power spectrum of different frequency bands of the brain, neural oscillation patterns, synchronization and desynchronization of brain waves. For neurorehabilitation assessment, these features help capture immediate neural responses in rehabilitation training and evaluate the recovery progress of neural functions. Neural activity features (such as sample entropy, timing features, etc.) can reveal the neural state of brain functional areas, help evaluate whether specific brain areas have restored normal functions during the rehabilitation process, and provide accurate feedback on rehabilitation progress. Through the extraction of functional connectivity features of functional magnetic resonance imaging data, the functional connection relationship between different brain regions can be revealed, reflecting the coordinated activities between brain regions during the rehabilitation process, especially the reconstruction of neural network connections across brain regions. Functional connectivity features help analyze the impact of rehabilitation training on brain area networks. In neurorehabilitation, the remodeling of brain functional networks is an important sign of rehabilitation. The extraction of functional connectivity features helps to evaluate the dynamic changes of brain functional networks, especially in terms of motor control and cognitive function recovery, and can intuitively display the connection recovery between different brain regions. By fusing neural activity characteristics (EEG) and functional connectivity characteristics (fMRI), it is possible to simultaneously utilize the high temporal resolution of EEG and the high spatial resolution of fMRI, providing more comprehensive neural feature data for neurorehabilitation. It can not only monitor neural activity in real time, but also locate the functional recovery of brain regions in detail.

[0118] Preferably, the neural activity feature extraction is specifically:

[0119] Step S211: performing artifact removal on the EEG data of the neurorehabilitation patient to obtain EEG artifact removed data;

[0120] Specifically, the EEG signal is subjected to independent component analysis to separate signals from different sources, such as neural activity signals and artifact signals (such as eye movement, muscle activity, heartbeat artifacts, etc.). By analyzing the timing diagram and power spectrum of the independent components based on a preset threshold, the artifact signals are identified and removed from the EEG signal. After removing the artifacts, the remaining independent component signals are reconstructed to obtain the EEG data after artifact removal.

[0121] Step S212: extracting time domain features and frequency domain features from the EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data;

[0122] Specifically, the average value of each channel in the entire time series is calculated to reflect the central trend of the signal. The variance of each channel is calculated to indicate the amplitude of signal fluctuation. By detecting positive and negative peaks, the extreme points of the signal are found to reflect the intensity of neural activity. The EEG time series feature data is obtained, including mean, variance and peak features.

[0123] The signal is converted from the time domain to the frequency domain using the fast Fourier transform (FFT) to calculate the power in each frequency band (δ wave, θ wave, α wave, β wave, γ wave). By calculating the phase difference between each electrode, the phase synchronization characteristics between different brain regions, such as the phase locking value (PLV), are obtained. The EEG frequency domain feature data, including the power and phase synchronization characteristics of different frequency bands, are obtained.

[0124] Step S213: performing sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data;

[0125] Specifically, the sample entropy calculation is performed on the time series features and frequency domain features. The steps are as follows: Select the embedding dimension and tolerance. Divide the time series into segments with a length equal to the embedding dimension, and calculate the similarity of each segment with other segments. Count the number of similar segments and calculate the sample entropy value. The lower the sample entropy value, the stronger the regularity of the time series; the higher the sample entropy value, the greater the complexity of the time series. The characteristic sample entropy data is obtained to characterize the complexity of the EEG data.

[0126] Step S214: Reconstruct the phase space according to the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data to obtain the neural activity characteristic data.

[0127] Specifically, based on Takens' theorem, the phase space is constructed using sample entropy data, time series feature data, and frequency domain feature data. Time delays and embedding dimensions are generated by preset parameters, and the one-dimensional time series is embedded into the high-dimensional phase space by the time delay coordinate method. Using time delay embedding, the features of each time point are mapped to a high-dimensional space to form a multidimensional feature trajectory, representing the neural activity state at different time points. Dynamic patterns of neural activity are identified in the phase space, such as stable attractors, periodic trajectories, or chaotic trajectories, reflecting different neural activity states. Neural activity feature data is obtained, which contains the trajectory of the EEG signal in the phase space and can be used for further neurorehabilitation evaluation.

[0128] By removing artifacts (such as eye movement, electromyographic noise, etc.), the signal quality of EEG data can be greatly improved in the present invention, ensuring that subsequent feature extraction is based on pure neural activity signals, reducing the interference of artifacts on the analysis results, and improving the accuracy of neural activity features. Artifact removal techniques, such as independent component analysis (ICA) or filtering processing, can retain key neural activity information while removing interference. Through time domain feature extraction (such as mean, peak, variance, etc.), the amplitude changes and waveform characteristics of EEG signals can be captured to reflect the instantaneous neural activity of the brain. Frequency domain feature extraction (such as power spectral density, band energy, etc.) can reveal neural activity patterns in different frequency bands (such as Alpha, Beta waves, etc.), which helps to analyze neural activity in different states of the brain (such as relaxation, concentration, etc.). Combining the extraction of time domain and frequency domain features, it can provide multi-level feature data for the comprehensive evaluation of neural activity and enhance the comprehensive evaluation ability of the patient's neural state. Sample entropy calculation can quantify the complexity of time series data (such as EEG time domain and frequency domain features) and help identify the nonlinear characteristics of neural signals. For example, during the rehabilitation process, the activity patterns of the nervous system often change from disorder to order, and from complex to simple, and these changes can be accurately quantified through sample entropy. Higher sample entropy values ​​usually indicate an increase in the complexity of neural activity, reflecting the complex process of the brain from an impaired state to partial recovery during rehabilitation. Sample entropy provides a new perspective for the neurorehabilitation process and can effectively monitor changes in neural activity. Phase space reconstruction can reveal the potential nonlinear dynamic behavior of the nervous system by mapping neural activity feature data into a higher-dimensional phase space, effectively revealing the periodicity, chaotic behavior or attractor structure of neural activity, and helping to more deeply understand the complex dynamic process of neural activity during rehabilitation.

[0129] Preferably, the sample entropy calculation is specifically as follows:

[0130] Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively;

[0131] Specifically, the EEG time series feature data is divided according to the time window. For example, a time window of fixed length (such as 100 time points) is selected to divide the time series data into multiple subsequences, each subsequence representing the neural activity within a time window. The EEG frequency domain feature data is divided according to the frequency segment. The power value of each frequency segment can be used as a subsequence. The frequency domain feature data can also be divided according to the time window to extract the frequency features within each time window. The EEG time series division data and the EEG frequency domain division data are obtained respectively, and these subsequences will be used as inputs for subsequent similarity calculations.

[0132] Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively;

[0133] Specifically, the Euclidean distance or Manhattan distance is used to calculate the similarity between each two time series subsequences. The specific process is to calculate the distance of each subsequence in the multidimensional space. The smaller the distance, the higher the similarity. Set the time series similarity threshold and count the number of other subsequences whose distance to a subsequence is less than the time series similarity threshold.

[0134] For the frequency domain partitioned data, the same distance calculation method is used to compare the similarity of frequency subsequences. The distance between each subsequence and other subsequences in the frequency domain space is calculated, and the similarity of their power spectra or phase synchronization is compared. Similarly, the frequency domain similarity threshold is set, and the number of other subsequences whose distance to a certain frequency domain subsequence is less than the frequency domain similarity threshold is counted.

[0135] Based on the above steps, the EEG time series similarity data and the EEG frequency domain similarity data are obtained respectively.

[0136] Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively;

[0137] Specifically, sample entropy is defined as a measure of the complexity of a time series, which measures the similarities between different subsequences in a sequence.

[0138] For each EEG time series subsequence, the number of subsequences with a distance less than r in the similarity calculation is counted, and the similarity ratios of subsequences with lengths of m and m+1 are calculated respectively. The time series sample entropy data is calculated using the sample entropy formula.

[0139] For the EEG frequency domain subsequence, the similarity ratio of the frequency domain subsequences with lengths of m and m+1 is calculated in the same way as the time series sample entropy calculation to obtain the frequency domain sample entropy data. The EEG time series sample entropy data and the EEG frequency domain sample entropy data are obtained respectively, indicating the complexity of the signal under different modalities.

[0140] The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the characteristic sample entropy data.

[0141] Specifically, in order to ensure that the time series sample entropy data and the frequency domain sample entropy data can be compared on the same scale, the sample entropy data is standardized, such as mean normalization or Z-score normalization. The time series sample entropy data and the frequency domain sample entropy data are weighted fused using weighted averaging or multimodal feature fusion technology. The weights can be set based on the importance or empirical parameters of different modalities. The fused feature sample entropy data is obtained, which integrates the complexity of the EEG signal in the time series and frequency domains for in-depth analysis of neural activity.

[0142] In the present invention, by dividing the time series features and frequency domain features of the electroencephalogram into vector sequences, the neural activity is analyzed from the two dimensions of time and frequency, and the changes in neural activity at different levels can be captured. The divided vector sequence can adapt to the complexity of the data more flexibly, especially in different brain regions and states, the feature performance may be different. By dividing the vector sequence, it is helpful to carefully analyze the local and global features of the electroencephalogram signal, and improve the delicacy and accuracy of the analysis. Through similarity calculation, similar patterns in the electroencephalogram time series and frequency domain data can be quantified, which is very important for detecting specific brain states (such as changes in brain activity during rehabilitation). High similarity indicates the stability or consistency of the data, and low similarity may indicate irregular changes in neural activity or a functional reconstruction process. Similarity calculation can help identify trend changes or mutation points in electroencephalogram signals, which is crucial for neurorehabilitation assessment because it can reveal the recovery or change trajectory of brain function during rehabilitation. Sample entropy is an important tool for quantifying the complexity and unpredictability of time series. By calculating the sample entropy of electroencephalogram time series data and frequency domain data, the complexity of brain neural activity can be evaluated. Higher sample entropy values ​​generally reflect the diversity and flexibility of brain activity, while lower sample entropy values ​​reflect stable neural activity patterns. Sample entropy calculation is not limited to time series features, but also extends to frequency domain features. Through this dual-dimensional complexity analysis, it is possible to more comprehensively evaluate the changes in the brain during neurorehabilitation. Frequency domain sample entropy reveals the complexity of neural activity in different frequency bands, and in many cases can supplement the deficiencies of time domain data.

[0143] Preferably, the phase space reconstruction is specifically:

[0144] The average mutual information is calculated based on the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data to obtain the time delay parameter data;

[0145] Specifically, for each pair of time series data, the average mutual information of adjacent time points is calculated to evaluate the dependency of adjacent time points. The average mutual information gradually decreases with the increase of time delay, and the position where the mutual information first drops to the local minimum is selected as the time delay parameter. The time delay parameter data is obtained to provide the necessary time delay information for phase space reconstruction.

[0146] According to the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data, principal component analysis and maximum variance dimension screening are performed to obtain the embedded dimension data;

[0147] Specifically, principal component analysis is performed on the feature sample entropy data, EEG time series and frequency domain feature data to find the direction of the maximum variance in the data. PCA finds the principal component that explains the maximum variance of the data by performing eigenvalue decomposition on the data covariance matrix. The number of principal components that explain more than 90% of the total variance is selected as the embedding dimension.

[0148] According to the time delay parameter data and the embedding dimension data, the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data are phase-space encoded to obtain feature sample entropy encoded data, EEG time series feature encoded data and EEG frequency domain feature encoded data respectively;

[0149] Specifically, the feature sample entropy data, EEG time series and frequency domain feature data are time-delayed and embedded using the time delay parameter and embedding dimension. According to Takens' theorem, the time series can be reconstructed into a multidimensional vector. Each time point is converted into a multidimensional vector to form feature sample entropy coded data, EEG time series feature coded data and EEG frequency domain feature coded data. Feature sample entropy coded data, EEG time series feature coded data and EEG frequency domain feature coded data are obtained respectively to prepare for the subsequent phase space construction.

[0150] Constructing a multimodal joint phase space according to feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data;

[0151] Specifically, the feature sample entropy coded data, the EEG time series feature coded data and the EEG frequency domain feature coded data are combined to construct a multimodal joint phase space. The multimodal feature vector at each time point is composed of coded data of different modalities. The fused multimodal data is mapped in a high-dimensional space to construct a multimodal phase space, which can reflect the dynamic changes of neural activity in different modalities. The multimodal joint phase space data is obtained, which integrates multimodal information and forms a complete phase space of neural activity.

[0152] The multimodal joint phase space data is processed by phase space trajectory to obtain neural activity feature data.

[0153] Specifically, in the multimodal phase space, the phase space trajectory is processed to analyze the trajectory pattern of neural activity, such as periodic trajectory, chaotic trajectory or stable attractor. Relevant dynamic features are extracted from the phase space trajectory, such as the average distance of the trajectory, the density of the trajectory distribution, the radius of the attractor, etc., to characterize the complexity and stability of neural activity. The neural activity feature data are obtained, which are used for further analysis and the construction of the neural rehabilitation model, reflecting the dynamic behavior characteristics of neural activity under multimodality.

[0154] In the present invention, by using average mutual information (AMI) to calculate the time delay parameter, the information of time dependence in the electroencephalogram signal can be captured more accurately. AMI can quantify the dependency of adjacent points in the feature data and automatically determine the optimal time delay parameter, avoiding the subjectivity of manually setting the time delay. Reasonable time delay parameters ensure that the dynamics and complexity of the signal features in the subsequent phase space reconstruction can be fully displayed, helping to reveal the key change points of neural activity in the rehabilitation process. Through principal component analysis (PCA) and screening the maximum variance dimension, the selection of the embedding dimension is more objective and data-driven, which can automatically extract the most representative information, reduce redundant features and noise, and improve the efficiency of data analysis. In the phase space reconstruction process, the embedding dimension is a key parameter. The dimension selection after PCA optimization can ensure that the reconstructed phase space retains the key information in the original data and avoids the computational complexity caused by too high dimensions. Phase space coding can uniformly map sample entropy data, time series feature data and frequency domain feature data into the phase space to form a unified coding representation, which helps to capture the interdependence between the features and effectively build a more complete neural activity model. Phase space encoding helps reproduce the nonlinear dynamic characteristics of brain activity. By embedding feature data into a higher-dimensional space, complex neural activity patterns and trajectories can be revealed, revealing potential nonlinear behaviors in the recovery process of the nervous system. By jointly constructing a multimodal phase space with encoded data of different modalities (sample entropy, time series features, frequency domain features), a comprehensive analysis of multidimensional data can be achieved, providing more comprehensive and multi-angle information for the evaluation of brain function, and helping to understand the interaction between the modalities. The multimodal joint phase space can provide richer feature dimensions and reduce the limitations of single-modal analysis. By combining spatiotemporal and frequency information, this method can more accurately reflect the key dynamic changes in the process of neurorehabilitation. By performing trajectory processing on the multimodal joint phase space, the motion trajectory of neural activity in high-dimensional space can be analyzed. Phase space trajectories can reveal the periodicity, attractor structure or chaotic behavior of neural activity, helping to better understand the changing patterns of brain function during rehabilitation.

[0155] Preferably, the functional connectivity feature extraction is specifically:

[0156] Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data;

[0157] Specifically, standard brain maps (such as AAL and Harvard-Oxford brain area maps) are used to divide functional magnetic resonance imaging (fMRI) data into brain regions. Based on these maps, the brain is divided into multiple functional areas, each of which corresponds to a specific neural function. The functional magnetic resonance imaging data of the patient's neurorehabilitation patients are aligned to a standard space (such as MNI space) to ensure that the patient's brain region division is consistent with the standard brain map. Brain region division data are obtained, which include brain region labels corresponding to each voxel in the functional magnetic resonance imaging.

[0158] Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data;

[0159] Specifically, for each divided brain region, the corresponding BOLD signal time series is extracted from it. The BOLD signal reflects the change of blood oxygen level in the brain region over time. The voxels of each brain region are averaged to obtain the average BOLD signal time series of the brain region, reducing the impact of voxel-level noise on functional connectivity analysis. Time series blood oxygen level data is obtained, which reflects the change of average blood oxygen level in each brain region over time.

[0160] Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data;

[0161] Specifically, the Pearson correlation coefficient was used to calculate the correlation between the time-series blood oxygen levels in different brain regions. The sliding window technique was used to calculate the dynamic correlation, that is, to calculate the time-series correlation between brain regions in different time periods to capture the dynamic changes in the functional connections between brain regions. Time-series blood oxygen correlation data were obtained, which reflected the functional connection strength between different brain regions throughout the entire time period.

[0162] The functional connection matrix is ​​constructed according to the time-series blood oxygen correlation data to obtain the functional connection matrix data;

[0163] Specifically, a functional connectivity matrix is ​​constructed based on the time-series blood oxygen correlation data. Each element of the matrix represents the correlation coefficient between two brain regions, and the matrix dimension is N×N, where N is the number of brain regions. The functional connectivity matrix is ​​standardized to remove noise and individual differences. Fisher transformation can be used to normalize the correlation coefficient to obtain the functional connectivity matrix data, which reflects the functional connection strength between all brain regions.

[0164] Extracting whole-brain functional connection features and local functional connection features from the functional connection matrix data to obtain whole-brain functional connection feature data and local functional connection feature data;

[0165] Specifically, based on the functional connectivity matrix, the features of whole-brain functional connectivity are extracted, such as the average functional connectivity strength of the whole brain, the overall density of the functional connectivity network, etc. These features can describe the functional integration level of the whole brain. The graph theory features of the functional connectivity network, such as global efficiency and small-worldness, are extracted to reflect the topological structure of the whole-brain network.

[0166] Based on the functional connectivity matrix, local functional connectivity features such as local clustering coefficient and local efficiency are extracted, which can reflect the local connectivity pattern of specific brain regions. Modularity analysis is used to divide brain regions into communities, identify functionally tightly connected brain region groups, extract the internal connectivity features of these communities, and obtain local functional connectivity feature data.

[0167] The whole-brain functional connection feature data and the local functional connection feature data are integrated to obtain the neural functional connection feature data.

[0168] Specifically, the whole-brain functional connectivity feature data and local functional connectivity feature data are integrated, and different features are merged into a comprehensive feature vector using weighted average method or feature level fusion technology. The integrated features are standardized to ensure that features from different sources can be compared on the same scale. The neural functional connectivity feature data is obtained, which combines the whole-brain and local functional connectivity information and fully describes the brain functional connectivity pattern of neurorehabilitation patients.

[0169] In the present invention, by dividing the functional magnetic resonance imaging data into brain regions, the different functional areas of the brain can be accurately divided, the functions of different brain regions can be analyzed in detail, and the basis for the subsequent analysis of functional connections can be provided to ensure the spatial accuracy of the data. Extracting the time-series blood oxygen level data (BOLD signal) of each brain region can dynamically reflect the changes in oxygen demand of neuronal activity, capture the activity level of different regions of the brain during task execution or rehabilitation, and help evaluate the functional recovery of brain regions, especially the impact on the brain during rehabilitation training. By performing correlation calculations on the time-series blood oxygen level data of each brain region, the functional connectivity between different brain regions can be revealed, reflecting the intensity of collaborative work or synchronous activity between brain regions, and helping users understand the network structure and changes between brain regions during rehabilitation.

[0170] Temporal correlation calculations can help discover the remodeling process of the brain's functional network. Especially in neurorehabilitation, the increase or decrease of functional connectivity can reflect the recovery of brain function. Changes in brain region connectivity during rehabilitation may be an important signal for the reconstruction of brain function. The functional connectivity matrix is ​​a mathematical representation of the strength of brain region connections, which can provide a full picture of the brain's functional network. Through the visualization of the matrix, the connection patterns between different brain regions can be intuitively displayed, helping to discover the overall change trend of the brain's functional network during rehabilitation.

[0171] The functional connectivity matrix can serve as the basis for analyzing brain functional networks and support subsequent network feature extraction, such as the connectivity of the whole brain network, the network clustering coefficient, and the small-worldness, so as to more comprehensively understand the overall architecture of the brain network and its changes. The whole-brain functional connectivity features can capture the overall network characteristics of the brain, reflect how different brain regions work together during the rehabilitation process, and help evaluate the recovery of the overall brain function, especially in cognition, attention, motor control and other functions involving a wide range of brain regions.

[0172] Local functional connectivity features can focus on the connectivity changes of specific brain regions and provide refined local network analysis. For example, focusing on local connections related to motor function and language function can provide a more accurate basis for evaluation and can detect local functional reconstruction during rehabilitation. By integrating whole-brain functional connectivity features and local functional connectivity features, multi-level analysis of brain functional networks can be achieved. Whole-brain connectivity provides a macro perspective, while local connectivity provides a micro perspective, which helps to more comprehensively understand the functional recovery of the brain.

[0173] Preferably, the multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data, and step S3 is specifically:

[0174] Performing modality classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data;

[0175] Specifically, the multimodal feature data of neurorehabilitation patients are classified by first labeling them according to the modal source of the data. For example, the feature data from EEG is labeled as "EEG modality", the feature data from fMRI is labeled as "fMRI modality", and the features from other sources are labeled as corresponding modalities. Since the features have been labeled with the source modality, these labels can be used directly for modality classification. This classification process can be performed by a simple rule classifier (such as a decision tree classifier) ​​to classify the feature data of different modalities into their respective modality categories. The first multimodal feature classification data is obtained, indicating the modality category (such as EEG, fMRI, etc.) to which each feature belongs.

[0176] The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

[0177] Specifically, in the multimodal feature data, the features are labeled by type. For example, the time domain features in EEG are labeled as "time domain features", and the frequency domain features are labeled as "frequency domain features"; in fMRI data, the functional connection features can be labeled as "connection features", and the graph theory features are labeled as "graph theory features". Use machine learning classification algorithms such as support vector machines (SVM) or random forests to classify feature types according to the statistical properties of the features (such as mean, variance, frequency, etc.). The algorithm classifies the features into corresponding type categories based on the trained classification model. According to the statistical properties of each feature type, dimension selection and dimensionality reduction are performed to optimize classification efficiency and reduce redundant features. The second multimodal feature classification data is obtained, indicating the specific types of different modal features (such as time domain features, frequency domain features, connection features, etc.).

[0178] In the present invention, by performing modal classification on the multimodal feature data of neurorehabilitation patients, it is possible to effectively distinguish the data according to its source (such as electroencephalogram, functional magnetic resonance imaging, etc.), ensure that different types of signals are processed separately, reduce confusion during data fusion, and help to more accurately analyze the contribution of each modality. After modal classification, the data of each modality can be processed, analyzed and modeled independently, so that the model can perform targeted feature selection and optimization between different modalities, thereby improving the efficiency and flexibility of data processing. By classifying multimodal feature data by feature type (such as time domain features, frequency domain features, complexity features, etc.), the organization of the data can be made clearer. Each feature type can represent different physiological signal manifestations, for example, time domain features reflect instantaneous changes, frequency domain features reveal periodic activities, and complexity features reflect nonlinear behavior.

[0179] Preferably, step S4 is specifically:

[0180] Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data;

[0181] Specifically, the preset neurological patient knowledge graph contains medical knowledge related to neurorehabilitation, pathological characteristics, functional connection patterns, rehabilitation treatment processes, etc. The nodes in the knowledge graph represent different entities of the nervous system (such as brain regions, neural networks, rehabilitation methods, etc.), and the edges represent the relationships between these entities (such as functional connections, pathological associations, etc.).

[0182] Fitting methods include: using graph neural networks to embed multimodal feature classification data into knowledge graphs. Through graph convolution operations, each feature is fitted with nodes and edges in the knowledge graph to obtain an associated representation of the feature and the graph. Alternatively, similarity metrics such as cosine similarity or Euclidean distance are used to calculate the similarity between multimodal features and knowledge graph nodes to find the most similar knowledge nodes.

[0183] Through GNN or similarity calculation, the fitting data of multimodal features and knowledge graphs are generated, and the matching degree of each multimodal feature and the corresponding knowledge node is recorded. The multimodal feature knowledge fitting data of neurological patients is obtained, which represents the position and association information of each multimodal feature in the knowledge graph.

[0184] The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

[0185] Specifically,

[0186] The introduction of the knowledge graph in the present invention presents the medical knowledge and expert experience in the field of neurorehabilitation in a systematic and structured manner, becoming a core part of the present invention. Through the rich information stored in the knowledge graph, such as the type of nerve injury, rehabilitation pathway, standardized treatment plan, patient recovery stage, etc., the system can have knowledge-driven capabilities in the data processing process, greatly improving the accuracy of data analysis, especially the processing efficiency of complex rehabilitation cases. The feature data is labeled by fitting the multimodal feature knowledge of neurological patients, so that the labeling process not only depends on the original multimodal data features, but also relies on the medical knowledge and rehabilitation experience provided by the knowledge graph. Through the knowledge fitting process, the labeling process becomes more accurate, and the most appropriate label tag can be automatically matched according to the specific feature status of the patient.

[0187] Preferably, the present application also provides a system for constructing a neurorehabilitation assessment model, which is used to execute the method for constructing a neurorehabilitation assessment model as described above. The system for constructing a neurorehabilitation assessment model includes:

[0188] Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data;

[0189] A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data;

[0190] A modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data;

[0191] The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

[0192] The beneficial effects of the present invention are:

[0193] 1. Data from different modalities provide a multidimensional perspective of neural activity, such as the high temporal resolution of EEG and the high spatial resolution of functional magnetic resonance imaging, which can make up for the limitations of a single modality and provide richer basic information for subsequent neurorehabilitation assessment. Multimodal data can reflect the dynamic changes in the patient's brain function at different time scales, spatial scales, and physiological scales, allowing the evaluation model to not only detect short-term neural responses, but also capture the long-term brain function reconstruction process.

[0194] 2. Time domain features reflect instantaneous changes in signals, frequency domain features reveal the intensity of neural activity in different frequency bands, and complexity features such as sample entropy can quantify the nonlinear behavior of signals, ensuring that the model can fully capture subtle changes in neural activity during neurorehabilitation. By extracting different types of features from multimodal data, the model can obtain rich input information, thereby improving the accuracy and robustness of the evaluation. During the training process, the model can be interpreted based on these different types of features to help analyze the complex dynamics of brain activity and identify key neural changes in rehabilitation. By combining time domain, frequency domain, and complexity features, the amount of feature information for each patient has increased from an average of 200 dimensions to 350 dimensions, and the information gain has increased by about 75%, providing more input data for accurate training of the model.

[0195] 3. Modality classification helps to distinguish data sources of different modalities (such as EEG and fMRI), ensuring the pertinence of feature processing and analysis. Feature type classification further classifies features by their types (such as time domain, frequency domain, complexity, etc.) to facilitate the rational selection and use of features. The classified data is more structured and data processing is more efficient.

[0196] 4. Through fitting calculations, the knowledge graph can help identify the patient's individualized neurological injury and rehabilitation progress. For example, for different injury types or rehabilitation speeds, the system can match the most similar historical rehabilitation cases from the knowledge graph and generate corresponding rehabilitation path recommendations based on this. From the perspective of data processing, the rehabilitation plan is more in line with the patient's actual situation, and can provide differentiated neurological rehabilitation assessment data for different patients, avoiding a "one-size-fits-all" treatment model. Through the intelligent fitting and labeling of the knowledge graph, the prediction accuracy of the model has increased from the original 87% to 96%, an increase of 9 percentage points.

[0197] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0198] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for constructing a neurorehabilitation assessment model, characterized in that: The following steps are involved: Step S1: Acquire multimodal data of neurorehabilitation patients; Step S2: extracting features based on the multimodal data of the neurorehabilitation patients to obtain multimodal feature data of the neurorehabilitation patients; Step S3: classifying the multimodal feature data of the neurorehabilitation patient to obtain multimodal feature classification data; Step S4: using the multimodal feature classification data to label the multimodal feature data of the neurorehabilitation patient, and obtaining the multimodal feature labeling data of the neurorehabilitation patient, so as to assist in the construction of the neurorehabilitation assessment model.

2. The method according to claim 1, characterized in that: Step S1 is specifically as follows: Collect EEG data of neurorehabilitation patients through EEG equipment; Functional magnetic resonance imaging data of neurorehabilitation patients were obtained by acquiring functional magnetic resonance imaging through a high-resolution fMRI scanner; The EEG data and functional magnetic resonance imaging data of neurorehabilitation patients are integrated to obtain multimodal data of neurorehabilitation patients.

3. The method according to claim 2, characterized in that Step S2 is specifically as follows: Extract neural activity features from the EEG data of neurorehabilitation patients to obtain neural activity feature data; Extract functional connectivity features from functional magnetic resonance imaging data of neurorehabilitation patients to obtain neural functional connectivity feature data; The neural activity feature data and the neural functional connection feature data are fused to obtain the multimodal feature data of neurorehabilitation patients.

4. The method according to claim 3, characterized in that The neural activity feature extraction is specifically as follows: Performing artifact removal on the EEG data of neurorehabilitation patients to obtain EEG artifact-removed data; Extracting time domain features and frequency domain features of the EEG artifact removal data to obtain EEG time series feature data and EEG frequency domain feature data; Perform sample entropy calculation on the EEG time series feature data and the EEG frequency domain feature data to obtain feature sample entropy data; Phase space reconstruction is performed based on characteristic sample entropy data, EEG time series characteristic data and EEG frequency domain characteristic data to obtain neural activity characteristic data.

5. The method according to claim 4, characterized in that The sample entropy calculation is as follows: Performing vector sequence division on the EEG time series feature data and the EEG frequency domain feature data to obtain EEG time series division data and EEG frequency domain division data respectively; Performing similarity calculation on the EEG time series division data and the EEG frequency domain division data to obtain EEG time series similarity data and EEG frequency domain similarity data respectively; Sample entropy is calculated based on the EEG time series similarity data and the EEG frequency domain similarity data to obtain EEG time series sample entropy data and EEG frequency domain sample entropy data respectively; The EEG time series sample entropy data and the EEG frequency domain sample entropy data are integrated to obtain the characteristic sample entropy data.

6. The method according to claim 5, characterized in that The phase space reconstruction is as follows: The average mutual information is calculated based on the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data to obtain the time delay parameter data; According to the characteristic sample entropy data, the EEG time series characteristic data and the EEG frequency domain characteristic data, principal component analysis and maximum variance dimension screening are performed to obtain the embedded dimension data; According to the time delay parameter data and the embedding dimension data, the feature sample entropy data, the EEG time series feature data and the EEG frequency domain feature data are phase-space encoded to obtain feature sample entropy encoded data, EEG time series feature encoded data and EEG frequency domain feature encoded data respectively; Constructing a multimodal joint phase space according to feature sample entropy coding data, EEG time series feature coding data, and EEG frequency domain feature coding data to obtain multimodal joint phase space data; The multimodal joint phase space data is processed by phase space trajectory to obtain neural activity feature data.

7. The method according to claim 3, characterized in that The functional connection feature extraction is specifically as follows: Perform brain region division on functional magnetic resonance imaging data of neurorehabilitation patients to obtain brain region division data; Extracting time-series blood oxygen level data from the brain region division data to obtain time-series blood oxygen level data; Performing time series correlation calculation on the time series blood oxygen level data to obtain time series blood oxygen correlation data; The functional connection matrix is ​​constructed according to the time-series blood oxygen correlation data to obtain the functional connection matrix data; Extracting whole-brain functional connection features and local functional connection features from the functional connection matrix data to obtain whole-brain functional connection feature data and local functional connection feature data; The whole-brain functional connection feature data and the local functional connection feature data are integrated to obtain the neural functional connection feature data.

8. The method according to claim 1, characterized in that The multimodal feature classification data includes first multimodal feature classification data and second multimodal feature classification data, and step S3 is specifically as follows: Performing modality classification on the multimodal feature data of the neurorehabilitation patient to obtain first multimodal feature classification data; The multimodal feature data of the neurorehabilitation patients are classified into feature types to obtain second multimodal feature classification data.

9. The method according to claim 1, characterized in that: Step S4 is specifically as follows: Perform fitting calculations based on the multimodal feature classification data and the preset neurological patient knowledge graph data to obtain neurological patient multimodal feature knowledge fitting data; The multimodal feature data of neurological patients are labeled using the multimodal feature knowledge fitting data of neurological patients to obtain the multimodal feature labeled data of neurological rehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

10. A system for constructing a neurorehabilitation assessment model, characterized in that: Used to execute the method for constructing a neurorehabilitation assessment model as claimed in claim 1, the system for constructing a neurorehabilitation assessment model comprises: Neurorehabilitation patient multimodal data acquisition module, used to obtain neurorehabilitation patient multimodal data; A neurorehabilitation patient multimodal feature extraction module is used to extract features based on the neurorehabilitation patient multimodal data to obtain the neurorehabilitation patient multimodal feature data; A modal feature classification module is used to classify the multimodal feature data of neurorehabilitation patients to obtain multimodal feature classification data; The multimodal feature labeling module for neurorehabilitation patients is used to label the multimodal feature data of neurorehabilitation patients using multimodal feature classification data to obtain multimodal feature labeling data of neurorehabilitation patients for auxiliary work in building a neurorehabilitation evaluation model.

Images