System and method for monitoring multi-modal vestibular function assessment and balance compensation

Abstract

The application discloses a kind of multi-modal vestibular function evaluation and balance compensation monitoring system and method, it is related to brain-computer interface technical field.The specific embodiment of the method includes: obtaining the BOLD signal of each brain region obtained by subject performing fMRI examination and the electroencephalogram signal of multiple leads obtained by performing EEG examination;Determine the magnetic resonance activation intensity data of each brain region for optokinetic stimulation task to select vestibular response related region from each brain region, determine the lead corresponding to vestibular response related region as target lead;From the electroencephalogram signal of target lead, extract various electroencephalogram feature data of each vestibular response related region for optokinetic stimulation task, fuse various electroencephalogram feature data with magnetic resonance activation intensity data to generate multi-modal aggregation vector, determine the evaluation result of subject vestibular function central response based on multi-modal aggregation vector.The embodiment can quantify the degree of vestibular function loss and balance compensation.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and in particular to a multimodal vestibular function assessment and balance compensation monitoring system and method. Background Technology

[0002] The vestibular system is an important sensory system in the human body, responsible for maintaining balance and postural control. Vestibular dysfunction can lead to dizziness, balance disorders, and a significantly increased risk of falls, severely impacting patients' quality of life. Epidemiological statistics show that approximately 35% of adults over 40 years of age have experienced symptoms related to vestibular dysfunction. Clinically, conditions such as vestibular neuritis, Meniere's disease, labyrinthine concussion, and post-acoustic neuroma surgery can all cause varying degrees of vestibular dysfunction.

[0003] Following vestibular dysfunction, balance is primarily maintained through vestibular compensation (VC), a mechanism employed by the central nervous system to adapt and compensate for damage. VC has two phases: static and dynamic. Static symptoms appear immediately after unilateral peripheral vestibular injury, with the resolution time varying by species. In injured animals, complete compensation is typically achieved within about two weeks, while in patients, it takes longer. Dynamic compensation, on the other hand, is a lengthy and generally incomplete process. It involves the coordinated work of various brain regions, including the reorganization of brainstem and cerebellar pathways, and the readjustment of eye movement and postural control programs to restore balance and motor control. Compensation for static symptoms relies on the restoration of firing balance in both vestibular nuclei neurons. Compensation for dynamic symptoms primarily relies on substitution, adaptation, and habituation.

[0004] Currently, routine clinical methods for assessing vestibular function mainly include: video head impulse test (vHIT) to evaluate high-frequency vestibular-ocular reflexes, caloric test to evaluate low-frequency vestibular function, vestibular evoked myogenic potential (VEMP) to evaluate otolith organ function, and rotational test, etc. While these methods provide a basis for vestibular function assessment, they only offer qualitative results and cannot quantify or accurately assess vestibular function loss and the degree of balance compensation. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a multimodal vestibular function assessment and balance compensation monitoring system and method, which can quantify the degree of vestibular function loss and balance compensation.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for assessing multimodal vestibular function and monitoring balance compensation is provided.

[0007] The multimodal vestibular function assessment and balance compensation monitoring method of this invention includes: acquiring oxygenation level-dependent BOLD signals for each brain region obtained from a functional magnetic resonance imaging (fMRI) examination of the brain, and electroencephalogram (EEG) signals from multiple leads obtained from an EEG examination; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task; acquiring the task timing function of the visual-motor stimulation task, and based on the task timing function and the BOLD signals for each brain region... The system determines the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task, selects multiple vestibular response-related regions from each brain region based on the magnetic resonance activation intensity data, and determines the leads corresponding to the vestibular response-related regions as target leads; and extracts multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task from the EEG signals of the target leads, fuses the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, and determines the assessment result of the subject's vestibular function based on the multimodal aggregation vector.

[0008] Optionally, the optomotor stimulation task is viewed by the subject and includes multiple subtasks that are repeatedly performed. Each subtask includes a resting segment, a clockwise rotation segment, a resting segment, and a counterclockwise rotation segment of equal duration, performed sequentially. In the clockwise rotation segment, a black and white striped disc rotates clockwise at a preset speed. In the counterclockwise rotation segment, the disc rotates counterclockwise at the preset speed. The fMRI examination and the EEG examination are performed at the same start time as the corresponding optomotor stimulation task.

[0009] Optionally, determining the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task based on the task timing function and the BOLD signal of each brain region includes: determining the hemodynamic response function (HRF) of each brain region of the subject; performing convolution between the HRF and the task timing function to obtain the baseline signal change curve of each brain region; and subtracting the BOLD signal of each brain region from the baseline signal change curve of the corresponding brain region to obtain the magnetic resonance activation intensity data of each brain region.

[0010] Optionally, the step of selecting multiple vestibular response-related regions from various brain regions based on the magnetic resonance activation intensity data includes: determining a preset number of brain regions with the highest magnetic resonance activation intensity as preliminary selection regions based on the magnetic resonance activation intensity data; and determining the preliminary selection regions located in a preset set of vestibular-related regions as the vestibular response-related regions.

[0011] Optionally, the EEG feature data of any vestibular response-related region includes: frequency domain feature data; and the frequency domain features include: the power spectral density of each target lead corresponding to the vestibular response-related region at a preset frequency point, the relative power of each target lead corresponding to the vestibular response-related region in a preset frequency band, and the response index of each target lead corresponding to the vestibular response-related region in a preset frequency band, wherein the response index characterizes the relative power difference between the clockwise rotation segment and the counterclockwise rotation segment relative to the adjacent resting segment.

[0012] Optionally, the EEG feature data further includes: time-frequency feature data; and the time-frequency features include: short-time Fourier transform features of the difference signals of each target lead corresponding to the vestibular response-related region in a preset frequency band, and event-related spectrum perturbation features of the difference signals, wherein the difference signals are the differences between the EEG signals of the clockwise and counterclockwise rotation segments relative to the adjacent resting segments.

[0013] Optionally, the EEG feature data further includes: brain network connectivity feature data; and the brain network connectivity feature includes: phase lock value features of any two vestibular response-related regions in a preset frequency band, wherein the phase lock value features characterize the signal synchronization degree of the target leads of the any two vestibular response-related regions in the preset frequency band.

[0014] Optionally, fusing the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregated vector includes: inputting the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data into a pre-trained data fusion model to obtain the multimodal aggregated vector output by the data fusion model; determining the assessment result of the subject's vestibular function based on the multimodal aggregated vector includes: performing linear operations on each component in the multimodal aggregated vector to obtain the subject's vestibular compensation efficiency index; comparing the vestibular compensation efficiency index with at least two preset thresholds to determine the assessment result of the subject's vestibular function from at least three preset assessment results; and the method further includes: generating vestibular function assessment results of the subject at multiple assessment times to monitor the subject's balance compensation.

[0015] To achieve the above objectives, according to another aspect of the present invention, a multimodal vestibular function assessment and balance compensation monitoring system is provided.

[0016] The multimodal vestibular function assessment and balance compensation monitoring system of this invention includes: an acquisition unit for acquiring oxygenation level-dependent BOLD signals of various brain regions obtained from functional magnetic resonance imaging (fMRI) of the brain, and electroencephalogram (EEG) signals of multiple leads obtained from an electroencephalogram (EEG) of the brain; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task; and a lead localization unit for acquiring the task timing function of the visual-motor stimulation task, and based on the task timing function and the BOLD signals of various brain regions... The OLD signal determines the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task. Based on the magnetic resonance activation intensity data, multiple vestibular response-related regions are selected from each brain region, and the leads corresponding to the vestibular response-related regions are determined as target leads. An evaluation unit is used to extract multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task from the EEG signal of the target leads, fuse the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, and determine the evaluation result of the subject's vestibular function based on the multimodal aggregation vector.

[0017] To achieve the above objectives, according to another aspect of the present invention, an electroencephalogram (EEG) control system is provided.

[0018] The EEG control system of this invention includes: a data acquisition unit, configured to acquire oxygenation level-dependent BOLD signals of various brain regions obtained from functional magnetic resonance imaging (fMRI) of the brain, and EEG signals from multiple leads obtained from electroencephalography (EEG); wherein the subject performs the fMRI and the EEG simultaneously with a preset visual-motor stimulation task; and a lead localization unit, configured to acquire the task timing function of the visual-motor stimulation task, and determine the dependence of each brain region on the visual-motor stimulation based on the task timing function and the BOLD signals of each brain region. The system includes: magnetic resonance activation intensity data for the task; selection of multiple significantly activated regions from various brain regions based on the magnetic resonance activation intensity data; determination of the leads corresponding to the significantly activated regions as target leads; and an execution unit for extracting multiple EEG feature data of each significantly activated region for the visual-motor stimulation task from the EEG signals of the target leads; fusing the multiple EEG feature data of each significantly activated region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector; performing linear operations on each component of the multimodal aggregation vector to obtain the subject's operational intention; and sending the obtained operational intention to a pre-connected controlled device.

[0019] To achieve the above objectives, according to another aspect of the present invention, an electronic device is provided.

[0020] An electronic device according to the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multimodal vestibular function assessment and balance compensation monitoring method provided by the present invention.

[0021] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0022] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal vestibular function assessment and balance compensation monitoring method provided by the present invention.

[0023] To achieve the above objectives, according to another aspect of the present invention, a computer program product is provided.

[0024] One computer program product of the present invention includes a computer program that, when executed by a processor, implements the multimodal vestibular function assessment and balance compensation monitoring method provided by the present invention.

[0025] According to the technical solution of the present invention, the embodiments described above have the following advantages or beneficial effects: The computer device first acquires the BOLD signals of various brain regions obtained from a subject's fMRI scan and the EEG signals of multiple leads obtained from an EEG scan. The subject performs the fMRI scan and the EEG scan simultaneously with a preset visuomotor stimulation task. Next, the computer device acquires the task timing function of the visuomotor stimulation task. Based on the task timing function and the BOLD signals of each brain region, it determines the magnetic resonance activation intensity data of each brain region in response to the visuomotor stimulation task. Based on the magnetic resonance activation intensity data, it selects multiple vestibular response-related regions from each brain region and identifies the leads corresponding to these vestibular response-related regions as target leads. Subsequently, the computer device extracts multiple EEG feature data of each vestibular response-related region in response to the visuomotor stimulation task from the EEG signals of the target leads. It then fuses these multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector. Based on this multimodal aggregation vector, it determines the assessment result of the subject's vestibular function. Through the above steps, when subjects simultaneously view the same optomotor stimulation task during fMRI and EEG examinations, strong consistency and easily acquired activation data can be generated on both the fMRI and EEG sides. This facilitates the localization of vestibular response-related areas and the extraction of relevant data, and promotes the fusion of fMRI and EEG feature data, enabling accurate assessment and quantification of vestibular function and balance compensation. Furthermore, embodiments of the present invention can deeply combine high spatial resolution fMRI and high temporal resolution EEG examinations based on the same optomotor stimulation task, forming a complementary spatiotemporal fusion framework to comprehensively characterize changes in vestibular function and balance compensation in subjects. The above methods of the present invention can be applied in EEG control scenarios, fusing fMRI and EEG based on the same optomotor stimulation task to collect brain signals from subjects to accurately infer their operational intentions, thereby controlling controlled devices.

[0026] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0027] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the main steps of the multimodal vestibular function assessment and balance compensation monitoring method in this embodiment of the invention; Figure 2 This is a schematic diagram of the disk in the visual-motor stimulation task according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the components of the monitoring system for multimodal vestibular function assessment and balance compensation in an embodiment of the present invention; Figure 4 This is an exemplary system architecture diagram that can be applied thereto according to embodiments of the present invention; Figure 5 This is a schematic diagram of the electronic device structure used to implement the monitoring method for multimodal vestibular function assessment and balance compensation in the embodiments of the present invention. Detailed Implementation

[0028] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] The vestibular system is an important sensory system in the human body, responsible for maintaining balance and postural control. Vestibular dysfunction can lead to dizziness, balance disorders, and a significantly increased risk of falls, severely impacting patients' quality of life. Epidemiological statistics show that approximately 35% of adults over 40 years of age have experienced symptoms related to vestibular dysfunction. Clinically, conditions such as vestibular neuritis, Meniere's disease, labyrinthine concussion, and post-acoustic neuroma surgery can all cause varying degrees of vestibular dysfunction.

[0030] Following vestibular dysfunction, balance is primarily maintained through vestibular compensation (VC), a mechanism employed by the central nervous system to adapt and compensate for damage. VC has two phases: static and dynamic. Static symptoms appear immediately after unilateral peripheral vestibular injury, with the resolution time varying by species. In injured animals, complete compensation is typically achieved within about two weeks, while in patients, it takes longer. Dynamic compensation, on the other hand, is a lengthy and generally incomplete process. It involves the coordinated work of various brain regions, including the reorganization of brainstem and cerebellar pathways, and the readjustment of eye movement and postural control programs to restore balance and motor control. Compensation for static symptoms relies on the restoration of firing balance in both vestibular nuclei neurons. Compensation for dynamic symptoms primarily relies on substitution, adaptation, and habituation.

[0031] Currently, routine clinical methods for assessing vestibular function mainly include: video head impulse test (vHIT) to evaluate high-frequency vestibular-ocular reflexes, caloric test to evaluate low-frequency vestibular function, vestibular evoked myogenic potential (VEMP) to evaluate otolith organ function, and rotational test, etc. These methods have provided important evidence for the diagnosis of vestibular disorders over the past few decades; however, existing vestibular function assessment techniques have the following fundamental limitations: (1) Lack of standardized and quantitative assessment: Currently, vestibular examination lacks standardized quantitative assessment methods. Existing assessments mainly rely on semi-quantitative parameters, such as vHIT gain value and unilateral attenuation index (UW) of the caloric test. These indicators reflect the residual function of peripheral input, making it difficult to assess the severity of vestibular function loss and the degree of compensation, and they are not a direct measure of central compensatory efficiency. Clinically, there is an urgent need for a quantitative index that can comprehensively assess the residual level of peripheral vestibular input, central response speed, and cortical network reorganization efficiency to guide the development of individualized rehabilitation strategies and efficacy evaluation.

[0032] (2) The results of the examination are mostly subjective and the applicable population is limited: the interpretation of the results depends on the operator's experience; the cold and heat test may induce severe vertigo and autonomic nervous system response, which some patients cannot tolerate. vHIT and VEMP are difficult to use in infants, young children, patients with cognitive impairment and during acute vertigo attacks, and the results depend on the operator's experience and have limited repeatability.

[0033] (3) Inability to assess central compensatory status: Following peripheral vestibular injury, the brain can compensate through sensory substitution (e.g., enhanced vision, proprioceptive dependence) and central remodeling (e.g., transsynaptic reorganization of vestibular nuclei). In clinical practice, two patients with similar levels of vestibular dysfunction may have drastically different clinical symptom severity and recovery outcomes—the root cause being individual differences in central compensatory capacity. Current examinations primarily assess the peripheral vestibular system and cannot assess central compensatory status, which is the most critical limitation—current examinations are all static assessments and cannot answer the following key clinical questions: • After functional decline on the affected side, what is the degree of compensation on the contralateral side or in the central nervous system? • Is the patient in a state of "well-compensated" or "decompensated"? • Is compensation improving after rehabilitation therapy? • Has vision or proprioception replaced impaired vestibular function? (4) Lack of temporal and spatial integration: Vestibular processing is a highly dynamic process involving multi-level integration from peripheral receptors (millisecond-level electrical signals) to cortical networks (second-level blood oxygenation response). EEG has millisecond-level temporal resolution and can capture the dynamic changes in vestibular-evoked neural electrical activity, but its spatial localization accuracy is low and is affected by the volume conductor effect, making it difficult to accurately locate the source cortical region. fMRI has millimeter-level spatial resolution and can accurately depict the activation atlas of the vestibular cortical network (including the PIVC of the parietal insula, the posterior insula, the medial vestibular nucleus, etc.), but its temporal response is constrained by blood oxygenation dynamics, with a delay of 2-6 seconds, and cannot reflect the time sequence of real neural discharge. In current clinical practice, there is no stimulation paradigm and experimental conditions suitable for EEG and fMRI to assess vestibular function and balance compensation. Currently, they are mostly in the research exploration stage and are usually used as independent methods, lacking an effective spatiotemporal fusion framework for combining EEG and fMRI analysis.

[0034] In summary, the existing technology has not yet provided a multimodal fusion method that can simultaneously assess peripheral vestibular injury, identify central compensatory pathways, and quantify compensatory efficiency.

[0035] To address the above issues, this invention provides a multimodal vestibular function assessment and balance compensation monitoring system and method, achieving a "spatiotemporal binding" assessment of vestibular compensation. It uses high spatial resolution fMRI to locate vestibular response-related regions and high temporal resolution EEG to extract EEG feature data from these regions, achieving precise cortical-level localization. This spatial-temporal mutual constraint mechanism solves the technical challenge of a single modality simultaneously satisfying both localization accuracy and temporal resolution. This invention constructs a quantifiable mathematical model of compensation efficiency. By calculating the Vestibular Compensation Efficiency Index (VCRI), which reflects fMRI activation intensity, EEG frequency domain characteristics, EEG temporal frequency characteristics, and brain network connectivity characteristics, and its relationship with a threshold, it automatically outputs a classification of the degree of compensation. This quantitative index compensates for the shortcomings of existing semi-quantitative assessment methods, providing objective and repeatable numerical evidence for clinical decision-making.

[0036] It should be noted that, unless otherwise specified, the embodiments of the present invention and the technical features thereof can be combined with each other.

[0037] Figure 1 This is a schematic diagram of the main steps of the multimodal vestibular function assessment and balance compensation monitoring method in this embodiment of the invention.

[0038] like Figure 1 As shown, the multimodal vestibular function assessment and balance compensation monitoring method of this invention can be executed by computer equipment (such as a server, personal computer, mobile smart terminal, etc.), and the specific execution steps are as follows: Step S101: The computer device acquires the blood oxygenation level-dependent BOLD signal of each brain region obtained from the subject's functional magnetic resonance imaging (fMRI) examination, and the electroencephalogram (EEG) signals of multiple leads obtained from the subject's electroencephalogram (EEG) examination; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task.

[0039] In this embodiment of the invention, the visuomotor stimulation task refers to a dynamic visual stimulation task. This visuomotor stimulation task is viewed by the subject and includes multiple sub-tasks that are repeatedly performed. Each sub-task includes equal-duration segments executed sequentially: a resting segment, a clockwise rotation segment, a resting segment, and a counter-clockwise rotation segment. Please refer to [link to relevant documentation]. Figure 2 In the clockwise rotation segment, the black and white striped disc rotates clockwise at a preset speed; in the counterclockwise rotation segment, the disc rotates counterclockwise at the preset speed; and the fMRI examination and the corresponding optomotor stimulation task start at the same time.

[0040] For example, the visual-motor stimulation task is as follows: a black and white striped disc rotates clockwise / counterclockwise at a constant speed (45° / s); a cycle is formed by "rest for 20.2 seconds - visual stimulation of the black and white striped disc rotating clockwise for 20.2 seconds - rest for 20.2 seconds - visual stimulation of the black and white striped disc rotating counterclockwise for 20.2 seconds", and this cycle is repeated 6 times.

[0041] In practical applications, subjects can observe a shielded projection screen located behind the magnet frame through a dedicated non-magnetic mirror mounted on the MRI head coil. Visual stimuli are controlled by a computer outside the shielded room and projected in real-time via fiber optic transmission or a high-frequency shielded projection system. This system is connected to the MRI scan sequence via a TTL pulse synchronization trigger, ensuring precise synchronization between the start time of the task paradigm and the time series of the MRI imaging.

[0042] Preferably, after acquiring the BOLD signal, the computer device can perform known preprocessing operations such as format conversion, temporal correction, head motion correction, spatial normalization, and spatial smoothing to obtain the required data. After acquiring the EEG signal, the computer device can perform known preprocessing operations such as filtering, bad conduction removal, independent component analysis, and block segmentation. The brain regions mentioned above can be regions divided according to any applicable granularity and rules.

[0043] Step S102: The computer device acquires the task timing function of the visual-motor stimulation task, determines the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task based on the task timing function and the BOLD signal of each brain region, selects multiple vestibular response-related regions from each brain region according to the magnetic resonance activation intensity data, and determines the leads corresponding to the vestibular response-related regions as target leads.

[0044] The above task timing function is used to characterize the corresponding time of the resting segment, clockwise rotation segment, and counterclockwise rotation segment. In specific applications, the steps for the computer device to determine the magnetic resonance activation intensity data are as follows: The computer device first determines the hemodynamic response function (HRF) of each brain region of the subject, and performs convolution on the HRF and the task timing function to obtain the baseline signal change curve of each brain region; the computer device subtracts the BOLD signal of each brain region from the baseline signal change curve of the corresponding brain region to obtain the magnetic resonance activation intensity data of each brain region.

[0045] In this embodiment of the invention, the computer device selects vestibular response-related regions according to the following steps. First, based on the magnetic resonance activation intensity data, the computer device identifies a predetermined number of brain regions with the highest magnetic resonance activation intensity as initial selection regions. The magnetic resonance activation intensity can be the average intensity within the region calculated based on the magnetic resonance activation intensity data. Then, the computer device determines the initial selection regions located within a predetermined set of vestibular-related regions as the vestibular response-related regions. Exemplarily, the set of vestibular-related regions includes: the parietal and temporoparietal junction, the frontal lobe, the prefrontal lobe, and the central region.

[0046] Step S103: Extract multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task from the EEG signals of the target lead, fuse the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, and determine the assessment result of the subject's vestibular function based on the multimodal aggregation vector.

[0047] Optionally, the EEG characteristic data of any vestibular response-related region is obtained by frequency domain analysis, which includes: frequency domain characteristic data; the frequency domain characteristics include: the power spectral density of each target lead corresponding to the vestibular response-related region at a preset frequency point, the relative power of each target lead corresponding to the vestibular response-related region in a preset frequency band, and the response index of each target lead corresponding to the vestibular response-related region in a preset frequency band, wherein the response index characterizes the relative power difference between clockwise and counterclockwise rotation segments relative to adjacent resting segments. Adjacent resting segments refer to resting segments that are adjacent to and earlier than either the clockwise or counterclockwise rotation segments.

[0048] Specifically, the frequency domain analysis first uses power spectral density (PSD) to estimate the energy distribution of the EEG signal at different frequencies. The role of PSD is to quantify the steady-state brain oscillations in the 20.2 s rotating block and assess whether the overall energy of α and θ is enhanced or weakened by rotation relative to resting state. Considering the long duration of the block design, the power spectrum is estimated using the Welch method to improve the stability and robustness of the spectral estimation.

[0049] Based on the PSD results, the relative power of each frequency band and the difference between rotation and resting state were further calculated. For the alpha band, a decrease in relative power during rotation usually indicates enhanced alpha desynchronization, reflecting the deinhibition of cortical inhibition and enhanced integration processing under visual-motor input; for the theta band, an increase in relative power during rotation indicates enhanced brain activity related to self-motor perception, conflict processing, and cognitive control. Therefore, the core feature of frequency domain analysis is the alpha / theta response index under left-handed (counterclockwise) and right-handed (clockwise) rotation conditions in the ROI.

[0050] Where Pxx(f) represents the power spectral density at frequency f. Let w[n] represent the EEG signal at the nth sampling point within the k-th window function segment, where k represents the segment number, K represents the total number of segments participating in the averaging, n represents the sampling point number, N represents the number of sampling points in each segment, w[n] represents the window function, U represents the window function energy normalization factor, Fs represents the sampling frequency, j represents the imaginary unit, b represents the preset frequency band, and f l and f h RP represents the lower and upper frequency limits of this frequency band, respectively. b RP represents the relative power of frequency band b. b The denominator of the formula corresponds to the total power within the preset total analysis frequency band; and RRI represents the relative power of the rotating segment and the adjacent resting segment within frequency band b, respectively. b This indicates the response index of rotation relative to adjacent resting periods. When RRI... α When <0, it indicates that the α power is lower than the resting power under rotating conditions, i.e., α desynchronization enhancement; when RRI θ A value greater than 0 indicates an increase in θ power under rotational conditions. These parameters constitute the primary output of frequency domain analysis.

[0051] Optionally, the EEG feature data further includes: time-frequency feature data; and the time-frequency features include: short-time Fourier transform features of the difference signals of each target lead corresponding to the vestibular response-related region in a preset frequency band, and event-related spectrum perturbation features of the difference signals, wherein the difference signals are the differences between the EEG signals of the clockwise and counterclockwise rotation segments relative to the adjacent resting segments.

[0052] Specifically, frequency domain analysis can only provide the average spectral distribution over the entire analysis period, failing to reveal the evolution of brain activity over time. Given the long duration of visual rotation stimuli and the potential for significant differences in brain responses at the start, adaptation, and end of rotation, short-time Fourier transform (STFT) was further employed for time-frequency analysis. This method characterizes the signal simultaneously in both time and frequency dimensions by repeatedly calculating the spectrum within a local time window.

[0053] In time-frequency analysis, the gray-screen resting segment adjacent to each rotation segment is used as a local baseline. The dynamic changes of the α and θ frequency bands in the early, middle, and late stages of rotation are examined in detail. If α decreases rapidly at the beginning of rotation, it usually indicates a rapid desynchronization response caused by visual-motor stimuli. If θ gradually increases during the continuous rotation phase, it is more likely to reflect the accumulation of self-motor perception, conflict processing, or persistent cognitive load. Based on the STFT results, the event-related spectral perturbation (ERSP) can be further calculated to represent the dynamic power increase or decrease relative to the baseline.

[0054] Among them, STFT x (τ,f) represents the local spectral representation of signal x(t) at time τ and frequency f; x(t) represents the target lead signal or the difference signal between the rotating segment and the adjacent resting segment to be analyzed; t represents the continuous time variable; τ represents the center time of the sliding time window; f represents the frequency; h(t-τ) represents the window function shifted around τ; j represents the imaginary unit; P(t,f) represents the power at time t and frequency f obtained by time-frequency transformation. base (f) represents the average power at frequency f during the baseline resting period; ERSP(t,f) represents the logarithmic change in power relative to the baseline. When ERSP(t,f)>0, it indicates that the power at that moment and frequency is higher than the baseline; when ERSP(t,f)<0, it indicates that the power is lower than the baseline. The core features extracted at the time-frequency level based on the above method include: the average value, peak value and occurrence time of α-ERSP and θ-ERSP in the early, middle and late stages of rotation, as well as the time-frequency trajectory differences under left-handed and right-handed conditions.

[0055] As a preferred embodiment, the EEG feature data further includes: brain network connectivity feature data; the brain network connectivity features include: phase lock value features of any two vestibular response-related regions in a preset frequency band, the phase lock value features characterizing the signal synchronization degree of the target leads of the any two vestibular response-related regions in the preset frequency band.

[0056] Specifically, in addition to local power variations, embodiments of the present invention also employ phase locking value (PLV) analysis to analyze functional connectivity between different brain regions. PLV reflects the degree of synchronization between brain regions within a certain frequency band by characterizing the consistency of the instantaneous phase difference between two EEG signals. For vestibular-related processing, analyzing power variations in a single brain region is insufficient; it is also necessary to examine whether there is synchronous pattern reorganization between the visual cortex, parietal integration area, and frontal cortex control area.

[0057] In practice, PLV calculations are based on channel cluster time series obtained from MRI ROI mapping, focusing on analyzing synchronization changes between different regional units in the α and θ bands. Significant changes in the α band PLV matrix typically indicate a reconstruction of the synchronization pattern in the visual-spatial integration network; enhanced brain network connectivity in the θ band may reflect higher levels of conflict monitoring and network collaboration under rotational stimulation. Therefore, the core features ultimately derived from functional connectivity analysis include differences in connectivity strength between vestibular-related brain regions in the α / θ bands.

[0058] in, and represents the instantaneous phase of electrode m and electrode n at time t, respectively; m and n represent the two different electrode numbers; t represents the time sampling point or phase sequence index; T represents the total number of time points involved in the calculation; i represents the imaginary unit; || represents taking the complex modulus. The value ranges from 0 to 1. The closer the value is to 1, the more stable the phase relationship between the two regions in this frequency band and the higher the degree of synchronization. The closer the value is to 0, the more unstable the phase relationship between the two regions and the lower the degree of synchronization.

[0059] Preferably, the computer device generates a multimodal aggregated vector according to the above steps: inputting the various EEG feature data of each vestibular response-related region and the magnetic resonance activation intensity data into a pre-trained data fusion model to obtain the multimodal aggregated vector output by the data fusion model. The above data fusion model can be trained using any known and applicable machine learning algorithm and a known model training method.

[0060] The computer device determines the vestibular function assessment result of the subject according to the following steps: performing linear operations on each component of the multimodal aggregated vector to obtain the vestibular compensation efficiency index (VCRI) of the subject; comparing the VRI with at least two preset thresholds to determine the subject's vestibular function assessment result from at least three preset assessment results. For example, a VCRI ≥ 1.2 indicates good compensation, and the current approach can be maintained; a VCRI between 0.6 and 1.2 indicates partial compensation, representing a critical window for rehabilitation training; a VCRI < 0.6 indicates decompensation, requiring adjustment of the treatment strategy.

[0061] In addition, the computer device can monitor the balance compensation by generating vestibular function assessment results of the subject at multiple assessment times, thereby monitoring the subject's balance compensation over a certain period of time.

[0062] It should be noted that the technical solutions of this invention, including the collection, updating, analysis, processing, use, transmission, and storage of user personal information, all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0063] For the foregoing method embodiments, they are described as a series of actions for ease of description. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, and some steps may actually be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential for implementing the present invention.

[0064] To facilitate better implementation of the above-described solutions of the embodiments of the present invention, a related system for implementing the above-described solutions is also provided below.

[0065] Please see Figure 3As shown, the multimodal vestibular function assessment and balance compensation monitoring system provided in this embodiment of the invention may include: an acquisition unit 301, used to acquire the blood oxygen level dependent BOLD signal of each brain region obtained from a functional magnetic resonance imaging (fMRI) examination of the brain, and the electroencephalogram (EEG) signals of multiple leads obtained from an EEG examination; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task; a lead localization unit 302, used to acquire the task timing function of the visual-motor stimulation task, and based on the task timing function and the signals of each brain region... The BOLD signal of the brain region determines the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task. Based on the magnetic resonance activation intensity data, multiple vestibular response-related regions are selected from each brain region, and the leads corresponding to the vestibular response-related regions are determined as target leads. The evaluation unit 303 is used to extract multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task from the EEG signal of the target leads, fuse the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, and determine the evaluation result of the subject's vestibular function based on the multimodal aggregation vector.

[0066] In this embodiment of the invention, the optomotor stimulation task is viewed by the subject. The optomotor stimulation task includes multiple sub-tasks that are repeatedly executed. Each sub-task includes a resting segment, a clockwise rotation segment, a resting segment, and a counterclockwise rotation segment of equal duration, executed sequentially. In the clockwise rotation segment, a black and white striped disc rotates clockwise at a preset speed. In the counterclockwise rotation segment, the disc rotates counterclockwise at the preset speed. The fMRI examination and the EEG examination start at the same time as the corresponding optomotor stimulation task.

[0067] In this embodiment of the invention, the lead localization unit 302 is further configured to: determine the hemodynamic response function (HRF) of each brain region of the subject; perform convolution on the HRF and the task timing function to obtain the basic signal change curve of each brain region; and subtract the BOLD signal of each brain region from the basic signal change curve of the corresponding brain region to obtain the magnetic resonance activation intensity data of each brain region.

[0068] In this embodiment of the invention, the lead localization unit 302 is further configured to: determine a preset number of brain regions with the highest magnetic resonance activation intensity as preliminary selection regions based on the magnetic resonance activation intensity data; and determine the preliminary selection regions located in a preset set of vestibular associated regions as the vestibular response-related regions.

[0069] In this embodiment of the invention, the EEG feature data of any vestibular response-related region includes: frequency domain feature data; and the frequency domain features include: the power spectral density of each target lead corresponding to the vestibular response-related region at a preset frequency point, the relative power of each target lead corresponding to the vestibular response-related region in a preset frequency band, and the response index of each target lead corresponding to the vestibular response-related region in a preset frequency band, wherein the response index characterizes the relative power difference between the clockwise rotation segment and the counterclockwise rotation segment relative to the adjacent resting segment.

[0070] In this embodiment of the invention, the EEG feature data further includes: time-frequency feature data; and the time-frequency features include: short-time Fourier transform features of the difference signals of each target lead corresponding to the vestibular response-related region in a preset frequency band, and event-related spectrum perturbation features of the difference signals, wherein the difference signals are the differences between the EEG signals of the clockwise and counterclockwise rotation segments relative to the adjacent resting segments.

[0071] In this embodiment of the invention, the EEG feature data further includes: brain network connectivity feature data; and the brain network connectivity feature includes: phase lock value features of any two vestibular response-related regions in a preset frequency band, wherein the phase lock value features characterize the signal synchronization degree of the target leads of the any two vestibular response-related regions in the preset frequency band.

[0072] In this embodiment of the invention, the evaluation unit 103 is further configured to: input the various EEG feature data of each vestibular response-related region and the magnetic resonance activation intensity data into a pre-trained data fusion model to obtain the multimodal aggregation vector output by the data fusion model; perform linear operations on each component of the multimodal aggregation vector to obtain the vestibular compensation efficiency index of the subject; compare the vestibular compensation efficiency index with at least two preset thresholds to determine the vestibular function evaluation result of the subject from at least three preset evaluation results; and generate the vestibular function evaluation result of the subject at multiple evaluation times to monitor the subject's balance compensation status.

[0073] According to the technical solution of the present invention, the data provided by fMRI is used as the spatial prior for EEG analysis. First, significantly activated brain regions (vestibular response-related regions) are extracted based on the fMRI data corresponding to the same optomotor stimulation task. Then, the key analysis electrodes (electrodes in the target leads) are determined on the EEG side according to the mapping relationship, thereby limiting the EEG analysis to the target network related to the vestibule. In this way, the high spatial resolution of fMRI is used to improve the specificity and interpretability of EEG feature extraction.

[0074] This invention also provides an EEG control system, comprising: a data acquisition unit, configured to acquire oxygenation level-dependent BOLD signals of various brain regions obtained from functional magnetic resonance imaging (fMRI) of the brain, and EEG signals from multiple leads obtained from electroencephalography (EEG); wherein the subject performs the fMRI and the EEG simultaneously with a preset visual-motor stimulation task; and a lead localization unit, configured to acquire a task timing function of the visual-motor stimulation task, and determine the relationship between each brain region and the visual-motor stimulation task based on the task timing function and the BOLD signals of each brain region. The system includes: magnetic resonance activation intensity data for a visomotor stimulation task; selection of multiple significantly activated regions from various brain regions based on the magnetic resonance activation intensity data; determination of the leads corresponding to the significantly activated regions as target leads; and an execution unit for extracting multiple EEG feature data of each significantly activated region for the visomotor stimulation task from the EEG signals of the target leads; fusing the multiple EEG feature data of each significantly activated region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector; performing linear operations on each component of the multimodal aggregation vector to obtain the subject's operational intention; and sending the obtained operational intention to a pre-connected controlled device.

[0075] In this embodiment of the invention, the optomotor stimulation task is viewed by the subject. The optomotor stimulation task includes multiple sub-tasks that are repeatedly executed. Each sub-task includes a resting segment, a clockwise rotation segment, a resting segment, and a counterclockwise rotation segment of equal duration, executed sequentially. In the clockwise rotation segment, a black and white striped disc rotates clockwise at a preset speed. In the counterclockwise rotation segment, the disc rotates counterclockwise at the preset speed. The fMRI examination and the EEG examination start at the same time as the corresponding optomotor stimulation task.

[0076] In this embodiment of the invention, the lead localization unit is further configured to: determine the hemodynamic response function (HRF) of each brain region of the subject; perform convolution on the HRF and the task timing function to obtain the basic signal change curve of each brain region; and subtract the BOLD signal of each brain region from the basic signal change curve of the corresponding brain region to obtain the magnetic resonance activation intensity data of each brain region.

[0077] In this embodiment of the invention, the lead localization unit is further configured to: determine a preset number of brain regions with the highest magnetic resonance activation intensity as significantly activated regions based on the magnetic resonance activation intensity data.

[0078] In this embodiment of the invention, the EEG feature data of any significantly activated region includes: frequency domain feature data; and the frequency domain features include: the power spectral density of each target lead corresponding to the significantly activated region at a preset frequency point, the relative power of each target lead corresponding to the significantly activated region in a preset frequency band, and the response index of each target lead corresponding to the significantly activated region in a preset frequency band, wherein the response index characterizes the relative power difference between the clockwise rotation segment and the counterclockwise rotation segment relative to the adjacent resting segment.

[0079] In this embodiment of the invention, the EEG feature data further includes: time-frequency feature data; and the time-frequency features include: short-time Fourier transform features of the difference signals of each target lead corresponding to the significant activation region in a preset frequency band, and event-related spectrum perturbation features of the difference signals, wherein the difference signals are the differences between the EEG signals of the clockwise and counterclockwise rotation segments relative to the adjacent resting segments.

[0080] In this embodiment of the invention, the EEG feature data further includes: brain network connectivity feature data; and the brain network connectivity feature includes: phase lock value features of any two significantly activated regions in a preset frequency band, wherein the phase lock value features characterize the signal synchronization degree of the target leads of the any two significantly activated regions in the preset frequency band.

[0081] In this embodiment of the invention, the execution unit is further configured to: input the multiple EEG feature data of each significantly activated region and the magnetic resonance activation intensity data into a pre-trained data fusion model to obtain the multimodal aggregated vector output by the data fusion model; and perform linear operations on each component of the multimodal aggregated vector to obtain the subject's operational intention (which may be one of a number of preset operational intentions).

[0082] Figure 4 An exemplary system architecture 400 is shown, which can be applied to the multimodal vestibular function assessment and balance compensation monitoring method or the multimodal vestibular function assessment and balance compensation monitoring system of the present invention.

[0083] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, network 404, and server 405 (this architecture is merely an example; the components included in a specific architecture may be adjusted according to the specific application). Network 404 serves as the medium for providing a communication link between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0084] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as vestibular function assessment applications (for example only).

[0085] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0086] Server 405 can be a server that provides various services, such as a backend server that supports the vestibular function assessment application operated by the user using terminal devices 401, 402, and 403 (for example only). The backend server can process received vestibular function assessment requests and feed back the processing results (such as assessment results - for example only) to terminal devices 401, 402, and 403.

[0087] It should be noted that the multimodal vestibular function assessment and balance compensation monitoring method provided in the embodiments of the present invention is generally executed by server 405, and correspondingly, the multimodal vestibular function assessment and balance compensation monitoring system is generally set in server 405.

[0088] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0089] The present invention also provides an electronic device. The electronic device of this invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multimodal vestibular function assessment and balance compensation monitoring method provided by the present invention.

[0090] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing an electronic device according to embodiments of the present invention. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0091] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0093] In particular, according to the embodiments disclosed in this invention, the processes described in the above main step diagrams can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the main step diagrams. In the above embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit 501, it performs the functions defined in the system of this invention.

[0094] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The units described in the embodiments of the present invention can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including: an acquisition unit, a lead localization unit, and an evaluation unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit can also be described as "a unit that provides BOLD signals and EEG signals to the lead localization unit."

[0097] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to perform the following steps: acquiring oxygenation level-dependent BOLD signals for various brain regions obtained from a functional magnetic resonance imaging (fMRI) examination of the brain, and electroencephalogram (EEG) signals from multiple leads obtained from an EEG examination; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task; acquiring a task timing function for the visual-motor stimulation task, and based on the task timing function... The magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task is determined by the BOLD signal of each brain region. Multiple vestibular response-related regions are selected from each brain region based on the magnetic resonance activation intensity data, and the leads corresponding to the vestibular response-related regions are determined as target leads. In addition, multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task are extracted from the EEG signal of the target leads. The multiple EEG feature data of each vestibular response-related region are fused with the magnetic resonance activation intensity data to generate a multimodal aggregation vector. The assessment result of the subject's vestibular function is determined based on the multimodal aggregation vector.

[0098] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the multimodal vestibular function assessment and balance compensation monitoring method provided by the present invention.

[0099] According to the technical solution of the present invention, when subjects simultaneously view the same optomotor stimulation task during fMRI and EEG examinations, respectively, it can generate highly consistent and easily acquired activation data on both the fMRI and EEG sides. This is beneficial for the localization of vestibular response-related areas and the extraction of relevant data, and for the fusion of fMRI and EEG feature data, enabling accurate assessment and quantification of vestibular function and balance compensation. Furthermore, the embodiments of the present invention can deeply combine high spatial resolution fMRI examinations with high temporal resolution EEG examinations based on the same optomotor stimulation task, forming a complementary spatiotemporal fusion framework to comprehensively characterize changes in vestibular function and balance compensation in subjects. The above methods of the embodiments of the present invention can be applied in EEG control scenarios, fusing fMRI and EEG based on the same optomotor stimulation task to acquire brain signals from subjects to accurately infer their operational intentions, thereby controlling controlled devices.

[0100] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for multimodal vestibular function assessment and balance compensation monitoring, characterized in that, include: The study acquires BOLD signals for blood oxygenation levels in various brain regions obtained from functional magnetic resonance imaging (fMRI) of the brain, and electroencephalogram (EEG) signals from multiple leads obtained from an electroencephalogram (EEG) of the brain. The subjects perform the fMRI examination and the EEG examination simultaneously with the pre-defined optomotor stimulation task. The task timeline function of the visual-motor stimulation task is obtained; based on the task timeline function and the BOLD signal of each brain region, the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task is determined; multiple vestibular response-related regions are selected from each brain region according to the magnetic resonance activation intensity data; and the leads corresponding to the vestibular response-related regions are determined as target leads; and, Multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task are extracted from the EEG signals of the target leads. The multiple EEG feature data of each vestibular response-related region are fused with the magnetic resonance activation intensity data to generate a multimodal aggregation vector. The assessment result of the subject's vestibular function is determined based on the multimodal aggregation vector.

2. The method according to claim 1, characterized in that, The optomotor stimulation task is viewed by the subject and includes multiple sub-tasks that are repeatedly performed. Each sub-task includes the following segments of equal duration and performed sequentially: a resting segment, a clockwise rotation segment, a resting segment, and a counterclockwise rotation segment. In the clockwise rotation segment, the black and white striped disk rotates clockwise at a preset speed; in the counterclockwise rotation segment, the disk rotates counterclockwise at the preset speed; and, The fMRI examination and the corresponding optomotor stimulation task were started at the same time. The EEG examination and the corresponding optomotor stimulation task were started at the same time.

3. The method according to claim 1, characterized in that, The determination of magnetic resonance activation intensity data for each brain region in response to the visuomotor stimulation task based on the task timing function and the BOLD signal of each brain region includes: Determine the hemodynamic response function (HRF) for each brain region of the subject, and perform convolution between the HRF and the task time-series function to obtain the baseline signal change curves for each brain region; and, The BOLD signal of each brain region is subtracted from the baseline signal change curve of the corresponding brain region to obtain the magnetic resonance activation intensity data of each brain region.

4. The method according to claim 1, characterized in that, The selection of multiple vestibular response-related regions from various brain regions based on the magnetic resonance activation intensity data includes: Based on the magnetic resonance activation intensity data, a predetermined number of brain regions with the highest magnetic resonance activation intensity are identified as initial selection regions; and, The initial regions located in the preset set of vestibular associated regions are determined as the vestibular response-related regions.

5. The method according to claim 2, characterized in that, EEG characteristics of any vestibular response-related region include: frequency domain characteristics; and, The frequency domain features include: the power spectral density of each target lead corresponding to the vestibular response-related region at a preset frequency point, the relative power of each target lead corresponding to the vestibular response-related region in a preset frequency band, and the response index of each target lead corresponding to the vestibular response-related region in a preset frequency band. The response index characterizes the relative power difference between the clockwise and counterclockwise rotation segments relative to adjacent resting segments.

6. The method according to claim 5, characterized in that, The EEG feature data also includes: time-frequency feature data; and, The time-frequency features include: short-time Fourier transform features of the difference signals of each target lead corresponding to the vestibular response-related region in a preset frequency band, and event-related spectrum perturbation features of the difference signals. The difference signals are the differences between the EEG signals of the clockwise and counterclockwise rotation segments and the adjacent resting segments.

7. The method according to claim 6, characterized in that, The EEG feature data also includes: brain network connectivity feature data; and... The brain network connectivity features include: the phase lock value features of any two vestibular response-related regions in a preset frequency band, wherein the phase lock value features characterize the signal synchronization degree of the target leads of any two vestibular response-related regions in the preset frequency band.

8. The method according to claim 1, characterized in that, The step of fusing the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregated vector includes: inputting the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data into a pre-trained data fusion model to obtain the multimodal aggregated vector output by the data fusion model; The determination of the subject's vestibular function assessment result based on the multimodal aggregated vector includes: performing a linear operation on each component of the multimodal aggregated vector to obtain the subject's vestibular compensation efficiency index; comparing the vestibular compensation efficiency index with at least two preset thresholds to determine the subject's vestibular function assessment result from at least three preset assessment results; and... The method further includes generating vestibular function assessment results for the subject at multiple assessment times to monitor the subject's balance compensation.

9. A multimodal vestibular function assessment and balance compensation monitoring system, characterized in that, include: The acquisition unit is used to acquire the blood oxygenation level-dependent BOLD signal of each brain region obtained from the subject's functional magnetic resonance imaging (fMRI) examination, and the electroencephalogram (EEG) signals of multiple leads obtained from the subject's electroencephalogram (EEG) examination; wherein the subject performs the fMRI examination while performing a preset visual-motor stimulation task, and the subject performs the EEG examination while performing the visual-motor stimulation task. A lead localization unit is used to acquire the task timing function of the visual-motor stimulation task, determine the magnetic resonance activation intensity data of each brain region for the visual-motor stimulation task based on the task timing function and the BOLD signal of each brain region, select multiple vestibular response-related regions from each brain region according to the magnetic resonance activation intensity data, and determine the leads corresponding to the vestibular response-related regions as target leads; and, An evaluation unit is used to extract multiple EEG feature data of each vestibular response-related region for the visual-motor stimulation task from the EEG signals of the target lead, fuse the multiple EEG feature data of each vestibular response-related region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, and determine the evaluation result of the subject's vestibular function based on the multimodal aggregation vector.

10. A brainwave control system, characterized in that, include: The acquisition unit is used to acquire the blood oxygenation level-dependent BOLD signal of each brain region obtained by the subject performing functional magnetic resonance imaging (fMRI) of the brain, and the electroencephalogram (EEG) signals of multiple leads obtained by the subject performing EEG; wherein the subject performs the fMRI while performing a preset optomotor stimulation task, and the subject performs the EEG while performing the optomotor stimulation task; A lead localization unit is used to acquire the task timing function of the optomotor stimulation task, determine the magnetic resonance activation intensity data of each brain region for the optomotor stimulation task based on the task timing function and the BOLD signal of each brain region, select multiple significantly activated regions from each brain region according to the magnetic resonance activation intensity data, and determine the leads corresponding to the significantly activated regions as target leads; and, The execution unit is configured to extract multiple EEG feature data of each significantly activated region for the visual-motor stimulation task from the EEG signal of the target lead, fuse the multiple EEG feature data of each significantly activated region with the magnetic resonance activation intensity data to generate a multimodal aggregation vector, perform linear operations on each component of the multimodal aggregation vector to obtain the subject's operation intention, and send the obtained operation intention to a pre-connected controlled device.

Images